Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2019-02

New feature: SQLDB Rest bridge

8 messages · 4 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Michael Van Canneyt

Hello,

We received some feedback about Pas2JS - our effort to bring Pascal to the browser - that easy database connectivity is an issue, hampering easy development.

To alleviate the problem I have created the SQLDB Rest bridge.

This is a set of components that allow you to expose any database supported by FPC's SQLDB as a REST service. It is designed to be simple to use, yet extensible.

A database can be exposed using a single line of code:
  FDisp.ExposeDatabase('PostGreSQL','localhost','expensetracker','me','secret',Nil,[foFilter,foInInsert,foInUpdate,foOrderByDesc]);

But no code is needed, the whole can be configured in the IDE if so desired.

This is not meant as a general REST service framework, (you can look for instance at the Brook framework for this) but is meant to easily expose SQL databases using a REST protocol.

Not-so-Short summary of possibilities:

SQL/REST:

* Every REST resource is defined by up to 4 SQL statements, corresponding to the 4 CRUD operations. But statements can be auto-generated.

* The CRUD Statements can be auto generated on the fly, based on a table name and field definitions.

* For every field, an alias can be provided.

* The following client-side types are available, automatically mapped from the native database type:

String, 32/64-bit integer, float, Boolean, Date, Time, DateTime, Blob

* Support for a sequence to generate IDs is built-in.

* Full control over which operations (GET,POST,PUT,DELETE) are allowed.

* Resources are collected in a schema.

* Multiple schemas can be attached to the service.

* Multiple databases (connections) can be defined.

* A schema can be tied to a single connection, or connections can share a schemas (use case: a different connection per client).

* Business processor components can be hooked to a resource, to make it easier to implement business logic. (not yet committed)

* SQL Statements can contain parameters, values for the parameters will be picked up from the request URL.

* Support to get custom datasets.

* Support for client-side provided SQL SELECT statements (optional, disabled by default)

* Full support for configuration through an <code>.ini</code> file out of the box.

HTTP:
* Authentication is handled using the HTTP protocol.

* Basic authentication is included by default, but is completely pluggable.

* Basic authentication can look up valid users in a database (by default the database being exposed)

* Output format can be fixed or detected per request (?fmt=format). Detection based on content-type is also available.

* list of fields to include in output can be specified in the URL: ?fl=field1,field2

* Field list of fields to exclude from output can be specified in the URL: ?fe=field1,field2

* Various input/output formats are available out of the box

JSON (the default),XML,CSV, CDS (Format used by Delphi Datasnap)

* A factory pattern is used, new formats can be added at will.

* Simple URL schemes. 2 basic schemes are available: BASEURL/Resource[/ID]
BASEURL/Connection/Resource[/ID]

* Support for introspection/discovery or metadata.

* limit,offset URL parameters for paging of results: ?limit=10&Offset=50

* A maximum limit can be enforced.

When the SQL statements support it, the limit and offset are translated to SQL fetch/offset clauses.

* Sort order can be specified in the request URL, if the resource definition allows it, using?sort=fieldname ( The ability to sort can be specified on a field basis)

* Filtering can be specified on a field basis in the request URL, and the filter operation is translated to SQL Where: ?MyField=10
?MyField_null=1 ?MyField_null=0
?MyField_lt=10 ?MyField_lte=10
?MyField_gt=10 ?MyField_gte=10

* The ability to filter on a field can be specified in the schema for each field.

* The request can specify whether metadata should be included in the response: ?metadata=1

* The request can specify whether the result should be human-readable (nicely formatted) or not: ?humanreadable=1

A more complete explanation is in the WIKI

http://wiki.freepascal.org/SQLDBRestBridge

2 demo programs exist, more are planned (there is even a Delphi demo showing how to read the data with Delphi's TClientDataset)

Lazarus integration is also available, the components are compilable with FPC 3.0.4.

This is an initial version (fully functional), several extensions are planned:
- Zero-config API
(a single generic binary can be shipped, no config files should be needed)
- ADO packages
- Integrate in lazarus Database desktop for easy development
- Integrate in pas2JS compile server for easy development.

Remarks, comments, suggestions welcome.
(to a lesser extent: bugreports ;) )

Michael.

Reply by email

Sven Barth
to Michael Van Canneyt

Michael Van Canneyt via lazarus schrieb am
So., 24. Feb. 2019, 17:46:

Show quoted text 13 lines
Hello, We received some feedback about Pas2JS - our effort to bring Pascal to the browser - that easy database connectivity is an issue, hampering easy development. To alleviate the problem I have created the SQLDB Rest bridge. This is a set of components that allow you to expose any database supported by FPC's SQLDB as a REST service. It is designed to be simple to use, yet extensible.

Cool! Great work!
Now I only need to get the parts for implementing an Interface using RTTI working and I'd be all but set :D

HTTP:

Show quoted text 7 lines
* Authentication is handled using the HTTP protocol. * Basic authentication is included by default, but is completely pluggable. * Basic authentication can look up valid users in a database (by default the database being exposed)

How would one check whether the user is allowed to execute an action?

Regards,
Sven

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190225/3a4fc591/attachment-0001.html>

Reply by email

Michael Van Canneyt
to Sven Barth
Sven Barth wrote 20 lines
Michael Van Canneyt via lazarus schrieb am So., 24. Feb. 2019, 17:46: > > Hello, > > We received some feedback about Pas2JS - our effort to bring Pascal to the > browser - that easy database connectivity is an issue, hampering easy > development. > > To alleviate the problem I have created the SQLDB Rest bridge. > > This is a set of components that allow you to expose any database supported > by FPC's SQLDB as a REST service. It is designed to be simple to use, yet > extensible. > Cool! Great work! Now I only need to get the parts for implementing an Interface using RTTI working and I'd be all but set :D

You mean for an RPC mechanism ?

Isn't Invoke() working already ? I have a JSON RPC implemention on my list, and thought to use Invoke() for that.

Show quoted text 11 lines
HTTP: > * Authentication is handled using the HTTP protocol. > > * Basic authentication is included by default, but is completely pluggable. > > * Basic authentication can look up valid users in a database (by default > the database being exposed) > How would one check whether the user is allowed to execute an action?

There is an OnAllowResource event, after the authentication. One of the parameters is the operation: GET, PUT etc.
If you set the 'allow' to false, you'll get a 403, 'Forbidden'. It's not yet committed, because I wanted that together with the business processor component, which will make implementing all kinds of events simpler. There are stilll several improvements waiting to be tested and committed.

Michael.

Reply by email

Sven Barth
to Michael Van Canneyt

Michael Van Canneyt via lazarus schrieb am
Mo., 25. Feb. 2019, 08:07:

Show quoted text 34 lines
On Mon, 25 Feb 2019, Sven Barth via lazarus wrote: > Michael Van Canneyt via lazarus schrieb am > So., 24. Feb. 2019, 17:46: > >> >> Hello, >> >> We received some feedback about Pas2JS - our effort to bring Pascal to the >> browser - that easy database connectivity is an issue, hampering easy >> development. >> >> To alleviate the problem I have created the SQLDB Rest bridge. >> >> This is a set of components that allow you to expose any database supported >> by FPC's SQLDB as a REST service. It is designed to be simple to use, yet >> extensible. >> > > Cool! Great work! > Now I only need to get the parts for implementing an Interface using RTTI > working and I'd be all but set :D You mean for an RPC mechanism ? Isn't Invoke() working already ? I have a JSON RPC implemention on my list, and thought to use Invoke() for that.

RTTI.Invoke() itself is working as is TRTTIMethod.Invoke(), so the server side wouldn't be a problem, but what I'm still missing is TVirtualInterface for the client side.

Show quoted text 21 lines
> > HTTP: >> * Authentication is handled using the HTTP protocol. >> >> * Basic authentication is included by default, but is completely pluggable. >> >> * Basic authentication can look up valid users in a database (by default >> the database being exposed) >> > > How would one check whether the user is allowed to execute an action? There is an OnAllowResource event, after the authentication. One of the parameters is the operation: GET, PUT etc. If you set the 'allow' to false, you'll get a 403, 'Forbidden'. It's not yet committed, because I wanted that together with the business processor component, which will make implementing all kinds of events simpler. There are stilll several improvements waiting to be tested and committed.

Great! Looking forward to that!

Regards,
Sven

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190225/6c8ff82d/attachment.html>

Reply by email

Mattias Gaertner
to Sven Barth
[...] RTTI.Invoke() itself is working as is TRTTIMethod.Invoke(), so the server side wouldn't be a problem, but what I'm still missing is TVirtualInterface for the client side.

Do you mean fpc's unit RTTI?
pas2js has TVirtualInterface.

Mattias

Reply by email

silvioprog
to Michael Van Canneyt: We received some feedback about Pas2JS - our effort to bring Pascal to the browser - that…

On Sun, Feb 24, 2019 at 1:46 PM Michael Van Canneyt via lazarus <

Show quoted text 14 lines
wrote: Hello, We received some feedback about Pas2JS - our effort to bring Pascal to the browser - that easy database connectivity is an issue, hampering easy development. To alleviate the problem I have created the SQLDB Rest bridge. This is a set of components that allow you to expose any database supported by FPC's SQLDB as a REST service. It is designed to be simple to use, yet extensible.

This is amazing news! 😀

Thanks the effort to make it available for the community.

I'm going to upgrade my laz/fpc copy and test it at home...

best,

Silvio Clécio -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190225/bb78d625/attachment.html>

Reply by email

silvioprog
to Michael Van Canneyt: You mean for an RPC mechanism ? Isn't Invoke() working already ? I have a JSON RPC…

On Mon, Feb 25, 2019 at 4:07 AM Michael Van Canneyt via lazarus <

wrote:

[...]

Isn't Invoke() working already ? I have a JSON RPC implemention on my list, and thought to use Invoke() for that.

Sven implemented it for Windows and it is working fine. I've implemented it for Linux (sysv x64) and it is almost done, we just need some fixes in the assembly code to handle functions with results of 128-bit. Sven helped me a lot (thanks dude! :-) ) to understand all the tests.

(I'm finishing a small project at company and I'll return to this work to fix invoke() for Linux :-) )

Silvio Clécio -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190225/691fbdd2/attachment-0001.html>

Reply by email

Sven Barth
to Mattias Gaertner: On Mon, 25 Feb 2019 11:24:29 +0100 Do you mean fpc's unit RTTI? pas2js has…

Mattias Gaertner via lazarus schrieb am
Mo., 25. Feb. 2019, 12:22:

Show quoted text 11 lines
On Mon, 25 Feb 2019 11:24:29 +0100 Sven Barth via lazarus wrote: >[...] > RTTI.Invoke() itself is working as is TRTTIMethod.Invoke(), so the > server side wouldn't be a problem, but what I'm still missing is > TVirtualInterface for the client side. Do you mean fpc's unit RTTI? pas2js has TVirtualInterface.

Yes, I mean FPC's RTTI unit. After all one could use this also for native FPC clients ;)

Regards,
Sven

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190225/28ed6508/attachment.html>

Reply by email