Hi,

Just over 3 years ago I asked about FPC support for creating 3-tier database applications. The answer was basically: it's not possible (unless you roll your own Midas / Datasnap code).

TClientDataset also missed some vital features, and the state of WebServices was also not complete.

Here is the original message thread:

So after 3 year, what has changed? Is the missing parts already implement, and is a Midas / Datasnap style 3-tier application possible with FPC?

I also believe Borland/CodeGear/Embarcadero released the Midas source code in post Delphi 7 versions. Anybody have success in compiling that code with FPC? Yes I know you can't release such code, and yes I know Datasnap will probably only work with Windows. But from my Kylix 3 manual it mentions TSoapConnection and WebServices for multi-tier development, so was wondering if FPC had no alternative, if I could possibly compile that Kylix 3 code using FPC - anybody tried this before?

Regards,
- Graeme -

In our previous episode, Graeme Geldenhuys said:

Just over 3 years ago I asked about FPC support for creating 3-tier database applications. The answer was basically: it's not possible (unless you roll your own Midas / Datasnap code).

- kbmMW http://components4developers.com/
the author was on the Lazarus day in 2010.

Afaik sb was also working on porting midware (www.overbyte.be), but while I haven't look at it for years I assume that is very windows centric.

I take 3-tier above as a general framework for 3-tier, not necessarily a Datasnap compatible layer.

I also believe Borland/CodeGear/Embarcadero released the Midas source code in post Delphi 7 versions.

No, starting only with D2010 afaik, and only enterprise and higher SKUs

Anybody have success in compiling that code with FPC?

That would be very hard since it is C++.

Since Delphi XE2 last December you can make datasnap apps for OS X and very experimental for iOS. Linux is said to follow in the next release.

However that is based on Firemonkey's ownerdrawn framework, and thus needs a separate codebase for the GUI. Firemonkey IS available for windows too though.

Note that stuff like datasource remains the same, but how non visual db aware components are databound to FM visual components differs totally.

There is no grid, but the stringgrid like has some databinding properties.

To be honest, it looked a bit convoluted to me during the demo, but that might just be the inexperience.

Out of the box: no.

Midas is written in C++, so that's not going to happen.

I got sidetracked on web-development, so the 3-tier support per-se is missing.

OTOH the web-development part has resulted in a ready-to-use packet transport layer. It's inefficient though, since it uses JSON or XML, but that can easily be adapted to support a 'binary' packet.

And it would only work with HTTP as the transport layer.

Michael.

Marco van de Voort wrote

- kbmMW http://components4developers.com/ the author was on the Lazarus day in 2010.

Thanks, I'll take a look.

I take 3-tier above as a general framework for 3-tier, not necessarily a Datasnap compatible layer.

Correct, it doesn't need to be compatible with Delphi. Any 3-tier support will do.

:-) Thanks for correctly me, I didn't know the details of each of those.

However that is based on Firemonkey's ownerdrawn framework, and thus needs a separate codebase for the GUI. Firemonkey IS available for windows too though.

FireMonkey seems to be missing many vital parts for a actual project development (lets ignore those silly 3D demos they love to show - which doesn't reflect real world applications). FireMonkey has no grids, no TAction, many missing events etc etc. It seems like a large "rush job" from Embarcadero.

Also FireMonkey seems more geared towards mobile device applications (iPhone, iPad, Android), and not desktop business applications.

Regards,
- Graeme -

Out of the box: no.

OK, thanks. Do you know if TClientDataset has improved at all?

Midas is written in C++, so that's not going to happen.

I didn't know that.

OTOH the web-development part has resulted in a ready-to-use packet transport layer. It's inefficient though, since it uses JSON or XML, but that can easily be adapted to support a 'binary' packet.

I just finished watching a CodeRage 5 Datasnap demo. The guy said that XML packet transport is extremely slow (because XML is generally hard to parse). Simply changing to CSV packet format gave a 20x speed improvement, but obviously CSV is not self-describing.

Is parsing JSON any faster than XML? Sorry if this is a stupid question, but I know near zero about JSON.

OK, so it seems worth my while to port the tiOPF Remote Persistence Layer to FPC+Synapse (or maybe lNet) then. Based on an old message from Peter Hinrichsen, tiOPF can take care of everything for me, without 3rd party components or libraries, including some state information. He also mentioned that [at the time of his message] it was well unit tested and already used for 4 years in a production environment. The current tiOPF Remote Persistence Layer uses Indy's HTTP client & server components and has two data packet formats it supports (one being more compressed). I'll dive into that code then and see what I can manage with FPC.

Regards,
- Graeme -

Marco van de Voort wrote

- kbmMW http://components4developers.com/ the author was on the Lazarus day in 2010.

Strange, nowhere on the kbmMW website do I see any mention of Free Pascal Compiler or Lazarus IDE support.

http://www.components4programmers.com/products/kbmmw/featurematrix/supporteddatabaseapi.htm

http://www.components4programmers.com/products/kbmmw/featurematrix/supportedides.htm

It mentions Kylix 3 support for under Linux, but not FPC. Maybe his website is just out of date?

Regards,
- Graeme -

It depends on a number of factors.

I did extensive testing of the speed difference in XML and binary messaging.

One of the conclusions was that you can gain an overall speed improvement of a factor 6 when switching from XML to binary messaging. Not using HTTP but a self-made TCP/IP protocol gets you another factor.

SOAP over HTTP (the standard for webservices) has the incredible advantage that it is cross-platform, and meanwhile any language can handle it. The incredible downside is that it is slow and bulky.

Our client/server apps used HTTP/SOAP it when we just started. It took us less than a week to realize this was a very bad idea, so we switched to TCP/IP and binary messages. Never looked back since.

(and this was before I did the extensive testing)

Is parsing JSON any faster than XML? Sorry if this is a stupid question, but I know near zero about JSON.

Yes it is.
It's much less verbose also, which means the transported packets are much smaller.

If you do, then I'd be very interested in hearing about your progress.

Michael.

Yes. I know that the support is ready, but not widely advertized, because lazarus lacks binary package support.

Michael.

factor 6 when switching from XML to binary messaging. Not using HTTP but a self-made TCP/IP protocol gets you another factor.

Are both the binary message format and self-made TCP/IP protocol publicly available? Maybe that might be worth implementing in tiOPF too, as an optional setting to improve the remote persistence layer performance.

Yes it is. It's much less verbose also, which means the transported packets are much smaller.

Excellent, I'll try and add JSON support to tiOPF too. A quick look on the web at the JSON syntax, it does indeed looks much more human readable and "simpler" than XML. I'm not a big fan of XML at all.

If you do, then I'd be very interested in hearing about your progress.

I'll keep you informed. I'll probably start on this early next week.

Regards,
- Graeme -

I created a FastCGI based server that handles JSON requests from a Win32/Linux GUI app, that uses a custom made ORM similar to tiOPF and it works really fast, even on slow-long distance networks. I never had to do this, but as most modern http servers support gzip compression, one alternative to binary formats is to enable compression on server side, and decompress on client side.

This approach has the advantage of JSON readability and the small size of binary format.

Leonardo M. Ramé
http://leonardorame.blogspot.com

That's exactly what we do also.
But in the case of large packages (we have datasets of 30.000 records), the JSON is really slow.

The browser chokes already on a dataset of 3000 records, when using ExtJS =-)

Michael.

----- Original Message -----

In those cases, we use pagination. We allways ask for record 1-100, 101-200, and so on.  
Leonardo M. Ramé
http://leonardorame.blogspot.com

Leonardo M. Ramé wrote

----- Original Message -----

<...snip...>

>> I created a FastCGI based server that handles JSON requests from a >> Win32/Linux GUI app, that uses a custom made ORM similar to tiOPF

and it

>> works really fast, even on slow-long distance networks. I never

had to do

>> this, but as most modern http servers support gzip compression, one >> alternative to binary formats is to enable compression on server

side, and

>> decompress on client side. >> >> This approach has the advantage of JSON readability and the small

size of

ExtJS =-)

> > Michael. > In those cases, we use pagination. We allways ask for record 1-100,

101-200, and so on.

Leonardo M. Ramé

Exactly.
Also, with ExtJS 4.0 they made a point of significantly speeding up most things.
Usually, the problem with ExtJS 3.x is not with the slowness of JSON itself compared to binary communication, but with the thousands of DOM elements you need to create to display such a big set of data using javascript on client browser side.
Of course, it is still a 4.0.x version at the moment, so I would wait until 4.1 at least before using it in a production environment.

AB

I know it's a bit late, only just noticed this thread, but we support FPC with DA and RO too. www.remobjects.com/da

Carlo Kok wrote

I know it's a bit late, only just noticed this thread, but we support FPC with DA and RO too. www.remobjects.com/da

Thanks for pointing out another alternative. I'll go ahead and read about DA now. For my immediate needs, I am abstracting and implementing the tiOPF (n-tier via embedded HTTP server) solution for use with Free Pascal.