Hi,

I've been searching on this topic for couples of months but found no good solution yet so I thought you guys might have some ideas.

What I am now having is a Delphi client-server SQLite application that uses socket communications (which i'm porting file by file to FPC/Lazarus so as to be able to run on Linux). The client (single client currently) sends commands to the socket server to retrieve information from SQLite, to update records, and to interface with devices on the PC. This is done at high speeds so thousands of records can be retrieved at one go in a second.

Now, my customer needs me to provide a multi-user web interface (with login session for every user). The most obvious way will be to rewrite the whole client in pure PHP but this will not be easy as I'll need to:

1. Convert sqlite into Postgresql or firebird so that multiple clients can connect to the database at the same time.
2. Get my Delphi programmer to learn PHP.

Obviously, this is not the most desirable way, so I went about searching and found out that AJAX might be able to work with FPC. From what I've read, I reckon that I'll need to create an AJAX client and a server coded in FPC (which I'll need to somehow convert from my existing server). A couple months back, I've also experimented with ExtPascal but it couldn't work.

For AJAX/RIA, there seem to be three market leaders. Here a summary of Internet posted comments for them which I have gathered (Note: i've not tried them myself, at least directly).

1. Qooxdoo.org - this seems to be a popular open source framework, free for commercial use, but is less popular than Extjs (which Extpascal use) and seems to have got less features. It provides an AJAX to RPC Server option but there's no pascal example for RPC Server. There are comments made by people on the Internet that Qooxdoo is tied to Python, but i don't know if Python has to be installed on the deployment server itself. Qooxdoo also comes in the form of VCL (for use in RADPHP or Delphi4PHP) - I don't know if they are based on the latest version or if they work with RPC Server.

2. ExtJS - this looks like the market leader judging from the size of the community and the Alex rank for sencha.com. It is only free for GNU deployments. The developer license is priced around US$500, and distribution is royalty free i think. I also came across another commercial framework (can't remember the name) that charges couple hundreds per distribution so this sounds cheap in comparison! Like Qooxdoo, it also comes with RPC framework, which they call ExtDirect, and like Qooxdoo, there's no pascal example. I've tested ExtJS indirectly when using Extpascal, the interface generated by ExtJS seems to be quite slow - I've not really done an equivalent comparison with Qooxdoo, but Qooxdoo seems to be a little quicker.

3. JQuery - this seems to be a basic JS framework as compared to the previous two. Can't meet my requirement.

I am curious to know if any of you have used any of the RIA tools using FPC and their suitability. Please share if you have found a better way. Thanks.

Rgds,
Joshua

Joshua,

Now, my customer needs me to provide a multi-user web interface (with login session for every user).

what about using RDP (remote desktop protocol) ?

It works very well. And your windows application becomes a web application is minutes :)

See also

http://www.microsoft.com/windowsxp/using/networking/expert/northrup_03may16.mspx

>

What I am now having is a Delphi client-server SQLite application that uses socket communications (which i'm porting file by file to FPC/Lazarus so as to be able to run on Linux). The client (single client currently) sends commands to the socket server to retrieve information from SQLite, to update records, and to interface with devices on the PC. This is done at high speeds so thousands of records can be retrieved at one go in a second.
<<

could you please expand on this ? Maybe privately ?

It seems something I have been searching for months

Thank you

Duilio

I work with ExtJS and FPC extensively currently. you'll notice that there are ExtDirect support files in FPC: see packages/fcl-web. There are examples available.

Since I need this for commercial work, you're guaranteed of good support :-)

ExtJS and Ext.Direct are a breeze. The Store for ExtJS data controls is controlled by a FPC WebDataProvider - essentially a no-code solution. All server side coding can be done in RAD.

For my part, ExtJS would be the way to go.

Michael.

Did you try ExtPascal, too ?

Did you get it working with FastCGI ?

Did you get "Server Events" working with the means for that ExtJS is said to provide (but ExtPascal did not support last time I checked) ?

Thanks,
-Michael

Yes. It is a good start and contains some good ideas, but it suffers from some major design flaws that IMHO prevent it's widespread use. I don't see the possibility (or the desire from the maintainer) to fix the flaws.

Did you get it working with FastCGI ?

I work only with FastCGI, and so does extpascal.

Did you get "Server Events" working with the means for that ExtJS is said to provide (but ExtPascal did not support last time I checked) ?

Not sure what you mean by that. But Fcl-Web supports all you need to work with ExtJS. I have meanwhile 4 apps up and running.

Michael.

Michael Van Canneyt wrote

Not sure what you mean by that.

Sorry, but my old brain does not come up with the correct name of the method (which I do know, though). :(

Here a (FastCGI) polling access is generated by the client java script, and the CGI-Application does not respond so it hangs. The server responds immediately to the still hanging request whenever an asynchronous "server-event" arises. The client java script gets this response and acts appropriately. There are some precautions that the requests does not hang so long that the WebServer times out and kills it.

This is necessary to overcome the asymmetry of the http protocol.

IMHO "Server-Events" are necessary to design a CGI program with some complexity.

Moreover "server-Internal-Events" like from Timers, serial or "direct" TCP/IP communication, thread-notification, ... are very desirable and should be able to fire "server-events" towards the client.

-Michael

Michael, where is WebDataProvider, it's a Lazarus component? I can't find in Lazarus component palette.

Is there a wiki or manual page about it?

ExtJS has no such mechanism to my knowledge, but I'm not a reference.

This is necessary to overcome the asymmetry of the http protocol. IMHO "Server-Events" are necessary to design a CGI program with some complexity.

Well, I create seriously complex applications. No such mechanism was necessary till now.

Moreover "server-Internal-Events" like from Timers, serial or "direct" TCP/IP communication, thread-notification, ... are very desirable and should be able to fire "server-events" towards the client.

Why ? What could you possibly need to do that the server needs to contact the client ? Everything you need can be programmed in the client, in my experience.

You must be making some very weird applications that you need such mechanisms :-)

The general rule here in my company is:
Every call to the server is a call too much.

Let alone a 'persistent' connection...

Michael.

The components are in fcl-web/src/webdata, in trunk.

In lazarus you must install the lazwebextra package. ("extra", because it needs FPC 2.5.1, and as such should not interfere with the fixes branch)

Is there a wiki or manual page about it?

Unfortunately not yet; but the fcl-web directory contains some demo programs.

I should make a diagram that explains how it works. Alas, too little time and too much ideas :/

Michael.

Nice!, I installed LazWebExtra and found it also includes many TExt... components.

I'll take a look at it later today.

Michael Van Canneyt wrote

ExtJS has no such mechanism to my knowledge, but I'm not a reference.

last time I checked some months ago it was said to be a new feature, but should work, even though I did not find anybody who has tested it.

Hmm, How to do notify the user that the server has finished complex work in the background without blocking the interface and/or creating excessive http polling traffic ?

An easy to do (non complex) example for a good application for such a method is a server that allows tow client to chat.

Above trivial example ("chat server")

Real-world: Imagine an "embedded" program automatically working with some hardware interfaces and/or serial and/or TCP/IP connections. Now you want to provide it with a "web" interface that allows a "user" to connect to it and do configuration and monitoring. For monitoring this method is essential: many "states" in the program permanently change due to the work of the program and the external situation. These state changes need to be shown in the remote browser with low latency and without excessive communication overhead.

You must be making some very weird applications that you need such mechanisms :-)

I do "embedded" stuff all the time :)

-Michael

Also check out dojo JS framework:

http://www.dojotoolkit.org/

Hi Duilio,

Thanks for your suggestion, but RDP is not an option as the users are really non-IT end users, not the administrator.

I'm not the programmer :), i just wrote the high level specs, but you can email me.

Define excessive polling ?

In my experience: keeping a connection open IS excessive communication overhead and very unreliable to boot.

I thought like you, 12 years ago. Till reality kicked in. Faulty switches, company firewalls and whatnot. Not all servers/firewalls support HTTP 1.1 (for a persistent connection) etc. Too many problems.

So you end up with polling anyway. 10 polls per second is enough to fool the human eye and doesn't stress the server too much.

Michael.

Hi Michael, thanks, will check out.

Did you get fpWeb/FastCGI + Windows + Apache (with fcgid module) running?

I tried a simple example without success

I did not tried the apache fastcgi module ( fcgid)

Luiz

Well, normally I use mod_fastcgi, but I also got fcgid working on linux.

mod_fastcgi works on windows and on linux. I prefer this, because it has the ExternalFastcgiServer option, which is very convenient.

Michael.

Thanks. I used fcgid because they deprecated fastcgi. I will give a try with the later to see if works (I will also test with IIS)

Luiz

Aren't ExtJS Ajax calls asynchronous? The JavaScript code continues after firing a call to the server. There is a "callback" event that we can use when the server sent the reply back (and process success/failure events on the call).
At least I had to do some things to NOT give back the control to the user over the client side until the response arrives from the FPC server CGI/FCGI app (like disabling all controls for example).

IMO Exjs and FPC/Lazarus works superbly (you need FPC and Lazarus from the trunk SVN at the moment for it). You do not need

Please, someone could start a wiki page for starters? Some kind of tutorial for a start... with let?s say a create/edit/delete/browse in a db table, a login form, a report (builded in pdf)... I don?t know.
I guess understanding this is enaught to keep going... Also server setup should be there.
May be this is allready there... but an introduction page will be grait! I have a lot of experience writing applications in delphi and something with Lazarus and FPC for desktop, but in my case I want to learn what Michael is doing, web applications with fcl-web + extjs. Somebody told that will try to see if this works with IIS... does it?

Thanks!

Horacio

Hi Horacio, thanks for suggesting, even a text point form step-by-step instruction, listing the entire process from coding the client, the server to deployment on a web server like Apache will help a lot.

Joshua,

>Thanks for your suggestion, but RDP is not an option as the users are really non-IT end users, not the administrator.

not sure you got the point.

Rgds

Duilio

Some kind of tutorial for a start...

Leonardo Rame blog is full of precious gems

http://leonardorame.blogspot.com/2010/02/web-20-programming-with-object-pascal.html

(and many other articles)

Rgds

Duilio

Since those articles, FPC's FCL-web and Lazarus' fpweb/lazwebextra has more native ExtJS support built in (SVN Trunk)
Only a fraction of Lazarus/FPC side coding is needed compared to it.

AB

Forgot to mention that there are some basic demos created by Michael under fpc/packages/fcl-web/examples/webdata/

Michael Van Canneyt wrote

Define excessive polling ?

IMHO, any "dump" polling is excessive. IMHO, a decent method needs to "sleep" most of the time in a polling cycle (not generating CPU or network traffic) and the asynchronous event needs to "wake" the process. This is the only way to provide both low latency and low overhead.

Yep ! HTTP is essentially not suitable for this. You would be much better off using normal bidirectional TCP/IP communication. It's only because of the dump IP departments that this silly http add-ons are necessary.

-Michael

ABorka wrote

You do not need course not. ExtPascal is just a library that might help to use ExtJS

more conveniently.

-Michael

... But unfortunately currently introduces too many problems to be usable. A pity, because it contains some great ideas.

Michael.

Horacio Jamilis wrote

Somebody told that will try to see if this works with IIS... does it?

In fact a colleague of mine did something like this with IIS, But on a strictly "commercial" level using payed tools such as Delphi and RemObjects.

He did a small "ISAP-Gateway" (a DLL to be called by ISS) by using the RemObjects "ISAPI protocol" feature. This "gateway" communicates with a free running program using RemObject's inter process communication features.

Very comfortable and flexible and decently fast.

Rem,Objects is specified to work with FP, so maybe doing this without Delphi might be possible.

-Michael

Even so, we use TCP/IP communication: same problems. Faulty firewalls and switches. I suspect that in 25% of cases, things go wrong if you work over internet. Hardly noticeable for HTTP with its small requests (and of course all ISPs make sure HTTP traffic gets priority) but a major mess if you use regular TCP/IP. You're better off with UDP these days.

Things may be better on intranets, but for internet persistent connections are a faraway dream. We had to spend lots of time on a error-detection + reconnect scheme to work around all problems.

Michael.

Currently, for IIS there are 2 options:

- Use fastcgi support for IIS. It has the drawback that IIS chooses to start/stop your app at will (as per the fastcgi specs)
- Create the server app as a fastcgi service, and use the CGI gateway to connect to it.

I still have IIS ISAPI module planned (I need it for work too, so that should be there before the end of the year).

Michael.

Finally in a short non-clouded moment my brain produced the name of the "standard" fastcgi "reverser event" method: "Comet".

regarding ExtJS: this seems to be called "Ext.Direct" and should be available in ExtJS 3.0 see e.g. :

http://ajaxian.com/archives/ext-js-30-roadmap-and-developer-tool http://fritzthomas.com/javascript/extjs/322-interesting-features-for-extjs-on-their-roadmap/

Does this trigger any more comments ?

-Michael

Hi Michael, have you explored the possibility of using ExtDesigner for the frontend?

http://www.sencha.com/products/designer/

Rgds,
Joshua

Hi Michael, have you explored the possibility of using ExtDesigner for the frontend?

http://www.sencha.com/products/designer/

Rgds,
Joshua

Joshua Lim wrote

Hi Michael, have you explored the possibility of using ExtDesigner for the frontend? http://www.sencha.com/products/designer/

I have, and it works fine with fcl-web.
The front-end is totally separate from the back-end.

Michael.

Last time I tried (july) this didn't work. For FastCGI I used direct socket-access, but locking on Windows works too differently from Linux to make this work.

So I had to come with a solution for that, but haven't worked on it since.

Joost.

I committed some fixes so fastcgi works on windows as well ? At least it works here ?

Michael.

That's possible. That's why I said it didn't work in july.

Joost.