Hi,

In revision 31011 I committed support for a HTTP server application in Lazarus, in the lazwebextra package. This is similar in functionality to the embweb package by Joost, but it does not use any external packages: you only need the latest FPC for this (at least rev. 17633).

It has an option to be able to serve regular files out-of-the-box. This is implemented as a regular fpWeb HTTP module, which simply sends a file if it finds it. This module is implemented in a (new) unit fpwebfile. It works similar to the Apache server's Alias directive:

  RegisterFileLocation('files','/some/existing/directory');

will map /files/index.html to /some/existing/directory/index.html and send it. Multiple locations can be registered.
if a mime.types file is present, the correct content-type is sent.

The FPC packages/fcl-web/examples/httpapp directory contains an example of how to use all this.

This means that FPC can now be used out of the box to implement web applications/pages as
- CGI.
- FastCGI.
- Apache module.
- Standalone webserver.
IIS ISAPI support is planned.

Comments/suggestions welcome.

Michael.

Great!.

Michael, I added a webmodule to testhttp and got two problems.

1 - Could not determine HTTP module for request "". Even if I call it the same way as I use with fpCGI and fpEmbWeb.

2 - The 2nd time I run the program I get. ESocketError "Binding of socket failed: 8080".

I'm running Ubuntu 11.04 - 64bits.

That is strange, because it works fine here ?
Can you send a test program please ? (in private) then I will test.

Did you test the sample program ? Does it happen there too ?

2 - The 2nd time I run the program I get. ESocketError "Binding of socket failed: 8080".

Yes. This can happen sometimes, and it is normal.

You need to wait till the kernel releases the address again. Waiting 20 seconds is usually enough. (I agree it is annoying ;) )

It is on my todo list to add setsocketopt(SO_REUSEADDR) support to ssockets. This will remedy the problem. I will try to do it this weekend.

I'm running Ubuntu 11.04 - 64bits.

Good choice. I run the same system ;-)

Michael.

Leonardo M. Ram? wrote

Michael, I added a webmodule to testhttp and got two problems. 1 - Could not determine HTTP module for request "". Even if I call it the same way as I use with fpCGI and fpEmbWeb.

This error usually happens when the project main program does not have the webmodule unit in it's uses statement.

I managed to reproduce the error; The error is only apparent if there are multiple modules registered. I committed a fix in 17647.

This raises the question again of the FPWeb behaviour for

http://somehost/path

Now, 'Path' is interpreted as the name of the action for the default module.

For an embedded server, this is not logical. This should be the module name, not the action name.

Michael.

I disagree. The vast majority of web applications do not need multiple web modules (usually the default module is used), but they use multiple actions.
Therefore, action names should have priority over module names if only one of them is passed.

AB

Hm. My experience is very different:
I didn't make a web application yet that had a single module. All had at least 3 or 4... And they're all in production.

Except some demo applications for FPC, they usually have 1 :-)

Maybe we should introduce 'TWebHandler.PreferModuleName : Boolean', which is False by default.

Michael.

And you do not specify the actions in the URLs when multiple web modules are in your application? I think actions are always specified when web modules are in the calling URL. But it is not true the other way around.

AB

Well, I never use actions. I always use the OnRequest handler of the module itself.

Michael.

ABorka, that's not the case of my program. I declared my webmodule's unit in the main program.

Heh, then we use fpweb differently (as I am sure other people prefer doing it in many additional ways). :)
I use one single web module with dozens of actions compared to multiple web modules with a single request handler that sorts out things. Much simpler and shorter calling URLs needed this way (an important thing when encrypting the action name and request Query parameters), and for me this way gives a nice organized source code in the web module, as all functions are nicely separated into action event handlers. I am sure there are many perfectly good ways - other that the two of us are using - to use fpweb.

In my humble opinion, if someone uses Web Actions (and they are there and highly visible in the object inspector - not like web modules -, as well as Delphi-converts most probably know about them already) then Actions have higher priority than web modules in the calling URLs. Logically, - as a design paradigm - a web application has more actions than web modules (not that I can really put a definition on why and when multiple web modules would be needed...).
However, if someone is not using Web Actions at all, then this is not true in the case they use multiple web modules.

But, since your suggestion to add a TWebHandler.PreferModuleName (Default is False) - or similar - would not break any current projects, it would be an easy addition for the above cases.

I also have a question: Could you please list the function of the multiple web modules in your web applications? I never really saw a good use for multiple web modules, but they were implemented more than 10 years ago in Delphi already too, so there must be a reason for that.

AB

I separate the different functionalities in web modules. Look at the fpwebdata and jsonrpc parts in fcl-web.

I have one module, responsible for handling all data requests. Another module which handles RPC requests.
A third which takes care of server management routines and producing HTML/images.
A fourth that handles uploads/downloads of files.
etc.

Modules are rarely very long (a couple of thousand lines at most).

Usually, complete modules can be re-used in another application. This is harder to do if you are using actions.

Also, since there are several members in a team, each person usually works on different modules. This keeps the chances of conflicts in SVN minimal.

Michael.

You did expect this :)

What about TTimer and Application.QueueAsyncCall ?

Any chance to do a remote GUI via e.g. ExtJS ?

(Of course this Application Type is useful without this.)

Thanks,
-Michael

Michael Schnell wrote

You did expect this :) What about TTimer and Application.QueueAsyncCall ?

Not implemented, no plans for it either.

I just don't see the point, I'm sorry. You have not produced a single use case which convinces me that we are ever likely to need this in a webserver.

Please don't take this wrong, but I think you are programming totally the wrong way if you need these constructs in a *webserver*. It goes against everything I think a webserver should be or do. Therefor, I will not even consider implementation of these features. You are free to create patches, and I will consider them just as I consider other patches.

But I will not spend time on its design or implementation, it is useless to ask this question of me (if other people want to spend time on it, please...).

Any chance to do a remote GUI via e.g. ExtJS ?

It depends what you mean. I have ExtJS apps running with this. And they use timers, btw. But in the browser.

Michael.

I just don't see the point, I'm sorry. You have not produced a single use case which convinces me that we are ever likely to need this in a webserver.

I already did state the use cases I have in mind. But this might be a long time ago and I don't actively work on these projects any more right now. They might come up again some time, though, and I will be happy to be able to built them on your work.

The projects are (rather huge) "headless" embedded programs, that are ported from Delphi. They heavily use timers and threads (e.g. communication threads that are dedicated to sockets or serial interfaces). They are supposed to run on systems that don't feature X11 and/or Widget libraries. Usually they work without any active user interface, but for monitoring, debugging and configuration purposes they should be able to be accessed by an external PC via TCP/IP (nice: via a standard browser; less nice but more versatile: via a dedicated program running on the PC; also nice but requiring a lot more resources on the embedded site: via VNC).

Lately another use came up: a service in a newer version of Windows "Server" editions can't do GUI linking at all, while with older versions this was possible (our Delphi programs <purpose: see above> do this).

But I will not spend time on its design or implementation, it is useless to ask this question of me (if other people want to spend time on it, please...).

I see. If the main purpose of the program is CGI, you of course are right, If the programs' CGI accessibility is only used temporarily, TTimers and thread do make sense.

So it provides a decent "multiple event interface" to the programmer (even though these event only are fired by the remote browser). Great !

I do suppose that, I will be able to build on this once I will need to take on the task.

I suppose joining you Application ("Widget") Type and the fpGUI ("Widget") Type should result in something useful on that behalf. (Technically e.g. adding a new "ExtJS" Platform to fpGUI).

-Michael

Michael Schnell wrote

a service in a newer version of Windows "Server" editions..

And as discussed recently: a daemon running most of the time silently in the background on an Android device.

-Michael

By all means, you can do so, just don't expect me to create the event queue and whatnot.

None of this requires the use of event queues or timers ?

Lately another use came up: a service in a newer version of Windows "Server" editions can't do GUI linking at all, while with older versions this was possible (our Delphi programs <purpose: see above> do this).

I do client/server programming quite a lot. All our apps are 3-tier. Server services are heavily threaded. None of them has a timer on the server, or an event queue. (unless you see the socket listener as an event queue)

There may be one timer, for the database backup scheduler - and even this one could be done perfectly without timer. A simple sleep() would do, or schedule it externally with the system scheduler.

Michael.

By all means, you can do so, just don't expect me to create the event queue

Of course not if you don't need it for your projects. :)

As you know, for me it's a major design flow of the LCL, that the event queue is implemented separately within any Widget Type, instead of being a central object that offers appropriate hooks to all Widget Types (maybe "Windows" should better stay outside here.) With this in place, you would not need to bother :).

None of this requires the use of event queues or timers ?

Of course they do: the free running "embedded" functionality performs lot of stuff on in timed cycles and as a result of events generated by Threads.

Of course all this can be implemented according to other design paradigms, but the beauty of programming in Object Pascal / Lazarus / Delphi is doing as much as possible in main thread events and with that being freed from bothering about mutual access to objects (and using threads only very thoughtfully when it really makes sense), while not needing to bother about "busy" polling loops killing the system performance and/or leading to larger latency. OTOH, when porting Delphi projects, using this paradigm is a must anyway.

I do client/server programming quite a lot. All our apps are 3-tier. Server services are heavily threaded. None of them has a timer on the server, or an event queue. (unless you see the socket listener as an event queue)

I do know that there are other programming paradigms than the "event driven" one suggested by the Delphi creators (supposedly they did have forerunners) but ... (see above).

Thanks for listening.
-Michael

Michael Schnell wrote

... and using threads only very thoughtfully when it really makes sense..

E.g. receiving data from asynchronous links or TCP/IP is done in objects that contain an internally hidden thread that waits for incoming data. When data is received, a main thread event is fired and the programmer can happily use this data without bothering at all about thread synchronization.

Of course this works with any number of instances of such objects, while the application programmer does not see any

Well, if you really think it is so important: please implement it.

You cannot expect us to implement something which we see no use for.

But patches are always welcome. You get my guarantee that I will seriously consider your patch.

Michael.

Well, if you really think it is so important: please implement it.

As you might know, I did plan to do so and I did start doing it some years ago. I found that doing it decently (creating a commonly usable event queue outside of the "active NoGUI" Widget Type I intended to create), and doing the new Widget Type in a way that complies to the makeup of the existing Widget Types (I tired to use GTK2 as a model) is hard for an LCL-internal newbie. But unfortunately I was assigned to other (non-Pascal) tasks and hat to abandon the active work on this.

You cannot expect us to implement something which we see no use for.

I never did, but please bear with me, keeping this boiling, in case somebody m,ore knowledgeable than me sees the need, and might be interested in me helping.

But patches are always welcome. You get my guarantee that I will seriously consider your patch.

Thanks for your faith !

I'll get back to you on this issue, once I might be able to get on this task.
Up till then, I am ready to help Graeme, whenever he might be inclined to enhance his fpGUI design by exporting the event queue code to some common place and/or plan a "noGUI", "ExtJS", "ifi" or whatever platform.

-Michael

I happen to be familiar with Graeme's fpGUI code.

Since his event loop code is very GUI and OS specific, I doubt this path will lead anywhere. (I have warned about this before).

Michael.

Michael Schnell wrote

Up till then, I am ready to help Graeme, whenever he might be inclined to enhance his fpGUI design by exporting the event queue code to some common place and/or plan a "noGUI", "ExtJS", "ifi" or whatever platform.

I thought all fpgui event code is integrated into the widgetset? Maybe you can point me to the code?

There are some good cross platform event loop libraries out there (see libevent and libev). I suggest that you take a

That of course is not nice :(

OTOH, I feel that fpGUI is better suitable for a move in this direction than any other Widget Type's code.

I did take a deep look at the event queue code of MSEGUI. Same is perfect, but using it as it is would re-implement a lot of classes that already are existing in the LCL (it uses a cascade of "List" classes that finally the event FIFO is derived from). At that time I managed to do a working event queue with Martin's code, that I could compile with Lazarus as an enhancement to the noGUI Widget Type . But I stopped this design path, as I did not intend to suggest such a duplication of functionality.

-Michael

Henry Vermaak wrote

There are some good cross platform event loop libraries out there (see libevent and libev). I suggest that you take a feel that implementing the event queue itself is not really difficult.

The hard stuff is to do it's interface in a way that multiple Widget Types, using different OSes can use it instead of the built-in code. Mind that the Widget Type code (in a GUI enabled "Widget Type") needs to merge GUI events and program-generated events (e.g. QueuAsyncCall() performed by a thread.

-Michael

Michael Schnell wrote

> Since his event loop code is very GUI and OS specific, I doubt this > path will lead anywhere. > (I have warned about this before). That of course is not nice :(

Michael van Canneyt is correct. fpGUI doesn't have a loose standing event loop system like MSEgui apparently has (though I must admit I know very little about MSEgui - just heard the things you mentioned here in the past - so going on your word that something like that exists in MSEgui). fpGUI has an internal thread-safe event queue, which it populates with events it receives from the OS's GUI framework (GDI or X11), or from other threads (created by the developer). The processing of this queue is still tied in with the event loop that also collects message from the OS's GUI.

Maybe you got confused between the fpGUI event loop and fpGUI event queue?

Anyway, I must add that I do have an interest in a separated event loop... right down to each thread having it's own event loop. But this is still just something I am dreaming about and no time to implement yet. Plus there is still lots I want to research on the subject before I start writing any of this code.

Regards,
- Graeme -

Graeme Geldenhuys wrote

Anyway, I must add that I do have an interest in a separated event loop... right down to each thread having it's own event loop.

I feel that once a decent event queue object is implemented independently of any Widget set, same can be instantiated and used by the main thread (in the TApplication code, linking it in a way that GUI events work appropriately, if we are in a GUI enabled Widget Type). Same also can be instantiated by an appropriate TThread Type (TThreadApplication or something like this) too (not using GUI mechanisms to fire Events and using OS means to wait for something to be scheduled) to allow for event driven programming in other threads, too.

Is I said: I suppose the critical part is designing the proper interface of the class.

-Michael

Michael Schnell wrote

What about TTimer and Application.QueueAsyncCall ?

How are they related? One is an async synchronize() and timer is a windows message based.

Marco van de Voort wrote

How are they related? One is an async synchronize() and timer is a windows message based.

I am not talking about Windows.

-Michael

Marco van de Voort wrote

How are they related? One is an async synchronize() and timer is a ...

In fact I am not talking about implementation details but about what the application programmer sees.

For same both are main thread events (procedure calls) that are fired asynchronously from the main thread by some external source. (There are more of those besides the two I mentioned. )

The _implementation_ of such stuff needs to be done in the LCL and is done with (e.g.) Windows and GTK2 Application but not with several others.

-Michael

Michael Schnell wrote

Is I said: I suppose the critical part is designing the proper interface of the class.

I suggest you read some of the Qt 4.5+ documentation. I believe they have support for event loops inside thread object instances etc... Might not be 100% to want you want, but it might give you some new thoughts and ideas.

Regards,
- Graeme -

You mean just as a source of ideas ?!?!?

Mind that the purpose of my thought is _not_ being forced to use an external Widget set, but enable attaching to one, if desired.

-Michael

Michael Schnell wrote

You mean just as a source of ideas ?!?!?

Yes. Simply by reading some of their documentation, one can find some good ideas and designs. They have a huge resource of developers, and have clearly worked through the design problems.

Mind that the purpose of my thought is _not_ being forced to use an external Widget set, but enable attaching to one, if desired.

Not everything in Qt is "GUI" or "Widget set" sentric. A lot of the modules (or packages, or whatever they call it) is non-gui related...much like the Free Pascal's RTL and FCL.

Regards,
- Graeme -

Michael Van Canneyt wrote

I want to see if we can integrate it in a TCustomApplication descendent (TEventApplication), of which the LCL TApplication can be made a descendent. If so, then the NOGUI widget type will automatically have an event loop and timer.

Sounds nice !

I vote for a kind of "general" implementation:

An TEventQueue class defined in a central location so that it can be used by any Widget Type (AKA "interface" implementation, AKA TApplication class derived from TCustomApplication, I don't know how this is related to the "Application Type" that is selectable when creating a "New" project.)

The TEventQueue class should provide the necessary hooks that theoretically would allow all (non-Windows) Widget Type implementation to be re-implemented using them. Maybe a partly virtual TCustomEventQueue class might be sensible to define the hooks and the basic functionality, so that the final TEventQueue derived from same again is implemented in the Widget Type code. With this I suppose that any (future) Widget Type implementation would be able to reuse the active code. With that maintenance would be perfect and optimizing the central active EventQueue code would make much sense. (AFAIK there are several more or less optimized implementations of self-sizing FiFOs around that could be considered to be tested against any initial implementation.)

Maybe in fact a future TApplication could be based on the TEventApplication you are planning (instead of directly on TCustomApplication as it seems to be done right now). If this is the plan, again I suggest that TEventApplication is done in a way that theoretically all (non-Windows) Widget Type implementation could be re-implemented as a sibling of same, dumping the active event-queue code done in the individual Widget Type implementation.

Well, based on your various descriptions and questions, I think it can be used to create this widget type.

Sounds good :).

... This is exactly what I want to establish.

Please let me know if I can be helpful in any way.

For testing I already have a quite generally usable testing project.

Suggestions:

I think, it would be good enough to implement
TEventApplication.QueueAsnycCall() as a central entry to the queue. Other mechanisms (such as TThread.Synchronize(), PostMessage(), and PostThreadMessage() ) can just use it. This would facilitate the code maintenance.

As you are a very active core member of the community, maybe you can improve the FPC RTL, too. Here, TThread should be enhanced by the TThread.Queue() procedure. Same is implemented in Delphi since years (but not decently documented in the Delphi help of all versions). TThread.Queue() would be nothing but a call to QueueAsnycCall with one argument.

Thanks again,
-Michael

That is because the eventloop principles are generally somewhat dependant on the type of application.