Hello,

Using the TCustomHTTPApplication class, I just set the "Threaded" property to true, and my application works in threaded mode. But, how to do the same in TCustomFCgiApplication class?

To test it, compile and run the code below. After, open two tabs in your browser. In tab 1, open the "http://localhost/test?op=loop" URL, in tab 2, open the "http://localhost/test" URL. Notice that the tab 2 will wait 10 seconds to show the result.

program project1;
{$mode objfpc}{$H+}
uses
{$IFDEF UNIX}
  CThreads,
{$ENDIF}
  CustWeb, HttpDefs, CustFCGI, Classes, SysUtils;
type
  { TMyFCGIApplication }
  TMyFCGIApplication = class(TCustomFCgiApplication)
  protected
    function InitializeWebHandler: TWebHandler; override;
  end;
  { TMyFCGIHandler }
  TMyFCGIHandler = class(TFCGIHandler)
  public
    procedure HandleRequest(ARequest: TRequest; AResponse: TResponse);
override;
  end;
  { TMyFCGIApplication }
  function TMyFCGIApplication.InitializeWebHandler: TWebHandler;
  begin
    Result := TMyFCGIHandler.Create(Self);
  end;
  { TMyFCGIHandler }
  procedure TMyFCGIHandler.HandleRequest(ARequest: TRequest; AResponse:
TResponse);
  var
    I: Integer;
  begin
    if ARequest.QueryFields.Values['op'] = 'loop' then
    begin
      for I := 1 to 10 do
        Sleep(1000);
      AResponse.Contents.Text := 'Done!';
    end
    else
      AResponse.Contents.Text :=
        'Now: ' + FormatDateTime('yyyy-mm-dd hh:nn:ss.zzz', Now);
  end;
begin
  with TMyFCGIApplication.Create(nil) do
    try
      Port := 8080;
      Run;
    finally
      Free;
    end;
end.

Thank you!

Oops, I forgot an additional information: i'm using Lazarus 1.2.6 / FPC 2.6.4 and FCGI as proxy (usgin native modules) in Apache 2.4 on Windos 7 64 bits.

This is currently not supported.

If you use fastcgi in classical environment, the Apache webserver will start as many instances of the fastcgi executable as it considers necessary to handle the load.

Only if you use proxy fastcgi (i.e. a separately running binary listening on a fixed port, not started in Apache), then threading may be useful.

To support threading requires a major rework of the fastcgi implementation, and probably also changes to fpweb (which is not thread-safe).

This is on my todo list, but you are free to try and supply a patch.

Michael.

Am I wrong thinking, that using "proxy FCGI" projects are a lot easier to manage, anyway, as here you can use normal debugging means (e.g. Lazarus) ? And with a permanently running independent executable, you also can do a lot of stuff that might be difficult to accomplish with a program started and killed by the Server.

Hence, when considering doing a CGI in fpc (a project which I finally, did not start though) I decided to use proxy CGI.

A problem might be that (AFAIK) IIS does not support proxy FCGI.

-Michael

Not really.
On windows they must be service programs, which are an absolute horror to debug.

And with a permanently running independent executable, you also can do a lot of stuff that might be difficult to accomplish with a program started and killed by the Server.

Well, I switched from proxy fcgi to normal fastcgi for performance reasons. It simply scales better.

With all the virtualization happening these days, introducing state on the server is a very bad idea anyway, so I would not recommend this approach.

As for debugging:
it rarely happens that I must debug an actual running fastcgi program. All code is unit tested, and the unit tests run in a CLI program.

The nice thing of web is that it is simply "text in, text out". Can perfectly be simulated on the command-line.

Michael.

Michael Van Canneyt wrote

On windows they must be service programs, which are an absolute horror to debug.

Yep ;-) .

To do such a thing on Windows (IIS) with Delphi, my (not Linux expert) colleagues very successfully created a simple "ISAPI" CGI that uses a bidirectional communication stream with a normal Delphi program (that has an option to be started as a service if the administrator wants that). The system is usually installed on virtualized Web server boxes.

-Michael

In our previous episode, Michael Van Canneyt said:

Stability wise it is best, but depending on technology this is not a hard _must_. The ISAPI framework that I once used allowed the main application to be both app and service, and for debugging purposes one used the application form.

The ISAPI dll was a simple URL decoder that then passed the url on to the right app via ipc.

I know. My own proxy fastcgi apps worked like that.

But if you are on a headless server, then a windows service is the only really viable way.

You can start messing with run entries in the registry, but the problem remains the same: the app must be running when the webserver starts. Best way to achieve that is a service.

The alternative is to let the webserver control the program instances. And except for debugging purposes, this is still the best approach IMHO.

Michael.

On Fri, Jan 30, 2015 at 4:46 AM, Michael Van Canneyt <

Perfect.

Do you know if FastCGI proxy accepts a multi-thread structure? I got this question after I did a test on nginx and Apache. I put a processing in a thread, and even then the server was waiting for the end of the first processing. =/

It's interesting we check it, because if they do not support multi-threaded implementations, we would have unnecessary work.

To support threading requires a major rework of the fastcgi implementation,

I'll test my drafts this weekend. I'm very interested in this feature, so now it is priority in my test list. =)

On Fri, Jan 30, 2015 at 8:57 AM, Michael Van Canneyt <

I think you should test your fcgi as proxy again after we implement the multi-thread support.

A tool as AB, that simulate multiple requests per second, it will have a low performance in in currenty implementation of proxy fcgi, because when AB call a second request, for example, the first probably will be still processing, since the interval between requests in AB is too short.

On Fri, Jan 30, 2015 at 11:01 AM, Michael Van Canneyt <

But it is very easy to solve it. In my case, when I talk about debug an application running, it isn't about debug the final executable itself, but its implementation. If I start my EXE via Apache, it is impossible to debug it. Using proxy, I can add another server in my HTTP server (nginx allows to add more than one), something like "fcgi_pass <IP of my development machine>/dev/myapp", so I open my browser and access the URL of the server that is running publicly (probably on a machine in the clouds), and this server will call the application running here on my PC. I know my local application will be a replica of the one already published, taking the time/commits from the GIT log.

I know all this. I did this myself :)

Debugging is currently not the main problem IMHO. Performance is.

So I am glad you are investigating threaded fastcgi, I will be one of the first to test :)

Although I am sure this will raise other problems when converting existing fastcgi processes: such as DB access which is not thread safe,
cached data access which must be made thread safe etc. etc. etc.

Michael.

Yes! ...

silvioprog wrote

On Fri, Jan 30, 2015 at 4:46 AM, Michael Van Canneyt < > wrote:

[...]

Do you know if FastCGI proxy accepts a multi-thread structure?

"answered Jan 10 '14 at 5:42
<http://stackoverflow.com/users/1048480/avibodha>
Avibodha <http://stackoverflow.com/users/1048480/avibodha>

I can't answer your exact questions about mod_fcgid, but Apache has moved on and the preferred method now in Apache 2.4 is to use mod_proxy_fcgi, http://httpd.apache.org/docs/trunk/mod/mod_proxy_fcgi.html.

It does handle multiple concurrent requests. Typically you might pass requests to the php-fpm handler."
Source: http://stackoverflow.com/a/21036947

Funny how they word it: 'Apache has moved on', when in fact they are saying 'we now do less work for you'. How this is 'moving on' escapes me :-)

Meanwhile there are now 3 modules that handle the FastCGI protocol: mod_fastcgi, mod_fcgid, mod_proxy_fcgi. Probably the maintainers of mod_fcgid/mod_fastcgi quit, and their successor didn't understand the code, so they wrote a new module :)

The mod_fastcgi module is the first and presumably still the best: it handles everything (and more) that mod_fcgid and mod_proxy_fcgi handle separately; the latter 2 each only handle one way of working.

But we can see how the wind blows, so we'd better get started on this multithreading approach :(

Michael.

[off topic:] ... But no easy/proven way of creating a "remote GUI" (including "server generated events") in the browser window by means of that :-( . Of course this is more a Lazarus issue than an fpc issue.

-Michael

On Fri, Jan 30, 2015 at 1:25 PM, Michael Van Canneyt <

Buddy, I finally could make a draft of a FastCGI application running in multi-thread in proxy mode. =)

It worked on:

. Windows 7 - 64 bits;
. nginx 1.7.9 com o módulo FastCGI distribuído nele;
. Apache 24 (httpd-2.4.12-win64-VC11.zip), que já vem com o módulo FastCGI
com suporte ao modo proxy;
. FPC do trunk.

And the steps to test it were:

. download and unzip the attached package, after unpacking it, enter the nginx directory and apply its settings in your copy (ainda não testei no Linux, mas farei isso em breve I haven't tested it on Linux yet, but I'll do it soon);
. open and run the project1.lpr project;
. open a tab in the browser and access http://localhost:81/test?op=loop, it is supposed to be frozen for 10 seconds;
. open another tab in the browser and access http://localhost:81/test, it is supposed to show the current time, even with the first tab frozen, this shows that the application is working in multi-thread.

Follow attached a unit called mycustfcgi.pp, that is a copy of the trunk custfcgi.pp. I've not created a patch because I believe that it will enter a new class in this unit, probably called TFCGIThread, and it won't be necessary a programmer treat the AcceptConnection method externally and not create extra classes, like "TFastCGIThread" and "TFastCGIRouter" as in the example.

I believe that with some tips of you, I can make this implementation by myself and then send a patch.

It would be great if this feature has already come in FPC 3.0, which is about to be released. =)

Thank you!

Oops... damn Ctrl+Enter in Gmail! =D

On Tue, Feb 10, 2015 at 5:44 PM, silvioprog wrote: [...]

nginx 1.7.9 with its distributed FastCGI module.

. Apache 24 (httpd-2.4.12-win64-VC11.zip), que já vem com o módulo FastCGI com suporte ao modo proxy;

Apache 24 (httpd-2.4.12-win64-VC11.zip), that is comes with the FastCGI
module with support for proxy mode;

. FPC do trunk.

(I haven't tested it on Linux yet, but I'll do it soon)

XD

silvioprog wrote

Buddy, I finally could make a draft of a FastCGI application running in multi-thread in proxy mode. =)

Congratulations !!

I will test this ASAP :)

Follow attached a unit called mycustfcgi.pp, that is a copy of the trunk custfcgi.pp. I've not created a patch because I believe that it will enter a new class in this unit, probably called TFCGIThread, and it won't be necessary a programmer treat the AcceptConnection method externally and not create extra classes, like "TFastCGIThread" and "TFastCGIRouter" as in the example.

Yes, Ideally, it should be simply a property 'Threaded' in TFastCGIApplication.

But, I have looked at the code, and I have several remarks/questions: you create threads as requests come in.

This can easily explode. I would prefer an approach where there is a thread pool, with a size that can be set at startup, or at least a maximum number of threads to start.

Do you think you can change the design so it uses a thread pool ?

Then: you create an instance of the FastCGIHandler for the request, but you do not copy the handler's properties (event handlers and so on). How is then for example the eventhandler OnIdle called ?

Maybe it is better to refactor the code so there is a separate 'FastCGIIOHandler' object or something like it.

Another question.
As I see it, you create a thread per accepted connection. Each connection can handle multiple requests.
All requests on a connection are handled within the same thread.

What about an approach where there is a pool of threads that handles requests ?

I believe that with some tips of you, I can make this implementation by myself and then send a patch. It would be great if this feature has already come in FPC 3.0, which is about to be released. =)

We will see, I think this is a big feature, and that is "bad" because big merges to fixes branch are not easily accepted.

Anyway: Great work !

Michael.

Please excuse me for not answering you before ...

On Wed, Feb 11, 2015 at 4:59 AM, Michael Van Canneyt <

Thank you! =)

Good! =)

Follow attached a unit called mycustfcgi.pp, that is a copy of the trunk

Yes I can, I had thought about a list of threads, but I did without it just to check how a multi-threaded application behaves in an HTTP server in proxy mode. =)

In the end implementation, we will have a thread-safe list, so we'll be able to add a property to limit the maximum number of threads allowed in the pool. And each item in the pool may also have an extra property (property Date: Pointer), allowing the programmer to use it to save some optional information (I plan to use it to save the each user session in the memory).

Then: you create an instance of the FastCGIHandler for the request, but you

do not copy the handler's properties (event handlers and so on). How is then for example the eventhandler OnIdle called ?

That's it. That code was just an example to see the "hello" on the screen, but the end implementation will have a clone of the request/response in each thread. =)

I have a question, what is the OnIdle used for? If I got it, it waits for more requests on the same thread. Is it right?

Watching the following code:

      If (poUseSelect in ProtocolOptions) then
        begin
        While Not DataAvailable do
          If (OnIdleNil) then
            OnIdle(Self);
        end;
...
    until Result or (FHandle=THandle(-1));

It seems that it is triggered when there is no data in the socket. If I'm right about this, I can create a OnIdle for each socket.

As we are going to change this part of the code, we could create the DoReadingData and DoWrittingData methods, they could be used to show a progress of sent and received data to the application. But it can be implemented later.

Maybe it is better to refactor the code so there is a separate

'FastCGIIOHandler' object or something like it.

So, If the OnIdle is used to wait for more requests on the same thread, we could create this class above, and it can be an attribute of the thread, handled in a loop in the thread, and to catch the OnIdle of triggerred in each thread, we could create a kind of "listener", that is just another property in the thread-safe list itself, avoiding thread deadlock.

Yes. I have a micro chat application that I created using my own protocol, and there I use a thread per request. But it is a structure that I created by myself, so it could be different from the structure and proposal of FastCGI. I came up with a question: how can I configure the FastCGI to accept more than one request per connection? Is it possible in proxy mode?! I saw the RequestsArray property and it drove me to think that you receive more than one request per connection.

I need to understand this structure to implement a listener, where each thread will have its own events and they could be accessed asynchronously.

What about an approach where there is a pool of threads that handles

requests ?

It will be a perfect approach!

I'll try to do it in the most performative and simple way as possible, then I'll send each change for you to evaluate, so when it becomes usable, we could apply it in the trunk. =)

I will also be very careful not to break the compatibility of custfcgi with legacy applications.

I believe that with some tips of you, I can make this implementation by

No problems buddy. =)

I'm planning to create an open source library (DLL/SO) to use the FPC FastCGI layer in other languages (at first in Delphi7 and XE7). I've already made a draft and it is working perfectly, but I want to make it multi-threaded, so I can use the FPC by branch fixes, applying my changes locally, and wait for them to be applied in the official repository.

Anyway: Great work !

Thank you. And I thank you for the great help you are providing me! =)

Pretty soon we'll have a robust structure to use FastCGI in multi-thread mode ...

On Sat, Feb 14, 2015 at 3:18 AM, silvioprog wrote: [...]

I came up with a question: how can I configure the FastCGI to accept more than one request per connection? Is it possible in proxy mode?! ...

I'm using Apache24 to debug the draft app, and today I've seen the following information on Apache documentation:

===

mod_proxy_fcgi <http://httpd.apache.org/docs/trunk/mod/mod_proxy_fcgi.html> disables
connection reuse by default, so after a request has been completed the connection will NOT be held open by that httpd child process and won't be reused. If the FastCGI application is able to handle concurrent connections from httpd, you can opt-in to connection reuse as shown in the following example:
Single application instance, connection reuse

ProxyPass /myapp/ fcgi://localhost:4000/ enablereuse=on

Source: http://httpd.apache.org/docs/trunk/mod/mod_proxy_fcgi.html. ===

I downloaded the httpd-2.4.12-win64-VC11.zip package, now I'm going to debug my app using this option ...

I was able to make the example works in "enablereuse=on" mode! =)

It was not necessary to change the mycustfcgi.pas, it is the same as the previous example.

This made the application more responsive, now I hold the F5 key on the browser and I can see the millisecond changing on the screen, even with another tab "locked" in a 10-second loop. In the first example it was not possible.

In that second example was added the "KeepConnection" property. If it is False, the application will work as the first example that I sent: each connection is closed in the end of request. If it is True, it will only open a new thread if the first thread was still occupied in any processing, otherwise the AcceptConnection of the application will be used.

I'm really impressed with the great performance achieved, please check this log below (Chrome debug the result):

KeepConnection = False

Oops ... I sent an incorrect example ...

Please use this new example in attached, it is the correct example.

On Sat, Feb 14, 2015 at 11:51 PM, silvioprog wrote: [...]

And it work fine in nginx too (it is very fast =o ), using this configuration:

    upstream fastcgi_backend {
        server 127.0.0.1:8080;
        keepalive 8;
    }

location /test {
fastcgi_pass fastcgi_backend;
fastcgi_keep_conn on;
...
}

Loading time: 4 ms

Very nice.

Some remarks:

- KeepConnection logic already existed. It is part of the fastcgi protocol.

Normally, the apache server must send a keepconnection option as part of the request. See the 'FKeepConnectionAfterRequest' variable in TFCGIrequest, it is processed when the FCGI_BEGIN_REQUEST block comes in and is checked after each request.

The fact that you apparently needed to introduce special processing means the proxy module does not correctly set this option. To test, you can check the value of the flag in the request handler.

I also suggest you look at the existing implementation of KeepConnection and see where that needs to be changed, instead of introducing new KeepConnection handling.

- if you really need to introduce an additional flag, please put an extra flag in ProtocolOptions. Something like 'poDefaultKeepConnection'. Do not introduce new booleans 'options' when an option set exists...

- Don't change the constructor signature, that is bad practice.

   A TComponent constructor must be Create(AOwner : TComponent);
   if you really want to change it, just create another one:
   Constructur CreateCustom(AOwner : TComponent; KeepConnection : Boolean);
   For components, properties must always be settable after create,
   otherwise streaming does not work correctly.

Michael.

On Sun, Feb 15, 2015 at 7:11 AM, Michael Van Canneyt <

wrote:

[...]

I saw, and it works fine in nginx, but ...

- if you really need to introduce an additional flag, please put an extra

flag in ProtocolOptions. Something like 'poDefaultKeepConnection'. Do not introduce new booleans 'options' when an option set exists...

... in Apache, using proxy mode, keepconn always returns true. =/ So we need to add a 'poProxyKeepConnection' flag.

Done. Please see the code in attached (some changes in mycustfcgi too).

After your tips, now I'm able to implement the changes in original custfcgi and send a patch. One more question before continuing: after implementing the pool, we will still need RequestsArray?

PS. please see a initial patch http://bugs.freepascal.org/view.php?id=27463.

Wait, I do not understand.
If keepconn always returns true, then all should work as-is ? Why do you need the flag ?

What happens if you set enablereuse=off in the apache config ?

Thank you.

After your tips, now I'm able to implement the changes in original custfcgi and send a patch. One more question before continuing: after implementing the pool, we will still need RequestsArray?

Normally yes, even more so than in the current implementation. IMHO you will maybe need 2 arrays: Incoming (request), outgoing(response).

- New Request comes in.
- No thread to handle request is available in pool.
- Request is accepted and put in incoming requestsarray.
- Check outgoing response array, send back any responses.
- When Thread finishes a request, puts it in outgoing responses array.

Details will depend heavily on the way the pool is handled.

Michael.

On Mon, Feb 16, 2015 at 7:23 AM, Michael Van Canneyt <

I tried everything:

- KeepAlive Off;
- enablereuse=Off;
- SetEnv proxy-nokeepalive 0;
- disablereuse=On.

I tried environment variables too. I tried to add the FCGI_KEEP_CONN flag to 0, and even so Apache return True. I happens only in Apache, it seems to be a bug, because I saw some people talking about this on StackOverflow. The "enablereuse" feature was recently implemented, so it possibly has bugs.

On nginx I've just removed the keepalive from upstram and all worked fine.

The poProxyKeepConnection flag will make the custfcgi hibrid, because the programmer could choose between using the configuration via server or application. So, add that. It worth. =)

- Don't change the constructor signature, that is bad practice.

Good.

But I don't know it we need two arrays, I think that a hash list will suit well, I'll explain it below ...

... I don't know if you tested the code that I sent you, but I've already done it, and in an unique array. =) The implemented structure works this way:

- the main acceptconn waits the first request;
- after receiving the first request, the execution comes from the main acceptconn, creating a new thread, that has an own acceptconn;
- the first open thread keep waiting for new requests in its own acceptconn, keeping the main acceptconn in 'stand-by'.

If an user make two or more later requests, it will call the acceptconn from the thread, and the main acceptconn will go on in stand-by. If a thread get in a long process (e.g: that 10 seconds loop), the main acceptconn it will call, that leaving from the stand-by status, creating a new thread, putting this new processing on it.

In short, if a request is short, only one thread is created and one acceptconn is used; if a new request takes some time to be processed, the application opens a new thread to process the other requests.

Another thing: IMHO, it is interesting to use hash lists instead of commom arrays. A hash list has a internal generic array, that implements the type of the array item according to the declared specialization. In some cases, we get more than 700 % of performance using hash lists instead of commons arrays. It happens due to the hash list uses a different logic to iterate items. Please, test this small comparation[1] (Commom: 00:00:07.554 vs Hash: 00:00:00.109) in your PC (considering that we will use only the 'add', 'get' and 'delete' methods of the list), realising the big advantage in use hash list (you can compare them even with common arrays). We could make a TFCGIThreadedList, using a critical section to turn it thread-safe (on the comparition I showed you how to do it).

It is interesting to enjoy this big performance of the hash list, so, in HTTP 1.1, the requests occur asynchronously: when a user open any page from a site, the download of files JS, CSS, HTML, PNG etc. they are done simultaneously (my calc of 150,000 iterations was supposing that 10,000 users was accessing a social network, and each user downloading in average 15 page files - CSS, JS, HTML and ajax GETs -, simultaneously).

[1] http://pastebin.com/ZM1DSLkG

I will check your code.

First of all, your sample code is simply wrong.

The threadlist is faster than the hashlist.

Change your code so that you
a) Set the capacity of the list.
b) add the items to the LIST, not the CommomList:

CommomList := TThreadList.Create;
   try
     List := CommomList.LockList; //
     List.Capacity:=Count+1;
     for I := 1 to Count do
     begin
       Item := TFCGIThread.Create;
       Item.Name := 'thread_' + IntToStr(I);
       List.Add(Item); // adding to LIST not COMMOMLIST
     end;
    CommomList.UnlockList; // unlock

If you do that, you'll see that the list is about 30% faster.

home: >./mytest
Hash: 00:00:00.752 vs Common: 00:00:00.526

So an ordinary (common) list is faster.

Then: when you do such tests, you must do 2 test runs, with the order of the tests reversed. The first of your 2 tests must always ask memory from the OS, because none is allocated yet. The second can re-use the memory and thus will be faster, this creates a skew in the result.

But then: where do you want to use a hash list ?

I don't believe that iterating over a hashlist is very useful, for that you can use a list.

A hashlist is only useful if you need to do lookups based on a key. So what do you want to look up ? Threads ? I don't think that creating 150000 threads is a very realistic example. I'm not sure the OS can pull this :) So, if you have 100 threads, this will be more than enough, and to use a hash list for 100 items seems a bit overkill. Maybe I am missing something ?

But if you still, after all this, want to use a hashlist: use one from contnrs.pp, not the generics one.

Michael.

On Tue, Feb 17, 2015 at 7:03 AM, Michael Van Canneyt <

Good! =)

After that I'll start the changes directly in custfcgi.

In short, if a request is short, only one thread is created and one

You are right.

Really, you are right! I made a confusion, because in other languages, most of the time, HashSet() is more fast than ArrayList(). In Pascal it seems a bit different (I need to do more tests after).

Then: when you do such tests, you must do 2 test runs, with the order of

Yes. I'll do it.

But then: where do you want to use a hash list ?

I don't want to use it anymore. Because you showed that an ordinary TFPList is more fast than a THashSet.

In this case, we could use a RTL ThreadList. Now I understand the internal code (it is very simple), so I can use it correctly.

I'll wait you check my last changes in
"FastCGI_Proxy_nginx_Apache24_III.zip", so, after that, I'll go back to the implementations directly in custfcgi, now using ThreadList. =)