win32, latest Lazarus and FPC
Any single web request call works OK, returning the content. When 2 browsers are open and calling the web module at the same time there is a crash happening after the 2nd one finishes working (just made a loop in the web action which lasts a few seconds to have both requests inside the server at the same time).
Details:
in /fpc/packages/fcl-web/src/fpapache.pp

Function TCustomApacheApplication.ProcessRequest(P: PRequest_Rec) : Integer;
Var
   Req : TApacheRequest;
   Resp : TApacheResponse;
begin
   Req:=TApacheRequest.CreateReq(Self,P);
   Try
     Resp:=TApacheResponse.CreateApache(Req);
     Try
       HandleRequest(Req,Resp);   <== call happens here OK but 2nd
concurrent request does not return, gets lost in the ether
     Finally
       Result:=OK;
       Resp.Free;
     end;
   Finally
     Req.Free;
   end;
end;
procedure TCustomApacheApplication.HandleRequest(ARequest: TRequest;
AResponse: TResponse);
Var
   MC : TCustomHTTPModuleClass;
   M  : TCustomHTTPModule;
   MN : String;
   MI : TModuleItem;
begin
   try
     MC:=Nil;
     If (OnGetModuleNil) then
       OnGetModule(Self,ARequest,MC);
     If (MC=Nil) then
       begin
       MN:=GetModuleName(ARequest);
       If (MN='') and Not AllowDefaultModule then
         Raise EFPApacheError.Create(SErrNoModuleNameForRequest);
       MI:=ModuleFactory.FindModule(MN);
       If (MI=Nil) and (ModuleFactory.Count=1) then
         MI:=ModuleFactory[0];
       if (MI=Nil) then
         begin
         Raise EFPApacheError.CreateFmt(SErrNoModuleForRequest,[MN]);
         end;
       MC:=MI.ModuleClass;
       end;
     M:=FindModule(MC); // Check if a module exists already
     If (M=Nil) then
       begin
         M:=MC.Create(Self);
         M.Name := '';//without this there's a crash due to same name
       end;
     M.HandleRequest(ARequest,AResponse);//calls the web action here,
works OK for both concurrent requests
   except
     On E : Exception do
       ShowRequestException(AResponse,E);
   end;
end;  <== everything works, but the 2nd concurrent request crashes after
here but before returning to the caller function

A little bit strange that this happens. The return content is prepared in both simultaneous calls properly. Any thoughts what could be the cause?

AB

After some more debugging the result is even stranger:

If I reduce the code to the bare minimum there is still a crash most of the time.

1. works always for 2 concurrent requests:
procedure TCustomApacheApplication.HandleRequest(ARequest: TRequest;
AResponse: TResponse);
Var
   MC : TCustomHTTPModuleClass;
   M  : TCustomHTTPModule;
   MN : String;
   MI : TModuleItem;
i:Integer;
begin
       MI:=ModuleFactory[0];
       MC:=MI.ModuleClass;
       M:=MC.Create(Self);
       M.Name := '';
aresponse.content := '<html>Hello</html>';
       for i := 0 to Maxint do;//time wasting loop, about 5 secs
end;
2. crashes randomly:
begin
       MI:=ModuleFactory[0];
       MC:=MI.ModuleClass;
       M:=MC.Create(Self);
       M.Name := '';
M.HandleRequest(ARequest,AResponse);
       for i := 0 to Maxint do;//time wasting loop, about 5 secs
end;//<= sometimes crash after here but before returning to caller function
3. always crashes:
begin
   try
       MI:=ModuleFactory[0];
       MC:=MI.ModuleClass;
       M:=MC.Create(Self);
       M.Name := '';
aresponse.content := '<html>Hello</html>';
       for i := 0 to Maxint do;//time wasting loop, about 5 secs
   except
     On E : Exception do
       ShowRequestException(AResponse,E);
   end;
end;//<= guaranteed crash after here but before returning to caller function

When M.HandleRequest(ARequest,AResponse); is used instead of aresponse.content := '<html>Hello</html>'; the crash is random at procedure return for the 2nd simultaneous caller.

When the procedure contains any try/except it is guaranteed not returning (crashing) after the work is finished for the 2nd caller (2nd simultaneaous request).

The first simultaneous caller always works without a problem.

Any suggestions?

Yes: please print the size of the Request_rec (or TRequest_Rec) and the same record in C. Compare if they are equal. Same for the module record. If they are not equal, then we know it is a problem with the pascal definition of this record. I've had to do this exercise about 30 times myself on various platforms to get it right. Maybe they changed the size again.

Michael.

Hi Michael,

Unfortunately, I do not have any C compilers - nor the knowledge to compile something with the apache header files in C. I manually compared the .h/.inc and the translated pascal structures/records for the two you have mentioned (module_struct and request_rec) and they seem to have the same fields, nothing added to them so far till Apache 2.2.9 .
For me in Windows XP 32bit/ FPC:
sizeof(request_rec)=416
sizeof(module_struct)=56

Not sure if it is a full list but in the apr header files (ap_mmn.h) there's a change list:
.
.snip I believe the translation happened for 2.2.3 so those are included .
+* 20051115.4 (2.2.4) Added ap_get_server_banner() and
* ap_get_server_description() (minor)
* 20051115.5 (2.2.5) Added ap_mpm_safe_kill() (minor)
* 20051115.6 (2.2.7) Added retry_set to proxy_worker (minor) +* 20051115.7 (2.2.7) Added conn_rec::clogging_input_filters (minor)
* 20051115.8 (2.2.7) Added flags to proxy_alias (minor) +* 20051115.9 (2.2.7) Add ap_send_interim_response API
* 20051115.10(2.2.7) Added ap_mod_status_reqtail (minor)
* 20051115.11(2.2.7) Add *ftp_directory_charset to proxy_dir_conf
* 20051115.12(2.2.8) Add optional function ap_logio_add_bytes_in() to mog_logio
* 20051115.13(2.2.9) Add disablereuse and disablereuse_set
* to proxy_worker struct (minor)
* 20051115.14(2.2.9) Add ap_proxy_ssl_connection_cleanup and
* add *scpool, *r and need_flush to proxy_conn_rec
* structure
* 20051115.15(2.2.9) Add interpolate_env to proxy_dir_conf and
* introduce proxy_req_conf.

The ones marked with '+' I added to my httpd.inc and httpd_protocol.inc files. The rest were for apache mods (mod_proxy, etc.) so ignored them. The crash did not go away. At least on Windows, Apache loads the module and executes it sometimes. On Linux the apache modules compiled don't even load.

AB

Try installing an older Apache.

I installed here with XAMMP, Apache 2.2.9 and it just quits without any error message when loading my Pascal module.

But when using the standard Apache 2.2.6 in Mandriva 2008 it works. Actually I can't get it to show the page (some config problem?), but it loads without problems, apache keeps working.

I think something changed in an incompatible way. This is pretty unexpected, as I hoped that they would keep compatibility between all 2.2.x series.

I'm not sure that something is changed to make it incompatible. The apache modules compiled on windows are only crashing if multiple requests are coming at the same time, and they are crushing at function/procedure returns (Yes, on Linux it is even worse, since no module gets even loaded by apache. That might be some C API translation problem).

Even if reduced to this small code it crushes (but only if multiple calls are inside this code) on windows:
===========================
Function TCustomApacheApplication.ProcessRequest(P: PRequest_Rec) : Integer;

Var
   Req : TApacheRequest;
   Resp : TApacheResponse;
   i:integer;
begin
   Req:=TApacheRequest.CreateReq(Self,P);
   Try
     Resp:=TApacheResponse.CreateApache(Req);
     Try
for i := 0 to maxint do ;//Time wasting for multiple requests at the
same time to run this code
//      HandleRequest(Req,Resp);
//      If Not Resp.ContentSent then
//        Resp.SendContent;
     Finally
       Result:=OK;
       Resp.Free;//<= crashes after here randomly for 2nd simultaneous
caller
     end;
   Finally
     Req.Free;
   end;
end;
==============================
Sometimes when removing all the try/finally pairs from the function it
works.
But then adding more code, even as simple as DateTimeToStr(Now), makes
it crash. Sometimes more code can be added but then it will crash inside
that code somewhere at function/procedure returns.

AB

Maybe you have to initialise the threading system so all the necessary locks in the system unit get activated. Something like

{$ifdef unix}
uses
   cthreads;
{$endif}

...

function dummythread(p: pointer): ptrint;
begin
end;
initialisation
  WaitForThreadTerminate(BeginThread(@dummythread));
end.

Jonas

This is a known problem; the heap manager does not function correctly in a DLL when an external program (apache, in this case) creates new threads.

Changing the heap manager to the C heap manager (just add 'uses cmem' as the first line in the project) may solve the problem, but it is not guaranteed; there was another bug in the memory manager which prevented that from working correctly. Maybe it has been fixed meanwhile.

Michael.

Michael Van Canneyt wrote

This is a known problem; the heap manager does not function correctly in a DLL when an external program (apache, in this case) creates new threads.

The heap manager is simply not thread safe until the FPC threading system has been initialised (by starting a thread).

Jonas

It is not that simple. The heap manager cannot handle situations where threads have been started by code that is not aware of the FPC threading mechanisms - such as apache. I have done some changes for this, but they are not sufficient. PThreads has no mechanisms for dealing with this situation. Windows does (using the DLL_THREAD_ATTACH message).

Michael.

Ah yes, that's indeed a problem with the new 2.3.x heap manager. I don't think the 2.2.x heap manager cares though, since it uses a regular critical section/mutex.

Jonas

Could it be then related to the Apache method of working? Apache has a non-threaded version and several threaded versions.

In my Mandriva where it didn't crash I installed the non-threaded version. (not sure if it won't create threads anyway for something else, even if it works in a non-threaded principle).

I cannot find in the XAMMP website if they use the threaded or non-threaded version.

2.2.2 has/had other problems for Apache :-)

Michael.

Well, it seems we have some progress now.

The
WaitForThreadTerminate(BeginThread(@__dummythread), 300); seems to fix it on Windows. At least for 2 concurrent requests. For some reason Apache cannot seem to want to run more than 2 requests at a time. If more comes in at a time, the first 2 starts running and the rest have to wait till at least 1 current request is handled. Even when MaxConnectionsPerServer is set to higher than 2 in the registry.

uses cmem; did not fix the crash on Windows.

I will try to make apache modules load in Linux next so some tests can be done there too.

AB

For me in Windows XP 32bit/ FPC (apache modules load properly): sizeof(request_rec)=416
sizeof(module_struct)=56

For me in Ubuntu 8.04 32bit/ FPC (apache modules don't load): sizeof(request_rec)=412
sizeof(module_struct)=56

Used exactly the same files to compile a project on Ubuntu but for some reason it is 4 bytes shorter there for request_rec. This shouldn't effect the module loading at apache2 startup, right? Only module_struct incompatibility should give the "...Can't locate API module structure `mod_apache1' in file ..." error.

OK, after 4 days of pulling my (remaining) hair out I was able to compile a basic C apache module on Ubuntu 8.04 (apache 2.2.8).

It seems both fpc/lazarus (not working apache module) and the C compiled (working apache module) one shows

sizeof(request_rec) = 412
sizeof(module_struct) = 56

So the length seems to be the same for for both. Still, the Lazarus compiled apache module doesn't even load in apache (mod_hello.pp or any of the other examples included in fpc).
If the exports is included then it loads but doesn't work at all:

modified "mod_hello.pp" from fpc/packages/httpd22/examples/ {*******************************************************************
* Test library of the Apache Pascal Headers
*******************************************************************} library mod_hello;

{*******************************************************************
* The mode must be objfpc on this unit because the unix code uses
* some extensions introduced on Free Pascal
*******************************************************************} {$ifdef fpc}

   {$mode objfpc}{$H+}

{$endif}

{$IFDEF WIN32}
   {$DEFINE WINDOWS}
{$ENDIF}
{$define Apache2_2}
uses SysUtils, httpd {$ifndef Apache1_3}, apr{$endif};
var
  test_module: module; public name 'test_module';
  default_module_ptr: Pmodule;
const
   MODULE_NAME = 'mod_hello.so';

{*******************************************************************
* Free Pascal only supports exporting variables on Windows *******************************************************************} { $ifdef WINDOWS}//commented out, exports work in Linux now exports

  test_module name 'test_module';

{ $endif}//commented out, exports work in Linux now .
.snip
.

Not sure why the Lazarus/fpc apache modules do not work on apache 2.2.8 or 2.2.9 on Ubuntu.
The above module is loaded, the handler registering function is called, but the handler never gets triggered when the module is called from a browser.

Nevermind, on Ubuntu the request_rec is only 384 bytes for FPC instead of 412.

But that still does not explain why the handler function is not even getting called by apache.

Hi Jonas,

This seems to work on Windows (and solves the problem), but on Linux WaitForThreadTerminate waits forever.

WaitForThreadTerminate(BeginThread(@__dummythread), 300);

apache just waits and doesn't start until I stop the apache loading with ctrl+c, as if it is in an infinite loop and waiting.

Any thoughts?

AB

Does this was fixed? I'm trying to deploy an apache 2.2.9 module, it compiles and loads ok (it doesn't crash apache) but its exported function isn't called, I'm using the same code ABorka used in this example.

This is my configuration:

Free Pascal Compiler version 2.2.4-3 [2009/06/03] for i386 Copyright (c) 1993-2008 by Florian Klaempfl

Linux debian 2.6.26-1-686 #1 SMP Fri Mar 13 18:08:45 UTC 2009 i686 GNU/Linux

Leonardo.

It should be working.

You have to make sure you delete the httpd13 and httpd20 directories from your fpc units folder (where httpd.o and .ppu is stored) so only the httpd22 folder remains there. Otherwise the wrong units might be linked with your module and then crash when called.

With the latest Lazarus SVN you can find some examples for web apache modules and cgi programs under the lazarus /components/fpweb/demo/ folder.

AB