Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2012-08

fpWeb long process progress

17 messages · 7 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Leonardo M. Ramé

As the subject says, I'm looking for a way to notify the caller of a time consuming process.

An example could be this:

procedure TMyHandler.TimeConsumingProcess(Sender: TObject; ARequest:
TRequest; AResponse: TResponse; var Handled: Boolean);
var
  I: Integer;
begin
  for I := 0 to 1000 do
  begin
    AResponse.Content := IntToStr(I);
    Sleep(1000);
  end;
  Handled := True;
end;

Of course this doesn't work. How can I send responses by intervals?.

Regards,

Reply by email

leledumbo
to Leonardo M. Ramé
How can I send responses by intervals?

AFAIK that's not the way web application works. It's the client that should be querying the server in a regular interval (using AJAX request perhaps). The server can track current progress status and return that right away when asked.

View this message in context: http://free-pascal-lazarus.989080.n3.nabble.com/Lazarus-fpWeb-long-process-progress-tp4025614p4025615.html Sent from the Free Pascal - Lazarus mailing list .

Reply by email

Leonardo M. Ramé
to leledumbo
leledumbo wrote 7 lines
> How can I send responses by intervals? AFAIK that's not the way web application works. It's the client that should be querying the server in a regular interval (using AJAX request perhaps). The server can track current progress status and return that right away when asked.

Yes, I know that, but I thought Lazarus have something easy to develop such process.

The way I'm thinking of implementing is this:

1) The client app asks the server (a CGI process) for a process ID, for example 123, by calling getProcId.
2) Using this ID, the client calls the time-consuming task, passing the ID. for example, runTask(Id). At this time, the server launches a program passing the ID for the task. This must be a multi-thread program, because it can be asked for running many tasks, and can be asked for the status of each task. The program only finishes when the last task is ended.
3) The client calss a getProcStatus(ID) method. The server, using IPC or Pipes, asks the already-launched process for its status.

Regards,

Reply by email

Marcos Douglas
to Leonardo M. Ramé
Leonardo M. Ram? wrote 28 lines
On 2012-08-13 15:15:56 -0700, leledumbo wrote: > > How can I send responses by intervals? > > AFAIK that's not the way web application works. It's the client that > should > be querying the server in a regular interval (using AJAX request > perhaps). > The server can track current progress status and return that right away > when > asked. > Yes, I know that, but I thought Lazarus have something easy to develop such process. The way I'm thinking of implementing is this: 1) The client app asks the server (a CGI process) for a process ID, for example 123, by calling getProcId. 2) Using this ID, the client calls the time-consuming task, passing the ID. for example, runTask(Id). At this time, the server launches a program passing the ID for the task. This must be a multi-thread program, because it can be asked for running many tasks, and can be asked for the status of each task. The program only finishes when the last task is ended. 3) The client calss a getProcStatus(ID) method. The server, using IPC or Pipes, asks the already-launched process for its status.

Your program is the main thread. For each request your program should create a new thread to process. Each thread synchronizes with the main thread (your program) about it your own status.

Already exists an example with the same idea (more or less)... in Pascal... called ExtPascal.

Marcos Douglas

Reply by email

Unknown author
to Leonardo M. Ramé: Yes, I know that, but I thought Lazarus have something easy to develop such process. The…
Leonardo M. Ram? wrote 24 lines
On 2012-08-13 15:15:56 -0700, leledumbo wrote: >> How can I send responses by intervals? > > AFAIK that's not the way web application works. It's the client that should > be querying the server in a regular interval (using AJAX request perhaps). > The server can track current progress status and return that right away when > asked. > Yes, I know that, but I thought Lazarus have something easy to develop such process. The way I'm thinking of implementing is this: 1) The client app asks the server (a CGI process) for a process ID, for example 123, by calling getProcId. 2) Using this ID, the client calls the time-consuming task, passing the ID. for example, runTask(Id). At this time, the server launches a program passing the ID for the task. This must be a multi-thread program, because it can be asked for running many tasks, and can be asked for the status of each task. The program only finishes when the last task is ended. 3) The client calss a getProcStatus(ID) method. The server, using IPC or Pipes, asks the already-launched process for its status.

I have code which does exactly that. It's used in production. It's unix-only, however (it relies on fork and exec).

If you're interested, I can send you the code.

Michael.

Reply by email

Michael Schnell
to Leonardo M. Ramé: As the subject says, I'm looking for a way to notify the caller of a time consuming…

AFAIK, a web application uses the plain old standard mechanism, a web server uses to work with a CGI application. it start the application and when same ends, the web server retrieves its output and sends it to the browser. So the web application just does not live long to be able to wait for anything.

To create a longer living web enabled process it takes a lot more effort.

You can try several approaches:

- purely propriety: Use a second process (a long living daemon) and have the web application communicate with same (e.g. via TCP/IP or Pipe)
- fast CGI: here the (Apache) Web Server does exactly this communication on it's own account
- ISAPI: here a (Microsoft) Web server communicates with a DLL

My colleagues do a combination of (1) and (3) with one of their (Delphi) project very successfully:

They created a very simple ISAPI DLL (using RemObjects to do the ISAPI communication)

They created a Windows service and used RemObjects (using the "Windows Message" transport) to have the ISAPI DLL communicate with the service.

(RemObjects is a commercial product that is specified to work as well with Delphi as with FreePascal.)

-Michael

Reply by email

Leonardo M. Ramé
to Michael Schnell: AFAIK, a web application uses the plain old standard mechanism, a web server uses to work…
Michael Schnell wrote 31 lines
AFAIK, a web application uses the plain old standard mechanism, a web server uses to work with a CGI application. it start the application and when same ends, the web server retrieves its output and sends it to the browser. So the web application just does not live long to be able to wait for anything. To create a longer living web enabled process it takes a lot more effort. You can try several approaches: - purely propriety: Use a second process (a long living daemon) and have the web application communicate with same (e.g. via TCP/IP or Pipe) - fast CGI: here the (Apache) Web Server does exactly this communication on it's own account - ISAPI: here a (Microsoft) Web server communicates with a DLL My colleagues do a combination of (1) and (3) with one of their (Delphi) project very successfully: They created a very simple ISAPI DLL (using RemObjects to do the ISAPI communication) They created a Windows service and used RemObjects (using the "Windows Message" transport) to have the ISAPI DLL communicate with the service. (RemObjects is a commercial product that is specified to work as well with Delphi as with FreePascal.) -Michael

Thanks Michael, I'm testing a different approach, based on a temporary file. It works this way:

1) The client asks for a TaskId, for example 1234.
2) With the TaskId generated by the server, the client calls a CGI method called runLongTask(myTaskId). This method writes a temporary file which name is the taskId.
3) The client can ask for status by using a method called getTaskStatus(myTaskId).

This way I can run a long task without having to create a daemon.

Regards,

Reply by email

Leonardo M. Ramé
to Unknown author: I have code which does exactly that. It's used in production. It's unix-only, however (it…
Show quoted text 35 lines
On Mon, 13 Aug 2012, Leonardo M. Ram? wrote: >On 2012-08-13 15:15:56 -0700, leledumbo wrote: >>>How can I send responses by intervals? >> >>AFAIK that's not the way web application works. It's the client that should >>be querying the server in a regular interval (using AJAX request perhaps). >>The server can track current progress status and return that right away when >>asked. >> > >Yes, I know that, but I thought Lazarus have something easy to develop >such process. > >The way I'm thinking of implementing is this: > >1) The client app asks the server (a CGI process) for a process ID, for >example 123, by calling getProcId. >2) Using this ID, the client calls the time-consuming task, passing the >ID. for example, runTask(Id). At this time, the server launches a >program passing the ID for the task. This must be a multi-thread >program, because it can be asked for running many tasks, and can be >asked for the status of each task. The program only finishes when the >last task is ended. >3) The client calss a getProcStatus(ID) method. The server, using IPC or >Pipes, asks the already-launched process for its status. I have code which does exactly that. It's used in production. It's unix-only, however (it relies on fork and exec). If you're interested, I can send you the code. Michael.

Yes please, send me the code.

Regards,

Reply by email

Unknown author
to Leonardo M. Ramé: Thanks Michael, I'm testing a different approach, based on a temporary file. It works…
Leonardo M. Ram? wrote 44 lines
On 2012-08-14 09:47:41 +0200, Michael Schnell wrote: > AFAIK, a web application uses the plain old standard mechanism, a > web server uses to work with a CGI application. it start the > application and when same ends, the web server retrieves its output > and sends it to the browser. So the web application just does not > live long to be able to wait for anything. > > To create a longer living web enabled process it takes a lot more effort. > > You can try several approaches: > > - purely propriety: Use a second process (a long living daemon) and > have the web application communicate with same (e.g. via TCP/IP or > Pipe) > - fast CGI: here the (Apache) Web Server does exactly this > communication on it's own account > - ISAPI: here a (Microsoft) Web server communicates with a DLL > > My colleagues do a combination of (1) and (3) with one of their > (Delphi) project very successfully: > > They created a very simple ISAPI DLL (using RemObjects to do the > ISAPI communication) > > They created a Windows service and used RemObjects (using the > "Windows Message" transport) to have the ISAPI DLL communicate with > the service. > > (RemObjects is a commercial product that is specified to work as > well with Delphi as with FreePascal.) > > -Michael Thanks Michael, I'm testing a different approach, based on a temporary file. It works this way: 1) The client asks for a TaskId, for example 1234. 2) With the TaskId generated by the server, the client calls a CGI method called runLongTask(myTaskId). This method writes a temporary file which name is the taskId. 3) The client can ask for status by using a method called getTaskStatus(myTaskId). This way I can run a long task without having to create a daemon.

That is exactly what I do.

Michael.

Reply by email

Michael Schnell
to Leonardo M. Ramé: Thanks Michael, I'm testing a different approach, based on a temporary file. It works…
Leonardo M. Ram? wrote
2) With the TaskId generated by the server, the client calls a CGI method called runLongTask(myTaskId).

I don't know if it's a goad idea to allow a standard CGI to do a looooong action before returning top the WebServer.

At least the WebServer might kill the child task after a few seconds if it thinks it has run wild.

Maybe it is possible to configure the appropriate Timeout in the web server parameters.

-Michael

Reply by email

Marcos Douglas
to Michael Schnell
Michael Schnell wrote 7 lines
On 08/14/2012 04:27 PM, Leonardo M. Ram? wrote: > > 2) With the TaskId generated by the server, the client calls a CGI method > called runLongTask(myTaskId). I don't know if it's a goad idea to allow a standard CGI to do a looooong action before returning top the WebServer.

+1

Show quoted text 5 lines
At least the WebServer might kill the child task after a few seconds if it thinks it has run wild. Maybe it is possible to configure the appropriate Timeout in the web server parameters.

IMHO, maybe is better to use CGI gateway to capture the requests and, for each one them, start a process -- this method will use a file to write the log of process, as Leonardo and Michael said.

Marcos Douglas

Reply by email

Leonardo M. Ramé
to Marcos Douglas
Marcos Douglas wrote
IMHO, maybe is better to use CGI gateway to capture the requests and, for each one them, start a process -- this method will use a file to write the log of process, as Leonardo and Michael said.

Yes, in my case, I know in advance the process wouldn't take more than one minute, but for longer processes the gateway is a nice idea.

Regards,

Reply by email

Michael Van Canneyt
to Leonardo M. Ramé
Leonardo M. Ram? wrote 8 lines
On 2012-08-14 11:47:41 -0300, Marcos Douglas wrote: > IMHO, maybe is better to use CGI gateway to capture the requests and, > for each one them, start a process -- this method will use a file to > write the log of process, as Leonardo and Michael said. > Yes, in my case, I know in advance the process wouldn't take more than one minute, but for longer processes the gateway is a nice idea.

Well, I use the code I sent to take backups or fix databases, which takes 30 minutes easily. The browser just polls ever so-and-so time.

Michael.

Reply by email

Michael Schnell
to Leonardo M. Ramé: Yes, in my case, I know in advance the process wouldn't take more than one minute, but…
Leonardo M. Ram? wrote
in my case, I know in advance the process wouldn't take more than one minute,

I feel that (without some special configuration) a normal web server will kill a standard CGI process that takes more than just a few seconds before returning.

Please check the specs of your Webserver and let us know what you find.

-Michael

Reply by email

Henry Vermaak
to Michael Schnell
Michael Schnell wrote 8 lines
On 08/15/2012 12:14 AM, Leonardo M. Ram? wrote: > > in my case, I know in advance the process wouldn't take more than one > minute, I feel that (without some special configuration) a normal web server will kill a standard CGI process that takes more than just a few seconds before returning.

Apache defaults to 300 seconds for its core timeout (which includes mod_cgi). Different modules have their own configuration settings, e.g. php have a max_execution_time, which defaults to 30 seconds afaik.

Henry

Reply by email