Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2007-11

daemonapp 100% cpu easting fix

21 messages · 7 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Steve Howe

Hello all,

While using the daemonapp.pp unit to code a daemon, I've found that applications that it eat 100% of the CPU. After tracking and debugging, I've found out that the problem is in the daemonapp's TDaemonThread.Execute() method:

procedure TDaemonThread.Execute;
begin
  If FDaemon.Start then
    begin
    FDaemon.Status:=csRunning;
    StartServiceExecute;
    if not FDaemon.Execute then
      begin
      While Not Terminated do
        CheckControlMessage(True);
      CheckControlMessage(False);
      end;
    end;
end;

The loop has no interval so it just consumes all free CPU. I think it should be modified like this:

procedure TDaemonThread.Execute;
begin
  If FDaemon.Start then
    begin
    FDaemon.Status:=csRunning;
    StartServiceExecute;
    if not FDaemon.Execute then
      begin
      While Not Terminated do
      begin
        CheckControlMessage(True);
        Sleep(50);
      end;
      CheckControlMessage(False);
      end;
    end;
end;

Someone could argue that the damon's thread priority could be changed instead to "idle", making the thread more cooperative, but since daemonapp.inc's TDaemonController.Main() method also creates internally a TDaemonThread, and that thread is not exposed, changing the daemon's thread wouldn't help either.

Does anyone have a better solution or comments ? Should I submit a patch ? If the changed is accepted and someone could do it for me, it would be even better.

Best Regards, Steve Howe

Reply by email

Mattias Gaertner
to Steve Howe
Show quoted text 24 lines
Hello all, While using the daemonapp.pp unit to code a daemon, I've found that applications that it eat 100% of the CPU. After tracking and debugging, I've found out that the problem is in the daemonapp's TDaemonThread.Execute() method: procedure TDaemonThread.Execute; begin If FDaemon.Start then begin FDaemon.Status:=csRunning; StartServiceExecute; if not FDaemon.Execute then begin While Not Terminated do CheckControlMessage(True); CheckControlMessage(False); end; end; end; The loop has no interval so it just consumes all free CPU.

Then CheckControlMessage has a bug. The 'True' means it should wait for the next message.
But I see no 'virtual' and no code in this function - strange (fpc 2.3.1).

Does anyone have a better solution or comments ? Should I submit a patch ? If the changed is accepted and someone could do it for me, it would be even better.

Mattias

Reply by email

Michael Van Canneyt
to Mattias Gaertner
Mattias Gaertner wrote 32 lines
On Sat, 17 Nov 2007 10:12:30 -0200 Steve Howe wrote: > Hello all, > > While using the daemonapp.pp unit to code a daemon, I've found that > applications that it eat 100% of the CPU. After tracking and > debugging, I've found out that the problem is in the daemonapp's > TDaemonThread.Execute() method: > > procedure TDaemonThread.Execute; > > begin > If FDaemon.Start then > begin > FDaemon.Status:=csRunning; > StartServiceExecute; > if not FDaemon.Execute then > begin > While Not Terminated do > CheckControlMessage(True); > CheckControlMessage(False); > end; > end; > end; > > The loop has no interval so it just consumes all free CPU. Then CheckControlMessage has a bug. The 'True' means it should wait for the next message. But I see no 'virtual' and no code in this function - strange (fpc 2.3.1).
You are right, I committed the 'fix' as initially proposed, but moved it to
the unix version of the CheckControlMessage;

Checking for this message is not yet implemented on unix. The idea is to open a 'control' socket using simpleIPC and to check on this socket for a message.

Michael.

Reply by email

Steve Howe
to Mattias Gaertner: On Sat, 17 Nov 2007 10:12:30 -0200 Then CheckControlMessage has a bug. The 'True' means…

Hello all,

Then CheckControlMessage has a bug. The 'True' means it should wait for the next message. But I see no 'virtual' and no code in this function - strange (fpc 2.3.1).

Yes, it can't be overridden and it's implementation is empty. I checked both truk and svn versions, they are binary identic. There are also other problems with that unit, but that one had to be patched.

Best Regards, Steve Howe

Reply by email

Bogusław Brandys
to Michael Van Canneyt: You are right, I committed the 'fix' as initially proposed, but moved it to the unix…
Michael Van Canneyt wrote 46 lines
On Sat, 17 Nov 2007, Mattias Gaertner wrote: > On Sat, 17 Nov 2007 10:12:30 -0200 > Steve Howe wrote: > >> Hello all, >> >> While using the daemonapp.pp unit to code a daemon, I've found that >> applications that it eat 100% of the CPU. After tracking and >> debugging, I've found out that the problem is in the daemonapp's >> TDaemonThread.Execute() method: >> >> procedure TDaemonThread.Execute; >> >> begin >> If FDaemon.Start then >> begin >> FDaemon.Status:=csRunning; >> StartServiceExecute; >> if not FDaemon.Execute then >> begin >> While Not Terminated do >> CheckControlMessage(True); >> CheckControlMessage(False); >> end; >> end; >> end; >> >> The loop has no interval so it just consumes all free CPU. > Then CheckControlMessage has a bug. The 'True' means it should wait for > the next message. > But I see no 'virtual' and no code in this function - strange (fpc > 2.3.1). You are right, I committed the 'fix' as initially proposed, but moved it to the unix version of the CheckControlMessage; Checking for this message is not yet implemented on unix. The idea is to open a 'control' socket using simpleIPC and to check on this socket for a message. Michael. _______________________________________________ fpc-devel maillist - http://lists.freepascal.org/mailman/listinfo/fpc-devel

Long time ago I had the same problem with delphi based NT service code which was using loop in ServiceMain procedure.Then I used such code :

  WaitForSingleObject(EndEvent,INFINITE);
  CloseHandle(EndEvent);

then in a control procedure SetEvent(EndEvent) was used to signal end of execution (SERVICE_CONTROL_STOP and SERVICE_CONTROL_SHUTDOWN options)

This trick is of course Windows only but it let me drop CPU usage from very high all the time to almost zero when working thread is not doing any CPU intensive task.

Boguslaw

Reply by email

Steve Howe
to Michael Van Canneyt: You are right, I committed the 'fix' as initially proposed, but moved it to the unix…

Hello all,

Show quoted text 11 lines
> Then CheckControlMessage has a bug. The 'True' means it should wait for > the next message. > But I see no 'virtual' and no code in this function - strange (fpc > 2.3.1). You are right, I committed the 'fix' as initially proposed, but moved it to the unix version of the CheckControlMessage; Checking for this message is not yet implemented on unix. The idea is to open a 'control' socket using simpleIPC and to check on this socket for a message.

Then the TDaemonThread.CheckControlMessage() method should be declared virtual, so that messages could actually be checked for, shouldn't it ?

Best Regards, Steve Howe

Reply by email

Michael Van Canneyt
to Steve Howe
Steve Howe wrote 15 lines
Hello all, > > Then CheckControlMessage has a bug. The 'True' means it should wait for > > the next message. > > But I see no 'virtual' and no code in this function - strange (fpc > > 2.3.1). > > You are right, I committed the 'fix' as initially proposed, but moved it to > the unix version of the CheckControlMessage; > > Checking for this message is not yet implemented on unix. The idea is to > open a 'control' socket using simpleIPC and to check on this socket for a > message. Then the TDaemonThread.CheckControlMessage() method should be declared virtual, so that messages could actually be checked for, shouldn't it ?

No, why ? It must simply be implemented globally by default?

If you want to override the default, then of course it should be virtual.

Michael.

Reply by email

Steve Howe
to Michael Van Canneyt

Hello all,

Show quoted text 6 lines
> Then the TDaemonThread.CheckControlMessage() method should be declared > virtual, so that messages could actually be checked for, shouldn't it ? No, why ? It must simply be implemented globally by default? If you want to override the default, then of course it should be virtual.

Yes, I could want to override the default for several reasons, such as changing the sleep time. Would it hurt to make it virtual ?

Best Regards, Steve Howe

Reply by email

Michael Van Canneyt
to Steve Howe
Steve Howe wrote 10 lines
Hello all, > > Then the TDaemonThread.CheckControlMessage() method should be declared > > virtual, so that messages could actually be checked for, shouldn't it ? > > No, why ? It must simply be implemented globally by default? > > If you want to override the default, then of course it should be virtual. Yes, I could want to override the default for several reasons, such as changing the sleep time. Would it hurt to make it virtual ?

No, so I will do so :-)

Michael.

Reply by email

Alvise Nicoletti
to Steve Howe: While using the daemonapp.pp unit to code a daemon, I've found that applications that it…

Steve Howe ha scritto:

Show quoted text 56 lines
Hello all, While using the daemonapp.pp unit to code a daemon, I've found that applications that it eat 100% of the CPU. After tracking and debugging, I've found out that the problem is in the daemonapp's TDaemonThread.Execute() method: procedure TDaemonThread.Execute; begin If FDaemon.Start then begin FDaemon.Status:=csRunning; StartServiceExecute; if not FDaemon.Execute then begin While Not Terminated do CheckControlMessage(True); CheckControlMessage(False); end; end; end; The loop has no interval so it just consumes all free CPU. I think it should be modified like this: procedure TDaemonThread.Execute; begin If FDaemon.Start then begin FDaemon.Status:=csRunning; StartServiceExecute; if not FDaemon.Execute then begin While Not Terminated do begin CheckControlMessage(True); Sleep(50); end; CheckControlMessage(False); end; end; end; Someone could argue that the damon's thread priority could be changed instead to "idle", making the thread more cooperative, but since daemonapp.inc's TDaemonController.Main() method also creates internally a TDaemonThread, and that thread is not exposed, changing the daemon's thread wouldn't help either. Does anyone have a better solution or comments ? Should I submit a patch ? If the changed is accepted and someone could do it for me, it would be even better.

Don't know if that helps, but can this problem be related to the "strange behaviour" I notifyed in this mailing list with the topic: "[lazarus] [fpc] strange behaviour (crash) in a closing linux daemon" the 07/11/2007

?

Reply by email

Michael Van Canneyt
to Micha Nelissen
Micha Nelissen wrote 5 lines
Michael Van Canneyt wrote: > On linux, you can only do polling. Of course you don't need polling, but the current architecture is only suited for polling.

On linux, everything comes down to polling. The only non-polling solution, signals, are not suitable since you don't know in which thread the signal ends up, and you must start using complicated inter-thread communication.

In practice, polling is simply much more easy and understandable.

Michael.

Reply by email

Micha Nelissen
to Michael Van Canneyt
Michael Van Canneyt wrote
On linux, everything comes down to polling. The only non-polling solution, signals, are not suitable since you don't know in which thread the signal ends up, and you must start using complicated inter-thread communication.

Use fpsigaction to mask the signal in the threads you don't want it? Perhaps this is already done so that all signals end up in main thread.

Anyway, I thought you mentioned a socket? You can do a blocking read on that?

Micha

Reply by email

Michael Van Canneyt
to Micha Nelissen
Micha Nelissen wrote 7 lines
Michael Van Canneyt wrote: > On linux, everything comes down to polling. The only non-polling solution, > signals, are not suitable since you don't know in which thread the signal > ends up, and you must start using complicated inter-thread communication. Use fpsigaction to mask the signal in the threads you don't want it? Perhaps this is already done so that all signals end up in main thread.

I don't think sigaction works on the thread level ?

Anyway, I thought you mentioned a socket? You can do a blocking read on that?

Ok, I see there is a difference in my and your interpretation of polling :-)

Yes, you can, but you'll need a fpselect() for the timeout as it is now: The thread which handles control messages uses a polling mechanism, this could be changed to a blocking mechanism since it is running in a separate thread anyway.

Michael.

Reply by email

Steve Howe
to Michael Schnell: Is it really necessary to do polling (with or without sleep) here ? Avoiding polling in…

Hello all,

Show quoted text 6 lines
> such as > changing the sleep time. Would it hurt to make it virtual ? Is it really necessary to do polling (with or without sleep) here ? Avoiding polling in any case would improve the performance and reduce intrusiveness.

On Windows you have alternatives such as someone posted in thread, but on Linux/Unix you must to do polling as far as know.

Anyway polling with sleep will consume very few cpu, acts very well overall and it's more portable.

Best Regards, Steve Howe

Reply by email

Bogusław Brandys
to Steve Howe
Steve Howe wrote 12 lines
Hello all, >> such as >> changing the sleep time. Would it hurt to make it virtual ? > Is it really necessary to do polling (with or without sleep) here ? > Avoiding polling in any case would improve the performance and reduce > intrusiveness. On Windows you have alternatives such as someone posted in thread, but on Linux/Unix you must to do polling as far as know. Anyway polling with sleep will consume very few cpu, acts very well overall and it's more portable.

Yes.On windows you could create event and wait for it to be signaled.This particularly move response for loop to the windows core and makes a huge difference in CPU usage.

Good luck.

Boguslaw

Reply by email

Michael Schnell
to Michael Van Canneyt: On linux, everything comes down to polling. The only non-polling solution, signals, are…
On linux, everything comes down to polling. The only non-polling solution, signals, are not suitable since you don't know in which thread the signal ends up, and you must start using complicated inter-thread communication.

There are lots of concepts in Linux that theoretically could be used for signaling. E.G. multiple message queues can be used with select() to handle multiple events. OTOH whenever a 3rd party widget system like QT is involved I suppose we need to use whatever this provides

In practice, polling is simply much more easy and understandable.

Right but technically it offers the lowest performance and unnecessarily uses the most CPU power.

-Michael

Reply by email

Michael Schnell
to Bogusław Brandys: Yes.On windows you could create event and wait for it to be signaled.This particularly…
Yes.On windows you could create event and wait for it to be signaled.This particularly move response for loop to the windows core and makes a huge difference in CPU usage.

Of course there are many zero overhead signaling methods in Linux. AFAIK, several inter thread / inter process communication methods are already done in some unit in the FP rtl.

-Michael

Reply by email