Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2014-01

respawn a daemon in Linux

19 messages · 4 people ·

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Michael Schnell

Hi Linux Experts.

Since years I am running an fpc program as a kind of daemon on a PC Linux server.

I simply defined a "respawn" line in /etc/inittab and with any start of the server, the program happily automatically runs as a daemon.

Now I migrated the the PC Linux server to an ARM based "QNAP" NAS box, which comes with a propriety - but rather normal - Linux distribution. Hence I compiled the program with the appropriate ARM fpc and it happily runs on this box, as well.

Now the problem is that I can't edit the inittab file on this box, as it is in a RAM file system that gets recreated with any start.

But I was able to install an init.d system that allows for running any script after booting. Thus I need to find a way to run my program as a deamon.

Of course I do know about daemonapp.pp and with that, I suppose I could easily have the program daemonize itself. But the program is written in a way that it exits whenever some kind of error happens (and this does occur, as it accesses multiple sites via TCP/IP. So it relies on the "respawn" feature, inittab provides.

I would appreciate any idea on how to create some kind of "respawn" algorithm within this program or with a second program - maybe just a script. In fact I could use "cron" for this, as happily the system does allow me to edit crontab (which here actually is /etc/config/crontab and the /etc/config/ is symlinked to a directory residing in a special location on a hard disk.

-Michael

Mark Morgan Lloyd
to Michael Schnell
Michael Schnell wrote 14 lines
Hi Linux Experts. Since years I am running an fpc program as a kind of daemon on a PC Linux server. I simply defined a "respawn" line in /etc/inittab and with any start of the server, the program happily automatically runs as a daemon. I would appreciate any idea on how to create some kind of "respawn" algorithm within this program or with a second program - maybe just a script. In fact I could use "cron" for this, as happily the system does allow me to edit crontab (which here actually is /etc/config/crontab and the /etc/config/ is symlinked to a directory residing in a special location on a hard disk.

Does it exit in a sufficiently-controlled state that you could simply exec a new copy at the end of the finalization block? The available parameters should be accessible in /proc.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]
Michael Van Canneyt
to Mark Morgan Lloyd
Mark Morgan Lloyd wrote 19 lines
Michael Schnell wrote: > Hi Linux Experts. > > Since years I am running an fpc program as a kind of daemon on a PC Linux > server. > > I simply defined a "respawn" line in /etc/inittab and with any start of the > server, the program happily automatically runs as a daemon. > I would appreciate any idea on how to create some kind of "respawn" > algorithm within this program or with a second program - maybe just a > script. In fact I could use "cron" for this, as happily the system does > allow me to edit crontab (which here actually is /etc/config/crontab and > the /etc/config/ is symlinked to a directory residing in a special location > on a hard disk. Does it exit in a sufficiently-controlled state that you could simply exec a new copy at the end of the finalization block? The available parameters should be accessible in /proc.

No need to modify the program. You can simply script it as

#!/bin/bash
while [ 1 ]; do
yourprogram youroptions
done

Michael.

Tony Whyman
to Michael Schnell: Hi Linux Experts. Since years I am running an fpc program as a kind of daemon on a PC…

Michael,

This should be a classic Unix program with two separate parent and child processes, where the child process runs your program and the parent is responsible for monitoring the child and respawning it when it stops. The basic algorithm of the parent is:

Begin
  ...
  daemon(0,0):
..
  repeat
    fork();
   if child then exec child program;
   else
     wait for child exit;
  log child completion;
 until sig term received;
...
End.

You need to set up signal handling to check for the sig term. Most examples will be in 'C' but you should be able to write the same algorithm in Pascal.

Good luck.

Tony Whyman
MWA

Michael Schnell wrote 37 lines
Hi Linux Experts. Since years I am running an fpc program as a kind of daemon on a PC Linux server. I simply defined a "respawn" line in /etc/inittab and with any start of the server, the program happily automatically runs as a daemon. Now I migrated the the PC Linux server to an ARM based "QNAP" NAS box, which comes with a propriety - but rather normal - Linux distribution. Hence I compiled the program with the appropriate ARM fpc and it happily runs on this box, as well. Now the problem is that I can't edit the inittab file on this box, as it is in a RAM file system that gets recreated with any start. But I was able to install an init.d system that allows for running any script after booting. Thus I need to find a way to run my program as a deamon. Of course I do know about daemonapp.pp and with that, I suppose I could easily have the program daemonize itself. But the program is written in a way that it exits whenever some kind of error happens (and this does occur, as it accesses multiple sites via TCP/IP. So it relies on the "respawn" feature, inittab provides. I would appreciate any idea on how to create some kind of "respawn" algorithm within this program or with a second program - maybe just a script. In fact I could use "cron" for this, as happily the system does allow me to edit crontab (which here actually is /etc/config/crontab and the /etc/config/ is symlinked to a directory residing in a special location on a hard disk. -Michael _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Michael Schnell
to Mark Morgan Lloyd: Does it exit in a sufficiently-controlled state that you could simply exec a new copy at…
Mark Morgan Lloyd wrote
Does it exit in a sufficiently-controlled state

Yep. If nothing completely unexpected happens, it writes a message in a log file and cleanly exits. Of course there _might_ be memory leaks, an unexpected kill signal or other queer stuff that could kill the application.

that you could simply exec a new copy at the end of the finalization block?

Using SysUtils.ExecuteProcess ? Do you think that this will work cleanly, even though the dying process would be the parent of the new one ? AFAIR, Linux performs a "reparent" at this point and assign the grand-parent. So it really might work.

Is the finalization block executed even when a "nasty" exception like "kill signal", "nonexisting address", or "division by zero" happens ?

The available parameters should be accessible in /proc.

Yep. In fact I would need a parameter that activates this feature, so that the program also can be run normally. No problem here.

-Michael

Michael Schnell
to Michael Van Canneyt: No need to modify the program. You can simply script it as #!/bin/bash while [ 1 ]; do…
Michael Van Canneyt wrote
#!/bin/bash while [ 1 ]; do yourprogram youroptions done

I feel this will start multiple parallel copies of the program and not wait until the previous one dies.

Moreover I understand that this loop will never exit. Hence I can't use it an init.d script, but in the calling script need to do something based on "bash ... &" to have the "respawn" script run in parallel.

-Michael

Michael Schnell
to Tony Whyman: This should be a classic Unix program with two separate parent and child processes, where…
Tony Whyman wrote 7 lines
Michael, This should be a classic Unix program with two separate parent and child processes, where the child process runs your program and the parent is responsible for monitoring the child and respawning it when it stops. The basic algorithm of the parent is: ....

This really sounds like the decent way to do this !

In fact at best someone (me) should modify the demonapp.pp unit in the fpc library to allow for this kind of re-spawning.

Would the fpc team accept such a patch ?

Thanks,
-Michael

Michael Van Canneyt
to Michael Schnell: I feel this will start multiple parallel copies of the program and not wait until the…
Michael Schnell wrote 8 lines
On 01/14/2014 11:32 AM, Michael Van Canneyt wrote: > #!/bin/bash > while [ 1 ]; do > yourprogram youroptions > done I feel this will start multiple parallel copies of the program and not wait until the previous one dies.

Of course it will wait.
A script always waits till a command exits.

Moreover I understand that this loop will never exit. Hence I can't use it an init.d script, but in the calling script need to do something based on "bash ... &" to have the "respawn" script run in parallel.

I didn't suggest you'd use this as a init.d script.

In your init.d script you need launch the above script (call it loopmyprogram) in the usual manner, in the background.

All this is IMHO preferable to modifying your program, which may introduce additional bugs, unforeseen cases etc...

Michael.

Michael Van Canneyt
to Michael Schnell: This really sounds like the decent way to do this ! In fact at best someone (me) should…
Michael Schnell wrote 15 lines
On 01/14/2014 11:35 AM, Tony Whyman wrote: > Michael, > > This should be a classic Unix program with two separate parent and child > processes, where the child process runs your program and the parent is > responsible for monitoring the child and respawning it when it stops. > The basic algorithm of the parent is: > .... This really sounds like the decent way to do this ! In fact at best someone (me) should modify the demonapp.pp unit in the fpc library to allow for this kind of re-spawning. Would the fpc team accept such a patch ?

No.

There are a lot of other factors involved, which you cannot control in such a general-purpose component.

I had thought about introducing this when doing the implementation, but decided against it.

Michael.

Michael Van Canneyt
to Michael Schnell
Michael Schnell wrote
On 01/14/2014 11:55 AM, Michael Van Canneyt wrote: > I had thought about introducing this when doing the implementation, but > decided against it. Thanks for the warning.

You're welcome.

It also explains why I would go for the scripting solution. A lot more safe.

Michael.

Michael Schnell
to Michael Van Canneyt: Of course it will wait. A script always waits till a command exits. I didn't suggest…
Michael Van Canneyt wrote
Of course it will wait. A script always waits till a command exits.

I see. Thanks (silly me).

In your init.d script you need launch the above script (call it loopmyprogram) in the usual manner, in the background.

Is there a more recommended "usual manner" to launch a script in the background than using "&" (which I did not test with scripts yet, just with applications)

All this is IMHO preferable to modifying your program, which may introduce additional bugs, unforeseen cases etc...

It seems to be really simple so I'll just do a test tonight.

-Michael

Mark Morgan Lloyd
to Michael Van Canneyt: No need to modify the program. You can simply script it as #!/bin/bash while [ 1 ]; do…
Michael Van Canneyt wrote 10 lines
> Does it exit in a sufficiently-controlled state that you could simply > exec a new copy at the end of the finalization block? The available > parameters should be accessible in /proc. No need to modify the program. You can simply script it as #!/bin/bash while [ 1 ]; do yourprogram youroptions done

With caution. You need to make sure that it doesn't try to restart if it's received a signal that the system's going down because of a power failure etc., where the last thing you want is a daemon respawning wildly and wasting CPU time that would be much better spent shutting everything else down in a controlled fashion.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]
Michael Van Canneyt
to Michael Schnell: I see. Thanks (silly me). Is there a more recommended "usual manner" to launch a script…
Michael Schnell wrote 9 lines
On 01/14/2014 11:52 AM, Michael Van Canneyt wrote: > Of course it will wait. A script always waits till a command exits. I see. Thanks (silly me). > In your init.d script you need launch the above script (call it > loopmyprogram) in the usual manner, in the background. Is there a more recommended "usual manner" to launch a script in the background than using "&" (which I did not test with scripts yet, just with applications)

using "&" is the recommended manner.

Whether you launch a script or an actual program is entirely irrelevant.

Michael.

Michael Schnell
to Michael Van Canneyt
Michael Van Canneyt wrote
Whether you launch a script or an actual program is entirely irrelevant.

We are discussing Linux, not Windows. So we will get what we expect :-) .

-Michael

Mark Morgan Lloyd
to Michael Schnell
Michael Schnell wrote 6 lines
On 01/14/2014 12:07 PM, Mark Morgan Lloyd wrote: > You need to make sure that it doesn't try to restart if it's received > a signal that the system's going down > I suppose the bash running the "loopmyprogram" script will get the kill signal, as well and stop right away.

They'll get a TERM first, and five seconds later a KILL. You don't want to restart after that first signal, because if you wait until the second one you risk leaving files etc. associated with the program in an indeterminate state. I've had Qemu break a database inside NTFS by refusing to honour a shutdown.

So you can't restart after a KILL, shouldn't after a TERM, might not want to after an INT (which, as we were discussing a few days ago, includes ^C from the keyboard) but probably do want to restart after a HUP. At the very least, you need to make sure that your program propagates why it's stopped to whatever's started it.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]
Michael Schnell
to Mark Morgan Lloyd
Mark Morgan Lloyd wrote 13 lines
They'll get a TERM first, and five seconds later a KILL. You don't want to restart after that first signal, because if you wait until the second one you risk leaving files etc. associated with the program in an indeterminate state. I've had Qemu break a database inside NTFS by refusing to honour a shutdown. So you can't restart after a KILL, shouldn't after a TERM, might not want to after an INT (which, as we were discussing a few days ago, includes ^C from the keyboard) but probably do want to restart after a HUP. At the very least, you need to make sure that your program propagates why it's stopped to whatever's started it.

Thanks a lot. I'll do some tests.....

-Michael

Michael Schnell
to Mark Morgan Lloyd: They'll get a TERM first, and five seconds later a KILL. You don't want to restart after…
Mark Morgan Lloyd wrote 6 lines
They'll get a TERM first, and five seconds later a KILL. You don't want to restart after that first signal, because if you wait until the second one you risk leaving files etc. associated with the program in an indeterminate state. I've had Qemu break a database inside NTFS by refusing to honour a shutdown.

I'll insert a "sleep 10" in the loop and hope for the best :-) .

-Michael