Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2008-02

fpfork

16 messages · 5 people ·

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Michael Schnell

As I intend to port FP to a non MMU architecture at some point in the future I came across this issue:

In the RTL there are some functions that do a fork. The RTL supports as well calling fork() as vfork() API (via fpFork and fpVFork).

Here a setting USE_VFORK is acknowledged to decide which one is to be used.

Questions:
- not in all instances the difference is provided. sometimes fork() is done unconditionally
- Why is fork , all systems can do vfork() (with
_at_least_ the same performance as fork() and thus the difference is not needed. Regarding the function there is a difference between fork() and vfork() only if exec?() is _not_ called immediately afterwards, which seems to be the case in the FP RTL.
- vfork seems to be used in BSD and SUNOS. Why just in these systems ?
- when doing a non-MMU (µCLinux) architecture fork is not available. What is the best way of handling that situation ? (Should we introduce a UCLINUX OS flavor ?)

-Michael

Reply by email

Jonas Maebe
to Michael Schnell
Michael Schnell wrote 8 lines
Questions: - not in all instances the difference is provided. sometimes fork() is done unconditionally - Why is fork , all systems can do vfork() (with _at_least_ the same performance as fork() and thus the difference is not needed. Regarding the function there is a difference between fork() and vfork() only if exec?() is _not_ called immediately afterwards, which seems to be the case in the FP RTL.

In some cases memory is still allocated between the fork and the exec. You cannot use vfork in such a case. That code still has to be rewritten so it does not allocate any memory.

- vfork seems to be used in BSD and SUNOS. Why just in these systems ?

Correction: it is only used on those systems if the RTL is libc- based. And the reason is that our thread variable implementation and its use in the syscall implementations conflicts with using vfork.

- when doing a non-MMU (µCLinux) architecture fork is not available. What is the best way of handling that situation ? (Should we introduce a UCLINUX OS flavor ?)

Does it already have posix_spawn?

Jonas

Reply by email

Michael Schnell
to Jonas Maebe
Jonas Maebe wrote
In some cases memory is still allocated between the fork and the exec. You cannot use vfork in such a case. That code still has to be rewritten so it does not allocate any memory.

Do you suggest that some improvement should be done there ? Of course I will try to do this for the other architectures once I'll get started with that task and fully understand the implications. I feel that a better solution would be e.g. to drop the fpVFork function altogether and do the difference (if necessary at all) in the fpFork function.

Show quoted text 6 lines
> - vfork seems to be used in BSD and SUNOS. Why just in these systems ? Correction: it is only used on those systems if the RTL is libc-based. And the reason is that our thread variable implementation and its use in the syscall implementations conflicts with using vfork.

Don't you use the pthread library for the implementation of pascal threads ? (Threads and inter thread communications is another issue that I'll need to investigate - and maybe improve - when I'll start to do the "embedded pascal" project.)

Show quoted text 6 lines
> - when doing a non-MMU (µCLinux) architecture fork is not available. > What is the best way of handling that situation ? (Should we > introduce a UCLINUX OS flavor ?) Does it already have posix_spawn?

I don't know. The expert are just trying to get a 2.6. µCLinux version out of the door and I don't even have any hardware to test yet.

Thanks for your answers !

-Michael

Reply by email

Leonardo M. Ram�
to Michael Schnell

I've found an strange problem in Linux when executing a simple threaded program (Synapse's httpserver demo) compiled with fpc 2.2.0 from
ftp://ftp.freepascal.org/pub/fpc/snapshot/fixes/source/fpcbuild.zip

When I execute the program with "./httpserver", the program starts stopped. When I compile in Win32, with an older FPC it works ok.

Does anybody faced this issue?

Leonardo M. Ramé
http://leonardorame.blogspot.com

____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page.
http://www.yahoo.com/r/hs

Reply by email

Jonas Maebe
to Michael Schnell: Do you suggest that some improvement should be done there ? Of course I will try to do…
Michael Schnell wrote 6 lines
Jonas Maebe wrote: > > In some cases memory is still allocated between the fork and the > exec. You cannot use vfork in such a case. That code still has to > be rewritten so it does not allocate any memory. Do you suggest that some improvement should be done there ?

Yes.

Show quoted text 5 lines
Of course I will try to do this for the other architectures once I'll get started with that task and fully understand the implications. I feel that a better solution would be e.g. to drop the fpVFork function altogether and do the difference (if necessary at all) in the fpFork function.

I don't understand what you mean here. Both vfork and fork have certain semantics, and you cannot replace one with the other in all cases.

Show quoted text 9 lines
>> - vfork seems to be used in BSD and SUNOS. Why just in these >> systems ? > > Correction: it is only used on those systems if the RTL is libc- > based. And the reason is that our thread variable implementation > and its use in the syscall implementations conflicts with using > vfork. Don't you use the pthread library for the implementation of pascal threads ?

Yes. But thread variable support is more than just calling some libpthread functions.

Jonas

Reply by email

Michael Schnell
to Jonas Maebe
Show quoted text 7 lines
> I feel that a better solution would be e.g. to drop the fpVFork > function altogether and do the difference (if necessary at all) in > the fpFork function. I don't understand what you mean here. Both vfork and fork have certain semantics, and you cannot replace one with the other in all cases.

If we can do the improvement mentioned before, and as I suppose there is no instance where fork is not used to start another program (via exec?() ), it should be possible to do the code in all appropriate locations in a way that the functional differences between fork() and vfork() are not triggered. Thus a single function "fpFork" (or similar) should be sufficient. Thus any OS-introduced differences could be handled here at a central point instead of scattered over the code of multiple source files.

Yes. But thread variable support is more than just calling some libpthread functions.

So I don't understand why it seemingly is possible to _always_ use fork in _some_ location of the code.

I don't understand why threading support has anything to do with starting an external program. (But right now I don't need to anyway and BSD is out of my reach. I'll come back to you on that issue once I really get involved.)

-Michael

Reply by email

Marco van de Voort
to Leonardo M. Ram�: I've found an strange problem in Linux when executing a simple threaded program (Synapse's
Show quoted text 8 lines
I've found an strange problem in Linux when executing a simple threaded program (Synapse's httpserver demo) compiled with fpc 2.2.0 from ftp://ftp.freepascal.org/pub/fpc/snapshot/fixes/source/fpcbuild.zip When I execute the program with "./httpserver", the program starts stopped. When I compile in Win32, with an older FPC it works ok. Does anybody faced this issue?

If it uses threading, do you USE the cthreads unit ?

Reply by email

Leonardo M. Ram�
to Marco van de Voort

Yes, of course. If I don't include CThreads the program complains about including it.

--- Marco van de Voort wrote 14 lines
> I've found an strange problem in Linux when executing a simple threaded program (Synapse's > httpserver demo) compiled with fpc 2.2.0 from > ftp://ftp.freepascal.org/pub/fpc/snapshot/fixes/source/fpcbuild.zip > > When I execute the program with "./httpserver", the program starts stopped. When I compile in > Win32, with an older FPC it works ok. > > Does anybody faced this issue? If it uses threading, do you USE the cthreads unit ? _______________________________________________ fpc-devel maillist - http://lists.freepascal.org/mailman/listinfo/fpc-devel

____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page.
http://www.yahoo.com/r/hs

Reply by email

Leonardo M. Ram�
to Marco van de Voort

Yes, I run as root. Also I tested using different ports, I.E. 85, 3337, 10000.

--- Marco van de Voort wrote 9 lines
> Yes, of course. If I don't include CThreads the program complains about including it. Then, do you run as root? Binding to below 1024 ports might not be allowed as plain user. _______________________________________________ fpc-devel maillist - http://lists.freepascal.org/mailman/listinfo/fpc-devel

____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page.
http://www.yahoo.com/r/hs

Reply by email

Marco van de Voort
to Leonardo M. Ram�
Yes, I run as root. Also I tested using different ports, I.E. 85, 3337, 10000.

Then I'm running out of steam. I've hardly any experience with Synapse, and even with Indy I mostly ran on either Windows or FreeBSD, and not so much Linux.

I can imagine that on modern distro's there might also policy enforcers (selinux etc) that have something to say about binding, but I've no experience with them. You could try to temporarily turn off as many as you can find

Reply by email

Leonardo M. Ram�
to Marco van de Voort

I'll try on another box with Suse 10.2 (I've tested on Fedora Core 5) to see what happen...

Thanks Marco.

--- Marco van de Voort wrote 15 lines
> Yes, I run as root. Also I tested using different ports, I.E. 85, 3337, > 10000. Then I'm running out of steam. I've hardly any experience with Synapse, and even with Indy I mostly ran on either Windows or FreeBSD, and not so much Linux. I can imagine that on modern distro's there might also policy enforcers (selinux etc) that have something to say about binding, but I've no experience with them. You could try to temporarily turn off as many as you can find _______________________________________________ fpc-devel maillist - http://lists.freepascal.org/mailman/listinfo/fpc-devel

____________________________________________________________________________________ Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

Reply by email

Joost van der Sluis
to Leonardo M. Ram�

To turn off selinux on fedora core 5, just type 'setenforce permissive' in a terminal.

Op maandag 11-02-2008 om 05:35 uur [tijdzone -0800], schreef Leonardo M. Ramé:

Show quoted text 32 lines
I'll try on another box with Suse 10.2 (I've tested on Fedora Core 5) to see what happen... Thanks Marco. --- Marco van de Voort wrote: > > Yes, I run as root. Also I tested using different ports, I.E. 85, 3337, > > 10000. > > Then I'm running out of steam. I've hardly any experience with Synapse, and > even with Indy I mostly ran on either Windows or FreeBSD, and not so much > Linux. > > I can imagine that on modern distro's there might also policy enforcers > (selinux etc) that have something to say about binding, but I've no > experience with them. You could try to temporarily turn off as many as you > can find > _______________________________________________ > fpc-devel maillist - > http://lists.freepascal.org/mailman/listinfo/fpc-devel > ____________________________________________________________________________________ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ _______________________________________________ fpc-devel maillist - http://lists.freepascal.org/mailman/listinfo/fpc-devel

Reply by email

Jonas Maebe
to Michael Schnell: If we can do the improvement mentioned before, and as I suppose there is no instance…
Michael Schnell wrote 12 lines
> I don't understand what you mean here. Both vfork and fork have > certain semantics, and you cannot replace one with the other in all > cases. If we can do the improvement mentioned before, and as I suppose there is no instance where fork is not used to start another program (via exec?() ), it should be possible to do the code in all appropriate locations in a way that the functional differences between fork() and vfork() are not triggered. Thus a single function "fpFork" (or similar) should be sufficient. Thus any OS-introduced differences could be handled here at a central point instead of scattered over the code of multiple source files.

The single code point would have to implement "vfork+exec" semantics. You cannot do this for fork/vfork on their own, because they also have other uses besides executing an external program (e.g. fork is also used for daemons).

Show quoted text 7 lines
> Yes. But thread variable support is more than just calling some > libpthread functions. So I don't understand why it seemingly is possible to _always_ use fork in _some_ location of the code. I don't understand why threading support has anything to do with starting an external program.

After every system call, we store error code in the rtl's errno equivalent. This errno is a thread variable.

I don't know what exactly the problem is (possibly that the child overwrites the parent's errno, or maybe sometimes you have to allocate memory or change other global state when looking up a threadvar), but Daniël tried to add vfork support to the Linux/syscall rtl and said that he failed because "It seems our threadvar handling is vfork incompatible"

Jonas

Reply by email

Michael Schnell
to Jonas Maebe
Show quoted text 5 lines
The single code point would have to implement "vfork+exec" semantics. You cannot do this for fork/vfork on their own, because they also have other uses besides executing an external program (e.g. fork is also used for daemons).

OK, If possible "fork+exec" would be nice if any use of fork in the RTL can be avoided. If not and if any use of fork that does not allow replacement with vfork can be avoided a combined fp_V_Fork might be good.

Show quoted text 11 lines
After every system call, we store error code in the rtl's errno equivalent. This errno is a thread variable. I don't know what exactly the problem is (possibly that the child overwrites the parent's errno, or maybe sometimes you have to allocate memory or change other global state when looking up a threadvar), but Daniël tried to add vfork support to the Linux/syscall rtl and said that he failed because "It seems our threadvar handling is vfork incompatible"

I'm storing this for future reference.

Thanks,
-Michael

Reply by email

Leonardo M. Ram�
to Joost van der Sluis: To turn off selinux on fedora core 5, just type 'setenforce permissive' in a terminal. Op…

Thanks Joost. But, I remember this worked ok some time ago, in the same machine. Anyway, I'll try 'setenforce permissive' tomorrow.

--- Joost van der Sluis wrote 44 lines
To turn off selinux on fedora core 5, just type 'setenforce permissive' in a terminal. Op maandag 11-02-2008 om 05:35 uur [tijdzone -0800], schreef Leonardo M. Ramé: > I'll try on another box with Suse 10.2 (I've tested on Fedora Core 5) to see what happen... > > Thanks Marco. > > --- Marco van de Voort wrote: > > > > Yes, I run as root. Also I tested using different ports, I.E. 85, 3337, > > > 10000. > > > > Then I'm running out of steam. I've hardly any experience with Synapse, and > > even with Indy I mostly ran on either Windows or FreeBSD, and not so much > > Linux. > > > > I can imagine that on modern distro's there might also policy enforcers > > (selinux etc) that have something to say about binding, but I've no > > experience with them. You could try to temporarily turn off as many as you > > can find > > _______________________________________________ > > fpc-devel maillist - > > http://lists.freepascal.org/mailman/listinfo/fpc-devel > > > > > > ____________________________________________________________________________________ > Be a better friend, newshound, and > know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ > > _______________________________________________ > fpc-devel maillist - > http://lists.freepascal.org/mailman/listinfo/fpc-devel > -- _______________________________________________ fpc-devel maillist - http://lists.freepascal.org/mailman/listinfo/fpc-devel

____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page.
http://www.yahoo.com/r/hs

Reply by email