Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2026-07

POpen en PClose

14 messages · 3 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Adriaan van Os

I am looking at the implementation of POpen and PClose in the runtime-library for Darwin (or more general for UNIX). It seems like it doesn't call into libc popen and pclose, but instead follows its own implementation in unix.pp in POpen_internal. I am just curious to know the reason.

Regards,

Adriaan van Os

Marco van de Voort
to Adriaan van Os

Op 20-7-2026 om 10:13 schreef Adriaan van Os via fpc-pascal:

Show quoted text 7 lines
I am looking at the implementation of POpen and PClose in the runtime-library for Darwin (or more general for UNIX). It seems like it doesn't call into libc popen and pclose, but instead follows its own implementation in unix.pp in POpen_internal. I am just curious to know the reason.

Phew, that was over twenty years ago, but afaik the reason is that libc popen and pclose us C FILE descriptors, which the RTL in general doesn't use to avoid a mix of functions with pascal and C file types.

So popen/pclose are a more high level (FILE * using) part of libc, while we tend to only use the basis.

Michael Van Canneyt
to Adriaan van Os: I am looking at the implementation of POpen and PClose in the runtime-library for Darwin…
Adriaan van Os wrote 5 lines
I am looking at the implementation of POpen and PClose in the runtime-library for Darwin (or more general for UNIX). It seems like it doesn't call into libc popen and pclose, but instead follows its own implementation in unix.pp in POpen_internal. I am just curious to know the reason.

- Because on linux we're independent of libc.
- IIRC the libc version uses the C FILE type, which is not compatible with the file descriptor we use elsewhere. Introducing that would require lots of wiring to handle both paths.

Michael.

Michael Van Canneyt
to Marco van de Voort: Op 20-7-2026 om 10:13 schreef Adriaan van Os via fpc-pascal: Phew, that was over twenty…
Marco van de Voort wrote 15 lines
Op 20-7-2026 om 10:13 schreef Adriaan van Os via fpc-pascal: > > I am looking at the implementation of POpen and PClose in the > runtime-library for Darwin (or more general for UNIX). It seems like it > doesn't call into libc popen and pclose, but instead follows its own > implementation in unix.pp in POpen_internal. I am just curious to know the > reason. > Phew, that was over twenty years ago, but afaik the reason is that libc popen and pclose us C FILE descriptors, which the RTL in general doesn't use to avoid a mix of functions with pascal and C file types. So popen/pclose are a more high level (FILE * using) part of libc, while we tend to only use the basis.

Glad our memories agree on this :-)

Michael.

Marco van de Voort
to Michael Van Canneyt

Op 20-7-2026 om 11:57 schreef Michael Van Canneyt via fpc-pascal:

Show quoted text 15 lines
>> >> I am looking at the implementation of POpen and PClose in the >> runtime-library for Darwin (or more general for UNIX). It seems like >> it doesn't call into libc popen and pclose, but instead follows its >> own implementation in unix.pp in POpen_internal. I am just curious >> to know the reason. >> > Phew, that was over twenty years ago, but afaik the reason is that > libc popen and pclose us C FILE descriptors, which the RTL in general > doesn't use to avoid a mix of functions with pascal and C file types. > > So popen/pclose are a more high level (FILE * using) part of libc, > while we tend to only use the basis. Glad our memories agree on this :-)

Well aside from the fact that FreeBSD was the architecture where the baseunix/unix RTL  was done. The port to Linux was only done later :-)

This shows the tension a bit between libc the system lib, and libc the C runtime. We are only interested in the system lib part.

Michael Van Canneyt
to Marco van de Voort
Marco van de Voort wrote 20 lines
Op 20-7-2026 om 11:57 schreef Michael Van Canneyt via fpc-pascal: >>> >>> I am looking at the implementation of POpen and PClose in the >>> runtime-library for Darwin (or more general for UNIX). It seems like >>> it doesn't call into libc popen and pclose, but instead follows its >>> own implementation in unix.pp in POpen_internal. I am just curious >>> to know the reason. >>> >> Phew, that was over twenty years ago, but afaik the reason is that >> libc popen and pclose us C FILE descriptors, which the RTL in general >> doesn't use to avoid a mix of functions with pascal and C file types. >> >> So popen/pclose are a more high level (FILE * using) part of libc, >> while we tend to only use the basis. > > Glad our memories agree on this :-) Well aside from the fact that FreeBSD was the architecture where the baseunix/unix RTL  was done. The port to Linux was only done later :-)

Correct, but popen/pclose existed before that in the linux unit. Pretty sure about that, because I implemented it there.

Michael.

Marco van de Voort
to Michael Van Canneyt

Op 20-7-2026 om 12:07 schreef Michael Van Canneyt via fpc-pascal:

Show quoted text 5 lines
> Well aside from the fact that FreeBSD was the architecture where the > baseunix/unix RTL  was done. The port to Linux was only done later :-) Correct, but popen/pclose existed before that in the linux unit. Pretty sure about that, because I implemented it there.

Yes, I think so.  The left overs of the linux/unix library were done relatively later (first baseunix was done, and everything was migrated),   and since the initial libc implementation of baseunix didn't introduce FILE *, it was logical to maintain the signatures.

Jonas  might even have started using the libc port for Darwin at that point (IIRC about half an year after the big bang), I can vaguely remember him helping me with some of those leftover  unix functions, which had some issues (iirc clone() usage and problems with a function call that had a deviation way of handling parameters (PIPE?)) .  Darwin and FreeBSD shared quite a lot of pedigree back then.

So it could even have been Jonas' decision to maintain the signatures, not mine.

Adriaan van Os
to Marco van de Voort: Op 20-7-2026 om 10:13 schreef Adriaan van Os via fpc-pascal: Phew, that was over twenty…
Marco van de Voort wrote 15 lines
Op 20-7-2026 om 10:13 schreef Adriaan van Os via fpc-pascal: > > I am looking at the implementation of POpen and PClose in the > runtime-library for Darwin (or more general for UNIX). It seems like > it doesn't call into libc popen and pclose, but instead follows its > own implementation in unix.pp in POpen_internal. I am just curious to > know the reason. > Phew, that was over twenty years ago, but afaik the reason is that libc popen and pclose us C FILE descriptors, which the RTL in general doesn't use to avoid a mix of functions with pascal and C file types. So popen/pclose are a more high level (FILE * using) part of libc, while we tend to only use the basis.

Ah, so that means that I couldn't just call naively into libc popen with FileRec.Handle ?

Regards,

Adriaan van Os

Michael Van Canneyt
to Adriaan van Os
Adriaan van Os wrote 21 lines
On 20/07/2026 11:40, Marco van de Voort via fpc-pascal wrote: > > Op 20-7-2026 om 10:13 schreef Adriaan van Os via fpc-pascal: >> >> I am looking at the implementation of POpen and PClose in the >> runtime-library for Darwin (or more general for UNIX). It seems like it >> doesn't call into libc popen and pclose, but instead follows its own >> implementation in unix.pp in POpen_internal. I am just curious to know the >> reason. >> > Phew, that was over twenty years ago, but afaik the reason is that libc > popen and pclose us C FILE descriptors, which the RTL in general doesn't > use to avoid a mix of functions with pascal and C file types. > > So popen/pclose are a more high level (FILE * using) part of libc, while we > tend to only use the basis. Ah, so that means that I couldn't just call naively into libc popen with FileRec.Handle ?

Indeed you cannot do that.

Michael.

Marco van de Voort
to Adriaan van Os: Ah, so that means that I couldn't just call naively into libc popen with FileRec.Handle ?…

Op 20-7-2026 om 15:13 schreef Adriaan van Os via fpc-pascal:

Show quoted text 7 lines
> > So popen/pclose are a more high level (FILE * using) part of libc, > while we tend to only use the basis. Ah, so that means that I couldn't just call naively into libc popen with FileRec.Handle ?

Indeed.   And FILE* is opague and thus target specific,  it would mean another structure to maintain, and to somehow to determine for a new *nix target. STAT and STATFS already give us grief enough.

Adriaan van Os
to Marco van de Voort
Marco van de Voort wrote 12 lines
Op 20-7-2026 om 15:13 schreef Adriaan van Os via fpc-pascal: >> >> So popen/pclose are a more high level (FILE * using) part of libc, >> while we tend to only use the basis. > > Ah, so that means that I couldn't just call naively into libc popen > with FileRec.Handle ? > Indeed.   And FILE* is opague and thus target specific,  it would mean another structure to maintain, and to somehow to determine for a new *nix target. STAT and STATFS already give us grief enough.

Right. So now I am calling PClose and it will hang. Why ? Well, PClose calls WaitProcess in unx.pp, which in turn calls fpWaitPid in a loop, which returns (on Darwin) 0 for a process that has died ....

   repeat
     r:=fpWaitPid(Pid,@s,0);
     if (r=-1) and (fpgeterrno=ESysEIntr) Then
       r:=0;
   until (r0);

In general, I find such a loop, that depends on result codes, quite risky. The POSIX standard says here
<https://pubs.opengroup.org/onlinepubs/9699919799/functions/wait.html>

  "If/wait/() or/waitpid/() return because the status of a child process is available, these functions shall return a value equal to the process ID of the child process. In this case, if the value of the argument/stat_loc/is not a null pointer, information shall be stored in the location pointed to by/stat_loc/. The value stored at the location pointed to by/stat_loc/shall be 0 if and only if the status returned is from a terminated child process that terminated by one of the following means:"

Regards,

Adriaan van Os

Adriaan van Os
to Adriaan van Os
Adriaan van Os wrote 38 lines
On 20/07/2026 16:19, Marco van de Voort via fpc-pascal wrote: > > Op 20-7-2026 om 15:13 schreef Adriaan van Os via fpc-pascal: >>> >>> So popen/pclose are a more high level (FILE * using) part of libc, >>> while we tend to only use the basis. >> >> Ah, so that means that I couldn't just call naively into libc popen >> with FileRec.Handle ? >> > Indeed.   And FILE* is opague and thus target specific,  it would > mean another structure to maintain, and to somehow to determine for a > new *nix target. STAT and STATFS already give us grief enough. Right. So now I am calling PClose and it will hang. Why ? Well, PClose calls WaitProcess in unx.pp, which in turn calls fpWaitPid in a loop, which returns (on Darwin) 0 for a process that has died ....   repeat     r:=fpWaitPid(Pid,@s,0);     if (r=-1) and (fpgeterrno=ESysEIntr) Then       r:=0;   until (r0); In general, I find such a loop, that depends on result codes, quite risky. The POSIX standard says here <https://pubs.opengroup.org/onlinepubs/9699919799/functions/wait.html>     "If/wait/() or/waitpid/() return because the status of a child process is available, these functions shall return a value equal to the process ID of the child process. In this case, if the value of the argument/stat_loc/is not a null pointer, information shall be stored in the location pointed to by/stat_loc/. The value stored at the location pointed to by/stat_loc/shall be 0 if and only if the status returned is from a terminated child process that terminated by one of the following means:"

This is nonsense. I apologize for the noise. The reason for the hang is not the loop but the child process that has not terminated.

By the way, how do I flush the output to the POpen child process ? I tried SysUtils.FileFlush with FileRec( File).Handle. but it failed.

Regards,

Adriaan van Os

Adriaan van Os
to Marco van de Voort: Op 20-7-2026 om 11:57 schreef Michael Van Canneyt via fpc-pascal: Well aside from the…
Marco van de Voort wrote 24 lines
Op 20-7-2026 om 11:57 schreef Michael Van Canneyt via fpc-pascal: >>> >>> I am looking at the implementation of POpen and PClose in the >>> runtime-library for Darwin (or more general for UNIX). It seems >>> like it doesn't call into libc popen and pclose, but instead >>> follows its own implementation in unix.pp in POpen_internal. I am >>> just curious to know the reason. >>> >> Phew, that was over twenty years ago, but afaik the reason is that >> libc popen and pclose us C FILE descriptors, which the RTL in >> general doesn't use to avoid a mix of functions with pascal and C >> file types. >> >> So popen/pclose are a more high level (FILE * using) part of libc, >> while we tend to only use the basis. > > Glad our memories agree on this :-) Well aside from the fact that FreeBSD was the architecture where the baseunix/unix RTL  was done. The port to Linux was only done later :-) This shows the tension a bit between libc the system lib, and libc the C runtime. We are only interested in the system lib part.

Another question. POpen has a file as input parameter. The description in unix.pp says

  "Starts the program in 'Prog' and makes it's input or out put the   other end of a pipe. If rw is 'w' or 'W', then whatever is written to   F, will be read from stdin by the program in 'Prog'. The inverse is true   for 'r' or 'R' : whatever the program in 'Prog' writes to stdout, can be   read from 'f'."

So, for write mode, it doesn't use StdOut but a file. Does this imply that (for writing) I can call POpen with multiple files, forking and running multiple programs, writing to each of these child processes through a different pipe ?

Regards,

Adriaan van Os

Michael Van Canneyt
to Adriaan van Os
Adriaan van Os wrote 41 lines
On 20/07/2026 12:02, Marco van de Voort via fpc-pascal wrote: > > Op 20-7-2026 om 11:57 schreef Michael Van Canneyt via fpc-pascal: >>>> >>>> I am looking at the implementation of POpen and PClose in the >>>> runtime-library for Darwin (or more general for UNIX). It seems >>>> like it doesn't call into libc popen and pclose, but instead >>>> follows its own implementation in unix.pp in POpen_internal. I am >>>> just curious to know the reason. >>>> >>> Phew, that was over twenty years ago, but afaik the reason is that >>> libc popen and pclose us C FILE descriptors, which the RTL in >>> general doesn't use to avoid a mix of functions with pascal and C >>> file types. >>> >>> So popen/pclose are a more high level (FILE * using) part of libc, >>> while we tend to only use the basis. >> >> Glad our memories agree on this :-) > > Well aside from the fact that FreeBSD was the architecture where the > baseunix/unix RTL  was done. The port to Linux was only done later :-) > > This shows the tension a bit between libc the system lib, and libc the > C runtime. We are only interested in the system lib part. Another question. POpen has a file as input parameter. The description in unix.pp says   "Starts the program in 'Prog' and makes it's input or out put the   other end of a pipe. If rw is 'w' or 'W', then whatever is written to   F, will be read from stdin by the program in 'Prog'. The inverse is true   for 'r' or 'R' : whatever the program in 'Prog' writes to stdout, can be   read from 'f'." So, for write mode, it doesn't use StdOut but a file. Does this imply that (for writing) I can call POpen with multiple files, forking and running multiple programs, writing to each of these child processes through a different pipe ?

If your question is, can you use POpen multiple times, then the answer is yes.

Michael.