Hi, I noted that Pierre is updating the OpenBSD port. 

Does anyone knows where can I find a bootstraping compiler for this platform?.  
Leonardo M. Ramé
http://leonardorame.blogspot.com

Leonardo M. Ramé wrote

Does anyone knows where can I find a bootstraping compiler for this platform?.

There is none that is usable for current trunk, you have to cross- compile.

Jonas

I tried to get i386-openbsd port up to date,

using mingw cross Binutils that I compiled myself

using openbsd sources.

Should I add those to ftp?

But I am stuck because I have a 64-bit OpenBSD virtual machine

(and no room to add a i386 version on my machine)

but all cross-compiled executables.

Is it possible to install an openbsd equivalent of lib32?

How do you compile i386 openbsd executables

using GCC on a amd64 machine?

Pierre

De :
De la part de Leonardo M.
Ramé
Envoyé : mercredi 5 octobre 2011 01:31
À : FPC developers' list
Objet : [fpc-devel] OpenBSD compiler

Hi, I noted that Pierre is updating the OpenBSD port.

Does anyone knows where can I find a bootstraping compiler for this platform?.

Leonardo M. Ramé
http://leonardorame.blogspot.com

________________________________

Pierre, I have one VM of OpenBSD 4.4 i386 running all day long. If you want I can let you access using ssh to it, otherwhise you could send me the executables and I can test them in that machine.

Leonardo M. Ramé
http://leonardorame.blogspot.com

In our previous episode, Pierre Free Pascal said:

I tried to get i386-openbsd port up to date,

I had a openbsd 4.8 VM lying around, so I tried my usual cross bootstrap routine (I'm lazy, you don't need crossbinutils on BSD systems since they can run the linux bootstrap compiler)

What I did (preparation):

0. checked out the repo on the openbsd VM
1. compiled a linux compiler on linux with -dCMEM, this so that the binary has a decent ELF "ident" section
2. Copy it over to the openbsd VM. chmod +x
3. ran
sysctl kern.emul.linux=1
as root to enable linux emulation
4. ran "pkg_add fedora_base" to install basic userland
5. ran "pkg_add gmake" to install GNU make
6. modified rtl/openbsd/makefile to also compile cmem
7. ran
gmake PP=~/linux-bootstrap-with-cmem-ppc386 OS_TARGET=openbsd BINUTILSPREFIX="" OPT=-gl
in rtl/openbsd to build rtl

First I tried to build the compiler straight:

gmake all PP=~/linux-bootstrap-with-cmem-ppc386 OS_TARGET=openbsd RTL=/fpc/fpc/rtl/units/i386-openbsd OPT="-gl" BINUTILSPREFIX=""

this yields a compiler that gives an exception in readdir (if I try to compile) or options.readparameters with no parameters.

If I run with -i or -h, the usual output appears with a crash in SYSTEM_REMOVE_FIXED_CHUNKS following it, which I assume is a heapmanager function calling munmap?

I then tried to build with CMEM (same line as above but with -dCMEM added to OPT) to work around potential memory allocation problems, but that didn't link on a bunch of operating_system_* missing symbols (envp/argv/argc), so I assume its startup code wasn't fixed.

In our previous episode, Marco van de Voort said:

Some status update after some play yesterday evening:

1. The commit from Pierre for the exit syscall seems to work. Exe's don't crash anymore.

2. Some other minor fixes for stat record (as of yet uncommitted, as I still have to check the many ifdefs introduced. I think that parts of bsd/ostypes.inc must move to OS specific includes in time, specially if the number of targets rises. Darwin is also moving in slightly incompatible ways, since I see some arm specific darwin ifdefs there.

3. fixed a bug in the compilers parameter processing that seems to be triggered more in OpenBSD (r19394)

4. The next big crash is, as usual for a new port, in fexpand/expandfilename/readdir.

netbsd/openbsd seem to only support getdirentries, not getdents (if I understood correctly getdirentries is more BSD tradition, getdents is more SysV). Also the telldir functionality might be slightly different among BSDs.

Probably the relevant library code needs to translated. Maybe is checking the libc implementation against the readdir implementation of 1.0.x (readdir.inc) enough. That's on the menu for me tonight.

5. The problems with cprt0.as were relatively mild, just adding declarations for the relevant variables and adding a _ to the fpucw reference. The result wouldn't link dynamically though, despite OpenBSD having a fairly recent linker for once (2.15, while old, is the same as FreeBSD uses, so shouldn't be a problem for normal operation). Or at least ldd would crash on the result.

Marco van de Voort wrote

In our previous episode, Marco van de Voort said: Some status update after some play yesterday evening:

.
.

4. The next big crash is, as usual for a new port, in fexpand/expandfilename/readdir.

.
.

I assume you probably meant FindFirst rather than FExpand/ExpandFileName, right? FExpand needs only GetDir and GetEnv/GetEnvironmentVariable which should hardly be a reason for a crash of any new port hopefully...

Tomas

In our previous episode, Tomas Hajny said:

Don't worry. I'm not dissing your fexpand :-)

The unix implementation of getdir can actually enumerate directories.

So the problem is in readdir, but on *nix ports the crash will be in expandfilename because getdir stresses readdir hard, and in the compiler that happens before the first findfirst.

The problem of readdir is simply that it is one of the few core functionalities with a sizable userland part, and the implementations can be fairly different across OSes. (with even two choices for the kernel call used)

I ran into it already with the original FreeBSD port (done just before 1.0, merged in slightly after 1.0) This because in the original Linux port, readdir WAS a system call.

Another such traditional issue is the handling of seek and truncate, because they are syscalls with 64-bit types. Some OSes that call syscalls with values in registers still call them as if they were on the stack, and insert extra zero parameters to "align", and some don't, with variations per architecture.

I just added a script rtl/openbsd/check_sys.sh to compare Free Pascal syscall numbers in rtl/openbsd/sysnr.inc to the values in /usr/include/sys/syscall.h

I fixed the trivial issues, but now I get those:
-bash-4.1$ ./check_sys.sh
problem for SYS_dup expected 41 line is "#define SYS_dup 41" problem for SYS_sigaltstack expected 53 line is "#define SYS_sigaltstack 288"
problem for SYS_semsys expected 169 line is " 169 is compat_10 osemsys */" problem for SYS_msgsys expected 170 line is " 170 is compat_10 omsgsys */" problem for SYS_shmsys expected 170 line is " 170 is compat_10 omsgsys */" problem for SYS_stat35 expected 188 line is " 188 is compat_35 stat35 */" problem for SYS_fstat35 expected 189 line is " 189 is compat_35 fstat35 */" problem for SYS_lstat35 expected 190 line is " 190 is compat_35 lstat35 */" problem for SYS_getdirentries35 expected 196 line is "#define SYS_ogetdirentries 196"
problem for SYS_xfspioctl expected 208 line is "#define SYS_nnpfspioctl 208" problem for SYS_semop35 expected 222 line is " 222 is compat_35 semop */" problem for SYS_shmget35 expected 231 line is " 231 is compat_35 shmget */" problem for SYS___semctl35 expected 257 line is " 257 is compat_35 semctl35 */"
problem for SYS_shmctl35 expected 258 line is " 258 is compat_35 shmctl35 */"
problem for SYS_msgctl35 expected 259 line is " 259 is compat_35 msgctl35 */"
problem for SYS_getpeereid expected 273 line is " 273 is compat_o47 getpeereid */"
problem for SYS_signalstack expected 288 line is "#define SYS_sigaltstack 288"

The ones with compat are probably OK,
but what about SYS_sigaltstack and SYS_signalstack, they seem to be inversed!

I tried to cross-compile the compiler,
but it crashes inside a fsearch call...

Pierre

FExpand/ExpandFileName,

be

fairly different across OSes. (with even two choices for the kernel call used) I ran into it already with the original FreeBSD port (done just before

1.0,

merged in slightly after 1.0) This because in the original Linux port, readdir WAS a system call. Another such traditional issue is the handling of seek and truncate,

because

they are syscalls with 64-bit types. Some OSes that call syscalls with values in registers still call them as if they were on the stack, and

insert

In our previous episode, Pierre Free Pascal said:

The ones with compat are probably OK,

No, they are old syscalls only kept for compatibility. That means you also have to use the corresponding old syscalls.

I also think the getdents=getdirentries "alias" is not entirely correct.

It is possible that what we see here in trunk is not the same as what worked, long,long ago in the fixes1_0 branch, since in that time release branches were not kept as close in sync to trunk as nowadays.

I tried to cross-compile the compiler, but it crashes inside a fsearch call...

Probably readdir again, I'll work on it on and off as I have time the coming days.

You were right, getdirentries has one more parameter! I adapted bsd/ossysc.inc

coming

days.

Too late... (at least hopefully)
I just achieved a successful 'gmake cycle'
on i386 openbsd.

Pierre

Pierre, glad to hear that. What compiler did you use to bootstrap?.

I've downloaded one from FreePascal's ftp, but it gives me "Bad system call" when I run it.

Leonardo M. Ramé
http://leonardorame.blogspot.com

In our previous episode, Leonardo M. Ram? said:

Pierre, glad to hear that. What compiler did you use to bootstrap?. I've downloaded one from FreePascal's ftp, but it gives me "Bad system call" when I run it.

As said, easiest is to use a linux binary. All three BSDs can run linux binaries using a fairly thin compatibility layer (sometimes called linuxator).

Pierre, glad to hear that. What compiler did you use to bootstrap?.

I used a mingw compiled Openbsd Binutils source
(available in */usr/src/gnu/usr.bin/Binutils/binutils on a BSD machine with source installed).

(with small mods to

I've downloaded one from FreePascal's ftp, but it gives me "Bad system

call"

when I run it.

I uploaded these new cross-binutils to
ftp://ftp.freepascal.org/pub/fpc/contrib/cross/mingw in file
binutils-2.15-win32-i386-openbsd.zip

In the hope that this helps...

Pierre

----- Original Message -----

Thanks, the compiler runs ok. But I can't "gmake build all" using it.

I updated trunk from svn, then did:

gmake PP=~/i386-openbsd-ppc386 clean all

And got this:

(stripped irrelevant parts)
...
/bin/rm -f fpcmade.i386-openbsd Package.fpc ppas.sh script.res link.res /bin/rm -f *.s *_ppas.sh
gmake[7]: Leaving directory `/var/home/leonardo/fpc/rtl/openbsd' gmake[6]: Leaving directory `/var/home/leonardo/fpc/rtl' gmake -C /var/home/leonardo/fpc/rtl 'OPT=' all
gmake[6]: Entering directory `/var/home/leonardo/fpc/rtl' gmake -C openbsd all
gmake[7]: Entering directory `/var/home/leonardo/fpc/rtl/openbsd' as -o /var/home/leonardo/fpc/rtl/units/i386-openbsd/prt0.o i386/prt0.as as -o /var/home/leonardo/fpc/rtl/units/i386-openbsd/cprt0.o i386/cprt0.as /var/home/leonardo/i386-openbsd-ppc386 -Ur -FD -Ur -Xs -O2 -n -Fi../inc -Fi../i386 -Fi../unix -Fi../bsd -Fi../bsd/i386 -Fi../openbsd/i386 -FE. -FU/var/home/leonardo/fpc/rtl/units/i386-openbsd -di386 -dRELEASE -Us -Sg ../bsd/system.pp gmake[7]: *** [system.ppu] Bad system call (core dumped) gmake[7]: Leaving directory `/var/home/leonardo/fpc/rtl/openbsd' gmake[6]: *** [openbsd_all] Error 2
gmake[6]: Leaving directory `/var/home/leonardo/fpc/rtl' gmake[5]: *** [rtl] Error 2
gmake[5]: Leaving directory `/var/home/leonardo/fpc/compiler' gmake[4]: *** [next] Error 2
gmake[4]: Leaving directory `/var/home/leonardo/fpc/compiler' gmake[3]: *** [ppc1] Error 2
gmake[3]: Leaving directory `/var/home/leonardo/fpc/compiler' gmake[2]: *** [cycle] Error 2
gmake[2]: Leaving directory `/var/home/leonardo/fpc/compiler' gmake[1]: *** [compiler_cycle] Error 2
gmake[1]: Leaving directory `/var/home/leonardo/fpc' gmake: *** [build-stamp.i386-openbsd] Error 2

 
Leonardo M. Ramé
http://leonardorame.blogspot.com

Problem confirmed:
I use Sys call number 312 for getdirentries
but for 4.4, the last system call is 310.

I tested if getdirentries35 works OK,
I was able to do a gmake cycle on a 4.4 openbsd.

Committed in 19423.

Pierre

----- Original Message -----

Great!. I did an svn up, then gmake again, but the problem persists. What gmake options did you use?.

Maybe you should upload the new bootstrapping compiler to ftp://ftp.freepascal.org/pub/fpc/contrib/   
Leonardo M. Ramé
http://leonardorame.blogspot.com

In our previous episode, Leonardo M. Ram? said:

For me the one before this already worked.

I tried a "make all", and that failed in the fpmake of fcl-base on the pipe syscall. That's a know gotcha that I know still from 1.0.x times, so it was easily fixed. (*) If you tried "make all" before, don't forget to run make distclean though, to kill stale fpmake's.

The "make all" snapshot is still running for more than an hour now, and already way beyond the previous problem point. The slowliness is probably a combination of VM and slower than usual assembling and ARing without binwriter.

Ok, the make all just stopped, it breaks on pkgwget when compiling fppkg.

Anyway, if the snapshot runs to completion tomorrow, we'll see if we can upload it. Then we merge openbsd to fixes after 2.6.0-final is branched off, so that we have a 2.6 series starting compiler.

(*) FreeBSD's pipe command contains an optimization. The array is not passed to the syscall, and the results are stored in regs, not in the array. The wrapper code stores the regs in the array.

I added a i386 openbsd ppc386 executable in:
ftp://ftp.freepascal.org/pub/fpc/snapshot/v27/i386-openbsd/

In the hope that it could be useful for others too...

Pierre

pipe

syscall. That's a know gotcha that I know still from 1.0.x times, so it

was

easily fixed. (*) If you tried "make all" before, don't forget to run make distclean though, to kill stale fpmake's.

I surely spend some time trying to understand that failure... Great you already knew about it!

The "make all" snapshot is still running for more than an hour now, and already way beyond the previous problem point. The slowliness is probably

a

combination of VM and slower than usual assembling and ARing without binwriter. Ok, the make all just stopped, it breaks on pkgwget when compiling fppkg.

I got to the same point now...
I will try to launch a native openbsd testsuite overnight.

Anyway, if the snapshot runs to completion tomorrow, we'll see if we can upload it. Then we merge openbsd to fixes after 2.6.0-final is branched

off,

so that we have a 2.6 series starting compiler. (*) FreeBSD's pipe command contains an optimization. The array is not

passed

to the syscall, and the results are stored in regs, not in the array. The wrapper code stores the regs in the array.

The current FreeBSD code is i386/x86_64 only...
maybe it should use the normal code you added for other CPUs (even if those are not yet supported...)

Pierre

See
http://www.freepascal.org/testsuite/cgi-bin/testsuite.cgi?os=7&cpu=8&version =0&date=&submitter=&machine=&comment=&cond=

Marco, didn't you say that you fixed i386/cprt0.as for the missing operatingsystem_parameter vars?
Maybe you forgot to commit the fix,
anyway, I committed rev 19443.
I also restarted a testsuite adding
-Fl/usr/local/lib
as at least libiconv library is in that directory.

Pierre

pipe

syscall. That's a know gotcha that I know still from 1.0.x times, so it

was

a

off,

so that we have a 2.6 series starting compiler. (*) FreeBSD's pipe command contains an optimization. The array is not

passed

In our previous episode, Pierre Free Pascal said:

I already installed a 9.0 beta on powerpc :-)

I also installed NetBSD in a VM.

maybe it should use the normal code you added for other CPUs (even if those are not yet supported...)

I think we just soldier on target by target, see the ifdefs grow, and then start thinking about splitting heavily ifdefed parts of the include files in bsd/ over the targets. (and they can decide for themselves if they split them out over the arch dirs)

Some other points:
1) no I didn't commit cprt0 since the result didn't work.
2) Did you test with elf binwriter ?

in

You were right, it didn't help much:
http://www.freepascal.org/testsuite/cgi-bin/testsuite.cgi?os=7&cpu=8&version =0&date=&submitter=&machine=&comment=&cond=

2) Did you test with elf binwriter ?

It took me a long time to understand the problem, but I think I found it now:
OpenBSD ld seems to do nonsense if you do not explicitly give him a dynamic loader
it inserted /usr/lib/libc.so.1 as the dynamic loader... Adding an explicit -FL/usr/libexec/ld.so seems to allow at least tests/test/cg/tprinf to run successfully.

I will commit a fix setting a default dynamic linker for OpenBSD if this improves the results significantly!

Pierre

In our previous episode, Pierre Free Pascal said:

With the binwriter I meant not using external as/ld. I hope it will make the VM runs go a bit faster (they are well over an hour now)

I did a run last night and it stumbled on missing "bsd" unit, which holds the kqueue implementation that lnet needs. I quickchecked headers, and it seems to be mostly there in openbsd (give or take two constants) so I enabled it.

However the next run failed on an error in a script, so I don't know where it gets stuck now. I'll be home after noon, working at home the rest of the day, and then do some more runs.

Building a snapshot now works fine, shared linking and IDE inclusive. The IDE isn't built standardly yet though, so not in the snapshot

I uploaded an initial snapshot to

ftp://ftp.freepascal.org/pub/fpc/snapshot/v27/i386-openbsd/