Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2012-07

Cross compiling x86 on amd64, linking to crtbegin.o etc.

7 messages · 4 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Sven Barth
to Konstantin
Konstantin wrote 19 lines
Hello! Free Pascal has been designed to run on several platforms on many architectures which is a neat feature. The executable name is chosen accordingly - ppc386, ppcx64 etc. to be able to have many at the same time on the same system. When building from source it automatically detects the platform on which it is running. Unfortunately, when building on amd64 you end up with the ppcx64 compiler which can not compile for x86 targets. I'm using Free Pascal on Gentoo so I created ebuilds (see Gentoo bug report here <https://bugs.gentoo.org/show_bug.cgi?id=322971>) to compile FPC on amd64 machines for x86 as well if desired. Currently this involves compiling everything twice and then install it together. This way the advantage of the CPU to be able to run code in 32 bit and 64 bit on the same system can be used. So first question, is there a better way instead of compiling everything twice? Maybe something can be reused?

I'm not used to Gentoo's ebuilds, so I'll tell you from FPC's normal perspective:

if you do a "make all CPU_TARGET=i386" on a x86_64 Linux system then a x86_64 -> i386 cross compiler will be created with default system Linux and where the compiler binary will be called ppcross386 (instead of ppc386). [It works similar on Windows, other targets and other CPUs with the only exception that most non-i386 systems can not cross compile to i386, because of the missing Extended support; x86_64 on non-Windows can though] Only problem might be that the build process and/or the compiler does not find the correct binutils, so you might need to help it using BINUTILSPREFIX (e.g. "BINUTILSPREFIX=i686-linux-") and CROSSBINDIR (e.g. "CROSSBINDIR=/opt/gcc32" {just an example}). The crosscompiled compiler (and its units) is then installed using "make crossinstall CPU_TARGET=i386 INSTALL_PREFIX=/whereever/you/want".

Regards,
Sven

Reply by email

Konstantin
to Sven Barth
Show quoted text 37 lines
Am 11.07.2012 16:36, schrieb Konstantin: > Hello! > > Free Pascal has been designed to run on several platforms on many > architectures which is a neat feature. The executable name is chosen > accordingly - ppc386, ppcx64 etc. to be able to have many at the same > time on the same system. When building from source it automatically > detects the platform on which it is running. Unfortunately, when > building on amd64 you end up with the ppcx64 compiler which can not > compile for x86 targets. > > I'm using Free Pascal on Gentoo so I created ebuilds (see Gentoo bug > report here <https://bugs.gentoo.org/show_bug.cgi?id=322971>) to compile > FPC on amd64 machines for x86 as well if desired. Currently this > involves compiling everything twice and then install it together. This > way the advantage of the CPU to be able to run code in 32 bit and 64 bit > on the same system can be used. > > So first question, is there a better way instead of compiling everything > twice? Maybe something can be reused? I'm not used to Gentoo's ebuilds, so I'll tell you from FPC's normal perspective: if you do a "make all CPU_TARGET=i386" on a x86_64 Linux system then a x86_64 -> i386 cross compiler will be created with default system Linux and where the compiler binary will be called ppcross386 (instead of ppc386). [It works similar on Windows, other targets and other CPUs with the only exception that most non-i386 systems can not cross compile to i386, because of the missing Extended support; x86_64 on non-Windows can though] Only problem might be that the build process and/or the compiler does not find the correct binutils, so you might need to help it using BINUTILSPREFIX (e.g. "BINUTILSPREFIX=i686-linux-") and CROSSBINDIR (e.g. "CROSSBINDIR=/opt/gcc32" {just an example}). The crosscompiled compiler (and its units) is then installed using "make crossinstall CPU_TARGET=i386 INSTALL_PREFIX=/whereever/you/want".

Thanks for the information. Just a note - the Gentoo ebuilds are a kind of compile and install scripts similar to shell scripts so even when you're not used to they are easy to understand if you are familiar with the compile and install process of a specific software.

I played around a bit to see what happens. On my x86_64 system i get it compiled using the following:

make all CPU_TARGET=i386 BINUTILSPREFIX=x86_64-pc-linux-gnu-

So the answer to my first question is apparently: no, it must be compiled twice.

The resulting ppcross386 seems to generate 32 bit code as ppc386 does on a native system but there the same problem appears regarding my second question - the resulting program can not be linked as in link.res absolute paths are used for crti.o and crtn.o. Here the messages:

Linking program
/usr/bin/ld: warning: link.res contains output sections; did you forget -T? /usr/bin/ld: skipping incompatible /usr/lib/crti.o when searching for /usr/lib/crti.o
/usr/bin/ld: skipping incompatible /usr/lib/crti.o when searching for /usr/lib/crti.o
/usr/bin/ld: cannot find /usr/lib/crti.o
vdim.pas(17,2) Error: Error while linking
vdim.pas(17,2) Fatal: There were 1 errors compiling module, stopping Fatal: Compilation aborted

I checked where this comes from and found, that in compiler/systems/t_linux.pas the function TLinkerLinux.WriteResponseFile generates the linker response file link.res where it gives absolute paths of [crtbegin.o,] crti.o, [crtend.o] and crtn.o. The bad thing is that these paths are determined by the FPC function librarysearchpath.FindFile and as this function does not honor the target architecture. It always returns /usr/lib/..... as the default architecture version of these files is located there. This of course fails when compiling 32 bit on a 64 bit Gentoo machine where the 32 bit versions are in /usr/lib32/..... and linking fails due to invalid files.

So again my second question is, why are absolute paths used for crtbegin.o, crti.o, crtend.o and crtn.o? When no paths are specified the linker automatically chooses the right version and everything it seems to be OK. Is there any need for absolute paths?

I made a patch for this
https://bugs.gentoo.org/attachment.cgi?id=317900&action=diff to specify the object files without paths. It would be nice to have a solution and Gentoo devs would like to know from upstream what the "right" solution for this linking issue is. You are welcome to add comments to the Gentoo bug report above.

Have a nice day,
Konstantin

Reply by email

Jonas Maebe
to Konstantin
Konstantin wrote 10 lines
The bad thing is that these paths are determined by the FPC function librarysearchpath.FindFile and as this function does not honor the target architecture. It always returns /usr/lib/..... as the default architecture version of these files is located there. This of course fails when compiling 32 bit on a 64 bit Gentoo machine where the 32 bit versions are in /usr/lib32/..... and linking fails due to invalid files. So again my second question is, why are absolute paths used for crtbegin.o, crti.o, crtend.o and crtn.o?

Because based on the information provided to the compiler via command line parameters and the knowledge hardcoded into the compiler, those are the files that should be linked. Specifying their explicit path ensures that the linker will link the object files that compiler wants it to link, rather than some random files with the same names that the linker find somewhere else. Those files may be the wrong ones in your particular scenario, but most cases this strategy will prevent the exact opposite problem.

What you can do is add something like this to the default configuration file on 64 bit hosts:

#ifdef cpui386
#ifdef linux
# prevent searching the default library directories -Xd
# add the 32 bit-specific library directories
-Fl/usr/lib32
-Fl/other/32bit/directory
#endif
#endif

Patching the compiler source should not be necessary.

Jonas

Reply by email

Sergei Gorelkin
to Jonas Maebe

17.07.2012 2:51, Jonas Maebe пишет:

Show quoted text 30 lines
On 17 Jul 2012, at 00:36, Konstantin wrote: > The bad thing is > that these paths are determined by the FPC function > librarysearchpath.FindFile and as this function does not honor the > target architecture. It always returns /usr/lib/..... as the default > architecture version of these files is located there. This of course > fails when compiling 32 bit on a 64 bit Gentoo machine where the 32 bit > versions are in /usr/lib32/..... and linking fails due to invalid files. > > So again my second question is, why are absolute paths used for > crtbegin.o, crti.o, crtend.o and crtn.o? Because based on the information provided to the compiler via command line parameters and the knowledge hardcoded into the compiler, those are the files that should be linked. Specifying their explicit path ensures that the linker will link the object files that compiler wants it to link, rather than some random files with the same names that the linker find somewhere else. Those files may be the wrong ones in your particular scenario, but most cases this strategy will prevent the exact opposite problem. What you can do is add something like this to the default configuration file on 64 bit hosts: #ifdef cpui386 #ifdef linux # prevent searching the default library directories -Xd # add the 32 bit-specific library directories -Fl/usr/lib32 -Fl/other/32bit/directory #endif #endif Patching the compiler source should not be necessary.

I'm afraid this isn't entirely correct. The problems arise when using "-n" in the command line (one example is compiling fpmake for packages directory). In this case any paths from fpc.cfg are ignored. Moreover, compiler silently ignores the absense of crti.o and company. The resulting executable manages to work somehow, but it is mostly by chance.

Regards,
Sergei

Reply by email

Jonas Maebe
to Sergei Gorelkin
Sergei Gorelkin wrote
I'm afraid this isn't entirely correct. The problems arise when using "-n" in the command line (one example is compiling fpmake for packages directory). In this case any paths from fpc.cfg are ignored. Moreover, compiler silently ignores the absense of crti.o and company. The resulting executable manages to work somehow, but it is mostly by chance.

Then you'd have to pass extra CROSSOPT parameters to the compiler when building (it's not that uncommon having to specify extra parameters when building a cross-distribution). Unless of course there is a specification somewhere in the LSB (or whatever it's called today) that specifies the default locations of cross-libraries on multi-arch systems, but we can keep adding code to the compiler to cater to all possible distributions.

Jonas

Reply by email

Sergei Gorelkin
to Jonas Maebe
Jonas Maebe wrote 7 lines
On 17 Jul 2012, at 07:15, Sergei Gorelkin wrote: > I'm afraid this isn't entirely correct. The problems arise when using "-n" in the command line (one example is compiling fpmake for packages directory). In this case any paths from fpc.cfg are ignored. Moreover, compiler silently ignores the absense of crti.o and company. The resulting executable manages to work somehow, but it is mostly by chance. Then you'd have to pass extra CROSSOPT parameters to the compiler when building (it's not that uncommon having to specify extra parameters when building a cross-distribution). Unless of course there is a specification somewhere in the LSB (or whatever it's called today) that specifies the default locations of cross-libraries on multi-arch systems, but we can keep adding code to the compiler to cater to all possible distributions.

I'm facing this problem even when not cross-compiling (and it is in fact unrelated to cross-compiling). The issue is that the path to crti.o is known only to gcc, if it is installed in system. It isn't related to other libraries' path and it isn't known to ld. This path is queried by fpc install script and written to fpc.cfg. Later it can become invalid if gcc version is upgraded without reinstalling fpc (this is likely what happens at our test machines, see tests/packages/webtbs/tw14265 failing with "cannot find -lgcc").

I was thinking about quering this path at every ld invocation by running `gcc -print-libgcc-file-name`, likewise it can be queried for cross-compiling with `gcc -m32 -print-libgcc-file-name`. Hopefully doing so won't slow down much. And probably fpc shouldn't keep silence if it needs gcc files to link and cannot find them.

Anyway, since currenly I'm working on ELF linker (reached the state when packages directory builds :-), I'll almost inevitably collect all linking-related issues

Regards,
Sergei

Reply by email