Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2020-11

Issue with Compiling FPC and MacOS Big Sur

9 messages · 3 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Michael Ring

I recently upgraded to Big Sur on two devices and tried to compile trunk based on official 3.2.0 compiler freshly downloaded from sourceforge but compilation fails:

make all

....

/usr/local/bin/ppcx64 -Ur -Xs -O2 -n -Fux86_64 -Fusystems -Fu/Users/ring/devel/fpc/rtl/units/x86_64-darwin -Fix86_64 -FEx86_64/bin/x86_64-darwin -FUx86_64/units/x86_64-darwin -dRELEASE    -dx86_64 -dGDB -dBROWSERLOG -Fux86 -Fix86 pp.pas
ld: library not found for -lc
An error occurred while linking
pp.pas(280,36) Error: Error while linking
pp.pas(280,36) Fatal: There were 1 errors compiling module, stopping

Same thing happens when trying to compile from within fpcupdeluxe, Developer Tools are installed and up to date.

This may be related to a change in MacOS Big Sur, found a few comments on the possible issue:

https://github.com/dotnet/msbuild/issues/5454

there it says:

This will stop working in the next version of macOS:

New in macOS Big Sur 11 beta, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to |dlopen()| the path, which will correctly check for the library in the cache. (62986286)

In Linker script -lc is last entry:

x86_64-darwin $ cat link.res
-arch
x86_64
-macosx_version_min
10.8
-L/usr/lib/
-L./x86_64/
-L./systems/
-L/Users/ring/devel/fpc/rtl/units/x86_64-darwin/
-L./x86/
-L/usr/local/bin/
-lc

Reply by email

Michael Ring
to Michael Ring

Problem is more simple, path to SDK tools is missing in linker script:

-arch
x86_64
-macosx_version_min
10.8
-L/usr/lib/
-L./x86_64/
-L./systems/
-L/Users/ring/devel/fpc/rtl/units/x86_64-darwin/
-L./x86/
-L/usr/local/bin/
*-L/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/lib/* -lc

A permanent solution could be to add a call to xcrun to the link process to make sure (hopefully once and for all) that the SDK path is set in the linker-script:

compiler $ xcrun --show-sdk-path (master)fpc
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

compiler $ ls -l
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk (master)fpc lrwxr-xr-x  1 root  wheel  15 10 Dez  2019
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -> MacOSX10.15.sdk

shall I try to create a patch?

Michael

Michael Ring wrote 63 lines
I recently upgraded to Big Sur on two devices and tried to compile trunk based on official 3.2.0 compiler freshly downloaded from sourceforge but compilation fails: make all .... /usr/local/bin/ppcx64 -Ur -Xs -O2 -n -Fux86_64 -Fusystems -Fu/Users/ring/devel/fpc/rtl/units/x86_64-darwin -Fix86_64 -FEx86_64/bin/x86_64-darwin -FUx86_64/units/x86_64-darwin -dRELEASE    -dx86_64 -dGDB -dBROWSERLOG -Fux86 -Fix86 pp.pas ld: library not found for -lc An error occurred while linking pp.pas(280,36) Error: Error while linking pp.pas(280,36) Fatal: There were 1 errors compiling module, stopping Same thing happens when trying to compile from within fpcupdeluxe, Developer Tools are installed and up to date. This may be related to a change in MacOS Big Sur, found a few comments on the possible issue: https://github.com/dotnet/msbuild/issues/5454 there it says: This will stop working in the next version of macOS: New in macOS Big Sur 11 beta, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to |dlopen()| the path, which will correctly check for the library in the cache. (62986286) In Linker script -lc is last entry: x86_64-darwin $ cat link.res -arch x86_64 -macosx_version_min 10.8 -L/usr/lib/ -L./x86_64/ -L./systems/ -L/Users/ring/devel/fpc/rtl/units/x86_64-darwin/ -L./x86/ -L/usr/local/bin/ -lc _______________________________________________ fpc-devel maillist - https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Reply by email

Jonas Maebe
to Michael Ring: I recently upgraded to Big Sur on two devices and tried to compile trunk based on…
Michael Ring wrote
I recently upgraded to Big Sur on two devices and tried to compile trunk based on official 3.2.0 compiler freshly downloaded from sourceforge but compilation fails:

That looks like you did not install the Xcode command line tools *before* installing FPC. The installer should warn you about this if it detects that is the case. Previous FPC versions refused to install at all in this case, but I changed it into a warning because Apple kept changing things that broke this detection. It should function correctly on macOS 11.0/Big Sur though.

Jonas

Reply by email

Michael Ring
to Jonas Maebe

Hi Jonas, thank you for your answer!

I cannot say if I did see a warning or not when installing fpc-package but I did not reinstall commandline tools before or after installing the 3.2.0 compiler. They have been on all of my Macs for years.

What are you doing in the installer that creates that dependency?

My guess would be that you rewrite /etc/fpc.cfg (I think I saw on one of my Macs that the timestamp of that file changed), but this will not help for compiling the compiler as the compiler is beeing compiled with the flag -n which effectively turns off reading /etc/fpc.cfg.

I made compilation work by adding an OPT parameter to the build:

make clean all OPT="-Fl$(xcrun --show-sdk-path)/usr/lib -Ff$(xcrun --show-sdk-path)/System/Library/Frameworks"

or, without using $():

make clean all
OPT="-Fl/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -Ff/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks"

So I guess the best solution would be to change the compiler Makefiles to add the Fl/Ff parameter to the compile when on a Mac. I am currently investigating this, what do you think, could that solve the issue once and for all? (We have already had this kind of conversation several times over the last years, usually on crt1.o

Fixing the Makefile will make this work for already existing compilers, I'd still propose to add this path from within fpc, when the linker commands are generated.

Or am I missing something ?

I am updating now my older Mac that is still on Catalina to latest XCode, will be interesting to see how this computer behaves after the update, with older xcode installed I can do 'make clean all' without any issue...

I can also run my build command there, on this machine XCode is installed so the Path to CommandlineTools points to some other place, but things still seems to work fine.

Michael

Jonas Maebe wrote 14 lines
On 2020-11-14 11:03, Michael Ring via fpc-devel wrote: > I recently upgraded to Big Sur on two devices and tried to compile > trunk based on official 3.2.0 compiler freshly downloaded from > sourceforge but compilation fails: That looks like you did not install the Xcode command line tools *before* installing FPC. The installer should warn you about this if it detects that is the case. Previous FPC versions refused to install at all in this case, but I changed it into a warning because Apple kept changing things that broke this detection. It should function correctly on macOS 11.0/Big Sur though. Jonas

Reply by email

Jonas Maebe
to Michael Ring
Michael Ring wrote 5 lines
Hi Jonas, thank you for your answer! I cannot say if I did see a warning or not when installing fpc-package but I did not reinstall commandline tools before or after installing the 3.2.0 compiler. They have been on all of my Macs for years.

They usually get removed when you upgrade to a new macOS version.

What are you doing in the installer that creates that dependency? My guess would be that you rewrite /etc/fpc.cfg (I think I saw on one of

That is correct.

my Macs that the timestamp of that file changed), but this will not help for compiling the compiler as the compiler is beeing compiled with the flag -n which effectively turns off reading /etc/fpc.cfg.

Indeed, I glossed over the fact that you were rebuilding FPC.

make clean all OPT="-Fl/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -Ff/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks"

You should use the -XR parameter
(-XR/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk) instead of -Fl/Ff in this situation.

Show quoted text 13 lines
So I guess the best solution would be to change the compiler Makefiles to add the Fl/Ff parameter to the compile when on a Mac. I am currently investigating this, what do you think, could that solve the issue once and for all? (We have already had this kind of conversation several times over the last years, usually on crt1.o Fixing the Makefile will make this work for already existing compilers, I'd still propose to add this path from within fpc, when the linker commands are generated. Or am I missing something ? From the top of my head:

1) it won't work when cross-compiling on another platform, because xcrun won't exist there. So it would create (invisible) compiler behaviour differences depending on whether or not you're cross-compiling.
2) if someone compiles on macOS itself using a non-default SDK (e.g. to build i386 binaries on 10.15+, or PPC/PPC64 binaries on 10.6+), this should not be done. I could detect this by looking at whether or not a -XR parameter is specified, but if someone uses -Fl/Ff instead, I have no idea what would happen when mixing SDKs like that (and that, again, could lead to very hard to debug problems).

Jonas

Reply by email

Karoly Balogh (Charlie/SGR
to Jonas Maebe

Hi,

Jonas Maebe wrote 13 lines
> my Macs that the timestamp of that file changed), but this will not help > for compiling the compiler as the compiler is beeing compiled with the > flag -n which effectively turns off reading /etc/fpc.cfg. Indeed, I glossed over the fact that you were rebuilding FPC. > make clean all > OPT="-Fl/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib > -Ff/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks" You should use the -XR parameter (-XR/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk) instead of -Fl/Ff in this situation.

Yes, I already had to dothis in my build scripts on Mojave. I defined an "FIXMOJAVE" environment var, which contains that -XR/Library/... path you wrote above, and in all my build scripts I just do:

make OS_TARGET=foo CPU_TARGET=bar OPT="-other -args $FIXMOJAVE" CROSSOPT=...

It's not the most convenient thing, but it works.

Charlie

Reply by email

Jonas Maebe
to Karoly Balogh (Charlie/SGR
Show quoted text 6 lines
Yes, I already had to dothis in my build scripts on Mojave. I defined an "FIXMOJAVE" environment var, which contains that -XR/Library/... path you wrote above, and in all my build scripts I just do: make OS_TARGET=foo CPU_TARGET=bar OPT="-other -args $FIXMOJAVE" CROSSOPT=...

I also have macOS 10.14 (Mojave) and I don't need to specify any special options when building FPC. All libraries and object files are still available under /usr there, so I have no idea why it would be necessary in your case.

Jonas

Reply by email

Karoly Balogh (Charlie/SGR
to Jonas Maebe

Hi,

Jonas Maebe wrote 12 lines
On 15/11/2020 15:19, Karoly Balogh (Charlie/SGR) via fpc-devel wrote: > > Yes, I already had to dothis in my build scripts on Mojave. I defined an > "FIXMOJAVE" environment var, which contains that -XR/Library/... path you > wrote above, and in all my build scripts I just do: > > make OS_TARGET=foo CPU_TARGET=bar OPT="-other -args $FIXMOJAVE" CROSSOPT=... I also have macOS 10.14 (Mojave) and I don't need to specify any special options when building FPC. All libraries and object files are still available under /usr there, so I have no idea why it would be necessary in your case.

Me neither, but it's been like that ever since I upgrade to Mojave, that these libs and objects are no longer in /usr/lib on my system. I used Sierra on the same machine before, and that worked, then I updated to Mojave and some then-recent XCode/command line utilities in one go (without reinstall), and then the files were gone, I had to add this extra argument to my FPC builds, otherwise things just didn't work.

But as I don't really develop native software on my Mac, only use to crosscompile various versions of FPC, I never investigated further, it was goon enough for me.

Charlie

Reply by email

Jonas Maebe
to Karoly Balogh (Charlie/SGR
Show quoted text 5 lines
e neither, but it's been like that ever since I upgrade to Mojave, that these libs and objects are no longer in /usr/lib on my system. I used Sierra on the same machine before, and that worked, then I updated to Mojave and some then-recent XCode/command line utilities in one go (without reinstall), and then the files were gone

This suggests you did not in fact reinstall the command line utilities. That installer is what adds those files on that version of the OS.

Jonas

Reply by email