Hello,

I already dug through the archives but found nothing really related to a weird problem I have with a FPC shared lib which is used by a C shared lib which in turn is a plugin to a C main program.

The net effect is that the FPC library crashes (apparently when trying to allocate a FPC object) when just used as-is, but works wonderfully if I put a Write(' ') into the FPC library's begin end.

I'm not familiar with FPC runtime internals, so I can only guess that this Write (a WriteLn works as well, as long as it goes to stdout) triggers some initialisation code of the FPC runtime that is otherwise not called. I tried calling System.SysInitStdIO instead, hoping it would maybe initialize the same thing a Write() does, but it did not work, as other blind flight attempts to find a better fix than this Write() didn't either.

I guess I really need to understand how the FPC runtime expects to be initialized when located in a shared lib, but I couln't find any hint so far in the FPC docs nor in the list archives.

The FPC shared library is implicitly loaded as a dependency of the C shared lib plugin, which in turn is loaded by dlopen() from the C main program. I run the program on Mac OS X and Linux (but not on Windows).

I can modify the C shared lib plugin and the FPC shared lib, but not the main program (so it's out of question to have the FPC runtime linked into the main program).

I'd really be glad if someone has any pointers or ideas how to correctly fix this problem. I do understand shared lib technical details and problems in general, but am a FPC newbie and have no idea what the specifics regarding the FPC runtime might be.

Thanks for any help in advance!

Lukas

()

Lukas Zeller wrote

I guess I really need to understand how the FPC runtime expects to be initialized when located in a shared lib, but I couln't find any hint so far in the FPC docs nor in the list archives.

Which FPC version are you using? I fixed shared library initialisation/ finalisation for Mac OS X and Linux in FPC 2.2.4. The only things you still have to watch out for are
a) don't pass ansistrings/widestrings/other reference counted types from one FPC shared library to another (they use different memory managers, so you'll get crashes when one tries to free memory allocated by another), or alternatively make sure that all libraries use the "cmem" memory manager
b) don't use these libraries in multi-threaded applications. It is possible to get them to work, but hard. See below for a text I posted last year to the mac-pascal list. It should still apply though. It was also about applications rather than libraries, but simply put the initialisation code in the initialisation section of your library rather than in the "begin..end." block of the main program.

Jonas

***
To properly initialise the threading system (only has to happen once at the start of your program), it's probably the easiest to create and then immediately destroy a dummy thread. The simplest way to do this:

uses
   cthreads,classes;
begin
   tthread.create(false).waitfor;
   <rest of your code>
end.

So, to recapitulate:
* add the above code at the start of your program (it will set ismultithread to true, amongst other things)
* define this helper function:

procedure CleanupThread;
{$if defined(VER2_2_0) or defined(VER2_2_2)}
var
   tm: tthreadmanager;
begin
   getthreadmanager(tm);
   tm.ReleaseThreadVars;
end;
{$else}
begin
   DoneThread;
end;
{$endif}

* and make your thread functions like this:

function threadfunc(parameter: pointer): longint; cdecl; { change to
the signature expected by the Multiprocessing Services, but don't
forget the "cdecl;"! }
begin
InitThread(stacksize);
RealThreadFunc(..);
CleanupThread;
end;
***

Hello Jonas,

thanks for responding so quickly!

I am using FPC 2.2.4 on Mac OS X and the problem happens there.

The linux version runs on an external server, I'll ask the people running it there what FPC version they have and if the program really crashes without the Write(' ').

Yes, I read about this in the archives. This can't be the problem as there are only plain C strings (PChar) passed between the C part and the FPC shared lib.

The application is single-threaded. But thanks for the information about thread init altough it can't be the problem here.

So it must be something more profane.

And in fact it was - while re-testing everything for writing this email, by accident I found the following:

The Write in the library initialisation is required if any other part of the library wants to write to stdout later as part of the normally called library functions. As soon as there is no single Write or WriteLn called anywhere in the library, it works.

So for practical purposes of our project, the problem is solved (we don't want any stdout from the library anyway, the ones that caused the crash were quick hacks inserted for debugging).

For technical curiosity however I still wonder what interals of the runtime a Write or WriteLn in the library init code initializes correctly while the same Write called later in a normal library function makes the program crash.

More precisely, I think in the later call to Write it still tries to initialize something, but then has side effects on other data structures in the runtime, which for example make memory allocations crash - note that the Write/WriteLn themselves WORK, i.e. show on the console, but make the program crash shortly afterwards (as said, my observation was during allocation of a TObject, although the debugging situation was a bit difficult so I could be mistaken here).

Best Regards,

Lukas

Lukas Zeller wrote

> Which FPC version are you using? I fixed shared library > initialisation/finalisation for Mac OS X and Linux in FPC 2.2.4. I am using FPC 2.2.4 on Mac OS X and the problem happens there.

On which version of Mac OS X?

That is strange.

The RTL initialisation code is run before any code in the initialisation section is executed.

If you can create a test that reproduces the problem, please submit a bug report.

Jonas

Hi Jonas,

Jonas Maebe wrote

> I am using FPC 2.2.4 on Mac OS X and the problem happens there. On which version of Mac OS X?

Now Snow Leopard 10.6.1, but we have a parallel installation on Leopard 10.5.latest that also shows the problem.

Ok. I assume the RTL initialisation and the library initialisation block are BOTH run as part of the system level shared lib initialisation (DLL main entry point), in that order, correct?

I could. Actually it also happens without the in-between C shlib. A C main with a FPC shlib is sufficient.

I have a 10 lines sample program that shows the effect nicely. As a FPC community newbie - where should I post the sample and the bug report to? Not to this list, I assume...

Lukas

Hi Jonas,

A quick followup:

In the meantime I tested the supersimple test program that reliable crashes on 10.6 also on a 10.5 installation.
Now here it behaves correctly (?!)

HOWEVER, with the original, big program the problem also shows on 10.5. As I discovered another side effect (garbled function result, see readme.txt below) on 10.6, I assume there IS something within FPC that can cause internal memory damage - and it's a matter of luck if that leads to a full crash or not.

As the examples are so small, I post them here for further discussion:

--- hellolib.pas
--------------------------------------------------------------------------

library hellolib;
function ua:integer; cdecl;
begin
     writeln('using standard output from pascal lib');
     ua := 42;
end;
exports ua;
begin
     writeln('Comment out this, and the writeln above in function ua
will crash');
end.

--- hello.c
-------------------------------------------------------------------------------

#include <stdio.h>
extern int ua();
int main()
{
     printf ("hello world\n");
     printf ("universal answer = %d\n", ua());
     return 0;
}

--- readme.txt
----------------------------------------------------------------------------

Example program to show that when using Stdout (Write, WriteLn) from within a FPC
shared library which is called from a plain C main program, the program crashes
(exit code 103), UNLESS a Write/WriteLn of at least one character is done
in the FPC library's initialisation block (begin … end.)

The issue was directly reproducible on Mac OS 10.6 with this small test program,
but not on 10.5. We however had the problem on 10.5 as well with our real (much bigger)
program.
Looks like memory corruption of some sort.

Compile the example as is:
--------------------------

# - make the lib (no_dead_strip_inits_and_terms to make it run on Snow Leopard)
fpc -CD hellolib.pas -k"-no_dead_strip_inits_and_terms"

# - make the main
gcc -arch i386 -L`pwd` -lhellolib hello.c -o hello

Run the example with a WriteLn() in the library init code ---------------------------------------------------------

./hello

# Output should be:
Comment out this, and the writeln above in function ua will crash hello world
using standard output from pascal lib
universal answer = 42

# Output in Snow Leopard is (weird function result, stdout from the library is not visible at all)
hello world
universal answer = 65578

Run the example with COMMENTED OUT WriteLn() (i.e. EMPTY library init code)
---------------------------------------------------------------------------

./hello

# Program does not output anything, simply exits

echo $?
103

-------------------------------------------------------------------------------------------

Lukas

Lukas Zeller wrote

Now Snow Leopard 10.6.1, but we have a parallel installation on Leopard 10.5.latest that also shows the problem.

Note that on Mac OS X 10.6.x (or rather with Xcode 3.2, which ships with Mac OS X 10.6) there is a known linker bug that messes up FPC libraries. See http://bugs.freepascal.org/view.php?id=14471 for the details.

Ok. I assume the RTL initialisation and the library initialisation block are BOTH run as part of the system level shared lib initialisation (DLL main entry point), in that order, correct?

It depends on the OS:
* on Mac OS X, the shared library entry point is called, which then calls the RTL initialisation routine before executing any code you may have added
* on Linux, some assembler RTL code is called that sets up various things, which in turn calls the library entry point and things proceed as on Mac OS X.

The bug reporting form is at http://bugs.freepascal.org. You can post things to the mailing list, but if not immediately acted upon they tend to be forgotten. So using the website is always better.

Jonas

I assume that you tested under 10.6, and the strange results there are presumably due to the broken init code reordering by the linker. When I use a compiler that passes -no_order_inits to the linker (note that you cannot do this with 2.2.4 using -k-no_order_inits for the reasons explained in the bug report I referred to earlier), it works fine for me:

* with the writeln in the library init code:
$ ./hello
Comment out this, and the writeln above in function ua will crash hello world
using standard output from pascal lib
universal answer = 42

* without the writeln in the init code:
$ ./hello
hello world
using standard output from pascal lib
universal answer = 42

Note that the problem is the linking under 10.6, not the running. So a library compiled/linked under 10.6 will also fail on 10.5 (even if compiled with -k-mmacosx_version_min=10.5 and against the 10.5 SDK), while a library compiled/linked under 10.5 will actually work fine under 10.6.

Jonas

Hi Jonas,

Thanks for testing that and the explanation of Apple linker issues. Makes sense now - it's no problem for us to build on 10.5 for now until a future version of FPC can work around the problem or the Apple linker gets fixed. It looks like the former is the case :-)

The only thing not explained yet are the crashes we had on 10.5 (built there, run there) with the real program, which *looked* like it was the same problem (crash without the Write(), no crash with Write() in place). But given the facts that the sample program is working correctly on 10.5 it might have been something else. I'll have that checked again to make sure, and will report if I can reproduce it on 10.5 somehow.

Thanks again for your help!

Best Regards,

Lukas

Lukas Zeller wrote

Thanks for testing that and the explanation of Apple linker issues. Makes sense now - it's no problem for us to build on 10.5 for now until a future version of FPC can work around the problem or the Apple linker gets fixed. It looks like the former is the case :-)

I also have good hopes that the linker bug will be fixed by Apple in the next version of the Xcode tools. It's unfortunate that it costs $500/year to get access to beta versions of major new OS releases, because otherwise I could properly test earlier on and then the bugs could be fixed before the OS is released to the public (when 10.5 was initially released, there were also several problems). Beta versions of minor Xcode updates are fortunately available to everyone nowadays.

Thanks.

Thanks again for your help!

You're welcome. I've also updated the message on the website about the problems with 10.6. I had forgotten to change it once the real cause of the problems was discovered.

Jonas