I'm currently tinkering with shared libraries, using cmem, mainly on Linux. In one case the main-program code is multithreaded, but so far I'm not moving data between threads inside the library.

Am I correct in believing that unit-level variables in a shared library might end up being shared between multiple instances of the library, or are they guaranteed to be distinct?

If my belief is correct, I presume that I could get around this by using threadvars. But in this case, how would I best implement a variable that I wanted to be common across related threads (e.g. to track the number of a particular object being allocated/deallocated), but distinct across program invocations (i.e. two programs using the same shared library wouldn't clash)?

The discussion of libraries in ch16 of the Language reference guide is rather quiet on the semantics.

Am 26.06.2012 14:02 schrieb "Mark Morgan Lloyd" <

: I'm currently tinkering with shared libraries, using cmem, mainly on

Linux. In one case the main-program code is multithreaded, but so far I'm not moving data between threads inside the library.

Am I correct in believing that unit-level variables in a shared library

might end up being shared between multiple instances of the library, or are they guaranteed to be distinct?

If my belief is correct, I presume that I could get around this by using

threadvars. But in this case, how would I best implement a variable that I wanted to be common across related threads (e.g. to track the number of a particular object being allocated/deallocated), but distinct across program invocations (i.e. two programs using the same shared library wouldn't clash)?

A programs memory always belongs to itself (exceptions through mmap or kernel related thing excluded) and libraries are loaded into the program's memory. Their code sections might only exist once if the OS detects that it had already loaded the library somewhere, but it's data sections and also the memory it allocates during its lifetime belong to the program's memory.

This means for you: normal global variables in your library are not shared with other "instances" of this library (note: if a program loads library A and B and library A also loads B then there is only one "instance" of the library and its data). They are shared by all threads that use this library though. Having variables shared between different "instances" of a library is a different topic altogether.

Regards,
Sven

Thanks Sven. So to summarise: code might be shared, but this generally won't be relevant since it should be read-only. Data- local or global- won't be shared.

OK, so to emphasise that: library B's data will be common, irrespective of the location of the calling routine. Does this apply if the load is being done at runtime, i.e. the program is using dl (or whatever) to load A and B, and A also uses dl to load B?

Am 26.06.2012 17:15 schrieb "Mark Morgan Lloyd" <

Linux. In one case the main-program code is multithreaded, but so far I'm not moving data between threads inside the library.

>> >> Am I correct in believing that unit-level variables in a shared library

might end up being shared between multiple instances of the library, or are they guaranteed to be distinct?

>> >> If my belief is correct, I presume that I could get around this by

using threadvars. But in this case, how would I best implement a variable that I wanted to be common across related threads (e.g. to track the number of a particular object being allocated/deallocated), but distinct across program invocations (i.e. two programs using the same shared library wouldn't clash)?

>> >> The discussion of libraries in ch16 of the Language reference guide is

rather quiet on the semantics.

won't be relevant since it should be read-only. Data- local or global- won't be shared.

Correct. Shared code sections shouldn't matter normally.

of the location of the calling routine. Does this apply if the load is being done at runtime, i.e. the program is using dl (or whatever) to load A and B, and A also uses dl to load B?

It does not matter whether the library is loaded at program start or at runtime. It's always the same mechanism with the exception that in the first case your main function has not yet been called (which normally doesn't matter either).

Regards,
Sven

Apologies for my lousy message threading, but some are vanishing in the spamtrap.

Just being cautious :-) One final question if I may: noting Ludo's example elsewhere:

Can such an alternative entry point in the main unit be called by a shared library, i.e. either resolved at load time or with the main binary reopened like a library? Or is the only way to pass a main-program entry point to a shared library by using it as a parameter to a procedure?

I've used such things in a Windows-style .exe so a loader/binder would know how to generate absolute code for an embedded system, but have never tried exploiting it with a "real" OS.

Am 26.06.2012 20:09 schrieb "Mark Morgan Lloyd" <

shared library, i.e. either resolved at load time or with the main binary reopened like a library? Or is the only way to pass a main-program entry point to a shared library by using it as a parameter to a procedure?

I've used such things in a Windows-style .exe so a loader/binder would

know how to generate absolute code for an embedded system, but have never tried exploiting it with a "real" OS.

I can't tell you for *nix systems as I'm not experienced enough regarding their dynamic linkers, but I can tell you for Windows systems: If you export functions from a executable (not a DLL) you can get access to these methods from within a library the same way you'd do with a DLL (though I don't know what happens if you load a unrelated executable). As your executable is already loaded no further "instance" of it will be created.
There is also the possibility that you link to the executable at link time (in Pascal using the "procedure bla; external 'your.exe' name 'bla'" mechanism). I don't know whether you can build a circle here (in the sense of having the exe depend on that DLL as well) as I have not tested that that mechanism yet. Also there is only one case that I definitely know of that uses this link time variant: drivers on NT based systems (which are simply DLLs) link to ntoskrnl.exe and the kernel loads them dynamically. You can see this if you look into the ndk.pas unit of my Native NT port where the constant NTDLL is set to 'ntoskrnl.exe' instead of 'ntdll.dll' if it is compiled with KMODE defined.

Regards,
Sven

Superficial testing suggests that if the code in a .so (Linux, i386) tries to call an "extra" entry point exported from the main program, that a run-time (libdl) load of the library will fail. I'll keep on tinkering with this, I've not yet retrieved the exact error message and I'm interested in seeing what happens if the .so itself tries to use libdl to reopen the main program.

Short answer: works on Linux, but at run- (not load-time) only.

Longer answer: if a .so has an explicit dependency on the main program, i.e. a procedure marked extern, an attempt to load it using dlopen(...RTLDLAZY) will fail (return zero as a handle). The error message refers to an entry point of the form library_proc_types rather than program_proc_types, and I've not managed to override that using extern or extern/name in a way that makes everything happy.

Using LoadLibrary() -> dlopen() inside the .so, passing an *empty* string representing the main program, appears to allow exported entry points to be accessed (i.e. a shared library can access a procedure in the main program in exactly the same way that a main program can access a procedure in a shared library).

One thing that would be useful would be if DynLibs had a "get last error" function. I've implemented this myself for Linux and Windows and find it indispensable.