Hello.

I have problem to access methods from a C library.

For FreeBSD, the methods cannot be accessed (but ok for Linux and Windows).

In FreeBSD, LoadLibrary(aac.so) is working but all GetProcAdress() for each method fail.

I already ask help in FreeBSD forum :
https://forums.freebsd.org/threads/55305/

But without luck.

Could it be that something is wrong with mangle-name of exported methods of the library in FreeBSD ?

Does anybody have a idea why GetProcAdress() fail in FreeBSD (but ok for Linux and Windows) ?

Or must the library-code be compiled with some special instructions for fpc ?

Every comment or light is highly welcome (tips, must-know, etc,...).

PS: A solution is highly expected because there are other libraries, with same problems in FreeBSD, that wait to be used by fpc-FreeBSD.

Many thanks.

Fre;D

-----
Many thanks ;-)

Windows).

In FreeBSD, LoadLibrary(aac.so) is working but all GetProcAdress() for

each

of

the library in FreeBSD ?

You could check with objdump or nm what the exported symbols look like in the library. There are some systems that export C functions with a leading '_' for example.

Regards,
Sven

You could check with objdump or nm what the exported symbols look like in

the library. There are some systems that export > C functions with a leading '_' for example.

Hello Sven and thanks for answer.

Here result of nm on FreeBSD 64.

<http://free-pascal-general.1045716.n5.nabble.com/file/n5724532/aac_freebsd.jpg>

But it seems that the exported functions have the same names as for Linux or Windows (like "mp4ff_open_read", "mp4ff_close", ...)

Or do you see a "special" exported name ?

Many thanks.

-----
Many thanks ;-)

It would be helpful to see the code fragments in question. The declaration of the external functions if any and the code calling LoadLibrary() and GetProcAddress().

it would be helpful to see the code fragments in question. The declaration of the external functions if any and the code calling LoadLibrary() and GetProcAddress().

Hello Mark and thanks for help.

Here declaration to load-getprocess:
----------------------------------------------------- var

  mp4ff_open_read    : function(f : p_mp4ff_callback_t) : mp4ff_t; cdecl;
  libpointer : {$IFDEF MSWINDOWS}longword{$ELSE}{$IFDEF

CPU32}longword{$ELSE}PtrInt{$ENDIF}{$ENDIF};
...

libpointer := DynLibs.SafeLoadLibrary(PChar(mp4ff)); -> OK for FreeBSD64
too;-)

Pointer(mp4ff_open_read) :=
GetProcAddress(hMp4ff, pchar('mp4ff_open_read')); -> Here always = nil for FreeBSD 64 ;-( (but ok for Linux + Windows) ...
-----

Thanks.

Fre;D

PS: I have the same problem for other libraries in FreeBSD 64 -> GetProcAddress() fail.
PS2: It appends for some libraries in FreeBSD64, not all.

-----
Many thanks ;-)

When following the procedure calls I find:

<dynlibs.pas>
Function GetProcAddress(Lib : TlibHandle; const ProcName : AnsiString) :
Pointer;
begin
  Result:=GetProcedureAddress(Lib,Procname);
end;
</dynlibs.pas>

Next is:

<dynlibs.inc>
Function GetProcedureAddress(Lib : TLibHandle; const ProcName :
AnsiString) : Pointer;
begin
  Result:=dlsym(lib,pchar(ProcName));
end;
</dynlibs.inc>

Okay, man page says:

http://www.freebsd.org/cgi/man.cgi?query=dlsym&apropos=0&sektion=0&manpath=FreeBSD+10.2-RELEASE&arch=default&format=html

You could do some things, like trying to use the special handle RTLD_NEXT, or and much more important, call dlerror() after the failing dlsym(). It should tell _why_ the call failed.

PS: I have the same problem for other libraries in FreeBSD 64 -> GetProcAddress() fail. PS2: It appends for some libraries in FreeBSD64, not all.

Intresting. I know there are differences in the systems dlopen() and another similar variant installed by libtool, found in libbtdl regarding to my notes. Dunno if it matters. Libtools implementation does differentiate between OSs by itself (at compile time). Cold be a source of confusion...

When following the procedure calls I find:

...

Hello Mark.

Thanks for your light.

Huh, yes, IMO, there is something strange in dlsym/FreeBSD.

And asking something about C library/compiler is extremely sensible there on FreeBSD forum.
(FreeBSD needs 8 (!) different C compilers (clang, c++, CC, cc, cpp, ...) to compile himself).

OK, the solution is to wait for a solution.

Many thanks.

Fre;D

-----
Many thanks ;-)

Would you mind telling the result of dlerror()?

And asking something about C library/compiler is extremely sensible there on FreeBSD forum. (FreeBSD needs 8 (!) different C compilers (clang, c++, CC, cc, cpp, ...) to compile himself).

Not really. Since c++ is the same compiler as (g)cc, cpp is the C ppreprocessor an CC is mostly only used as a variable name in Makefiles and shell skripts set to the right compiler, that is clearly wrong.

There are in fact different C compilers which are all able to compile C or C++. There is actually the system compiler, which has been gcc and will be or is already clang. Some ports require newer versions of gcc, so sometimes there are two or three of them installed. But that's it, AFAIK.

Look:

cc -v

Using built-in specs.
Target: amd64-undermydesk-freebsd
Configured with: FreeBSD/amd64 system compiler
Thread model: posix
gcc version 4.2.1 20070831 patched [FreeBSD]

gcc -v

Using built-in specs.
Target: amd64-undermydesk-freebsd
Configured with: FreeBSD/amd64 system compiler
Thread model: posix
gcc version 4.2.1 20070831 patched [FreeBSD]

CC -v

Using built-in specs.
Target: amd64-undermydesk-freebsd
Configured with: FreeBSD/amd64 system compiler
Thread model: posix
gcc version 4.2.1 20070831 patched [FreeBSD]

c++ -v

Using built-in specs.
Target: amd64-undermydesk-freebsd
Configured with: FreeBSD/amd64 system compiler
Thread model: posix
gcc version 4.2.1 20070831 patched [FreeBSD]

cpp --version

cpp (GCC) 4.2.1 20070831 patched [FreeBSD]
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

All the same. ;)
Only that one differs:

clang -v

FreeBSD clang version 3.0 (tags/RELEASE_30/final 145349) 20111210 Target: x86_64-unknown-freebsd9.0
Thread model: posix

You can try looking at the manual pages, type in "man cc" and the other names.

Would you mind telling the result of dlerror()?

OK, I will follow your tips.
Write you later with the result.

There are in fact different C compilers which are all able to compile C

or C++.

Yes and nearly all that compilers are the same (why not use sym-link?)

-> "Why not use sym-link in place of copy 8x the same compiler (+- 8 megas) with different name " -> Dont ask this to FreeBSD forum -> It is sensible. (I already try)

-> "Why, in the script to create-compile the kernel of FreeBSD, different compilers are used (that are the same in fact).
Would it not be simpler to use only one compiler ? " -> Dont ask this to FreeBSD forum -> It is sensible. (I already try)

PS: I will become totally fan of FreeBSD when the C-war will end and when it will be easy to recompile the kernel (with only one compiler).

Many thanks for your excellent tips Marc.

Write you later.

Fre;D

-----
Many thanks ;-)

Hello Marc.

Huh, I have a dummy question...

How do you use dlopen(), dlsym() and dlerror() ?

On console, with ->

dlopen('/root/mylib.so')

Result = "Illegal command name..."

call dlerror() after the failing dlsym(). It should tell _why_ the call failed.

Re-huh..., it should be a great plus if a dlerror() was implemented in fpc too.

But maybe I am missing something, maybe dlopen(), dlsym() and dlerror() can be done by pascal code (and not via console, like I try).

Thanks.

Fre;D

-----
Many thanks ;-)

dlopen, dlsym and dlerror are *functions*, not console commands. Well, I never tried the latter, but it appears they are not on your system ;-)

Now, I don't know where exactly these functions are declared (in which unit, that is), but for debugging purposes, just add

    Function dlopen(filename: PChar; flags: cint): Pointer; cdecl; external;
    Function dlclose(handle: Pointer): cint; cdecl; external;
    Function dlsym(handle: Pointer; Name: PChar): Pointer; cdecl; external;
    Function dlerror: PChar; cdecl; external;

Somewhere in your code before you call them.

Ooops, thanks for answer but now I am completely loosed... ;-(

So those functions can be used in fpc code?
If so, must a library be loaded to access that functions (and what library)? Or are those functions defined in a fpc unit ?

I am in the dark.

Fre;D

-----
Many thanks ;-)

In our previous episode, Ewald said:

... or use unit dl.

I knew they were in some unit somewhere ;-) Thanks for the pointer!

See Marco's answer. So instead of using the above declarations, a simple `Uses dl;` should do the trick.

As to how to use them, now you do something like this:
    X:= GetProcedureAddress(...);
    Y:= GetProcedureAddress(...);
    Z:= GetProcedureAddress(...);
When one of these fails, a nil pointer is returned. dlerror will give
you more info on what failed where. So, if you know X returns nil for
some-yet-to-be-determined-reason, place the dlerror immediately after that:
    X:= GetProcedureAddress(...);
    WriteLn(dlerror);
    Y:= GetProcedureAddress(...);
    Z:= GetProcedureAddress(...);
Of course, you could choose to do something like this if you do not know
which call exactly fails:
    X:= GetProcedureAddress(...);
    If X = nil Then
        WriteLn('Z: ', dlerror);
    Y:= GetProcedureAddress(...);
    If Y = nil Then
        WriteLn('Y: ', dlerror);
    Z:= GetProcedureAddress(...);
    If Z = nil Then
        WriteLn('Z: ', dlerror);

It is really nothing more than a function call, quite similar to fpGetErrNo.

I am in the dark.

Being in the dark is positive, as one can always turn on the lights :-)

@ Marco and Marc thanks for help.

Sorry I do not have easy internet connection so I worked by my side.

I will try your tips.

By the way, here are my investigations:

____________________________________

Hello.

Ok, ok, understood and wow.

Added in code:

    Function dlopen(filename: PChar; flags: cint): Pointer; cdecl; external;
    Function dlclose(handle: Pointer): cint; cdecl; external;
    Function dlsym(handle: Pointer; Name: PChar): Pointer; cdecl; external;
    Function dlerror: PChar; cdecl; external;

and, magic, it works. ;-)

Now the result:

In code:

procedure loadmylib(lib : pchar);
var
ap1, ap2, ap3 : pointer;
'
begin
/// FreeBSD functions:
ap1 := dlopen(Pchar(lib), 0);
if ap1  nil then
writeln('dlopen() is OK') else
writeln('dlopen() is NOT OK') ;
ap2 := dlsym(ap1, Pchar('mp4ff_open_read');
if ap2  nil then
writeln('dlsym() is OK') else
writeln('dlsym() is NOT OK') ;
writeln(dlerror());
end;

---> result:
writeln('dlopen() is OK')
writeln('dlsym() is OK')
_ (no error)

--------------------------------------
Now with DynLibs:

procedure loadmylib(lib : pchar);
var
ap1: pointer;
hn : integer;
'
begin
/// with dynlibs
hn := dynlibs.loadlibrary(Pchar(lib));
if hn  0 then
writeln('loadlibrary() is OK') else
writeln('loadlibrary() is NOT OK') ;
ap1 := getprocedureaddress(hn, Pchar('mp4ff_open_read');
if ap1  nil then
writeln('getprocedureaddress() is OK') else
writeln('getprocedureaddress() is NOT OK') ;
end;

--> result:

loadlibrary() is OK
getprocedureaddress() is NOT OK

............

Conclusion: Maybe getprocedureaddress() has problems.

PS: Other tips are welcome.

Fre;D

-----
Many thanks ;-)

OOOps, in previos mail:
Thanks to Ewald, Marco and Marc.

Fre;D

-----
Many thanks ;-)

Just quick before they cut the signal...

Could dlsym(ap1, Pchar('mp4ff_open_read') be used in place of getprocedureaddress(hn, Pchar('mp4ff_open_read') ?

Because, if dlsym() works, maybe replacing all getprocedureaddress() with dlsym() will work?

And can loadlibrary() be used by dlsym() ( or dlsym() needs dlopen() ) ?

Thanks.

Fre;D

-----
Many thanks ;-)

Re-hello (and last for tonight).

Yep, yep, yep.

Replacing all GetProcedureAdress() with dlsym()

DOES THE TRICK ! ;-)

(but dlopen() is needed in place of Loadlibrary())

Excellent tip.

Many, many, many thanks for help.

(And, maybe, a check for DynLibs.GetProcedureAdress() is needed for FreeBSD).

And re-thanks for your bright light.

PS: This is a great victory who will solve (I hope) other library-related problems.
PS2: Are there limitations of using dlsym() vs GetProcedureAdress() ? PS3: If dlsym() is cross-platform too, should I update all my codes that use GetProcedureAdress() ?

Fre;D

-----
Many thanks ;-)

"fredvs" wrote

PS: This is a great victory who will solve (I hope) other library-related problems. PS2: Are there limitations of using dlsym() vs GetProcedureAdress() ? PS3: If dlsym() is cross-platform too, should I update all my codes that

use

GetProcedureAdress() ?

LoadLibrary and GetProcedureAddress internally use dlopen and dlsym respectively on Unix-like platforms. So it might be that either LoadLibrary or GetProcedureAddress calls its respective internal function incorrectly on FreeBSD and that needs to be fixed then.
And no, dlopen and dlsym are not cross platform (e.g. not on Windows) that's.why the LoadLibrary and GetProcedureAddress functions exist in the first place.

Regards,
Sven

Here you are using 0 for three mode argument. One difference I can see ist that uysing dynlibs.pas the mode is RTLD_LAZY like it should be, but that constant is defined as 1.

Does anyone now if this really makes a difference?

you can use

  Function GetLoadErrorStr: string;

here. Or simply dlerror() like in the other program, should work.

Huh, indeed, 0 is working in Linux but not in FreeBSD --> changed to 1 and all ok in FreeBSD.

Or simply dlerror() like in the other program, should work.

OK. here code + result of GetProcedureAddress(), maybe it could help for future fixes in fpc:

GetProcedureAddress(hn, pchar('mp4ff_open_read');
writeln(dlerror());

---> Result:

"Invalid shared object handle 0x2fb2800"

Many, many thanks to Marc, Marco, Ewald and Sven.

Fre;D

-----
Many thanks ;-)

Uhm, are you sure about this? I always thought GetProcedureAddress simply calls through to dlsym. If one works and the other doesn't, this needs some more investigation I think?

Also, in you last example, quoted here:

--------------------------------------
Now with DynLibs:

procedure loadmylib(lib : pchar);
var
ap1: pointer;
hn : integer;
'
begin
/// with dynlibs
hn := dynlibs.loadlibrary(Pchar(lib));
if hn  0 then
writeln('loadlibrary() is OK') else
writeln('loadlibrary() is NOT OK') ;
ap1 := getprocedureaddress(hn, Pchar('mp4ff_open_read');

****INDICATED POSITION****

if ap1  nil then
writeln('getprocedureaddress() is OK') else
writeln('getprocedureaddress() is NOT OK') ;
end;
--------------------------------------

Could you add a "writeln(dlerror());" ate the "****INDICATED POSITION****"?

Just curious about what is going on here...

PS: Why do you use "hn: Integer" instead of "hn: TLibHandle"? I don't know if it can be assumed that this handle will always be an integer (think, for example, pointer)? Just a thought.

So that's not the cause.

Doesn't tell much, I fear. I'll have to do same testing myself to find out.

Many, many thanks to Marc, Marco, Ewald and Sven.

You're welcome, thanks for reporting and testing.

Alright, ignore my previous mail, I'm lagging behind it seems ;-)

Could this be related to pointer trucation? The man page of dlopen tells us that the return value is a pointer (hence the return value of LoadLibrary has the same width). In your example you used an integer. Is sizeof(Integer) = sizeof(Pointer) of your system?

If for example, you are missing the upper four bytes of the pointer, the above message makes sense.

Good point:

uname -mrs

FreeBSD 9.1-STABLE amd64

./pointersize

pointersize: 8
size of integer: 2

cat pointersize.pas

program pointersize;
uses
  sysutils;
begin
  writeln('pointersize:     ' + inttostr(sizeof(pointer)));
  writeln('size of integer: ' + inttostr(sizeof(integer)));
end.

Doing it right it seems to work, I've tried three libs with one of them definitely not part of the OS but from a port:

program loadlibrary;
{$mode objfpc}{$H+}
uses
  sysutils, dynlibs;
var
  //libname: string = '/usr/lib/libalias.so';
  //fncname: string = 'AddLink';
  //libname: string = '/usr/lib/libipsec.so';
  //fncname: string = 'ipsec_dump_policy';
  libname: string = '/usr/local/lib/vlc/plugins/codec/libx264_plugin.so';
  fncname: string = 'vlc_entry__1_1_0g';
  libhandle: TLibHandle;
  pfnc: pointer;
begin
  libhandle := dynlibs.LoadLibrary(pchar(libname));
  if (libhandle = 0) then begin
    writeln('loading error: ' + GetLoadErrorStr);
    halt(1);
  end else
    writeln('library ' + libname + ' loaded.');
  pfnc := GetProcedureAddress(libhandle, pchar(fncname));
  if (pfnc = NIL) then
    writeln('got proc address error: ' + GetLoadErrorStr)
  else begin
    writeln('function ' + fncname + ' loaded.');
    writeln(inttostr(qword(pfnc)));
  end;
  UnloadLibrary(libhandle);
end.

I did not call any of those functions, but their nakmes are found and loaded by dlsym. For all three libraries the program worked fine.

For Fred:
You can check the names inside a dynamic liobrary using system tools:

nm -D /usr/lib/libipsec.so
nm -D /usr/local/lib/vlc/plugins/codec/libx264_plugin.so

The letters after the address tell the type of symbol, see

$ man nm

for an in detail explanation. Mostly those having a T are really inside the library, U means unresolved and so linked fgrom another lib.

--> https://github.com/fredvs/uos

Try SimplePlayer example (all FreeBSD libraries are included in uos package) --> choose /sound/test.m4a --> IT WORKS ;-)

Could you add a "writeln(dlerror());" ate the "****INDICATED POSITION****"? Just curious about what is going on here...

See what I noted in previous mail :

GetProcedureAddress(hn, pchar('mp4ff_open_read');
writeln(dlerror());

---> Result:
Invalid shared object handle 0x2fb2800

Why do you use "hn: Intege"hn: Integer" instead of "hn: TLibHandle"

Excellent question...
Because of... fpc...

In fpc 64 bit unix : result of dynlibs.loadlibrary() is... a integer.

And for fpc unix 64 bit : result is a TLibHandle.

I agree it is strange.

By the way, many thanks for your bright light Ewald.

Fre;D

-----
Many thanks ;-)

Thanks but... see my second mail in this topic... ;-)

Fre;D

-----
Many thanks ;-)

Re-hello.

To resume.

Here what is working perfectly for me:

 {$IF DEFINED(freebsd)}
...
 var
ap1 : pointer;
begin
  ap1 := dlopen(Pchar(mp4ff), 1);
  Pointer(mp4ff_open_read) :=  dlsym(ap1, pchar('mp4ff_open_read'));
...
 {$else}
...
 var
an1 : integer;
begin
  an1 := DynLibs.SafeLoadLibrary(PChar(mp4ff));
  Pointer(mp4ff_open_read) :=  GetProcAddress(an1,
pchar('mp4ff_open_read'));
...
 {$endif}

Fre;D

-----
Many thanks ;-)

That's totally OK, but the unit dynlibs is written for platform independence and I'd like to be able to use it on FreeBSD and at least Windows without having much $ifdefs and the like.

Have fun,
Marc

That's totally OK, but the unit dynlibs is written for platform independence and I'd like to be able to use it on FreeBSD and at least Windows without having much $ifdefs and the like.

Huh,... me too...
I am sure that fpc team will find the solution.
But, before it will be integrate into new version of fpc, $ifdefs is the fastest way to make my projects working.

Totally other thing...

Some time ago I noted here on fpc-mailing list that the size of fpc-libraries is enormous vs same code for a fpc-program. Somebody of fpc team promises to reduce the size (even without -fPIC).

I believe in the power of fpc-libraries (native or java-native), even more than in the power of fpc-program.

Sadly, the size of a fpc library is very, very, very big, even in last fpc 3.1.1 trunk. (this without debug and stripped, of course) Too big to compete with a c library (who does the same thing).

Does have fpc team project to reduce the size of fpc-libraries?

Many thanks.

Fre;D

-----
Many thanks ;-)

Well, if this is the case, then you should probably report a bug. The man page of at least linux and freeBSD indicate that it should be a pointer.

However, I would like to note that on linux, using revision 32828, TLibHandle is defined as a ptrint, which is correct.

Ooops,

How will I explain that ?...

In the pascal header of the library that gives problems this is defined : var libhandle : {$IFDEF MSWINDOWS}longword{$ELSE}{$IFDEF CPU32}longword{$ELSE}PtrInt{$ENDIF}{$ENDIF};

And in all the other library-headers that i use without problems it is defined like this:
var libhandle:TLibHandle=dynlibs.NilHandle;

Maybe (surely) the first definition is wrong.

I have to re-test the problematic library with
var libhandle:TLibHandle=dynlibs.NilHandle;

and see if it is ok for GetProcAdress().

Maybe that was the problem.

Write you later.

PS: Sorry for all the noise but it is not easy.
FreeBSD, in his pkg install a_library gives only the main dynamic library (.so).
All dependent-libraries are static (*.a).
So the end user has to re-compile all .a libraries to make it dynamic.

For the library in question: -> libmp4ff.a -->
gcc -shared -o libmp4ff.so.0.0.0 -Wl,--whole-archive libmp4ff.a -Wl,--no-whole-archive

Then you have a dynamic library.

Huh, not so easy because there is the C war...
So you have to make:
gcc -shared -o libmp4ff.so.0.0.0 -Wl,--whole-archive libmp4ff.a -Wl,--no-whole-archive
CC -shared -o libmp4ff.so.0.0.0 -Wl,--whole-archive libmp4ff.a -Wl,--no-whole-archive
clang -shared -o libmp4ff.so.0.0.0 -Wl,--whole-archive libmp4ff.a -Wl,--no-whole-archive
cpp -shared -o libmp4ff.so.0.0.0 -Wl,--whole-archive libmp4ff.a -Wl,--no-whole-archive

And test each result with nm libmp4ff.so.0.0.0.
Finally, cpp gives something "doable".

And the problem was to know if that compiled library was ok.

Now, thanks to your help, I am sure the library is ok.

I feel bad to not have seen that the (surely) wrong declaration of library_handle was maybe the cause of all the problems.

Fre;D

-----
Many thanks ;-)

Aaaaaargh.

Indeed, the declaration of variable for the library-handle was wrong. And FreeBSD is more strait than Linux and Windows.

I was focused on GetProcAddress but the bug was in LoadLibrary... ;-(

Changing --> var lib_handle:TLibHandle=dynlibs.NilHandle;
solves everything, even in FreeBSD.

I am really sorry for all the noise.
But, thanks to you, I am very happy to have learned interesting things about libraries (dlerror() is a pearl).

Conclusion, all ok for LoadLibrary and GetProcAddress() in FreeBSD too.

Many thanks my friends.

PS: Ewald --> you are very strong, well seen.

Fre;D

-----
Many thanks ;-)

Even on Win64 that would have been wrong as there TLibHandle is 64-bit as well and it also defines MSWINDOWS.

Regards,
Sven

Even on Win64 that would have been wrong as there TLibHandle is 64-bit as

well and it also defines MSWINDOWS.

Hello Sven.

I am unforgivable to not have seen that wrong declaration.

And to not have deeply check code from other before to commit it.

I apologize for all the noise.

Fre;D

-----
Many thanks ;-)

"fredvs" wrote

> Even on Win64 that would have been wrong as there TLibHandle is 64-bit

as

I'm not blaming you. These kind of mistakes can happen to the best ;) I only pointed out that this could have happened on other systems as well.

Regards,
Sven