Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2011-07

[semi-off-topic] some C questions arising while using h2pas

8 messages · 5 people ·

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Bernd

I am just trying to convert some openssl headers to Pascal (I know that there is an openssl unit in the fpc packages but it is missing the ECDSA functions which I will need later and which is main reason I am starting to look into openssl).

To get used to this entire thing I am beginning with something much more simple, I will try to get some sha hashing functions from openssl imported (I will also need them, so it would not hurt to start with them).

I am referring to the header file /usr/include/openssl/sha.h (pastebin here: http://pastebin.com/dS8DY2z9 )

After appyling h2pas i have the following: http://pastebin.com/v3S3vtFR

If I want to import functions in Pascal I would either declare them external and explicitly tell it the name of the library binary (this seems straightforward to me, I see no problems with this) or I load it dynamically at runtime and then ask for the addresses of the functions that I need and assign them to procedure variables (I have also done this already on some occasions and it is equally easy to understand and also raises no questions).

I am confused about something else:

When file and also when looking at the generated .pp file it seems that nowhere in the header there was any information about which .so (or .dll) to load or link against. It created empty bodies with { you must implement this function }. Now the question (and I am intentionally asking this on a Pascal list and not a C list to get an answer that better fits into my Pascal brain) how would the C compiler or linker know from where exactly to import these functions? Where is this information, how are these C folks doing these things, what kind of magic have they going on?

confused,
Bernd

Reply by email

Jonas Maebe
to Bernd
Bernd wrote
To get used to this entire thing I am beginning with something much more simple, I will try to get some sha hashing functions from openssl imported (I will also need them, so it would not hurt to start with them).

FPC comes with translated openssl headers, so other than for learning purposes you shouldn't have to do this.

Show quoted text 5 lines
Now the question (and I am intentionally asking this on a Pascal list and not a C list to get an answer that better fits into my Pascal brain) how would the C compiler or linker know from where exactly to import these functions?

On Unix-like systems, at static link time (i.e., at the end of compiling the program), the linker searches all libraries specified on the command line for the used symbols. If a symbol is not found, you get an error. Furthermore, at run time the dynamic linker again will search for those same symbols. Depending on the operating system and the dynamic linker settings, it will only search for those symbols in certain libraries or again in any library that's in the (dynamic) library path. The binary generally does not contain information about from which library a particular symbol comes (exceptions are Windows, and at least also Mac OS X to some extent).

http://blogs.embarcadero.com/eboling/2010/02/16/5656/ contains a fairly good overview of the symbol binding strategies on different platforms.

Jonas

Reply by email

ik
to Bernd: I am just trying to convert some openssl headers to Pascal (I know that there is an…
Bernd wrote 35 lines
I am just trying to convert some openssl headers to Pascal (I know that there is an openssl unit in the fpc packages but it is missing the ECDSA functions which I will need later and which is main reason I am starting to look into openssl). To get used to this entire thing I am beginning with something much more simple, I will try to get some sha hashing functions from openssl imported (I will also need them, so it would not hurt to start with them). I am referring to the header file /usr/include/openssl/sha.h (pastebin here: http://pastebin.com/dS8DY2z9 ) After appyling h2pas i have the following: http://pastebin.com/v3S3vtFR If I want to import functions in Pascal I would either declare them external and explicitly tell it the name of the library binary (this seems straightforward to me, I see no problems with this) or I load it dynamically at runtime and then ask for the addresses of the functions that I need and assign them to procedure variables (I have also done this already on some occasions and it is equally easy to understand and also raises no questions). I am confused about something else: When file and also when looking at the generated .pp file it seems that nowhere in the header there was any information about which .so (or .dll) to load or link against. It created empty bodies with { you must implement this function }. Now the question (and I am intentionally asking this on a Pascal list and not a C list to get an answer that better fits into my Pascal brain) how would the C compiler or linker know from where exactly to import these functions? Where is this information, how are these C folks doing these things, what kind of magic have they going on?

The information is located under /usrlib/pkgconfig/openssl.pc

It tells to the configure script how to generate the Makefile and how to link stuff. It just a gcc syntax that is pasted back to the Makefile.

I wrote a tool once (in Bash) that allow you properly search for a function inside an so to see who have it.
If you wish, you can find it here:
http://code.google.com/p/pasxlibbind/source/browse/trunk/find_function.sh

Show quoted text 7 lines
confused, Bernd _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/mailman/listinfo/fpc-pascal

Ido

Reply by email

Sven Barth
to Bernd: I am just trying to convert some openssl headers to Pascal (I know that there is an…
Bernd wrote 34 lines
I am just trying to convert some openssl headers to Pascal (I know that there is an openssl unit in the fpc packages but it is missing the ECDSA functions which I will need later and which is main reason I am starting to look into openssl). To get used to this entire thing I am beginning with something much more simple, I will try to get some sha hashing functions from openssl imported (I will also need them, so it would not hurt to start with them). I am referring to the header file /usr/include/openssl/sha.h (pastebin here: http://pastebin.com/dS8DY2z9 ) After appyling h2pas i have the following: http://pastebin.com/v3S3vtFR If I want to import functions in Pascal I would either declare them external and explicitly tell it the name of the library binary (this seems straightforward to me, I see no problems with this) or I load it dynamically at runtime and then ask for the addresses of the functions that I need and assign them to procedure variables (I have also done this already on some occasions and it is equally easy to understand and also raises no questions). I am confused about something else: When file and also when looking at the generated .pp file it seems that nowhere in the header there was any information about which .so (or .dll) to load or link against. It created empty bodies with { you must implement this function }. Now the question (and I am intentionally asking this on a Pascal list and not a C list to get an answer that better fits into my Pascal brain) how would the C compiler or linker know from where exactly to import these functions? Where is this information, how are these C folks doing these things, what kind of magic have they going on?

You missed some options of h2pas.

1. Did you give a name to the "-l" option? I don't know which library the functions reside in, but you should then use something like "-l libssl.so"
2. Using the options you passed you did not generate code for external functions, but basically a normal Pascal unit. For the first you need to use either the "-d" or the "-D" parameter. The first one will only add an "external;" to declarations while the second one will add "external libname name 'name of the declaration';" (where libname is the name you gave with "-l"). If you use the first one you need to add a "{$linklib thelibraryname}" (e.g. "{$linklib ssh}") in the second case it should work already.

I hope this helps, if not, feel free to ask :)

Regards,
Sven

Reply by email

Mark Morgan Lloyd
to Jonas Maebe: FPC comes with translated openssl headers, so other than for learning purposes you…
Jonas Maebe wrote 18 lines
On 15 Jul 2011, at 13:43, Bernd wrote: > To get used to this entire thing I am beginning with something much > more simple, I will try to get some sha hashing functions from openssl > imported (I will also need them, so it would not hurt to start with > them). FPC comes with translated openssl headers, so other than for learning purposes you shouldn't have to do this. > Now the question > (and I am intentionally asking this on a Pascal list and not a C list > to get an answer that better fits into my Pascal brain) how would the > C compiler or linker know from where exactly to import these > functions? On Unix-like systems, at static link time (i.e., at the end of compiling the program), the linker searches all libraries specified on the command line for the used symbols. If a symbol is not found, you get an error. Furthermore, at run time the dynamic linker again will search for those same symbols. Depending on the operating system and the dynamic linker settings, it will only search for those symbols in certain libraries or again in any library that's in the (dynamic) library path. The binary generally does not contain information about from which library a particular symbol comes (exceptions are Windows, and at least also Mac OS X to some extent). http://blogs.embarcadero.com/eboling/2010/02/16/5656/ contains a fairly good overview of the symbol binding strategies on different platforms.

I wonder if I could tack a question onto this. When I started looking at SPARC Solaris 10 and Solaris 8 earlier in the year, I found that the linker (which I think was gld) followed symlinks at build time rather than waiting until runtime. This meant that while I was able to build FPC on 10 without significant difficulty, I couldn't move the binaries back to 8: I had to rebuild with a hacked symlink to a temporary copy of one of the standard libraries (possibly libc, I can't remember for certain).

Is this what one would expect, or is it another ld glitch that might be platform-specific?

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Bernd
to Jonas Maebe: FPC comes with translated openssl headers, so other than for learning purposes you…

2011/7/15 Jonas Maebe :

FPC comes with translated openssl headers, so other than for learning purposes you shouldn't have to do this.

The problem currently is that it seems to provide only a subset of all functionality of openssl.

Specifically I am currently looking for an implementation of ECDSA (elliptic curve DSA signatures), this is not trivial (and probably not a good idea) to try to implement this from scratch myself and since I will later need openssl anyways (also for SHA512 which I will also need and also for SSL sockets in lNet) I thought the best way would be to use the implementations of these algorithms that are provided by openssl.

Of course it would be desirable to make a patch to add the needed declarations to the existing openssl unit but for this I first need to make myself familiar with this library and its usage and this is the reason I am currently experimenting with my own header translations.

Bernd

Reply by email

Jonas Maebe
to Mark Morgan Lloyd: I wonder if I could tack a question onto this. When I started looking at SPARC Solaris 10…
Mark Morgan Lloyd wrote
I wonder if I could tack a question onto this. When I started looking at SPARC Solaris 10 and Solaris 8 earlier in the year, I found that the linker (which I think was gld) followed symlinks at build time rather than waiting until runtime. This meant that while I was able to build FPC on 10 without significant difficulty, I couldn't move the binaries back to 8: I had to rebuild with a hacked symlink to a temporary copy of one of the standard libraries (possibly libc, I can't remember for certain). Is this what one would expect, or is it another ld glitch that might be platform-specific?

It is expected. The logic is:
a) you build the application using a certain set of headers that go with a certain version of the library. Hence, the program should be linked against that particular version of the library b) you don't want to have to change the build command every time a library gets updated. Hence, there is a symlink with a generic name going to the default version of the library (and on most Linux systems this symlink in fact is only created if you install the -devel version of a package, since it is only required when compiling programs) c) if a library changes in a way that breaks backwards compatibility as far as the api is concerned, its major version is increased so that applications built against a previous version of the library won't use the newer one and then crash. Also, since the new version has a different name, you can in principle have both the old and the new version installed concurrently so that both old apps and new apps still work (the system however breaks down when a single program uses e.g. libA and libB that in turn depend on respectively libfoo.1.so and libfoo.2.so, because both libfoos will probably have a bunch of symbols in common causing symbol resolution conflicts)

Jonas

Reply by email

Mark Morgan Lloyd
to Jonas Maebe
Jonas Maebe wrote 10 lines
On 15 Jul 2011, at 14:40, Mark Morgan Lloyd wrote: > I wonder if I could tack a question onto this. When I started looking at SPARC Solaris 10 and Solaris 8 earlier in the year, I found that the linker (which I think was gld) followed symlinks at build time rather than waiting until runtime. This meant that while I was able to build FPC on 10 without significant difficulty, I couldn't move the binaries back to 8: I had to rebuild with a hacked symlink to a temporary copy of one of the standard libraries (possibly libc, I can't remember for certain). > > Is this what one would expect, or is it another ld glitch that might be platform-specific? It is expected. The logic is: a) you build the application using a certain set of headers that go with a certain version of the library. Hence, the program should be linked against that particular version of the library b) you don't want to have to change the build command every time a library gets updated. Hence, there is a symlink with a generic name going to the default version of the library (and on most Linux systems this symlink in fact is only created if you install the -devel version of a package, since it is only required when compiling programs) c) if a library changes in a way that breaks backwards compatibility as far as the api is concerned, its major version is increased so that applications built against a previous version of the library won't use the newer one and then crash. Also, since the new version has a different name, you can in principle have both the old and the new version installed concurrently so that both old apps and new apps still work (the system however breaks down when a single program uses e.g. libA and libB that in turn depend on respectively libfoo.1.so and libfoo.2.so, because both libfoos will probably have a bunch of symbols in common causing symbol resolution conflicts)

Thanks Jonas, noted and understood. To summarise, a .h corresponds to a very specific version of a library; if there is a symlink to that library it is provided as a courtesy.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email