Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2020-06

OpenSSL v1.1.x DLL loading / Bug 37137

13 messages · 3 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Wayne Sherman

*Bug Reference:*
"OpenSSL v1.1.1 could not loaded on Windows platform" https://bugs.freepascal.org/view.php?id=37137

1) I can confirm that the new v1.1.x DLL names are correct. I also checked some of the builds linked from the OpenSSL binary's page. The OpenSSL commit that discusses the name changes:
https://github.com/openssl/openssl/commit/520f434b42c83d63d8777075eb66967618551d5b

2) There is a problem with the loading. The DLLs names have to be handled in pairs. If the first DLL fails to load (crypto/util DLL), then both should be skipped and the next pair tried. If the second DLL (SSL lib) fails to load, the first one needs to be unloaded and the pair skipped to try the next pair. Otherwise it is possible to load two DLLs that are not matched versions.

Reference master branch code:
https://github.com/graemeg/freepascal/blob/master/packages/openssl/src/openssl.pas#L5634

3) I recommend the newest libraries be tried first, working backwards towards the oldest.

Both #2 and #3 are demonstrated by this patch on the synapse mailing list: https://sourceforge.net/p/synalist/mailman/message/36933840/

<https://github.com/graemeg/freepascal/blob/master/packages/openssl/src/openssl.pas#L5634>
4) The bug report says "Fixed in Version: 3.2.0", but that appears to be incorrect.

Thanks,

Wayne

Reply by email

Michael Van Canneyt
to Wayne Sherman
Wayne Sherman wrote 18 lines
*Bug Reference:* "OpenSSL v1.1.1 could not loaded on Windows platform" https://bugs.freepascal.org/view.php?id=37137 1) I can confirm that the new v1.1.x DLL names are correct. I also checked some of the builds linked from the OpenSSL binary's page. The OpenSSL commit that discusses the name changes: https://github.com/openssl/openssl/commit/520f434b42c83d63d8777075eb66967618551d5b 2) There is a problem with the loading. The DLLs names have to be handled in pairs. If the first DLL fails to load (crypto/util DLL), then both should be skipped and the next pair tried. If the second DLL (SSL lib) fails to load, the first one needs to be unloaded and the pair skipped to try the next pair. Otherwise it is possible to load two DLLs that are not matched versions. Reference master branch code: https://github.com/graemeg/freepascal/blob/master/packages/openssl/src/openssl.pas#L5634

This needs to be checked.

3) I recommend the newest libraries be tried first, working backwards towards the oldest.

This is already so ? The version numbers are added from new to old ?

Show quoted text 7 lines
Both #2 and #3 are demonstrated by this patch on the synapse mailing list: https://sourceforge.net/p/synalist/mailman/message/36933840/ <https://github.com/graemeg/freepascal/blob/master/packages/openssl/src/openssl.pas#L5634> 4) The bug report says "Fixed in Version: 3.2.0", but that appears to be incorrect.

I have changed that to 4.0

Michael.

Reply by email

Wayne Sherman
to Michael Van Canneyt
PM Michael Van Canneyt wrote 5 lines
> > 3) I recommend the newest libraries be tried first, working backwards > towards the oldest. This is already so ? The version numbers are added from new to old ?

The libraries are currently loading in this order: First attempt: ssleay32.dll / libeay32.dll (older names, but not oldest) Second attempt (util): libcrypto-1_1-x64.dll (newest name) Third attempt (ssl): libssl32.dll (oldest name)
Fourth attempt (ssl): libssl-1_1-x64.dll (newest name)

They should be tried as pairs in this order (win64 shown):
1) libssl-1_1-x64.dll / libcrypto-1_1-x64.dll (newest)
2) ssleay32.dll / libeay32.dll (older)
3) libssl32.dll / libeay32.dll (oldest)

PM Michael Van Canneyt wrote 18 lines
On Wed, 24 Jun 2020, Wayne Sherman via fpc-devel wrote: > *Bug Reference:* > "OpenSSL v1.1.1 could not loaded on Windows platform" > https://bugs.freepascal.org/view.php?id=37137 > ... > 2) There is a problem with the loading. The DLLs names have to be handled > in pairs. If the first DLL fails to load (crypto/util DLL), then both > should be skipped and the next pair tried. If the second DLL (SSL lib) > fails to load, the first one needs to be unloaded and the pair skipped to > try the next pair. Otherwise it is possible to load two DLLs that are not > matched versions. > > Reference master branch code: > https://github.com/graemeg/freepascal/blob/master/packages/openssl/src/openssl.pas#L5634 This needs to be checked.

I ran a test. If a system has a stray file it can cause mismatched DLLs to load (which could result in strange bugs)
Please see attached image "OpenSSL_dll_loading_screenshot.png"

> 3) I recommend the newest libraries be tried first, working backwards > towards the oldest. This is already so ? The version numbers are added from new to old ?

The libraries are current loading in this order:
First attempt: ssleay32.dll / libeay32.dll (older name pair, but not oldest)
Second attempt (util): libcrypto-1_1-x64.dll (newest name) Third attempt (ssl): libssl32.dll (oldest name)
Fourth attempt (ssl): libssl-1_1-x64.dll (newest name)

They should be tried as pairs in this order (Win64 shown):
1) libssl-1_1-x64.dll / libcrypto-1_1-x64.dll (newest)
2) ssleay32.dll / libeay32.dll (older)
3) libssl32.dll / libeay32.dll (oldest)

> 4) The bug report says "Fixed in Version: 3.2.0", but that appears to be > incorrect. I have changed that to 4.0

From: https://www.openssl.org/policies/releasestrat.html openssl versions 1.1.0, 1.0.2, 1.0.1, 1.0.0 and 0.9.8 are no longer supported

So can Windows openssl v1.1 library loading be included in the next maintenance release (v3.2.1?) instead of waiting for v4.0?

Would you like me to have a go at a revised patch?

Thanks,

Wayne

Reply by email

Michael Van Canneyt
to Wayne Sherman
Wayne Sherman wrote 13 lines
On Thu, Jun 25, 2020 at 11:43 PM Michael Van Canneyt wrote: >> >> 3) I recommend the newest libraries be tried first, working backwards >> towards the oldest. > > This is already so ? The version numbers are added from new to old ? The libraries are currently loading in this order: First attempt: ssleay32.dll / libeay32.dll (older names, but not oldest) Second attempt (util): libcrypto-1_1-x64.dll (newest name) Third attempt (ssl): libssl32.dll (oldest name) Fourth attempt (ssl): libssl-1_1-x64.dll (newest name)

Oh, you mean the DLL names.
I thought you meant the version numbers. Yes, this can be changed too.

Show quoted text 6 lines
> > This needs to be checked. I ran a test. If a system has a stray file it can cause mismatched DLLs to load (which could result in strange bugs) Please see attached image "OpenSSL_dll_loading_screenshot.png"

We'll adapt the mechanism.

Show quoted text 5 lines
> I have changed that to 4.0 From: https://www.openssl.org/policies/releasestrat.html openssl versions 1.1.0, 1.0.2, 1.0.1, 1.0.0 and 0.9.8 are no longer supported

Maybe, but for backwards compatibility they must be supported in FPC.

So can Windows openssl v1.1 library loading be included in the next maintenance release (v3.2.1?) instead of waiting for v4.0?

No worries, 4.0 means we can consider it for the next fix release. That will be 3.2.2 (even numbers only for releases, odd indicates development sources)

Would you like me to have a go at a revised patch?

Sure. If you create a patch, upload it to the bugtracker and assign it to me (or mail here so I can have a look)

Michael.

Reply by email

Wayne Sherman
to Michael Van Canneyt

OpenSSL library loading fixes. Patch file is attached. Comments are welcome.

Summary:
1) For Windows and OS2, remove DLLSSLNameX and DLLUtilNameX variables. Instead use matched arrays of DLL filenames (SSL_DLL_Names and Crypto_DLL_Names)

2) Due to refactoring, remove LoadLibHack or LoadLib functions.

3) Remove LoadUtilLibrary function, UnloadSSLLib procedure, and UnloadUtilLib procedure since these were not being called internally and are unreachable externally.

4) Add new function TryLoadLibPair which attempts to load a matched pair of DLLs (SSL and Crypto). If either fails to load, UnloadLibraries is called to clean up.

5) Refactored LoadLibraries to use arrays of names for Windows and OS2, and to use the new TryLoadLibPair helper function for all platforms.

PM Michael Van Canneyt wrote 57 lines
On Fri, 26 Jun 2020, Wayne Sherman wrote: > On Thu, Jun 25, 2020 at 11:43 PM Michael Van Canneyt > wrote: >>> >>> 3) I recommend the newest libraries be tried first, working backwards >>> towards the oldest. >> >> This is already so ? The version numbers are added from new to old ? > > The libraries are currently loading in this order: > First attempt: ssleay32.dll / libeay32.dll (older names, but not oldest) > Second attempt (util): libcrypto-1_1-x64.dll (newest name) > Third attempt (ssl): libssl32.dll (oldest name) > Fourth attempt (ssl): libssl-1_1-x64.dll (newest name) Oh, you mean the DLL names. I thought you meant the version numbers. Yes, this can be changed too. >> >> This needs to be checked. > > I ran a test. If a system has a stray file it can cause mismatched > DLLs to load (which could result in strange bugs) > Please see attached image "OpenSSL_dll_loading_screenshot.png" We'll adapt the mechanism. >> I have changed that to 4.0 > > From: https://www.openssl.org/policies/releasestrat.html > openssl versions 1.1.0, 1.0.2, 1.0.1, 1.0.0 and 0.9.8 are no > longer supported Maybe, but for backwards compatibility they must be supported in FPC. > > So can Windows openssl v1.1 library loading be included in the next > maintenance release (v3.2.1?) instead of waiting for v4.0? No worries, 4.0 means we can consider it for the next fix release. That will be 3.2.2 (even numbers only for releases, odd indicates development sources) > > Would you like me to have a go at a revised patch? Sure. If you create a patch, upload it to the bugtracker and assign it to me (or mail here so I can have a look) Michael. _______________________________________________ fpc-devel maillist - https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Reply by email

Michael Van Canneyt
to Wayne Sherman
Wayne Sherman wrote 20 lines
OpenSSL library loading fixes. Patch file is attached. Comments are welcome. Summary: 1) For Windows and OS2, remove DLLSSLNameX and DLLUtilNameX variables. Instead use matched arrays of DLL filenames (SSL_DLL_Names and Crypto_DLL_Names) 2) Due to refactoring, remove LoadLibHack or LoadLib functions. 3) Remove LoadUtilLibrary function, UnloadSSLLib procedure, and UnloadUtilLib procedure since these were not being called internally and are unreachable externally. 4) Add new function TryLoadLibPair which attempts to load a matched pair of DLLs (SSL and Crypto). If either fails to load, UnloadLibraries is called to clean up. 5) Refactored LoadLibraries to use arrays of names for Windows and OS2, and to use the new TryLoadLibPair helper function for all platforms.

Thank you very much for the patch.
I will look at this ASAP.

Michael.

Reply by email

Wayne Sherman
to Michael Van Canneyt
AM Michael Van Canneyt wrote 26 lines
On Tue, 30 Jun 2020, Wayne Sherman via fpc-devel wrote: > OpenSSL library loading fixes. Patch file is attached. Comments are welcome. > > Summary: > 1) For Windows and OS2, remove DLLSSLNameX and DLLUtilNameX > variables. Instead use matched arrays of DLL filenames (SSL_DLL_Names > and Crypto_DLL_Names) > > 2) Due to refactoring, remove LoadLibHack or LoadLib functions. > > 3) Remove LoadUtilLibrary function, UnloadSSLLib procedure, and > UnloadUtilLib procedure since these were not being called internally > and are unreachable externally. > > 4) Add new function TryLoadLibPair which attempts to load a > matched pair of DLLs (SSL and Crypto). If either fails to load, > UnloadLibraries is called to clean up. > > 5) Refactored LoadLibraries to use arrays of names for Windows and > OS2, and to use the new TryLoadLibPair helper function for all > platforms. Thank you very much for the patch. I will look at this ASAP.

Hi Michael,

The OpenSSL library loading code still has the potential to load mismatched library versions. Did you get a chance to review the patch I sent?

Thanks

Reply by email

Michael Van Canneyt
to Wayne Sherman
Wayne Sherman wrote 34 lines
On Tue, Jun 30, 2020 at 12:38 AM Michael Van Canneyt wrote: > > On Tue, 30 Jun 2020, Wayne Sherman via fpc-devel wrote: > > > OpenSSL library loading fixes. Patch file is attached. Comments are welcome. > > > > Summary: > > 1) For Windows and OS2, remove DLLSSLNameX and DLLUtilNameX > > variables. Instead use matched arrays of DLL filenames (SSL_DLL_Names > > and Crypto_DLL_Names) > > > > 2) Due to refactoring, remove LoadLibHack or LoadLib functions. > > > > 3) Remove LoadUtilLibrary function, UnloadSSLLib procedure, and > > UnloadUtilLib procedure since these were not being called internally > > and are unreachable externally. > > > > 4) Add new function TryLoadLibPair which attempts to load a > > matched pair of DLLs (SSL and Crypto). If either fails to load, > > UnloadLibraries is called to clean up. > > > > 5) Refactored LoadLibraries to use arrays of names for Windows and > > OS2, and to use the new TryLoadLibPair helper function for all > > platforms. > > Thank you very much for the patch. > I will look at this ASAP. Hi Michael, The OpenSSL library loading code still has the potential to load mismatched library versions. Did you get a chance to review the patch I sent?

Not yet, I'm currently very busy.

The patch is invasive so I need some time to check this, meaning it will be for next weekend (not tomorrow, but next week).

Michael.

Reply by email

Michael Van Canneyt
to Michael Van Canneyt
Michael Van Canneyt wrote 13 lines
>> Thank you very much for the patch. >> I will look at this ASAP. > > Hi Michael, > > The OpenSSL library loading code still has the potential to load > mismatched library versions. Did you get a chance to review the patch > I sent? Not yet, I'm currently very busy. The patch is invasive so I need some time to check this, meaning it will be for next weekend (not tomorrow, but next week).

I made some time for this, it's checked in with revision 46987.

Note that this revision also enables support for LibreSSL (or whatever it is called) versions on linux/mac systems.

Thank you for your patch, and I'm sorry that it took me so long to get to it...

Michael.

Reply by email

Ondrej Pokorny
to Michael Van Canneyt
Michael Van Canneyt wrote 8 lines
I made some time for this, it's checked in with revision 46987. Note that this revision also enables support for LibreSSL (or whatever it is called) versions on linux/mac systems. Thank you for your patch, and I'm sorry that it took me so long to get to it...

Hello,

openssl doesn't build anymore (make fpc trunk):

Compiling openssl\BuildUnit_openssl.pp

Compiling FPC\trunk_src\packages\openssl\src\openssl.pas openssl.pas(5629,66) Error: Wrong number of parameters specified for call to "MakeLibName"
openssl.pas(5570,11) Error: Found declaration: MakeLibName(const AnsiString;const AnsiString):AnsiString(0);
openssl.pas(5710) Fatal: There were 2 errors compiling module, stopping Fatal: Compilation aborted

Ondrej

Reply by email

Michael Van Canneyt
to Ondrej Pokorny
Ondrej Pokorny wrote 23 lines
On 28.09.2020 15:34, Michael Van Canneyt via fpc-devel wrote: > I made some time for this, it's checked in with revision 46987. > > Note that this revision also enables support for LibreSSL (or whatever > it is > called) versions on linux/mac systems. > > Thank you for your patch, and I'm sorry that it took me so long to get > to it... Hello, openssl doesn't build anymore (make fpc trunk): Compiling openssl\BuildUnit_openssl.pp Compiling FPC\trunk_src\packages\openssl\src\openssl.pas openssl.pas(5629,66) Error: Wrong number of parameters specified for call to "MakeLibName" openssl.pas(5570,11) Error: Found declaration: MakeLibName(const AnsiString;const AnsiString):AnsiString(0); openssl.pas(5710) Fatal: There were 2 errors compiling module, stopping Fatal: Compilation aborted

Should be fixed in rev. 46991.

Michael.

Reply by email

Ondrej Pokorny
to Michael Van Canneyt
Michael Van Canneyt wrote 27 lines
On Mon, 28 Sep 2020, Ondrej Pokorny via fpc-devel wrote: > On 28.09.2020 15:34, Michael Van Canneyt via fpc-devel wrote: >> I made some time for this, it's checked in with revision 46987. >> >> Note that this revision also enables support for LibreSSL (or >> whatever it is >> called) versions on linux/mac systems. >> >> Thank you for your patch, and I'm sorry that it took me so long to >> get to it... > > Hello, > > openssl doesn't build anymore (make fpc trunk): > > Compiling openssl\BuildUnit_openssl.pp > > Compiling FPC\trunk_src\packages\openssl\src\openssl.pas > openssl.pas(5629,66) Error: Wrong number of parameters specified for > call to "MakeLibName" > openssl.pas(5570,11) Error: Found declaration: MakeLibName(const > AnsiString;const AnsiString):AnsiString(0); > openssl.pas(5710) Fatal: There were 2 errors compiling module, stopping > Fatal: Compilation aborted Should be fixed in rev. 46991.

Thank you!

Ondrej

Reply by email