Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2021-11

Fix for an annoying error

12 messages · 4 people ·

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

J. Gareth Moreton

Hi everyone,

For a while now I've had problems building the i386-win32 compiler under my 64-bit Windows system because one of the packages fails to build - this is because it thinks a statically-imported DLL (done through $linklib) is invalid. Technically it is, but this system DLL (for me, it is named COMMON.DLL and is in one of my system folders, so it's not something I can safely modify or replace) is for 64-bit Windows and is not recognised when building for 32-bit, and the only 'fix' is to comment out the culprit $linklib line in the package source.

I've submitted a merge request that generates a warning instead of an error if it detects the DLL is 64-bit for a 32-bit target and vice versa.  The checks, however, are very strict (it checks the COFF magic number, which contains a platform target, and the size of the optional header), and an error is still thrown if the DLL is actually corrupted or is for an unknown target.

A warning is still thrown because the program that uses it may malfunction, and might be indicative of a bug or a faulty search path for the DLL.

https://gitlab.com/freepascal.org/fpc/source/-/merge_requests/91

Gareth aka. Kit

This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

Reply by email

Tomas Hajny
to J. Gareth Moreton

On 2021-11-30 08:33, J. Gareth Moreton via fpc-devel wrote:

Hi Gareth,

Show quoted text 22 lines
For a while now I've had problems building the i386-win32 compiler under my 64-bit Windows system because one of the packages fails to build - this is because it thinks a statically-imported DLL (done through $linklib) is invalid. Technically it is, but this system DLL (for me, it is named COMMON.DLL and is in one of my system folders, so it's not something I can safely modify or replace) is for 64-bit Windows and is not recognised when building for 32-bit, and the only 'fix' is to comment out the culprit $linklib line in the package source. I've submitted a merge request that generates a warning instead of an error if it detects the DLL is 64-bit for a 32-bit target and vice versa.  The checks, however, are very strict (it checks the COFF magic number, which contains a platform target, and the size of the optional header), and an error is still thrown if the DLL is actually corrupted or is for an unknown target. A warning is still thrown because the program that uses it may malfunction, and might be indicative of a bug or a faulty search path for the DLL. https://gitlab.com/freepascal.org/fpc/source/-/merge_requests/91

Semantically, what's the difference between the two cases (a DLL for a different architecture, or a broken DLL)? In both cases the compiled executable may be used on other machines (with proper version of the respective DLL), but it will fail when running it on the machine used for compilation, correct? Shouldn't the compiler handle both cases the same way (possibly issuing a similar warning if the DLL is not found at all)?

Tomas

Reply by email

J. Gareth Moreton
to Tomas Hajny

That was a conundrum I was trying to answer when making the patch.  What is a warning and what is an error?

A lot of the verification code is dependent on compile-time constants whose values depend on the compiler's target platform, including header sizes, so any mismatch is an error case, and the checking of header sizes and the COFF magic number are done together, since the correct header size is dependent on the magic number's value.

I didn't look too much further beyond the verification code, but if it's able to detect if a DLL is 32-bit on a 64-bit system and vice versa, I feel that's a warning rather than error because, in my experience, it's down to a default system configuration (I got the error on a freshly-installed machine) rather than something untoward.  If the DLL is missing, that should probably be a warning too and I'll see if I can make that change.  A corrupted DLL is definitely an error though.

I didn't do a verification to see if the DLL is for AArch64, for example, since that situation is more unlikely.  It also starts to bloat the compiler code.

Either way, using a DLL for a different build of Windows is a warning at the very least because the project will probably break when you try to run it.

Gareth aka. Kit

Tomas Hajny wrote 41 lines
On 2021-11-30 08:33, J. Gareth Moreton via fpc-devel wrote: Hi Gareth, > For a while now I've had problems building the i386-win32 compiler > under my 64-bit Windows system because one of the packages fails to > build - this is because it thinks a statically-imported DLL (done > through $linklib) is invalid. Technically it is, but this system DLL > (for me, it is named COMMON.DLL and is in one of my system folders, so > it's not something I can safely modify or replace) is for 64-bit > Windows and is not recognised when building for 32-bit, and the only > 'fix' is to comment out the culprit $linklib line in the package > source. > > I've submitted a merge request that generates a warning instead of an > error if it detects the DLL is 64-bit for a 32-bit target and vice > versa.  The checks, however, are very strict (it checks the COFF magic > number, which contains a platform target, and the size of the optional > header), and an error is still thrown if the DLL is actually corrupted > or is for an unknown target. > > A warning is still thrown because the program that uses it may > malfunction, and might be indicative of a bug or a faulty search path > for the DLL. > > https://gitlab.com/freepascal.org/fpc/source/-/merge_requests/91 Semantically, what's the difference between the two cases (a DLL for a different architecture, or a broken DLL)? In both cases the compiled executable may be used on other machines (with proper version of the respective DLL), but it will fail when running it on the machine used for compilation, correct? Shouldn't the compiler handle both cases the same way (possibly issuing a similar warning if the DLL is not found at all)? Tomas _______________________________________________ fpc-devel maillist  -  https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

Reply by email

Michael Van Canneyt
to J. Gareth Moreton
J. Gareth Moreton wrote 24 lines
That was a conundrum I was trying to answer when making the patch.  What is a warning and what is an error? A lot of the verification code is dependent on compile-time constants whose values depend on the compiler's target platform, including header sizes, so any mismatch is an error case, and the checking of header sizes and the COFF magic number are done together, since the correct header size is dependent on the magic number's value. I didn't look too much further beyond the verification code, but if it's able to detect if a DLL is 32-bit on a 64-bit system and vice versa, I feel that's a warning rather than error because, in my experience, it's down to a default system configuration (I got the error on a freshly-installed machine) rather than something untoward.  If the DLL is missing, that should probably be a warning too and I'll see if I can make that change.  A corrupted DLL is definitely an error though. I didn't do a verification to see if the DLL is for AArch64, for example, since that situation is more unlikely.  It also starts to bloat the compiler code. Either way, using a DLL for a different build of Windows is a warning at the very least because the project will probably break when you try to run it.

Personally, I think that reverting to a warning was a mistake.

You have no guarantee that the resulting program will run if you ignore the "error":

Not on the system where you compiled, not on the system where you will run the program.

For your particular problem: I looked at the source code of the oracle unit. As far as I can see there is no need to include the common.dll in the linklib statements, all that should be linked is the oracle dll. It may be a better solution simply to remove that line.

Michael.

Reply by email

J. Gareth Moreton
to Michael Van Canneyt

Well, running the program on the target system isn't a guaranteed success even if there's no error on the compilation system because the DLL in question may be different (or missing).

If it turns out that common.dll is not required, then the maintainer of the package should probably remove the $linklib line, otherwise I'm not sure.  If I'm right in thinking, if the DLL is missing or incompatible, it won't read the exports table and so even if just a warning is raised, it may cause a cascade error if any of its functions are used in the unit because they're not declared.

Gareth aka. Kit

Michael Van Canneyt wrote 50 lines
On Tue, 30 Nov 2021, J. Gareth Moreton via fpc-devel wrote: > That was a conundrum I was trying to answer when making the patch.  > What is a warning and what is an error? > > A lot of the verification code is dependent on compile-time constants > whose values depend on the compiler's target platform, including > header sizes, so any mismatch is an error case, and the checking of > header sizes and the COFF magic number are done together, since the > correct header size is dependent on the magic number's value. > > I didn't look too much further beyond the verification code, but if > it's able to detect if a DLL is 32-bit on a 64-bit system and vice > versa, I feel that's a warning rather than error because, in my > experience, it's down to a default system configuration (I got the > error on a freshly-installed machine) rather than something > untoward.  If the DLL is missing, that should probably be a warning > too and I'll see if I can make that change.  A corrupted DLL is > definitely an error though. > > I didn't do a verification to see if the DLL is for AArch64, for > example, since that situation is more unlikely.  It also starts to > bloat the compiler code. > > Either way, using a DLL for a different build of Windows is a warning > at the very least because the project will probably break when you > try to run it. Personally, I think that reverting to a warning was a mistake. You have no guarantee that the resulting program will run if you ignore the "error": Not on the system where you compiled, not on the system where you will run the program. For your particular problem: I looked at the source code of the oracle unit. As far as I can see there is no need to include the common.dll in the linklib statements, all that should be linked is the oracle dll. It may be a better solution simply to remove that line. Michael. _______________________________________________ fpc-devel maillist - https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

Reply by email

Bart
to J. Gareth Moreton: For a while now I've had problems building the i386-win32 compiler under my 64-bit…
AM J. Gareth Moreton wrote 8 lines
For a while now I've had problems building the i386-win32 compiler under my 64-bit Windows system because one of the packages fails to build - this is because it thinks a statically-imported DLL (done through $linklib) is invalid. Technically it is, but this system DLL (for me, it is named COMMON.DLL and is in one of my system folders, so it's not something I can safely modify or replace) is for 64-bit Windows and is not recognised when building for 32-bit, and the only 'fix' is to comment out the culprit $linklib line in the package source.

I have had exactly the same problem on Win64 building 32-bit fpc. The culprit was the file packages\oracle\src\oraoci.pp

As it turned out the common.dll was in the wrong folder (64-bit in a 32-bit folder or vice-versa).
Most likely by a buggy installer of sorts.
I have been fiddling with the system and found a way to remove that dll, not even sure how I succeeded.

After disabling the dll and then removing it my Win10-64 just runs fine. And compiling fpc (32-bit) was OK again.

Before I succeeded in that, I even wrote a simple program that altered packages\oracle\src\oraoci.pp so it would build correctly and the restore it after build (I was getting a bit desperate and frustrated).

Bart

Reply by email

J. Gareth Moreton
to Bart

Yeah, that's the exact same file that I have problems with.

Gareth aka. Kit

Bart wrote 27 lines
On Tue, Nov 30, 2021 at 8:33 AM J. Gareth Moreton via fpc-devel wrote: > For a while now I've had problems building the i386-win32 compiler under > my 64-bit Windows system because one of the packages fails to build - > this is because it thinks a statically-imported DLL (done through > $linklib) is invalid. Technically it is, but this system DLL (for me, it > is named COMMON.DLL and is in one of my system folders, so it's not > something I can safely modify or replace) is for 64-bit Windows and is > not recognised when building for 32-bit, and the only 'fix' is to > comment out the culprit $linklib line in the package source. I have had exactly the same problem on Win64 building 32-bit fpc. The culprit was the file packages\oracle\src\oraoci.pp As it turned out the common.dll was in the wrong folder (64-bit in a 32-bit folder or vice-versa). Most likely by a buggy installer of sorts. I have been fiddling with the system and found a way to remove that dll, not even sure how I succeeded. After disabling the dll and then removing it my Win10-64 just runs fine. And compiling fpc (32-bit) was OK again. Before I succeeded in that, I even wrote a simple program that altered packages\oracle\src\oraoci.pp so it would build correctly and the restore it after build (I was getting a bit desperate and frustrated).
This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

Reply by email

Bart
to J. Gareth Moreton
PM J. Gareth Moreton wrote
Yeah, that's the exact same file that I have problems with.

That file links to many more libs trhat I don't have, which does not create a problem when building (it would of course not run). It's just that you happen to have a common.dll in the wrong place that makes the process grind to a halt.
Notice that windows will allways find that dll, even if you set %path% to only the path of the compiler and make.

I would suggest trying to remove the dll.
Since it is used by the system you have to hunt down which service needs it. (That was the hard part IIRC. I think I also discussed this on the fpc-devel list, musty have been august this year, since around that time I created my "unlinkcommondll" utility as a workaround.) The shutdown and disable that service.
The reboot.
Then rename the dll (so that you can restore it in the unlikely event that your windows stops functioning).

Bart

Reply by email

Bart
to Bart
PM Bart wrote
I think I also discussed this on the fpc-devel list, musty have been august this year, since around that time I created my "unlinkcommondll" utility as a workaround.

Indeed:
Building trunk of today fails on Windows: Error: Invalid DLL C:\WINDOWS\system32\common.dll, invalid header size Sat, Aug 21, 11:35 PM
fpc-devel ML

Here's how I managed to fix it.
===
Finally good news.

In taskmanager I had to stop one process,then go to services, find xtu3service.exe, right-click, then "open services", find that service again (now called XTUOCDriverService) in the newly opened window, right-click, select properties, then set Startup type (? "Opstart type") to Disabled.

After a reboot I was able to rename that common.dll to _common.dll, rebooted again.
All this as administrator of course.

Finally after all that I was able to build fpc for 32-bit. ===

And if that does not work maybe this can help?

Looking with Windows Explorer at the Common.dll in question: Productname: Intel(R) Extreme Tuning Utility
The guide on how to remove this utility:
https://www.intel.com/content/www/us/en/support/articles/000032459/processors/processor-utilities-and-programs.html That unfortunately did not work for me, hence the solution above.

At least it may give you some clues where to start.

Bart

Reply by email

J. Gareth Moreton
to Bart

Of course, I'm not fussed if the patch is rejected.  It was more of compensation for something that isn't always the user's fault and tended to block "make all".

Gareth aka. Kit

Bart wrote 38 lines
On Tue, Nov 30, 2021 at 7:53 PM Bart wrote: > I think I also discussed this on the > fpc-devel list, musty have been august this year, since around that > time I created my "unlinkcommondll" utility as a workaround. Indeed: Building trunk of today fails on Windows: Error: Invalid DLL C:\WINDOWS\system32\common.dll, invalid header size Sat, Aug 21, 11:35 PM fpc-devel ML Here's how I managed to fix it. === Finally good news. In taskmanager I had to stop one process,then go to services, find xtu3service.exe, right-click, then "open services", find that service again (now called XTUOCDriverService) in the newly opened window, right-click, select properties, then set Startup type (? "Opstart type") to Disabled. After a reboot I was able to rename that common.dll to _common.dll, rebooted again. All this as administrator of course. Finally after all that I was able to build fpc for 32-bit. === And if that does not work maybe this can help? Looking with Windows Explorer at the Common.dll in question: Productname: Intel(R) Extreme Tuning Utility The guide on how to remove this utility: https://www.intel.com/content/www/us/en/support/articles/000032459/processors/processor-utilities-and-programs.html That unfortunately did not work for me, hence the solution above. At least it may give you some clues where to start.
This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

Reply by email

Michael Van Canneyt
to J. Gareth Moreton: Well, running the program on the target system isn't a guaranteed success even if there's…

Gareth,

I removed the $linklib for common.dll.

Michael.

J. Gareth Moreton wrote 73 lines
Well, running the program on the target system isn't a guaranteed success even if there's no error on the compilation system because the DLL in question may be different (or missing). If it turns out that common.dll is not required, then the maintainer of the package should probably remove the $linklib line, otherwise I'm not sure.  If I'm right in thinking, if the DLL is missing or incompatible, it won't read the exports table and so even if just a warning is raised, it may cause a cascade error if any of its functions are used in the unit because they're not declared. Gareth aka. Kit On 30/11/2021 16:22, Michael Van Canneyt via fpc-devel wrote: > > > On Tue, 30 Nov 2021, J. Gareth Moreton via fpc-devel wrote: > >> That was a conundrum I was trying to answer when making the patch.  >> What is a warning and what is an error? >> >> A lot of the verification code is dependent on compile-time constants >> whose values depend on the compiler's target platform, including >> header sizes, so any mismatch is an error case, and the checking of >> header sizes and the COFF magic number are done together, since the >> correct header size is dependent on the magic number's value. >> >> I didn't look too much further beyond the verification code, but if >> it's able to detect if a DLL is 32-bit on a 64-bit system and vice >> versa, I feel that's a warning rather than error because, in my >> experience, it's down to a default system configuration (I got the >> error on a freshly-installed machine) rather than something >> untoward.  If the DLL is missing, that should probably be a warning >> too and I'll see if I can make that change.  A corrupted DLL is >> definitely an error though. >> >> I didn't do a verification to see if the DLL is for AArch64, for >> example, since that situation is more unlikely.  It also starts to >> bloat the compiler code. >> >> Either way, using a DLL for a different build of Windows is a warning >> at the very least because the project will probably break when you >> try to run it. > > Personally, I think that reverting to a warning was a mistake. > > You have no guarantee that the resulting program will run if you > ignore the > "error": > > Not on the system where you compiled, not on the system where you will > run the program. > > For your particular problem: I looked at the source code of the oracle > unit. > As far as I can see there is no need to include the common.dll in the > linklib statements, all that should be linked is the oracle dll. It > may be a better solution simply to remove that line. > > Michael. > > _______________________________________________ > fpc-devel maillist - > https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel -- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus _______________________________________________ fpc-devel maillist - https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Reply by email

J. Gareth Moreton
to Michael Van Canneyt

As long as it doesn't break something in the package... that works, thanks!  That's been the bane of i386-win32 testing for ages!

Gareth aka. Kit

Michael Van Canneyt wrote 87 lines
Gareth, I removed the $linklib for common.dll. Michael. On Tue, 30 Nov 2021, J. Gareth Moreton via fpc-devel wrote: > Well, running the program on the target system isn't a guaranteed > success even if there's no error on the compilation system because > the DLL in question may be different (or missing). > > If it turns out that common.dll is not required, then the maintainer > of the package should probably remove the $linklib line, otherwise > I'm not sure.  If I'm right in thinking, if the DLL is missing or > incompatible, it won't read the exports table and so even if just a > warning is raised, it may cause a cascade error if any of its > functions are used in the unit because they're not declared. > > Gareth aka. Kit > > On 30/11/2021 16:22, Michael Van Canneyt via fpc-devel wrote: >> >> >> On Tue, 30 Nov 2021, J. Gareth Moreton via fpc-devel wrote: >> >>> That was a conundrum I was trying to answer when making the patch.  >>> What is a warning and what is an error? >>> >>> A lot of the verification code is dependent on compile-time >>> constants whose values depend on the compiler's target platform, >>> including header sizes, so any mismatch is an error case, and the >>> checking of header sizes and the COFF magic number are done >>> together, since the correct header size is dependent on the magic >>> number's value. >>> >>> I didn't look too much further beyond the verification code, but if >>> it's able to detect if a DLL is 32-bit on a 64-bit system and vice >>> versa, I feel that's a warning rather than error because, in my >>> experience, it's down to a default system configuration (I got the >>> error on a freshly-installed machine) rather than something >>> untoward.  If the DLL is missing, that should probably be a warning >>> too and I'll see if I can make that change.  A corrupted DLL is >>> definitely an error though. >>> >>> I didn't do a verification to see if the DLL is for AArch64, for >>> example, since that situation is more unlikely.  It also starts to >>> bloat the compiler code. >>> >>> Either way, using a DLL for a different build of Windows is a >>> warning at the very least because the project will probably break >>> when you try to run it. >> >> Personally, I think that reverting to a warning was a mistake. >> >> You have no guarantee that the resulting program will run if you >> ignore the >> "error": >> >> Not on the system where you compiled, not on the system where you >> will run the program. >> >> For your particular problem: I looked at the source code of the >> oracle unit. >> As far as I can see there is no need to include the common.dll in the >> linklib statements, all that should be linked is the oracle dll. It >> may be a better solution simply to remove that line. >> >> Michael. >> >> _______________________________________________ >> fpc-devel maillist  -  >> https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel > > -- > This email has been checked for viruses by Avast antivirus software. > https://www.avast.com/antivirus > > _______________________________________________ > fpc-devel maillist  -  > https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel > _______________________________________________ fpc-devel maillist - https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Reply by email