Hi,

This is a odd one, but I'll try and explain as best I can.

This relates to the *.lpk packages in the tiOPF project. These packages haven't changed for years. But in recent months they sometimes fail to compile. I say sometimes, because it is very strange. On my native OSX and FreeBSD systems the 'tiopflcl.lpk' package fails to compile, but in my Linux and Windows VM that exact same package compiles without problems. Same version of FPC and tiOPF.

I first thought it was related to the Build Modes & "Additions and Overrides" changes, but the Linux VM that works is using a Lazarus v1.5 from 3 weeks ago.

I have two packages in the tiOPF project:

tiOPF.lpk // GUI independent code
tiOPFLCL.lpk // units that require LCL

Each package has a different unit output directory, and the second package depends on the first package. What I find very weird (and consider wrong) is that in my unit output directories I have duplicate compiled units (units found in the first package). I would have expected the second package to reuse the already compiled units from the first package. The compiler options are matched exactly between packages.

ls ./Compilers/FPC/lib/x86_64-freebsd/lcl
./Compilers/FPC/lib/x86_64-freebsd -1 | sort | uniq -c | sort

2 dzlib.o
2 dzlib.ppu
2 tiCompressZLib.o
2 tiCompressZLib.ppu
2 tiDataBuffer_BOM.o
2 tiDataBuffer_BOM.ppu
2 tiDataBuffer_Cli.o
2 tiDataBuffer_Cli.ppu
2 tiQueryCSV.o
2 tiQueryCSV.ppu
2 tiQueryDataset.o
2 tiQueryDataset.ppu
2 tiQuerySqldbIB.o
2 tiQuerySqldbIB.ppu
2 tiQuerySqldb.o
2 tiQuerySqldb.ppu
2 tiQueryTAB.o
2 tiQueryTAB.ppu
2 tiQueryTXTAbs.o
2 tiQueryTXTAbs.ppu
2 tiQueryXMLLight.o
2 tiQueryXMLLight.ppu
2 tiXML.o
2 tiXML.ppu
2 tiXMLToTIDataset.o
2 tiXMLToTIDataset.ppu

So why the duplicate units? Obviously this is what causes the checksum changed issue and ultimately makes FPC go pear shaped. But more puzzling is why this confusion happens on some systems, but not on others.

I opened each package and clicked the "Show Options" button. Here are the compiler settings for each package. As you can see, the second package does include the unit output directory from the first package (Compilers/FPC/lib/x86_64-freebsd), but why does it still decide to recompile some units (duplicates shown above). I can also add that the duplicates are only a subset of units found in the first package.

***** tiOPF *****
-MObjFPC -Scai -Cg -O1 -g -gl -l -vewnhibq -vm5024 -Fi/data/devel/Wisa/fpcpackages/tiopf/Core
-Fu/data/devel/Wisa/fpcpackages/tiopf/Core
-Fu/data/devel/Wisa/fpcpackages/tiopf/Options
-Fu/data/devel/Wisa/fpcpackages/tiopf/Compilers/FPC/ -FU/data/devel/Wisa/fpcpackages/tiopf/Compilers/FPC/lib/x86_64-freebsd/ -dLINK_CSV
-dLINK_TAB
-dLINK_FBLx
-dLINK_SQLDB_IB
-dLINK_SQLDB_PQx
-dLINK_SQLDB_Oraclex
-dLINK_SQLDB_SQLite3x
-dLINK_SQLDB_ODBCx
-dLINK_SQLDB_MySQL40x
-dLINK_SQLDB_MySQL41x
-dLINK_SQLDB_MySQL50x
-dLINK_XMLLIGHT
-dLINK_ZEOS_FBx
-dUseCThreads

***** tiOPF+LCL *****
-MObjFPC -Scai -Cg -O1 -g -gl -l -vewnhibq
-Fi/data/devel/Wisa/fpcpackages/tiopf/Core
-Fu/data/devel/Wisa/fpcpackages/tiopf/Options
-Fu/data/devel/Wisa/fpcpackages/tiopf/GUI
-Fu/data/devel/Wisa/fpcpackages/tiopf/GUI/LCL
-Fu/data/devel/lazarus/packager/units/x86_64-freebsd -Fu/data/devel/Wisa/fpcpackages/tiopf/Compilers/FPC/lib/x86_64-freebsd -Fu/data/devel/lazarus/components/lazutils/lib/x86_64-freebsd -Fu/data/devel/lazarus/lcl/units/x86_64-freebsd
-Fu/data/devel/lazarus/lcl/units/x86_64-freebsd/gtk2 -Fu/data/devel/Wisa/fpcpackages/tiopf/Compilers/FPC/ -FU/data/devel/Wisa/fpcpackages/tiopf/Compilers/FPC/lib/x86_64-freebsd/lcl/ -dLCL
-dLCLgtk2
-dUseCThreads

Does anybody know what could be going wrong here? In help would be very much appreciated. As a work around I used IDE Macros in my project, instead of Lazarus Packages, then my project compiles just fine. The other alternative is to create a new package just for my project that contains all required tiOPF units in a single package, and not use the packages included with tiOPF. Both workarounds are not ideal.

The problem doesn't occur just on my system. In the last week, somebody else also reported this issue in the tiOPF newsgroups. They use Linux. So the issue isn't even specific to OSX or FreeBSD either.

If you want to have a try at compiling these packages yourself, you can grab a copy of the code via Git, or download a snapshot via SourceForge.

git clone git://git.code.sf.net/p/tiopf/code tiopf

OR

via your web browser:
http://sourceforge.net/p/tiopf/code/ci/master/tarball

Regards,
- Graeme -

Graeme Geldenhuys wrote

This is a odd one, but I'll try and explain as best I can.

I can also add that I have cleared out all compiled units between tests. This made no difference on my failing systems.

Regards,
- Graeme -

Graeme Geldenhuys wrote

This is a odd one, but I'll try and explain as best I can.

One more piece of information.... my Lazarus versions:

Lazarus version | Compile Result ----------------------------+---------------------- Linux v1.5 r48595 | pass
OSX v1.4 | fail
FreeBSD v1.5 r49362 | fail

So as you can see my Linux setup worked, but OSX (older) and FreeBSD (newer) doesn't. It's all very inconsistent. All setups use the released FPC 2.6.4 compiler and latest tiOPF.

Regards,
- Graeme -

If you let the compiler choose between source and ppu file it often chooses source, because source rulez.

Make sure your packages have their own separate unit output directories (separate from sources and from other packages) and that the second package unit paths do not include any directories of the first package. Lazarus adds the unit output directory automatically. A normal package has only a "Unit output directory" and all other paths are empty.

Maybe a compiler bug.

The above path is shared. If it contains source files, remove it.

And that one.

It's on my todo list to generate warnings, if packages share source directories.
It is a common problem with "unit not found" messages, that users starts *adding* search paths, instead of cleaning up.

Agreed.

Mattias

Hi Mattias,

Many thanks for your response.

Mattias Gaertner wrote

If you let the compiler choose between source and ppu file it often chooses source, because source rulez.

That's what I assumed is happening.

Make sure your packages have their own separate unit output directories (separate from sources and from other packages)

That is already the case. The first packages outputs to:

lib/$(TargetCPU)-$(TargetOS)/

and the second package outputs to:

lib/$(TargetCPU)-$(TargetOS)/lcl

and that the second package unit paths do not include any directories of the first package.

This is a problem.

1) All units in tiOPF share a common tiDefines.inc include file that controls some behaviour of the framework and defines new compiler defines used throughout the framework. This include file lives in the <tiopf>/Core/ directory. So the second package needs that path to compile the tiOPF+LCL units.

2) The second package uses a unit tiDataset.pas (VCL and LCL specific) that is located in the <tiopf>/Options/ directory. So Lazarus automatically added the <tiopf>/Options/ directory to the unit search path otherwise it couldn't find that unit.

Any suggestions?

The Options directory contains optional units not required by the core framework, but often used. It contains things like alternative persistence layer for various database backends, alternative ID generators etc. The first package (eg: alternative persistence layers) compiles these units based on the LINK_xxx compiler defines found in the Other compiler options. So the first package needs the Options unit search path setting to know where to find these if they are enabled.

I don't specify them, Lazarus must be adding them somewhere. They don't appear in the "Compiler Options -> Paths" dialog.

Regards,
- Graeme -

Try in the IDE

Run->Clean up and Build

Often works for me with this sort of problem.

Tony

Sharing include files is ok.
Don't share unit paths.

Put often used units in a package. Share the ppu files, not the sources.

Then either it is a package output directory or the project added them via "Additions and Overrides".
Click on Compiler Options / Show Options / Inherited parameters to find out.

Mattias

Tony Whyman wrote

Run->Clean up and Build Often works for me with this sort of problem.

Thank you for the suggestion, but no that didn't help. I already did a similar thing with 'git clean -f -d' which removes everything that isn't committed.

Regards,
- Graeme -