I am using custom build of minimal FPC RTL and found possible bug. When unit 'system' (or some other?) compiled using -O2 (or higher) option in 'fpc.cfg', then application, compiled using this unit, increase memory usage every time BeginThread() function is called. This appears on FPC 3.2.0 and not happens in earlier versions. Tested on official FPC binary for Windows, target platform - 'win32'/'win64'.

Here is complete test environment - minimal RTL sources, FPC executables and test project: http://dl.free.fr/ckmK7KVPX

Steps to reproduce:

1) Unpack archive

2) Execute script 'FPC\src\make_RTL_32A.bat' to build RTL. Current 'FPC\fpc.cfg' contains -O2 option enabled

3) Execute script 'compile.bat' to build test project.

4) Execute 'test.exe', then run Windows 'Task Manager' to view used memory. Test project repeatedly creates thread with 500ms sleep. Every time thread created, memory used by 'test.exe' is increased

Repeat steps 2-4 after comment out '-O2' option in 'FPC\fpc.cfg' or replacing FPC binary with provided 'FPC\ppc386 3.0.4 - WORK.exe' (or your own) - memory will remains stable.

If you have configured own RTL compilation - here is just test project without FPC binaries and RTL sources: http://dl.free.fr/nHDbSOzOQ Just change FPC path's in 'compile.bat'.

Please anyone confirm this behaviour.

Answer for my previous message
(https://lists.freepascal.org/pipermail/fpc-devel/2021-January/043512.html).

Memory leak happens when unit 'rtl\win32\sysinitpas.pp' compiled with "-O2" or higher.
To eliminate this bug add "-O-" or "-OoNOREGVAR" to command-line when compile this unit. This disables optimizations when enabled in 'fpc.cfg', e.g:

%FPC_COMMAND% -O- -Mobjfpc win32\sysinitpas.pp

or

%FPC_COMMAND% -OoNOREGVAR -Mobjfpc win32\sysinitpas.pp

Hmmm, I've seen regvar problems before.  Can you provide the assembly dumps of that file under -O2 and -O2 -OoNOREGVAR respectively? It might reveal something.

Gareth aka. Kit

You can simply compile this file by yourself using your existing FPC installation.
1) Get 'rtl\win32\sysinitpas.pp' and 'rtl\win32\sysinit.inc' from repository or sources archive.
2) Create build script ('compile.bat'):

REM Change this path's to your FPC installation, e.g.: set FPC_DIR=c:\FPC
set FPC_BINARY=%FPC_DIR%\bin\i386-win32\ppc386.exe set UNITS_DIR=%FPC_DIR%\units\i386-win32\*

"%FPC_BINARY%" -Fu"%UNITS_DIR%" -O2 -Mobjfpc "sysinitpas.pp"

=======
Here is test project that show bug: http://dl.free.fr/ckmK7KVPX Test project with full environment (compiler and minimal RTL): http://dl.free.fr/nHDbSOzOQ

(https://lists.freepascal.org/pipermail/fpc-devel/2021-January/043512.html).

> > Memory leak happens when unit 'rtl\win32\sysinitpas.pp' compiled with > "-O2" or higher. > To eliminate this bug add "-O-" or

"-OoNOREGVAR" to command-line when

Hello,

Your test program runs fine when compiled with FPC 3.2.0 using the official RTL compiled with -O2. No memory leak. I suppose the issue is caused by your modifications to RTL. -O2 (regvars) just triggers the problem in the modified RTL code which is hidden when regvars are disabled.
You need to debug the thread code in your modified RTL by yourself and find out what exactly causes the memory leak and why. If there is indeed a bug in the regvars implementation you need to create a simple test program which triggers the bug. This is a common practice for such issues. Only such way it can be fixed.

Yuriy Sydorov,

RTL not modified. Just recompilation of 'sysinitpas.pp' unit with some options causes problem.

Here is another test project that uses your current official FPC installation with unmodified RTL:
http://dl.free.fr/ePtkQwgJJ

Test project contains source of 'sysinitpas.pp' (and required 'sysinit.inc', 'systlsdir.inc').
Used version 3.2.2 RC1. If you have another - get sources for your version.

Script 'compile.bat' compiles 'sysinitpas.pp' and then - test project 'test.pp'.
See comments inside.

'test_bad.exe' shows growing memory consumption.

Hello, Your test program runs fine when compiled with FPC 3.2.0 using the

official RTL compiled with -O2. No memory leak.

I suppose the issue is caused by your modifications to RTL. -O2

(regvars) just triggers the problem in the modified RTL code which is hidden when regvars are disabled.

Hello,

I've take a look at you RTL compile script and found the mess with the compiler options.

When compiling RTL you should disable reading of options from fpc.cfg using the -n switch. When compiling the system unit use the -Us switch. Additionally specify only CPU/Target and optimization options -O<x>. No other options are required for Windows RTL.

When RTL is properly compiled, even with -O2, your test program works without memory leaks.

Yuriy Sydorov,

Thank you, i will try this.

I've take a look at you RTL compile script and found the mess with the

compiler options.

When compiling RTL you should disable reading of options from fpc.cfg

using the -n switch.

without memory leaks.

Yuriy Sydorov,

Unfortunately I tried it and this doesn't work.
Problem only with 'sysinitpas.pp' unit.
He is not depends on other RTL units, but linked into final executable by compiler.
I suppose, compiler itself depends on internal layout of this unit. And a solution is to compile 'sysinitpas.pp' with the same options as 'ppc386.exe' was compiled, not RTL.

See my previuos demo: http://dl.free.fr/ePtkQwgJJ

When compiling RTL you should disable reading of options from fpc.cfg

using the -n switch.

When compiling the system unit use the -Us switch. Additionally specify only CPU/Target and optimization options -O. > No

other options are required for Windows RTL.

When RTL is properly compiled, even with -O2, your test program works

without memory leaks.

Yuriy Sydorov,

Hello,

You have not read closely what I wrote before:

"When compiling RTL you should disable reading of options from fpc.cfg using the -n switch. When compiling the system unit use the -Us switch. Additionally specify only CPU/Target and optimization options -O<x>. No other options are required for Windows RTL."

sysinitpas.pp is the part of the RTL. So in your example it is needed to be compiled as:

"%FPC_BINARY%" -Fu"%UNITS_DIR%" -FU"%SCRIPT_DIR%units" -n -O2 "sysinitpas.pp"

-n ensures the default state of the compiler settings. This is the intended state for compiling the RTL.

RTL must be compiled in the FPC mode. No other modes must be specified in the command line or fpc.cfg. You can't first pass the -Mdelphi switch and then override it with -Mfpc. -Mdelphi implicitly enables lot of other compiler options which are not reverted by -Mfpc.

Yuriy Sydorov,

Hello, You have not read closely what I wrote before:

Sorry, earlier I try your hint - and result was bad. Perhaps I did something wrong.
But I repeat it now - and it is working. Thank you again.

Finally my working build script look like this:

set UNITS_PATH=.\packages\*
set UNITS_PATH=.\objpas;.\objpas\*;.\inc;.\win;%UNITS_PATH%;.\win32 set INC_PATH=.\inc;.\objpas;.\objpas\*;.\win;.\win\wininc;.\i386;.\win32 set OUTPUT_PATH=..\units32a

set FPC_COMMAND=..\ppc386.exe -n -O3 -Twin32 -FU%OUTPUT_PATH% -Fu%UNITS_PATH% -Fi%INC_PATH%

%FPC_COMMAND% -Us win32\system.pp
%FPC_COMMAND% win32\classes.pp
%FPC_COMMAND% packages\rtl-extra\winsock.pp
%FPC_COMMAND% packages\winunits-base\comobj.pp
%FPC_COMMAND% packages\winunits-base\shellapi.pp
%FPC_COMMAND% packages\winunits-base\wininet.pp

%FPC_COMMAND% win\messages.pp
%FPC_COMMAND% packages\winunits-base\richedit.pp
%FPC_COMMAND% packages\paszlib\paszlib.pas

%FPC_COMMAND% inc\fpintres.pp
%FPC_COMMAND% win32\sysinitpas.pp

Yuriy Sydorov wrote

You can't first pass the -Mdelphi switch and then override it with -Mfpc. -Mdelphi implicitly enables lot of other compiler options which are not reverted by -Mfpc.

That feels like a bug.

Jonas

I've double checked this and the mode change seems to work fine. The source of the issue is that specifying -Mfpc enables inlining which triggers a bug in the Windows RTL. I've fixed the bug in r49400.

Yuriy.

Yuriy Sydorov wrote

I've double checked this and the mode change seems to work fine. The source of the issue is that specifying -Mfpc enables inlining which triggers a bug in the Windows RTL. I've fixed the bug in r49400.

Thanks!

Jonas