Hi *,

Sending the message to this list, but basically hoping in some help / response from Rika. ;-)

Since quite some time ago, attempts for full FPC release building for the OS/2 target from trunk fail while building packages using fpmake, in particular while trying to build the IDE. According to what I found, it fails due to not being able to allocate more memory from the operating system. However, the memory available on that machine is not used completely at that point in time - according to what I found, it seems that I actually run out of "handles" used by the operating system for tracking memory allocations rather than the memory as such. The solution for this issue recommended in some programmer forum (and also a general recommendation in the OS/2 API documentation) suggests that developers should allocate bigger blocks and use a so-called sub-allocations for smaller memory blocks in order to avoid this situation.

Obviously, this is something similar to what the FPC heap manager does, but OS/2 provides dedicated API for this sub-allocation directly at the operating system level rather than in a runtime library of a particular programming language implementation - the primary API functions for memory allocation and freeing are DosAllocMem/DosFreeMem (see e.g. https://www.edm2.com/os2api/Dos/DosAllocMem.html / https://www.edm2.com/os2api/Dos/DosFreeMem.html), but there's also a pair of DosSubAllocMem / DosSubFreeMem for this sub-allocation (https://www.edm2.com/os2api/Dos/DosSubAllocMem.html / https://www.edm2.com/os2api/Dos/DosSubFreeMem.html) - the general mechanism is briefly described e.g. at
https://www.edm2.com/index.php/32-Bit_Memory_Management_in_OS/2#Heaps_for_Small_Allocations, the original IBM documentation of OS/2 API (not the latest version, but anyway) is available online at
https://archive.org/details/os-2-warp-control-program-api (search for DosSubAllocMem there). I also found information that all requests for memory allocations made using the "big" DosAllocMem are rounded up to a multiple of 64 kB and thus the allocation requests should be preferably made in multiples of 64 kB to avoid wasting the memory (well, it's a bit more complex than this, the rounding may be reduced to 4 kB, but the price for that is that some OS/2 APIs wouldn't be able to work with such memory blocks - let's consider the 64 kB size for simplicity).

Now my questions are:

1) What is the smallest memory block allocated by the current FPC memory manager using SysOSAlloc?

2) If I need to incorporate use of the sub-allocation concept into the OS/2 RTL, do you have an idea how to do that efficiently? I suspect that it would require conditional OS/2 specific changes within the common (cross-platform) routines of the FPC heap manager. That doesn't sound very nicely, but maybe you have an idea how to do that in some not that much obtrusive way?

Thanks in advance for any ideas

Tomas

— Google search doesn’t seem to back that up: they say that DosAllocMem is only limited by the address space (?).

— The compiler started eating noticeably more memory after some change related to ctask.pas; that might be the culprit. For another data point (and the exact ctask.pas commit):
https://gitlab.com/freepascal.org/fpc/source/-/work_items/41670.

— Does -dLEGACYHEAP help you? :) If anything, it *increases* the number of OS allocations (4× asymptotically).

— OS allocations are roughly RoundUp(min(used memory div 16, GrowHeapSize2), 64 KB). The default GrowHeapSize2 is 1 MB, which kicks in after ≈50 such allocations. You can tweak GrowHeapSize2 to some extent (this ability was inherited from oldheap.inc and doesn’t seem useful; I’ve even considered removing it entirely). OS allocations start at a natural minimum of 64 KB, which is not configurable; the idea is that programs and especially threads can be tiny, in which case they benefit from fitting everything into one, or a few, 64 KB OS chunks.

— Second commit of
https://gitlab.com/freepascal.org/fpc/source/-/merge_requests/1498 should slightly improve the situation through a dedicated design for 32-bit sizes that leverages 8-byte alignment.

On 2026-06-27 18:26, runewalsh via fpc-devel wrote:

Hi Rika,

Thanks for your response!

— Google search doesn’t seem to back that up: they say that DosAllocMem is only limited by the address space (?).

I didn't find anything like that in the official documentation either, but I found it mentioned in some discussion forum, IIRC.

— The compiler started eating noticeably more memory after some change related to ctask.pas; that might be the culprit. For another data point (and the exact ctask.pas commit): https://gitlab.com/freepascal.org/fpc/source/-/work_items/41670.

Yes, I understand that. I'm not sure if I can expect any decrease there, although that would be nice, of course. However, as mentioned previously, I monitored amount of available physical memory (as reported by the operating system) at the moment when the allocation failed and it didn't show all the memory having been allocated.

— Does -dLEGACYHEAP help you? :) If anything, it *increases* the number of OS allocations (4× asymptotically).

I'll check that and let you know.

Interesting, especially the part clarifying that the allocated blocks grow depending on the overall amount of memory already used (used memory div 16)... That might mean that at certain point in time, the operating system may not be able to allocate a continuous block of memory of the requested size (if the memory became fragmented by previous allocations and deallocations), although the overall amount of free memory is still much higher than the newly requested block. Do I assume correctly that the memory manager wouldn't try allocating a smaller block in such a case? I'm not sure about the likelihood of this really happening, just an idea...

In addition, thinking about it a bit more, I just noticed another potential reason of the misalignment between the overall memory still available and the memory allocation failing... Originally, OS/2 didn't allow allocation of more than 512 MB of RAM per process (that limit has its reasons, but those go beyond this post). Later OS/2 versions provided a solution for overcoming that limitation, but that solution has its own gotchas which is the reason why I don't use that by default within the OS/2 RTL. It's possible that the trunk compiler hits this limit nowadays. :-( If this is the case, I might try allocation of the "high memory" (> 512 MB address space) area and see if that helps... If it does, it isn't an optimal solution (certain OS/2 API functions wouldn't be able to work with memory blocks from this "high memory" :-( ), but I believe that this shouldn't affect the compiler itself (it might easily affect the text mode IDE, though :-((( ). But let's see what happens...

— Second commit of https://gitlab.com/freepascal.org/fpc/source/-/merge_requests/1498 should slightly improve the situation through a dedicated design for 32-bit sizes that leverages 8-byte alignment.

OK, thanks.

Tomas

Hi,

Yeah, this was also my first reaction, when you were talking about getting an an OOM, that you might run into the per process limit of 512MB address range (which is in practice even less "available memory" because every process has shared memory range too, and what else, which to my understanding also counts into the 512MB limit.

But regardless of what the actual limit is - to take hundreds of MB to compile a Free Vision app or the IDE sounds a bit excessive. Don't we have a memory leak somewhere? I used to be able to compile this stuff on an Amiga with 128MB RAM... I understand that the compiler gained quite some features since, so internal structures might have grown, but have they really grown this much?

Charlie

 Do I assume correctly that the memory manager wouldn't try allocating

a smaller block in such a case?

No, if allocation for a good-looking size fails, it retries with the minimal one:
https://gitlab.com/freepascal.org/fpc/source/-/blob/9936961be6474a674b061b79f8893a5b1e50af1e/rtl/inc/heap.inc#L1210-1215 (which shouldn’t matter much as you’re going to get an OOM soon anyway).

30. června 2026 17:54:55 SELČ, Karoly Balogh via fpc-devel napsal:

Hi Charlie,

Indeed, it's the same address space

I can't comment the potential leaks, but I'm afraid they have - in a comment attached to a recent commit, Rika mentioned just the compiler compilation requiring almost 500 MB. That's clearly not really the case yet (at least not on a 32-bit machine) because compiler cycling still succeeds under OS/2, but adding the IDE with all its units and dependencies to the requirements of the embedded compiler might possibly get beyond that point...

Tomas