Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2014-08

Heap manager tuning

3 messages · 2 people ·

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

denisgolovan

Hi all

I am trying to debug an issue that seems to be related to FPC heap manager. Test machine - Linux x64.

It looks like really large memory "overhead" compared to cmem manager. Test load profile - 50 heavy objects with around 25 large (1Mb - 10Mb) dynamic arrays each. Those arrays are created in threads.

That allocation takes 10Gb+ under FPC heap manager. Further allocation becomes hardly possible (I have 16Gb on-board). Meanwhile, cmem allocates around 3Gb. So I can operate >3 times more object set.

I am pretty sure I don't have any memory leaks (heaptr is enabled all the time).

Any suggestions?
I am willing to sacrifice some allocation performance to decrease total memory consumption if it is necessary.

Regards, Denis Golovan
Jonas Maebe
to denisgolovan
denisgolovan wrote 6 lines
That allocation takes 10Gb+ under FPC heap manager. Further allocation becomes hardly possible (I have 16Gb on-board). Meanwhile, cmem allocates around 3Gb. So I can operate >3 times more object set. I am pretty sure I don't have any memory leaks (heaptr is enabled all the time). Any suggestions?

Try avoiding growing memory blocks small amounts at a time via reallocmem or setlength, and instead grow them in bigger chuncks. This will both increase the performance of your program and reduce the internal memory fragmentation.

Jonas

denisgolovan
Show quoted text 8 lines
On 09/08/14 23:01, denisgolovan wrote: Try avoiding growing memory blocks small amounts at a time via reallocmem or setlength, and instead grow them in bigger chuncks. This will both increase the performance of your program and reduce the internal memory fragmentation. Jonas

I don't have that allocation pattern here. Just a lot of large arrays allocating/deallocating (including local arrays for temporary results). Any Java-like knobs to play with? :)

Regards, Denis Golovan