Hi,

I'd like to know why one of my programs pircsrv written in FPC uses up so much memory. 107Mb of virtual memory (so it seems) and 7Mb of data. Here is the top stats...

top - 08:37:16 up 1 day, 18:20, 2 users, load average: 0.21, 0.15, 0.05
Tasks: 79 total, 1 running, 78 sleeping, 0 stopped, 0 zombie Cpu(s): 1.3% user, 0.4% system, 0.0% nice, 98.3% idle Mem: 126608k total, 104848k used, 21760k free, 10712k buffers Swap: 562424k total, 143360k used, 419064k free, 48740k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ Command 421 root 5 -10 62112 9560 8404 S 0.0 7.6 23:38.23 XFree86 2700 pircsrv 9 0 107m 7196 7184 S 0.0 5.7 0:57.98 pircsrv <------- this one! 23733 prologic 9 0 3632 3628 2896 S 0.0 2.9 0:00.20 gaim 301 root 9 0 4992 2800 2792 S 0.0 2.2 0:04.21 xfs 2720 prologic 9 0 3432 2784 2648 S 0.0 2.2 0:04.45 xchat 2287 nobody 9 0 3464 2540 2540 S 0.0 2.0 0:00.90 httpd 2337 prologic 8 0 2672 2420 2068 S 0.0 1.9 0:08.38 WindowMaker 23857 prologic 9 0 2260 2260 1756 S 0.0 1.8 0:00.30

This is the compilation line in the Makefile ...

debug: $(pircsrv)
fpc -bl -gg -gc -gl -S2 $(pircsrv)

The source is 764Kb, 67 source files, 18103 lines.

cheers
James

Try compiling with -ghl to check for memory leaks.

I'd like to know why one of my programs pircsrv written in FPC uses up so much memory. 107Mb of virtual memory (so it seems) and 7Mb of data. Here is the top stats...

Two possibilities:
- memory leaks. Proceed as Peter already explained.
- heap fragmentation. Use unit cmem, or if you use a lot of small same size reallocations, pool them and reuse (often even faster)

The second needs some explaining. The default FPC heapmanager never releases memory back to the system. Of course it tries to reuse the memory if possible, but sometimes with weird allocation schemes this can lead to a race in long running programs. (like daemons)

All this is because of performance reasons. Memory allocation is close to being the slowest thing a program can do itself (calling the OS is of course slower).

The cmem unit links in libc, and uses the C primary memory allocations routines. Note that if you use libc anyway for some reason, it will also avoid having two heaps. (a libc and a fpc-rtl one)

The libc memory manager is much slower, but has less problems with strange memory allocation patterns. It is also probably a bit more secure.

In general keep using the FPC memory manager, unless this kind of behaviour happens (usually only in daemon programs that allocate/deallocate memory all the time), and then either switch to libc, or try to fix the odd memory patterns. (e.g. by pooling)

I have. I do not fully understand the output though when the program exits. Still reading the documentation on the heaprtl

cheers
James

James Mills wrote

> Try compiling with -ghl to check for memory leaks. I have. I do not fully understand the output though when the program exits. Still reading the documentation on the heaprtl

It gives you a list of memory blocks that you allocated but never released, along with a backtrace that shows where those blocks were allocated.

Jonas

But a 100Mb odd memory leak ? How is this possible, my program only has a handlefull of getMem and reAllocMem calls throughout it.

Not releasing memory back to the system, isn't this bad ?

James

James Mills wrote

> - memory leaks. Proceed as Peter already explained. But a 100Mb odd memory leak ? How is this possible, my program only has a handlefull of getMem and reAllocMem calls throughout it.

That could explain it, reallocmem() could in certain cases cause this behaviour in 1.0.6 and earlier. Have you already tried a 1.0.7 snapshot? It should be fixed.

Jonas

But never released. So if the program released all memory then it wouldn't have any backtrace lines in the output log ?

Another question (I'm not an expert on linux :P), it outputs after the program exits. If mine goes into the background,
ie: ./pircsrv > log.out &
Would this be correct to get the ghl output ?
./pircsrv > log.out & 2>mem.txt

Also I'd like to see an example of a proper Makefile for FPC written programs, as I think most of my units are not compiled with debugging info...

thank you :)

James

Ooooh, darnnit well that might explain a few things. I've never used snapshots or installed FPC from the source. I use Debian, did use 1.0.4 then upgraded to SID and it had 1.0.6 .

When do you expect to upload a newer version into the Debian repositories ? If it's a long way down the track I'll remove 1.0.6 and install the latest snapshot instead.

cheers
James

We can't upload a release to debian. The debian maintainer is responsable for that. See the Changelog.debian.gz for his email. I tried to contact him a few months ago but got no reply :(

Also I'd like to see an example of a proper Makefile for FPC written programs, as I think most of my units are not compiled with debugging info...

You can use fpcmake to generate a Makefile. See the manuals for more information

Okay. Thanks. Been writing them by hand :P

cheers
James

Wasn't sure how it worked really. I'll contact him. I tried installing from the sources once, but failed. Is it hard to do so, I assume it has instructions ? And do you have to have FPC in order to compile from the source ?

cheers
James

James Mills wrote

But never released. So if the program released all memory then it wouldn't have any backtrace lines in the output log ?

Indeed. This is regardless the reallocmem inefficiency I mentioned before (ie. upgrading will not make any memory leaks you have now go away).

I think it should be

./pircsrv > log.out 2>mem.txt &

Jonas

The FPC heap manager does pooling.

k, just confirming the output of the heap trace unit.

ta

cheers
James