When building an application with Lazarus (r20153) using FPC (r13188) (on a Debian Lenny system) we are experiencing significant memory leaks. Earlier revisions of both Lazarus and FPC 2.3.1 experience the same problem.

Building and running our app with heaptrc enabled we find a very large number of blocks reporting memory allocated by fpc_dynarray_copy and fpc_dynarray_setlength as remaining allocated at application exit even though the variables to which the arrays are assigned either go out of scope, are reused or are destroyed at application exit (or before).

When the same application is built with with Delphi6, Kylix3 and Turbo Delphi there are no such memory leaks. With are building using FPC 2.3.1 with $MODE DELPHI.

Are we missing something here or might this be an FPC bug?

We'll file a bug report if necessary.

Many thanks for any advice!

Cheers, Bruce.

Bruce Tulloch wrote

Are we missing something here or might this be an FPC bug? We'll file a bug report if necessary.

A bug report with a compilable example program that demonstrates the problem would be helpful, thanks.

Jonas

PS: your message was held for moderation because you are not subscribed with this address to this list (it is subscribed to fpc- devel, though). If you're subscribed with another address, there's no problem. If you're not subscribed, it's best to mention this in mails you send, because otherwise people (other than moderators who saw that your message was held for moderation) won't add you in CC when replying. In any case, messages from this address to fpc-pascal will also be automatically accepted from now on.

Are we missing something here or might this be an FPC bug?

It depends on how you use HeapTrc. If via -gh, this might a bug. If you manually include in a uses clause, you have to be sure that you put it as the first one of your program. Otherwise, HeapTrc won't be able to detect what have been done by units loaded before it.

Thanks for that. We're using -gh. We're aware of the usage requirements of heaptrc.

The cause of this problem in our case is very subtle. So far we have been unable to reproduce it outside our (very large) application but we're working on isolating it in a smaller test program.

The problem looks like a dynamic array, created by Copy() inside an object's method, which is returned as the result of the method and assigned to a property (not a field) of another object (of a different class), is never released.

This is the case even when the property to which it's assigned, is assigned a new value later on (at which point by our accounting, the reference count of the first array should have dropped to zero).

We'll post more details or an example program when we have abstracted this problem from the rest of our application (in which application we have already eliminated all other known memory leaks using heaptrc).

Are we correct to assume that as soon as no variable in our program, be it global, static, object field or property, refers to a dynamic array, the dynamic array will be released. Are we also correct to assume that the reassignment of any variable referring to a dynamic array to a new value will cause the previous array value to be released (presuming nothing else refers to this array)?

Cheers, Bruce.

Yes.

One caveat is that the memory may not be immediately released after the last reference that you know of disappears, because there may still be temporary memory locations (implicitly created by the compiler while evaluating expressions) containing a reference. Such temporary locations will be finalised either when they are reused by the compiler later on, or when the function in which they were created by the compiler exits (such temporary locations can never survive the scope in which they were created by the compiler).

However, this could only cause the behaviour you are seeing if you were to call halt() somewhere in a deeply nested function and some temporary locations in a parent stack frame still contained references, or of you call halt() while some threads are still running that have local variables/temporaries that point to such variables.

Jonas

Ok, I don't think that caveat does not applies in this case, we neither call halt nor share these particular variables between threads (we do have threads in this application however). What we do see when freeing (ie, assigning to nil) some of these variables is a jump from recount = 1 (as we expect at this point in our application) to refcount = 7 on some blocks. There is no way (in our application) that there are 7 references to these blocks. We are investigating and report further as soon as we have some more concrete data to feed back to you.

Bruce.

PS: does the attribute "compilerproc" mean the function is defined to be called by the compiler implicitly? We have rebuilt the FPC RTL with debug enabled to further diagnose what's happening here (which is why we came across it).

It means that the procedure becomes invisible to regular Pascal code, and that it gets a particular internal symbol name. Such routines are indeed called implicitly by the compiler for various purposes.

Jonas

I think I've nailed and example of the problem; when Copy() is called with the result of another Copy() the reference count for the returned array is not managed correctly when compiled with -MDelphi.

A trivial example program, copytest.pas:

    program copytest;
    var
    S, D : array of Integer;
    begin
      SetLength(S,4000);
      D := Copy(Copy(S));
    end.

which when compiled with:

fpc -MDelphi -gl -gh copytest.pas

and run produces the following:

lenin:$ ./copytest
Heap dump by heaptrc unit
3 memory blocks allocated : 48024/48024
2 memory blocks freed : 32016/32016
1 unfreed memory blocks : 16008
True heap size : 262144
True free heap : 246048
Should be : 246072
Call trace for block $B7F4BF50 size 16008
$080480EA main, line 8 of copytest.pas
$080657A7

I hope this helps diagnose what's happening here.

Note that if compiled without -MDelphi it works correctly but in our case we need to use -MDelphi because our application is also built with Delphi and Kylix3.

Cheers, Bruce.

I've reported this as an FPC bug:

http://bugs.freepascal.org/view.php?id=13820

Cheers, Bruce.

Confirmed fixed in r13195 as tested with the application in which this was originally discovered, thanks Jonas.

Op dinsdag 26-05-2009 om 10:16 uur [tijdzone +1000], schreef Bruce Tulloch:

Confirmed fixed in r13195 as tested with the application in which this was originally discovered, thanks Jonas.

In that case, can you close the bug report? So we know that we don't have to look at this issue anymore?

Joost.

Hmm, I did. Apparently mantis did not register this? Done it again. Closed now. -b