Hi all,

I'm trying to learn FreePascal by writing a file indexer a la locate and I'm hitting more walls after receiving quick and clear help from Michael Van Canneyt regarding memory leaks.

In the last couple of hours I've seen memory leaks cropping up in the main LPR file, invalid pointer problems (=> think I was too aggressive cleaning up memory), list index out of bounds, reading past end of file errors (faulty "improvement" of data types in exif code?).

1. Could some kind soul tell me what I've been doing wrong if possible? It's probably in the search as well as in the exif code. I'd love to start with something that I can understand and work up from there - at this moment I feel like I'm chasing smoke. The code is at:
http://bitbucket.org/jb/flocate/downloads/

... but I'd rather prefer getting better at figuring out what I did wrong, however, when setting some breakpoints on my Lazarus 0.9.29 r23972 FPC 2.4.1 x86_64-win64-win32/win64 Free Pascal Compiler version 2.4.1 [2010/03/13] for x86_64 system, every time the IDE would inform me that the debugger had crashed. I didn't see instructions for debugging with Lazarus x64 on Windows. (I've also been running the fpc compiler only on Linux to see the effects of the compiler conditionals).

2. Is there anything I need to do to get visual debugging working? Some project options? I've fiddled with dwarf output etc but am not sure what should be enable and disabled? Should I switch to Lazarus 32bit? Please see heaptrc dump as well as last part of my debug output below.

Seems like I still need to get some basic details about the language regardless of checking the fpc reference manual, Marco Cantu's Essential Pascal/Delphi and the Delphi basics website...

Thanks in advance,
jb

stdout message:
exception at 000000010003EFE3:
Read past end of file.

stderr output snipped:
Debug: 31-3-2010 19:38:37: starting exif info
for:C:\Windows\Microsoft.NET\Framework64\v2.0.50727\ASP.NETWebAdminFiles\Images\ASPdotNET_logo.jpg Heap dump by heaptrc unit
156639 memory blocks allocated : 13406733/14127240 156636 memory blocks freed : 13406173/14126672 3 unfreed memory blocks : 560
True heap size : 1343488 (224 used in System startup) True free heap : 1342208
Should be : 1342312
Call trace for block $00000000035662A0 size 432
$0000000100038E2D TFILESEARCH__SEARCHPERFILTER, line 659 of search.pp $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp $000000010003915B TFILESEARCH__SEARCH, line 714 of search.pp $0000000100001AF5 FLOCATE__DORUN, line 140 of flocate.lpr Call trace for block $00000000035CAEF0 size 35
$0000000100030659
$00000001000324B1
$0000000100032616
$0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp Call trace for block $00000000011A5340 size 93
$0000000100038CBB TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp $000000010003915B TFILESEARCH__SEARCH, line 714 of search.pp $0000000100001AF5 FLOCATE__DORUN, line 140 of flocate.lpr $0000000100034148

Hello FPC-Pascal,

Wednesday, March 31, 2010, 7:52:43 PM, you wrote:

J> I'm trying to learn FreePascal by writing a file indexer a la locate and J> I'm hitting more walls after receiving quick and clear help from Michael J> Van Canneyt regarding memory leaks.
[...]
J> Call trace for block $00000000035CAEF0 size 35
J> $0000000100030659
J> $00000001000324B1
J> $0000000100032616
J> $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp J> $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp J> $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp J> $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp J> $0000000100038CCF TFILESEARCH__SEARCHPERFILTER, line 638 of search.pp

Your code looks too tricky, specially the AddFileToResult. You are using records to store the item, in theAddFileToResult you assign an ansistring to one or more fields, in this process a reference to the ansistring is generated, and this references are quite sure your memory leaks in some way as the allocation point by heaptrc is in the RTL (no traceback).

Instead using records, use a class and instead TList use TObjectList, this way when you invoke Res.Free all objects inside will be freed also, or when you delete an object it is automagically freed.

Working with records allocated with New and adding them to a list is a complex task to keep the track of each one, specially if you have to mix different record types.

If you still want to use records, before the Dispose(APResultRecord) set the ansistring fields to '', this will garantee that the reference is freed.

Or call Finalize() on them.

Flávio

Thanks José & Flávio,

Sounds like a good plan, though I liked the beauty/simplicity of dealing with records - unless you forget that ansistrings also have pointers ;) I had completely forgotten about them and assumed the compiler would deal with them behind the scenes. Obviously it doesn't.

I'll give it a go and get back to the list if I hit more troubles.

All, any advice/tricks on debugging? I think I'll try setting up Lazarus in an Ubuntu VM to see how that works in any case. Maybe that works...

Regards,
jb

Dear José & Flávio,

Thanks for the tips; I"ve taken José's advice and replaced the records with objects. This does make things easier as long as you think object-oriented; I was suffering from exhaustion and a deja-vu with QuickBasic record structures ;)

I have to test the code a bit more to see if it falls over on large filesystems and if it works on OSX (and possibly Maemo/Nokia N900 - hmmm, would have to have a cross compiler setup for that ;).

After that, it's off to the databases: storing the results into a Firebird 'll probably start with prepared SQL
statements; later I might look into tiOPF.

Then (by Christmas ;) probably a Lazarus GUI for searching through the results, running scans etc.

The code is on

http://bitbucket.org/jb/flocate/

Comments are welcome - I'm still learning so there must be tons of things I could have done better - but thanks to the list it's at least working!

Regards,

jb