Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2011-12

possible json parser memory leak

7 messages · 4 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

ik

Hello,

I'm using FPC 2.6 (rc) x86_64 on Linux with JSONParser.

I've written the following code:

-----------------------
uses fpjson, jsonparser, SysUtils, classes;
var
  parser    : TJSONParser;
  json_file : TFileStream;
begin
  json_file := TFileStream.Create('/tmp/test.json', fmOpenRead);
  json_file.position := 0;
  parser := TJSONParser.create(json_file);
  writeln('JSON: ', parser.Parse.AsJSON);
  FreeAndNil(parser);
  FreeAndNil(json_file);
end.
---------------------

It works well, but compiling with -gh gives the following return: $ ./test_json
JSON: { "Hello" : "World", "TheAnswerForWorldUniverseAndEverything" : 42, "Array" : ["a", 1, 3.14000000000000E+000] }
Heap dump by heaptrc unit
70 memory blocks allocated : 7060/7208
53 memory blocks freed : 6274/6408
17 unfreed memory blocks : 786
True heap size : 458752
True free heap : 455296
Should be : 455776
Call trace for block $00007FA261D57660 size 16
$000000000042BB8C
$000000000042B974
$000000000042BEA7
$000000000042B94F
$000000000042BCF8
$000000000042B92A
$000000000042B663
$0000000000400180
Call trace for block $00007FA261D575C0 size 16
$000000000042BAED
$000000000042B974
$000000000042BEA7
$000000000042B94F
$000000000042BCF8
$000000000042B92A
$000000000042B663
$0000000000400180
Call trace for block $00007FA261D67E40 size 32
$000000000041637A
$000000000042B94F
$000000000042BCF8
$000000000042B92A
$000000000042B663
$0000000000400180
Call trace for block $00007FA261D57520 size 16
$000000000042B918
$000000000042BEA7
$000000000042B94F
$000000000042BCF8
$000000000042B92A
$000000000042B663
$0000000000400180
Call trace for block $00007FA261D67D80 size 18
$0000000000499691
$000000000042B94F
$000000000042BCF8
$000000000042B92A
$000000000042B663
$0000000000400180
Call trace for block $00007FA261D67CC0 size 24
$000000000047C0D8
$000000000047BE34
$0000000000428B13
$000000000042BE45
$000000000042B94F
$000000000042BCF8
$000000000042B92A
$000000000042B663
Call trace for block $00007FA261D67C00 size 24
$0000000000428B13
$000000000042BE45
$000000000042B94F
$000000000042BCF8
$000000000042B92A
$000000000042B663
$0000000000400180
Call trace for block $00007FA261D57480 size 16
$000000000042BE45
$000000000042B94F
$000000000042BCF8
$000000000042B92A
$000000000042B663
$0000000000400180
Call trace for block $00007FA261D57340 size 16
$000000000042BAED
$000000000042B974
$000000000042BCF8
$000000000042B92A
$000000000042B663
$0000000000400180
$0000000000400180
Call trace for block $00007FA261D5F520 size 70
$000000000041637A
$000000000042B92A
$000000000042B663
$0000000000400180
Call trace for block $00007FA261D67B40 size 20
$000000000041637A
$000000000042B92A
$000000000042B663
$0000000000400180
Call trace for block $00007FA261D470C0 size 384
$000000000041637A
$000000000042B92A
$000000000042B663
$0000000000400180
Call trace for block $00007FA261D573E0 size 16
$000000000042B918
$000000000042BCF8
$000000000042B92A
$000000000042B663
$0000000000400180
Call trace for block $00007FA261D67A80 size 22
$0000000000499691
$000000000042BCF8
$000000000042B92A
$000000000042B663
$0000000000400180
Call trace for block $00007FA261D5F440 size 56
$000000000047EA32
$000000000042A53B
$000000000042BC43
$000000000042B92A
$000000000042B663
$0000000000400180
Call trace for block $00007FA261D67900 size 24
$000000000042A53B
$000000000042BC43
$000000000042B92A
$000000000042B663
$0000000000400180
Call trace for block $00007FA261D572A0 size 16
$000000000042BC43
$000000000042B92A
$000000000042B663
$0000000000400180

Is that a memory leak in the library, or am I missing something here ? Thanks,
Ido

Reply by email

silvioprog
to ik

2011/12/18 ik :

Show quoted text 5 lines
Hello, I'm using FPC 2.6 (rc) x86_64 on Linux with JSONParser. I've written the following code:

Use:

var
  parser: TJSONParser;
  json_data: TJSONData;
  json_file: TFileStream;
begin
  json_file := TFileStream.Create('test.json', fmOpenRead);
  parser := TJSONParser.Create(json_file);
  json_data := parser.Parse;
  WriteLn(json_data.AsJSON);
  json_data.Free;
  json_file.Free;
  parser.Free;
end;

Thx.

Silvio Clécio =============================================== Blog - <silvioprog.com.br> Twitter - <twitter.com/silvioprog> Facebook - <facebook.com/silvioclecio> LazSolutions - <code.google.com/p/lazsolutions> Lazarus-BR - <groups.google.com.br/group/lazarus-br?hl=pt-BR> ===============================================    * Conheça nosso canal IRC sobre Lazarus: #lazarus-br * ===============================================

Reply by email

Michael Van Canneyt
to ik: I'm using FPC 2.6 (rc) x86_64 on Linux with JSONParser. I've written the following code:…
ik wrote 18 lines
Hello, I'm using FPC 2.6 (rc) x86_64 on Linux with JSONParser. I've written the following code: ----------------------- uses fpjson, jsonparser, SysUtils, classes; var   parser    : TJSONParser;   json_file : TFileStream;  begin   json_file := TFileStream.Create('/tmp/test.json', fmOpenRead);   json_file.position := 0;   parser := TJSONParser.create(json_file);   writeln('JSON: ', parser.Parse.AsJSON);

You are not freeing the result of the parse operation ?

The user is responsible for freeing the result of .parse()

So you must do

Var
   D: TJSONData;
    parser := TJSONParser.create(json_file);
    D:=Parser.parse;
    writeln('JSON: ', D.AsJSON);
    D.free; // Free result of parse.

And while you're at it, you must always use try/finally blocks. Otherwise any exception (an error in the json) will result in your code not freeing the file stream or parser instance.

Michael.

Reply by email

ik
to silvioprog: 2011/12/18 ik : Use: var parser: TJSONParser; json_data: TJSONData; json_file:…
silvioprog wrote 25 lines
2011/12/18 ik : > Hello, > > I'm using FPC 2.6 (rc) x86_64 on Linux with JSONParser. > > I've written the following code: Use: var parser: TJSONParser; json_data: TJSONData; json_file: TFileStream; begin json_file := TFileStream.Create('test.json', fmOpenRead); parser := TJSONParser.Create(json_file); json_data := parser.Parse; WriteLn(json_data.AsJSON); json_data.Free; json_file.Free; parser.Free; end; Thx.

Thanks, that works.

Show quoted text 16 lines
-- Silvio Clécio =============================================== Blog - <silvioprog.com.br> Twitter - <twitter.com/silvioprog> Facebook - <facebook.com/silvioclecio> LazSolutions - <code.google.com/p/lazsolutions> Lazarus-BR - <groups.google.com.br/group/lazarus-br?hl=pt-BR> =============================================== * Conheça nosso canal IRC sobre Lazarus: #lazarus-br * =============================================== _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/mailman/listinfo/fpc-pascal

Reply by email

Frank Church
to silvioprog: 2011/12/18 ik : Use: var parser: TJSONParser; json_data: TJSONData; json_file:…
silvioprog wrote 26 lines
2011/12/18 ik : > Hello, > > I'm using FPC 2.6 (rc) x86_64 on Linux with JSONParser. > > I've written the following code: Use: var parser: TJSONParser; json_data: TJSONData; json_file: TFileStream; begin json_file := TFileStream.Create('test.json', fmOpenRead); parser := TJSONParser.Create(json_file); json_data := parser.Parse; WriteLn(json_data.AsJSON); json_data.Free; json_file.Free; parser.Free; end; Thx.

I am getting some issues with memory leaks and want to see if this topic sheds some light on the issue.

From the original posters question it appears that parse.Parse.AsJSON

create an object which needed to be free although there was no explicit Txxx.Create in the code.

writeln('JSON: ', parser.Parse.AsJSON);

Does this represent a common case where using AsXXX creates an object?

Is there some way of telling whether some types of functions create objects which need to be freed even although they do not offer handles which can be used to free them?

It looks to me that lot of leaky code is created which looks right unless you know more about the functions used create objects..

I am using the XML libraries and beginning to suspect that lots of this kind of problem are lurking within the code I am writing.

PS. Does every 'Call trace xxx' represent an object created that wasn't freed? I asked about this in another thread but haven't gotten a reply yet

Show quoted text 15 lines
-- Silvio Clécio =============================================== Blog - <silvioprog.com.br> Twitter - <twitter.com/silvioprog> Facebook - <facebook.com/silvioclecio> LazSolutions - <code.google.com/p/lazsolutions> Lazarus-BR - <groups.google.com.br/group/lazarus-br?hl=pt-BR> =============================================== * Conheça nosso canal IRC sobre Lazarus: #lazarus-br * =============================================== _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Frank Church ======================= http://devblog.brahmancreations.com

Reply by email

ik
to Frank Church
Frank Church wrote 40 lines
On 18 December 2011 17:34, silvioprog wrote: > 2011/12/18 ik : > > Hello, > > > > I'm using FPC 2.6 (rc) x86_64 on Linux with JSONParser. > > > > I've written the following code: > > Use: > > var > parser: TJSONParser; > json_data: TJSONData; > json_file: TFileStream; > begin > json_file := TFileStream.Create('test.json', fmOpenRead); > parser := TJSONParser.Create(json_file); > json_data := parser.Parse; > WriteLn(json_data.AsJSON); > json_data.Free; > json_file.Free; > parser.Free; > end; > > Thx. > > I am getting some issues with memory leaks and want to see if this topic sheds some light on the issue. From the original posters question it appears that parse.Parse.AsJSON create an object which needed to be free although there was no explicit Txxx.Create in the code. >writeln('JSON: ', parser.Parse.AsJSON); Does this represent a common case where using AsXXX creates an object?

Nop, AsJSON is a function with string. Parse is function that return a class. The thing is, that the function is designed to not to manage memory. It can not actually. If you wanted a memory manager it should have been property and not a function.

Show quoted text 5 lines
Is there some way of telling whether some types of functions create objects which need to be freed even although they do not offer handles which can be used to free them?

I think most of them, if not all. Because you can not manage memory that you do not own. When you return a created object and send it as a result. Then it can not follow it anymore.

It looks to me that lot of leaky code is created which looks right unless you know more about the functions used create objects..

The memory leak was in my code, due to usage of the library, not in the library. But only after I saw the answer, I noticed it was a function and not a property.

Show quoted text 39 lines
I am using the XML libraries and beginning to suspect that lots of this kind of problem are lurking within the code I am writing. PS. Does every 'Call trace xxx' represent an object created that wasn't freed? I asked about this in another thread but haven't gotten a reply yet > -- > Silvio Clécio > =============================================== > Blog - <silvioprog.com.br> > Twitter - <twitter.com/silvioprog> > Facebook - <facebook.com/silvioclecio> > LazSolutions - <code.google.com/p/lazsolutions> > Lazarus-BR - <groups.google.com.br/group/lazarus-br?hl=pt-BR> > =============================================== > * Conheça nosso canal IRC sobre Lazarus: #lazarus-br * > =============================================== > _______________________________________________ > fpc-pascal maillist - > http://lists.freepascal.org/mailman/listinfo/fpc-pascal > -- Frank Church ======================= http://devblog.brahmancreations.com _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/mailman/listinfo/fpc-pascal

Reply by email

ik
to Michael Van Canneyt: You are not freeing the result of the parse operation ? The user is responsible for…
Michael Van Canneyt wrote 42 lines
On Sun, 18 Dec 2011, ik wrote: Hello, > > I'm using FPC 2.6 (rc) x86_64 on Linux with JSONParser. > > I've written the following code: > > ----------------------- > uses fpjson, jsonparser, SysUtils, classes; > > var > parser : TJSONParser; > json_file : TFileStream; > > begin > json_file := TFileStream.Create('/tmp/test.**json', fmOpenRead); > json_file.position := 0; > parser := TJSONParser.create(json_file); > writeln('JSON: ', parser.Parse.AsJSON); > You are not freeing the result of the parse operation ? The user is responsible for freeing the result of .parse() So you must do Var D: TJSONData; parser := TJSONParser.create(json_file); D:=Parser.parse; writeln('JSON: ', D.AsJSON); D.free; // Free result of parse. And while you're at it, you must always use try/finally blocks. Otherwise any exception (an error in the json) will result in your code not freeing the file stream or parser instance.

The fruit of this questions:
http://translate.google.com/translate?sl=auto&tl=en&js=n&prev=_t&hl=en&ie=UTF-8&layout=2&eotf=1&u=http%3A%2F%2Fidkn.wordpress.com%2F2011%2F12%2F19%2Fworking-withh-json-using-fpc%2F&act=url:)

They translated very good but not perfect as you'll see. And they destroyed the source code there. I might upload it to github or something.

Show quoted text 6 lines
Michael. _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/mailman/listinfo/fpc-pascal

Ido

Reply by email