Greetings! I've been using fcl-passrc (PParser and PasTree) to create a tool to help me with coverage analysis. Using the example provided in the package directory as a starting point, I've created a tool which finds the "begin" line number for each function/procedure definition and inserts a line of code there to help me keep track which functions are called and which aren't in my coverage tests. I've got it working great and it's giving me some really useful information.

I was looking at the idea of increasing my granularity by doing this for every code block rather than just once at the beginning of every function. However, as I'm descending down through the parse tree, I'm finding that most of the TPasImplBlock objects have empty SourceFilename values and SourceLinenumber is 0. TPasProcedure's TProcedureBody's TPasImplBlock has the source file/line number information set, but if I descend any lower than that the deeper TPasImplBlock do not.

Could someone with insight into this area give me any idea of why this is?

Thanks,

-SG

Oversight, a bug.

I am currently working on the parser, I'll look into this.

Michael.

Oversight, a bug. I am currently working on the parser, I'll look into this. Michael.

Thanks, I appreciate it.

-SG

Please check revision 21909.

I haven't done much testing, because I didn't get to the test cases for statements yet.

Michael.

Michael wrote

Please check revision 21909. I haven't done much testing, because I didn't get to the test cases for statements yet.

Michael, I grabbed what's currently up in the svn trunk under fcl-passrc.

I am getting the source filename and line number information now, so that's great.

I did notice one problem, though... when FreeAndNil'ing the TPasModule object returned from ParseSource, I am able to get it to segfault with a simple example.

I dummied it down into this source file:

==================================================
unit timelib;
interface
implementation
var
  elapsedT   : array [0..999] of int64;
  ic         : longword;
initialization
  for ic := low(elapsedT) to high(elapsedT) do begin
    elapsedT[ic] := 0;
  end;
end.
==================================================

If you run the test_parser program (which I actually had to tweak to get it to compile) with that little unit as input, you'll get a segfault when doing the FreeAndNil(M).

I'll let you know if i run into anything else.

Thanks,

-SG

I think I found another problem with the latest revision of fcl-passrc in svn trunk.

Given this source file as input to the test_parser program:

===================================
unit timelib;
interface
implementation
procedure SortIndexHashTable;
begin
  try
  except
    if (h  nil) then begin
    end;
  end;
end;
end.
===================================
I get this output:
===================================
$ ./test_parser timelib.pas
Unit timelib;
Interface
Implementation
Procedure SortIndexHashTable;
begin
 try
  except
   if (hnil) then
     begin
     end;
;
  except
   if (hnil) then
     begin
     end;
;
 end;
end;
end.
===================================

I think it's going down through the except block twice... I don't know if this is a problem with test_parser.pp itself, or with the package.

-SG

Seth Grover wrote

I think I found another problem with the latest revision of fcl-passrc in svn trunk.

I should have fixed both problems you found. Please test, rev 21922.

Michael.

I should have fixed both problems you found. Please test, rev 21922. Michael.

It did fix those problems, but there's still a problem in the release given the following unit with a finalization section:

===========================================
unit testlib;
interface
implementation
finalization
  if Assigned(hello) then begin
  end;
end.
===========================================

Same sort of issue, it segfaults in the FreeAndNil of the TPasModule.

The problem with the double-parsed code in the except block appears to be fixed now, though.

-SG

I ran into another problem scenario, dealing with $include'd files.

Given these two files:

==iputils.pas========================================================
unit iputils;
{$mode objfpc}{$H+}
interface
{$define IPv6_Types}
{$include ip.inc}
{$undef IPv6_Types}
{$define IPv6_Interface}
{$include ip.inc}
{$undef IPv6_Interface}
implementation
{$define IPv6_Implementation}
{$include ip.inc}
{$undef IPv6_Implementation}
end.
=====================================================================
==ip.inc=============================================================
{$IFDEF IPv6_Types}
type IPv6Addr = array[0..15] of byte;
type IPv4Addr = longword;
{$ENDIF}
{$IFDEF IPv6_Interface}
function IsIPv4CompatibleIPv6Addr(const ipv6 : IPv6Addr) : boolean;
{$ENDIF}
{$IFDEF IPv6_Implementation}
function IsIPv4CompatibleIPv6Addr(const ipv6 : IPv6Addr) : boolean;
begin
  result := false;
end;
{$ENDIF}
=====================================================================

I get an error like this:

Syntax error at token "EOF" in file iputils.pas at line 21 column -7202378 line:21 column:-7202378 file:iputils.pas
An unhandled exception occurred at $000000000046FCA1 : EParserError : Syntax error at token "EOF" in file iputils.pas at line 21 column -7202378
$000000000046FCA1
$0000000000475C0C
$0000000000474C83
$0000000000474C3B
$0000000000474396
$00000000004091A4 line 1913 of test_parser.pp

Thanks,

-SG

Also, it looks like it gets a segfault if you try to $include a file that can't be found.

-SG

Fixed. Revision 21929.

The problems you're experiencing are because the parser didn't free huge amounts of objects, creating massive memory leaks. I already fixed several issues as I test the various parts. So now from time to time, a problem pops up when something is freed twice.

Michael.

Fixed. Undefine was not working correctly. Added tests.

Revision 21931.

Michael.

Seth Grover wrote

Also, it looks like it gets a segfault if you try to $include a file that can't be found.

Fixed in rev. 21932. Uninitialized result.

Michael.

Michael, I pulled down your latest revision (21934) which did fix the problems I had reported. Thanks!

I did find another one for you:
========================================
unit timelib;

{$mode objfpc}{$H+}
interface
implementation
procedure SmonthToCmonth (const sMonth : array of char;
                          const stDex  : longword;
                          out   cVal   : longword);
begin
  case sMonth[stDex] of
    'J' : cVal := 0;
  end;
end;
end.
========================================

Segfaults in the same way as the others, on the release.

Thanks,

-SG

Works fine here.

Michael.

Works fine here. Michael.

Hm, that's odd. I'll do a clean pull and clean build and let you know. Thanks.

-SG

Hm, it's still failing for me.

Here's what I get with test_parser from examples given this input:

===================================================
unit timelib;
{$mode objfpc}{$H+}
interface
implementation
procedure SmonthToCmonth (const sMonth : array of char;
                          const stDex  : longword;
                          out   cVal   : longword);
begin
  case sMonth[stDex] of
    'J' : cVal := 0;
  end;
end;
end.
===================================================

Here's the program output:

$ ./test_parser timelib.pas
Unit timelib;
Interface
Implementation
Procedure SmonthToCmonth(const sMonth: Array of Char; const stDex:
LongWord; out cVal: LongWord);
begin
 case sMonth[stDex] of
   'J':  cVal:= 0;
 end;
end;

end.
An unhandled exception occurred at $0000000000000000 : EAccessViolation : Access violation
$0000000000000000
$0000000000480B72 line 1872 of pastree.pp
$00000000004854D3 line 2740 of pastree.pp
$0000000000419892
$0000000000480B72 line 1872 of pastree.pp
$0000000000485186 line 2703 of pastree.pp
$0000000000419892
$0000000000480B72 line 1872 of pastree.pp
$0000000000419892
$0000000000485001 line 2672 of pastree.pp
$0000000000419892
$00000000004800AB line 1747 of pastree.pp
$0000000000419892
$000000000047DF2E line 1425 of pastree.pp
$0000000000484D5D line 2652 of pastree.pp
$0000000000419892
$000000000047DFDB line 1436 of pastree.pp

Here's the GDB backtrace:

(gdb) bt
#0 0x0000000000000000 in ?? ()
#1 0x0000000000419892 in SYSTEM_TOBJECT_$__FREE () #2 0x00007ffff7f965c0 in ?? ()
#3 0x000000000047d819 in RELEASE (this=0x7ffff7f965c0) at pastree.pp:1332 #4 0x0000000000480b72 in DESTROY (this=0x7ffff7f96540, vmt=0x0) at pastree.pp:1872
#5 0x00000000004854d3 in DESTROY (this=0x7ffff7f96540, vmt=0x1) at pastree.pp:2740
#6 0x0000000000419892 in SYSTEM_TOBJECT_$__FREE () #7 0x00007ffff7f96458 in ?? ()
#8 0x000000000047d819 in RELEASE (this=0x7ffff7f96540) at pastree.pp:1332 #9 0x0000000000480b72 in DESTROY (this=0x7ffff7f964c0, vmt=0x0) at pastree.pp:1872
#10 0x0000000000485186 in DESTROY (this=0x7ffff7f964c0, vmt=0x1) at pastree.pp:2703
#11 0x0000000000419892 in SYSTEM_TOBJECT_$__FREE () #12 0x00007ffff7fa6620 in ?? ()
#13 0x000000000047d819 in RELEASE (this=0x7ffff7f964c0) at pastree.pp:1332 #14 0x0000000000480b72 in DESTROY (this=0x7ffff7f9e2e0, vmt=0x1) at pastree.pp:1872
#15 0x0000000000419892 in SYSTEM_TOBJECT_$__FREE () #16 0x0000000000000002 in ?? ()
#17 0x000000000047d819 in RELEASE (this=0x7ffff7f9e2e0) at pastree.pp:1332 #18 0x0000000000485001 in DESTROY (this=0x7ffff7f86040, vmt=0x1) at pastree.pp:2672
#19 0x0000000000419892 in SYSTEM_TOBJECT_$__FREE () #20 0x00007ffff7fa6400 in ?? ()
#21 0x000000000047d819 in RELEASE (this=0x7ffff7f86040) at pastree.pp:1332 #22 0x00000000004800ab in DESTROY (this=0x7ffff7f961c0, vmt=0x1) at pastree.pp:1747
#23 0x0000000000419892 in SYSTEM_TOBJECT_$__FREE () #24 0xffffffffffffffff in ?? ()
#25 0x000000000047d819 in RELEASE (this=0x7ffff7f961c0) at pastree.pp:1332 #26 0x000000000047df2e in DESTROY (this=0x7ffff7fb6180, vmt=0x0) at pastree.pp:1425
#27 0x0000000000484d5d in DESTROY (this=0x7ffff7fb6180, vmt=0x1) at pastree.pp:2652
#28 0x0000000000419892 in SYSTEM_TOBJECT_$__FREE () #29 0x00007fffffffe0d0 in ?? ()
#30 0x000000000047d819 in RELEASE (this=0x7ffff7fb6180) at pastree.pp:1332 #31 0x000000000047dfdb in DESTROY (this=0x7ffff7f96140, vmt=0x1) at pastree.pp:1436
#32 0x0000000000419892 in SYSTEM_TOBJECT_$__FREE () #33 0x00000000006dd268 in U_SYSTEM_OUTPUT ()
#34 0x000000000044369e in SYSUTILS_FREEANDNIL$formal () #35 0x00007fffffffe0d0 in ?? ()
#36 0x00000000004099fa in main () at test_parser.pp:1985

I'm doing it in Linux compiling for x86_64. There wouldn't be any problem with me using this package source (pastree, paswrite, pparser, pscanner) with FPC 2.6.0 would there? I am using FPC 2.6.0, but just compiling this one package from svn in trunk separately.

-SG

Seth Grover wrote

Hm, it's still failing for me. Here's what I get with test_parser from examples given this input:

[]

I'm doing it in Linux compiling for x86_64. There wouldn't be any problem with me using this package source (pastree, paswrite, pparser, pscanner) with FPC 2.6.0 would there? I am using FPC 2.6.0, but just compiling this one package from svn in trunk separately.

I can reproduce the problem with 2.7.1 on another machine, with a fresh checkout.

Seems like something didn't get committed. I will check.

Michael

Fixed.

It seems that using the heaptrc unit masks some access violations.

Michael.

Fixed. It seems that using the heaptrc unit masks some access violations. Michael.

Good to know. That fix seemed to work, but there's an issue with having an "else" in a case statement. If you remove the else it works.

==========================
unit timelib;
{$mode objfpc}{$H+}
interface
implementation
procedure testproc;
var
  i : integer = 0;
begin
  case i of
    0 : begin
    end;
  else
  end;
end;
end.
==========================

Probably a similar thing to the others you've fixed.

Thanks a lot. I hope this is helping. I'll keep running this over my source files until we've found them all. :)

-SG

Fixed, plus a couple of others as well. Revision 21942.

Thanks, it's a great help.

Michael.

Fixed, plus a couple of others as well. Revision 21942. Thanks, it's a great help. Michael.

I pulled the latest this morning and ran it over my whole project (about 140 units, 600,000-ish lines of code) and didn't get any more segfaults and it seems like all the information I'm pulling out of the parse tree is correct. I'll let you know if I run into anything else with it.

Thanks a lot!

-SG