Hi,

I'm working on a project where I parse source code, then use the token tree to modify the source code, and write new units out again.

I found a few limitation in fcl-passrc which prevents rebuilding source code from a token tree, but it was easy to fix by introducing a few new tokens. I'll submit a patch for this on Monday.

Anyway, while working with fcl-passrc I noticed something really strange. The token list is in reverse order! Why is that? So to insert for instance a new uses clause in the Interface section, I have to do something like this:

  for i := tokenlist.Count-1 downto 0 do
  begin
     if tokenlist[i].token = tkUses then
        ....
  end;

Otherwise it finds the Implemenation section's uses clause first. It's not hard to work with it like this (if you know about it), I just find it odd and wondered if there is any technical reason why the token tree is in reverse.

Ehm. Are you sure this is about fcl-passrc ?

Where do you see the token list ? I searched for tokenlist, but didn't see it anywhere ? The scanner doesnt produce a list, and the parser is private.

Michael.

Michael Van Canneyt wrote

Ehm. Are you sure this is about fcl-passrc ?

Yes, but I have to apologize. While creating a test app to show the problem, I found my problem in my own code. Inserting into my list, instead of Adding to my list! Dope!!! :-(

My stupid mistake. Sorry for the false alarm.

Where do you see the token list ? I searched for tokenlist, but didn't see it anywhere ? The scanner doesnt produce a list, and the parser is private.

I maintained my own list of tokens and values, so I can rebuild the source code later. Here is the bit of code I use to parse the files. If there is a better way of doing this, please let me know - I'm rather new to the fcl-passrc code and how to use it.

TPasTokenList is my own class that holds a list of tokens and values for later usage.

function TPasTokenList.ParseSource(AFileName: string): boolean;
var
  pas: TPascalScanner;
  fr: TFileResolver;
  token: TToken;
begin
  Result := False;
  fr := TFileResolver.Create;
  pas := TPascalScanner.Create(fr);
  try
    pas.OpenFile(AFileName);
    repeat
      token := pas.FetchToken;
      Add(token, pas.CurTokenString);
    until pas.CurToken = tkEOF;
    Result := True;
  finally
    pas.Free;
    fr.Free;
  end;
end;

BTW:
I used this code with a recursive directory scanner. One of my little test apps take a directory as a parameter, then recursively runs through files and outputs the source file being parsed and OK or FAIL if it managed to parse it. A nice and quick way to run the fcl-passrc parser through it's paces of all the FPC source code, or any of my own projects.