More tests with the parser but it seems like it doesn’t know quite a bit of syntax. Are these not implemented yet?

1) class operators for records

EParserError: Expected "procedure" at token "operator" in file /Users/ryanjoseph/Developer/Projects/FPC/GLCanvas/GLCanvas.pas at line 115 column 7

class operator TVertex3.= (constref a, b: TVertex3): boolean;
begin
  result := (@a = @b);
end;

2) For..in loops

EParserError: Expected := or in at token "(" in file /Users/ryanjoseph/Developer/Projects/FPC/NewEngine/Sources/Examples/EmptyWindow.pas at line 138 column 14
  for pointer(entity) in entities do
    begin
      entity.Update;
      entity.Draw(renderer);
      renderer.PushBox(entity.GetHitBox, TRGBA.RedColor);
    end;

3) {$i settings} will not find the file “settings.inc”. This is valid in FPC but the parser seems to have other rules.

Regards,
Ryan Joseph

This can be.

For in loops should be supported, but I've never seen this before:

for pointer(entity) in entities do

What is for pointer(entity) in entities supposed to do ?

It can be that this syntax is not supported (no surprise, since I don't have a clue what this is supposed to do).

They should be the same.

As said, pas2js uses fcl-passrc, and pas2js handles inifiles just as FPC does.

You must provide valid include path and file resolvers, of course.

Michael.

It's a little early to conclude this. The code supports class operators. But see below.

I didn't know you could do this in a for in loop. FPC eats this ?

Not yet.

I have no idea how you call the parser. Include files most definitely work: the documentation generator uses fcl-passrc, and the RTL/FCL is full of include files.

Probably your invocation of the parser is simply wrong or missing some arguments.

Again, that works. These constructs can be found in the RTL/FCL and the documentation generator parses that (and so does pas2js).

Disregarding the typecast in the for..in, concluding that there are bugs is very premature. All constructs you mention are supported.

So the question is why you are getting errors for constructs that are supported ?

Did you specify the advanced record modeswitch in the sources ? Do you specify delphi mode when calling the parser ?

But the parser must be invoked correctly. If you don't invoke it correctly, then of course you can expect it to report errors. If you use the default 'ParseSource' function, that definitely does not cover all possible cases as handled by FPC. It will not observe whatever you have in .fpc.cfg or /etc/fpc.cfg etc.

Also, are you using trunk or 3.0.4 ? There have been a massive amount of updates to fcl-passrc.

Michael.

Michael Van Canneyt schrieb am So., 13. Okt. 2019, 08:20:

It currently does, but it might as well be a bug in FPC. I'll have to cross check with Delphi.

Could the missing ".inc" be the problem? I have not checked, but it could be that FPC always adds ".inc" if no extension is present.

Regards,
Sven

Delphi does not allow it:

[dcc32 Error] Project1.dpr(18): E1019 For loop control variable must be simple local variable

I don't think this should be allowed, either.

Well spotted. I didn't notice that the extension was missing.

That will definitely not be done in fcl-passrc.

I was not aware FPC does this either. It's not documented, that's for sure:

https://www.freepascal.org/docs-html/current/prog/progsu40.html

Delphi also does not look for .inc files, I checked. It only adds extension .pas

So, conclusion: FPC is way too lenient. ;-)

Michael.

Same.

This is how FPC looks for an include file:

=== code begin ===

            { try to find the file }
            found:=findincludefile(path,name,foundfile);
            if (not found) and (ExtractFileExt(name)='') then
             begin
               { try default extensions .inc , .pp and .pas }
               if (not found) then
found:=findincludefile(path,ChangeFileExt(name,'.inc'),foundfile);
               if (not found) then
found:=findincludefile(path,ChangeFileExt(name,sourceext),foundfile);
               if (not found) then
found:=findincludefile(path,ChangeFileExt(name,pasext),foundfile);
             end;

=== code end ===

Regards,
Sven

To be clear:

here I meant that 'currently that will definitely not be done in fcl-passrc.'

Thanks, I will adapt the mechanism in pas2js.

Michael.

This is a problem then because I wanted to make a language server for FPC but if the parser doesn’t understand accepted FPC syntax the whole thing falls apart. Can you make an FPC complient mode or something then? IMO if the parser included with the compiler doesn’t understand the same syntax as the compiler then we’re in trouble.

Here’s basically what I’m doing to parse. Nothing special and the engine is just a template that does just the minimum (like the examples in the package).

module := ParseSource(engine, ’bugs.pas', 'darwin', 'x86_64');

- This is a new version from 3 days ago.
- The record sections failed because they are inside a function. If I pull the record out of the function they work.
- Please test the operator because I don’t think the parser understands it.

{$mode objfpc}
{$modeswitch advancedrecords}
program bugs;
type
  TMyRecordA = record
    class operator = (constref a, b: TMyRecordA): boolean;
  end;
class operator TMyRecordA.= (constref a, b: TMyRecordA): boolean;
begin
  result := (@a = @b);
end;
procedure DoThis;
  type
    TMyRecordB = record
      private
        x, y, z: integer;
    end;
  begin
  end;
begin
end.

Regards,
Ryan Joseph

Why not?

I just had this problem on 9/26, search for “for-in loop cast”. What if I have a list of TObjects I want to iterate? I know TMyClass is safe to use but the compiler complains so I override it with a typecast. If you take that away I’ll have to do tons of casting outside of the loop.

var
  arr: specialize TFPGObjectList<TObject>;
  obj: TMyClass;
begin
  arr := specialize TFPGObjectList<TObject>.Create;
  arr.Add(TObject.Create);
  // Incompatible types: got "TObject" expected "TMyClass"
  for obj in arr do
    begin
    end;

Regards,
Ryan Joseph

Ryan Joseph schrieb am So., 13. Okt. 2019, 16:22:

Then specialize the list with the correct type.

This issue *will* be fixed and is not up to discussion.

Regards,
Sven

It should understand the same syntax, 100%.
If it does not, that's definitely a bug.

The missing extension you can easily fix by explicitly adding it.

But of course I will add it to fcl-passrc, I simply was not aware of the .inc. The documentation also must be adapted for this.

Here’s basically what I’m doing to parse. Nothing special and the engine is just a template that does just the minimum (like the examples in the package). module := ParseSource(engine, ’bugs.pas', 'darwin', 'x86_64');

At first sight, you need to do at least

module := ParseSource(engine, ’-Mdelphi bugs.pas', 'darwin', 'x86_64');

Because for the language constructs you are using you need mode delphi. (unless you have the necessary $mode and $modeswitch statements in code)

The parser by default does not enable mode delphi or objfpc for that matter, so you need to specify that on the 'command-line'.

- This is a new version from 3 days ago. - The record sections failed because they are inside a function. If I pull the record out of the function they work.

Ahah...

That needs to be tested then. This seems to be a bug. Can you please create a bug with a compilable example ?

- Please test the operator because I don’t think the parser understands it.

OK, that's strange, because I can see the code for handling operators, and operators are definitely parsed, the RTL/FCL are riddled with operators...

Then this needs to be tested and possibly fixed too. Please report this also as a bug with a testcase, so I don't forget.

I'll try to fix them ASAP.

Michael.

sed s/bug/bugreport/

Michael.

I’m out of time today but I’m make some examples later. Where do you want me to post them? Sorry I don’t get "sed s/bug/bugreport/“.

heres’ the list:

1) allow omission of extension for $include macro
2) record class operators
3) record sections in records nested in routines

Regards,
Ryan Joseph

Please post them in the bugtracker, one issue per problem.

so I expect 3 issues :-) I don't know if you can assign them to someone as reporter, but if you can you can assign them to me (Michael Van Canneyt)

As for "sed":

sed s/bug/bugreport/ : sed is the Stream EDitor command-line tool. the expression s/bug/bugreport/ substitutes the first occurrence of 'bug' with 'bugreport'

I wrote 'bug' when I meant 'bugreport'

Michael.

Here’s some code from the parser today. TFPList is being used because it’s a generic storage for various different kinds of classes.

  public
    Declarations: TFPList; // list of TPasElement
    // Declarations contains all the following:
    Attributes, // TPasAttributes
    Classes,    // TPasClassType, TPasRecordType
    Consts,     // TPasConst
    ExportSymbols,// TPasExportSymbol
    Functions,  // TPasProcedure
    Properties, // TPasProperty
    ResStrings, // TPasResString
    Types,      // TPasType, except TPasClassType, TPasRecordType
    Variables   // TPasVariable, not descendants
      : TFPList;

What do I do when I want to iterate over this now? I know that these types all descend from TPasElement so I want to use that, but no, the compiler complains I *must* use a pointer for the iterator. Why can’t I as the programmer tell the for loop I know what the enumerator is going to return?

var
  element: TPasElement;
begin
  // ERROR!
  for element in module.InterfaceSection.Functions do
    begin
    end;

I guess we’re supposed to just do our typecasts inside the loop now but how is this any better? Sorry I’m really not getting this.

var
  element: pointer;
begin
  for element in module.InterfaceSection.Functions do
    begin
      writeln(TPasElement(element).Name, ‘’, TPasElement(element).DoSomething);
    end;

Regards,
Ryan Joseph

Because the iterator returns a Pointer and not whatever you think it might return. If you know it's something else, you cast that inside the loop.

It's not about better or not, it's about correct: The for-in-loop expects a variable, not an expression, just like the normal for-loop does. And as this behavior is accidental it could just as well be that this breaks behind the scene. For example if we should optimize for-in some time in the future. Or it might trigger some runtime checks.

Regards,
Sven

[...] Because the iterator returns a Pointer and not whatever you think it might return. If you know it's something else, you cast that inside the loop.

I agree with Sven.

What about the other way round?

For example this is allowed: TControl(aButton):=aControl;
This works also when passing something to a "var" argument.

It would be consistent to allow it as the for-in-loop variable.

Mattias

That's a good way to shoot yourself in the foot.
If aControl is a TEdit, for example, this will also pass.

This works also when passing something to a "var" argument.

That's like giving someone 2 guns to shoot himself in the feet simultaneously.

No, seriously:

That's a really bad kludge because for good reasons, you can't pass descendents as a var argument, and unfortunately the typecast is sometimes used to work around this.

This is also an excellent way to shoot yourself in the foot.

Consider the following:

Procedure DoSomething(var A : TControl);
Var
  N : String;
begin
   if Assigned(a) then
      N:=A.Name;
   A:=TEdit.Create(Nil);
   A.Name:=N;
end;
var
   B : TButton;
begin
   DoSomething(TControl(B));
   // And now B contains a TEdit, despite the declaration
end;

The typecast is something you should not do in this case.

/* Cultural note: this is the reason FreeAndNil has an untyped argument :( */

It would be consistent to allow it as the for-in-loop variable.

As far as I know, you're (rightly) a proponent of not allowing people op shoot themselves in the foot ?

The typecast is something that should be discouraged, not encouraged.

So no, I don't think we should allow this.

Michael.

Mattias Gaertner via fpc-pascal schrieb
am Mo., 14. Okt. 2019, 10:39:

First you agree with me and then you provide in essence the same examples as Ryan? So what is it now?

Anyway as I wrote to Ryan: the for-in loop is defined as "for VAR in EXPR do", not "for EXPR1 in EXPR2 do". And even if we would limit it to typecasts then the next one would come and ask for this or that to be allowed as well.

No. Nothing of that. There is a variable in the for-in-loop and nothing more. Anything else currently is a bug and *will* break.

Regards,
Sven

Ryan gave several examples.
I agreed with you in the case of no-typecast:

var
  element: TPasElement;
begin
  for element in module.InterfaceSection.Functions do
You cannot do "element:=module.InterfaceSection.Functions[0]", so it
is consistent that the for-in does not allow it neither.

OTOH if there is a lossless typecast, then it could theoretically be treated similar to an assign and thus would be consistent - as in: it is not new concept for a Pascal programmer.

Anyway as I wrote to Ryan: the for-in loop is defined as "for VAR in EXPR do",

True in Delphi.
Not yet in FPC.

not "for EXPR1 in EXPR2 do".

The assignment operator is *not* defined as Expr1 := Expr2 .

And even if we would limit it to typecasts then the next one would come and ask for this or that to be allowed as well.

True, for example properties.

No. Nothing of that. There is a variable in the for-in-loop and nothing more. Anything else currently is a bug and *will* break.

Sure. I got the impression that Ryan proposed a consistent extension and got surprisingly strong rejections.

Mattias

On Oct 14, 2019, at 8:02 AM, Sven Barth via fpc-pascal wrote: No. Nothing of that. There is a variable in the for-in-loop and nothing more. Anything else currently is a bug and *will* break.

I don’t disagree with anything you’re saying but this is going to make for-in loops much harder to work with. Pascal isn’t like C++, C#, Python etc… where there are “auto” vars we can use for the iterator. Since I don’t want to typecast every single time I access the iterator I’m probably going to make a temporary iterator and then assign it to the real variable inside the loop.

var
  it: pointer;
  obj: TObject;
begin
  for it in list do
    begin
      obj := TObject(it);
      // continue on like before using “obj” instead of “it"
    end;

That’s our realistic best use case now but it requires 2 extra steps. I hope there’s a better solution to keep the for-in loops as easy to use as before.

Regards,
Ryan Joseph

Hi,

you could do

var
   it: pointer;
   obj: TObject absolute it;
begin
   for it in list do
     begin
       // continue on like before using “obj” instead of “it"
     end;

Cheers,
Benito

That's probably the best/easiest suggestion.

Michael.

That is a clever solution but it’s basically a sneaky trick around the compilers excessively strict typing. The programmer knows the correct type but there’s no way to tell that to the for loop.

Regards,
Ryan Joseph

Ehm... I can hardly believe my eyes...

Of course there is: make the list the correct type. No typecasting needed then.

One could argue that it's your own "laziness" in correct typing that forces you to use "tricks" further down the line.

Then blaming the need for these tricks on the compiler's 'excessively strict typing' is simply the world turned upside down.

Michael.

On Oct 15, 2019, at 6:13 PM, Michael Van Canneyt wrote: Of course there is: make the list the correct type. No typecasting needed then.

I guess I’ll get used to it but we have lots of code like this (snippet I posted earlier from the parser):

   Declarations: TFPList; // list of TPasElement
   // Declarations contains all the following:
   Attributes, // TPasAttributes
   Classes,    // TPasClassType, TPasRecordType
   Consts,     // TPasConst
   ExportSymbols,// TPasExportSymbol
   Functions,  // TPasProcedure
   Properties, // TPasProperty
   ResStrings, // TPasResString
   Types,      // TPasType, except TPasClassType, TPasRecordType
   Variables   // TPasVariable, not descendants
     : TFPList;

where we have generic lists that we cast later to get the correct type. All these now are going to require new specializations for each class and changing the declarations. In fact this means using TFPList is not really a good idea at all anymore unless you don’t plan to use for..in loops.

Regards,
Ryan Joseph

Typecasting is a "sneaky trick around the compilers excessively strict typing" (sic!) as well. I personally use "absolute" very often when I want neither typecasts nor another temporary variable to clutter my code.

Regards,
Sven