Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2021-04

Re: Nested function closures

22 messages · 8 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Ryan Joseph
Show quoted text 14 lines
On Apr 27, 2021, at 9:58 AM, Michael Van Canneyt wrote: Wait. I asked Sven to make sure that nested functions are under ALL circumstances usable as closures or can be used instead of anonymous functions. Pas2js already supports this, and I want FPC and Pas2JS to be compatible in this regard. So as Sven wrote, you would be duplicating effort, needlessly, since it has to work always... If the compiler can decide that the heap interface is not needed and optimize it away: so much the better. But I doubt this will be possible.

Maybe we're misunderstanding each other then. I'm responding to Sven when he says:

Getting rid of the interface only works in very narrow circumstances that are so seldom in real world code that it is not worth the effort.

I.e. all closures will be interface based because there are not enough real world use cases to support any alternative. Which is contrary to what you're saying that nested functions/closure will be compatible types. Sven can clear this up for us I guess.

Regards,
Ryan Joseph

Michael Van Canneyt
to Ryan Joseph
Ryan Joseph wrote 25 lines
> On Apr 27, 2021, at 9:58 AM, Michael Van Canneyt wrote: > > Wait. > > I asked Sven to make sure that nested functions are under ALL circumstances > usable as closures or can be used instead of anonymous functions. > > Pas2js already supports this, and I want FPC and Pas2JS to be compatible in > this regard. > > So as Sven wrote, you would be duplicating effort, needlessly, since it has > to work always... If the compiler can decide that the heap interface is not > needed and optimize it away: so much the better. But I doubt this will be > possible. Maybe we're misunderstanding each other then. I'm responding to Sven when he says: > Getting rid of the interface only works in very narrow circumstances that are so seldom in real world code that it is not worth the effort. I.e. all closures will be interface based because there are not enough real world use cases to support any alternative. Which is contrary to what you're saying that nested functions/closure will be compatible types.

Why is this contrary ?

As I understand it:

a closure is used for arguments of type 'reference to procedure'.

In delphi, this procedure can be a real method or an anonymous function.

So why should you not be able to use a local function instead of an anonymous function ?

How the closure is implemented internally is irrelevant from the programmer's point of view. As long as the signature is correct, presumably the compiler can figure out what needs to be done: create an interface (with all that implies) or not.

Michael.

Sven Barth
to Michael Van Canneyt
Michael Van Canneyt wrote 54 lines
On Tue, 27 Apr 2021, Ryan Joseph via fpc-pascal wrote: > > >> On Apr 27, 2021, at 9:58 AM, Michael Van Canneyt >> wrote: >> >> Wait. >> >> I asked Sven to make sure that nested functions are under ALL >> circumstances >> usable as closures or can be used instead of anonymous functions. >> >> Pas2js already supports this, and I want FPC and Pas2JS to be >> compatible in >> this regard. >> >> So as Sven wrote, you would be duplicating effort, needlessly, since >> it has >> to work always... If the compiler can decide that the heap interface >> is not >> needed and optimize it away: so much the better. But I doubt this >> will be >> possible. > > Maybe we're misunderstanding each other then. I'm responding to Sven > when he says: > >> Getting rid of the interface only works in very narrow circumstances >> that are so seldom in real world code that it is not worth the effort. > > > I.e.  all closures will be interface based because there are not enough > real world use cases to support any alternative.  Which is contrary to > what you're saying that nested functions/closure will be compatible > types. Why is this contrary ? As I understand it: a closure is used for arguments of type 'reference to procedure'. In delphi, this procedure can be a real method or an anonymous function. So why should you not be able to use a local function instead of an anonymous function ? How the closure is implemented internally is irrelevant from the programmer's point of view. As long as the signature is correct, presumably the compiler can figure out what needs to be done: create an interface (with all that implies) or not.

It will *always* create an interface. It's just how the compiler will wrap it.

Regards,
Sven

PS: And who was the one who switched to fpc-pascal? O.ó

Ryan Joseph
to Sven Barth
On Apr 27, 2021, at 2:17 PM, Sven Barth via fpc-pascal wrote: It will *always* create an interface. It's just how the compiler will wrap it.

But why would it do that when we could use an alternate code path that uses nested functions instead? Maybe I'm not being clear but we can do BOTH depending the situation when one is better than the other. This is just an optimization that leverages existing features that are already in the compiler anyways. Seems like a no-brainer to me.

Regards,
Ryan Joseph

Ryan Joseph
to Ryan Joseph
On Apr 27, 2021, at 2:23 PM, Ryan Joseph wrote: But why would it do that when we could use an alternate code path that uses nested functions instead? Maybe I'm not being clear but we can do BOTH depending the situation when one is better than the other. This is just an optimization that leverages existing features that are already in the compiler anyways. Seems like a no-brainer to me.

Here's what I'm proposing. Closures and nested functions should be symmetrical so that anonymous function bodies can be used without relying on the interfaces when they're not needed.

type
  TComparator = procedure (a, b: TEntity): Integer is nested;
procedure TMyClass.SortEntities(comparator: TComparator);
begin
end;
for i := 0 to entities.Count - 1 do
   begin
     value := entities[i];
     value.SortEntities(function(a, b: TEntity): integer
       begin
         // do stuff
       end
     );
   end;

Regards,
Ryan Joseph

Graeme Geldenhuys
to Ryan Joseph
Ryan Joseph wrote 5 lines
value.SortEntities(function(a, b: TEntity): integer begin // do stuff end );

It seem the beginning of the thread is missing, but I would like to comment on something here - purely based on the example code given.

The SortEntities() is defined with a signature of type TComparator, which in turn is a function that takes 2 arguments of type TEntity and returns a integer.

Why must the anonymous function repeat all that information again, as shown in the quoted code above? Can't the compiler figure all that out simply by inference? All the developer should have to do is give the 2 parameters a name, and a function body (block of code). Thus you could end up with something like this:

     value.SortEntities((a, b) ->
       begin
         // do stuff with a and b, then
         // return some integer.
       end
     );

Yes, I stole the "->" symbol from another language simply to separate the parameters from the function body. Replace that with any Pascal-like symbol.

Regards,
Graeme

Ryan Joseph
to Graeme Geldenhuys
Show quoted text 15 lines
On Apr 27, 2021, at 4:53 PM, Graeme Geldenhuys via fpc-pascal wrote: Why must the anonymous function repeat all that information again, as shown in the quoted code above? Can't the compiler figure all that out simply by inference? All the developer should have to do is give the 2 parameters a name, and a function body (block of code). Thus you could end up with something like this: value.SortEntities((a, b) -> begin // do stuff with a and b, then // return some integer. end );

I agree it's verbose but the syntax is already decided upon and implemented already.

Regards,
Ryan Joseph

Sven Barth
to Graeme Geldenhuys: It seem the beginning of the thread is missing, but I would like to comment on something…
Graeme Geldenhuys wrote 18 lines
On 27/04/2021 10:13 pm, Ryan Joseph via fpc-pascal wrote: > value.SortEntities(function(a, b: TEntity): integer > begin > // do stuff > end > ); It seem the beginning of the thread is missing, but I would like to comment on something here - purely based on the example code given. The SortEntities() is defined with a signature of type TComparator, which in turn is a function that takes 2 arguments of type TEntity and returns a integer. Why must the anonymous function repeat all that information again, as shown in the quoted code above? Can't the compiler figure all that out simply by inference? All the developer should have to do is give the 2 parameters a name, and a function body (block of code). Thus you could end up with something like this:

First: Pascal does not support type inference which would be required for this.

Second: the syntax is required for Delphi compatibility anyway - and the main idea *is* to be able to use libraries like Spring4D or OmniThreadLibrary in FPC so I don't want *anyone* to whine about this, thus it makes no sense to introduce a new syntax (as much as I'd love to as well).

Regards,
Sven

Ryan Joseph
On Apr 27, 2021, at 11:36 PM, Sven Barth wrote: Anyway, it would in principle be possible to convert an anonymous function to a "is nested" function, but that will only come *after* the whole implementation is here so that the chance for messing that core functionality (!) up is reduced.

Sorry I'm struggling with all these new terms myself. Yes that is what I'm proposing and what I demonstrated in that GitHub branch. We talked about this in 2018 but you said to wait until the "real thing" was finished so I let it be. I brought this up again now to say that when, and not before, the closures are finished and in trunk I will see how to integrate the "nested anonymous functions" into the new system.

To clarify for myself, we have a few concepts here:

- Closures: blocks which capture variables in scope and wrap them into a container (nested functions use a record I think and closures will use an interface).
- Anonymous procedures: Simply an anonymous procedure/function body which lacks a name and can be declared inside code blocks.
- 2 Procedure types: "reference to" and "is nested". On the caller side these are what control which kind of closure is used is used. Is that correct? So the anonymous function doesn't really become a closure until it's passed to a "reference to" or "is nested" variable.

So once the current work is done we have 2 kinds of closures but only one of those is compatible with anonymous functions and this is why I want the other closures (nested functions) to have the same pair of functionality.

Regards,
Ryan Joseph

Graeme Geldenhuys
to Sven Barth: First: Pascal does not support type inference which would be required for this. Second:…

Hello Sven,

Sven Barth wrote
Second: the syntax is required for Delphi compatibility anyway

Couldn't such verbose syntax be limited to {$mode delphi} behaviour, and then leave {$mode objfpc} free to experiment and introduce new less verbose syntax in the language.

Granted I fully understand that that would have the negative effect that many thing would have to be implemented twice. It just feels like FPC can't progress much with the Object Pascal language, because everything must be "Delphi compatible". :-( This result in FPC always being one step behind, and Embarcadero doesn't always make the best decisions for the language either.

[Please don't take this as a rant - it's not intended that way. It's just me being late to the party and asking some questions out of curiosity.]

Regards,
Graeme

Martin Frb
to Graeme Geldenhuys
Graeme Geldenhuys wrote 7 lines
Hello Sven, On 28/04/2021 6:32 am, Sven Barth via fpc-pascal wrote: > Second: the syntax is required for Delphi compatibility anyway Couldn't such verbose syntax be limited to {$mode delphi} behaviour, and then leave {$mode objfpc} free to experiment and introduce new less verbose syntax in the language.

Would omitting the type info not lead to issues with overloaded functions?

foo(function(a,b)

Foo could have lots of overloaded arguments, all taking a callback with 2 params.
Or it does not have them yet, but later.

Ryan Joseph
to Graeme Geldenhuys: Couldn't such verbose syntax be limited to {$mode delphi} behaviour, and then leave…
Show quoted text 5 lines
On Apr 28, 2021, at 10:43 AM, Graeme Geldenhuys via fpc-pascal wrote: Couldn't such verbose syntax be limited to {$mode delphi} behaviour, and then leave {$mode objfpc} free to experiment and introduce new less verbose syntax in the language.

Sven is having none of this and for good reason. This feature has been collecting dust as "almost done" for years now so I would consider ourselves lucky to get , Ryan Joseph

Graeme Geldenhuys
to Martin Frb: Would omitting the type info not lead to issues with overloaded functions?…
Martin Frb wrote
Would omitting the type info not lead to issues with overloaded functions?

Luckily others have already solved that problem. :-) Here is Java's JSR-355 and overloading is covered in Section F.

https://jcp.org/aboutJava/communityprocess/final/jsr335/index.html

The Java compiler does a three step processed to find the best match. Alternatively, the developer could also list the argument types to help out.

So to extend your example, the optional type could be added by the developer.

foo(function(a,b: integer))
foo(function(a,b: string))
foo(function(a: integer, b: string))

But even the Java developers found cases that were really hard or impossible to determine by the compiler alone, and opted to forgo overloading and change the interface. For example:

fooInteger(function(a,b))
fooString(function(a,b))

Not ideal, but more as a last resort.

Regards,
Graeme

Sven Barth
to Graeme Geldenhuys: Couldn't such verbose syntax be limited to {$mode delphi} behaviour, and then leave…

Graeme Geldenhuys via fpc-pascal schrieb
am Mi., 28. Apr. 2021, 19:00:

Show quoted text 16 lines
Hello Sven, On 28/04/2021 6:32 am, Sven Barth via fpc-pascal wrote: > Second: the syntax is required for Delphi compatibility anyway Couldn't such verbose syntax be limited to {$mode delphi} behaviour, and then leave {$mode objfpc} free to experiment and introduce new less verbose syntax in the language. Granted I fully understand that that would have the negative effect that many thing would have to be implemented twice. It just feels like FPC can't progress much with the Object Pascal language, because everything must be "Delphi compatible". :-( This result in FPC always being one step behind, and Embarcadero doesn't always make the best decisions for the language either.

You completely ignored my first point, which in this case is the much more significant one: Pascal does not support type inference.

It is impossible to implement a syntax like you're suggesting without major rework of the parser. That is something we're not willing to do with only the possible and in the grand scheme of things very minor benefit of a slimmer syntax.

Regards,
Sven

Sven Barth
to Martin Frb: Would omitting the type info not lead to issues with overloaded functions?…

Martin Frb via fpc-pascal schrieb am Mi.,
28. Apr. 2021, 19:26:

Show quoted text 17 lines
On 28/04/2021 18:43, Graeme Geldenhuys via fpc-pascal wrote: > Hello Sven, > > On 28/04/2021 6:32 am, Sven Barth via fpc-pascal wrote: >> Second: the syntax is required for Delphi compatibility anyway > Couldn't such verbose syntax be limited to {$mode delphi} behaviour, > and then leave {$mode objfpc} free to experiment and introduce new > less verbose syntax in the language. Would omitting the type info not lead to issues with overloaded functions? foo(function(a,b) Foo could have lots of overloaded arguments, all taking a callback with 2 params. Or it does not have them yet, but later.

It would have to derive the types from whatever is done inside the anonymous function and even then that could lead to situations where the parser would need to give up. I had already thought about such things as well when I had played around (in my mind) with a slimmer lambda syntax.

Regards,
Sven

Sven Barth
to Ryan Joseph: Sorry I'm struggling with all these new terms myself. Yes that is what I'm proposing and…

Ryan Joseph via fpc-pascal schrieb am
Mi., 28. Apr. 2021, 17:53:

Show quoted text 23 lines
> On Apr 27, 2021, at 11:36 PM, Sven Barth wrote: > > Anyway, it would in principle be possible to convert an anonymous function to a "is nested" function, but that will only come *after* the whole implementation is here so that the chance for messing that core functionality (!) up is reduced. Sorry I'm struggling with all these new terms myself. Yes that is what I'm proposing and what I demonstrated in that GitHub branch. We talked about this in 2018 but you said to wait until the "real thing" was finished so I let it be. I brought this up again now to say that when, and not before, the closures are finished and in trunk I will see how to integrate the "nested anonymous functions" into the new system. To clarify for myself, we have a few concepts here: - Closures: blocks which capture variables in scope and wrap them into a container (nested functions use a record I think and closures will use an interface).

Nested functions simply use the framepointer. They are using it like a record, but in essence it's simply the framepointer...

- Anonymous procedures: Simply an anonymous procedure/function body which

lacks a name and can be declared inside code blocks.

Correct.

- 2 Procedure types: "reference to" and "is nested". On the caller side

these are what control which kind of closure is used is used. Is that correct? So the anonymous function doesn't really become a closure until it's passed to a "reference to" or "is nested" variable.

To be precise there are two more: function/procedure variables (no special designator) and method variables ("of object"). Depending on what a anonymous function captures (or for the sake of it a nested function) it would be possible to assign it to such a type as well (for a function variable only global variables may be used, for a method variable additionally Self may be used).

So once the current work is done we have 2 kinds of closures but only one

of those is compatible with anonymous functions and this is why I want the other closures (nested functions) to have the same pair of functionality.

Once anonymous functions are supported we can improve their compatibility with other constructs.

Regards,
Sven

Mattias Gaertner
to Sven Barth: Graeme Geldenhuys via fpc-pascal schrieb am Mi., 28. Apr. 2021, 19:00: You completely…
[...] You completely ignored my first point, which in this case is the much more significant one: Pascal does not support type inference.

FPC does not.
Delphi does:
http://docwiki.embarcadero.com/RADStudio/Sydney/en/Inline_Variable_Declaration

begin
  var MyDictionary := TDictionary<string, Integer>.Create;
  for var I:=1 to 10 do ;
end.
It is impossible to implement a syntax like you're suggesting without major rework of the parser.

Indeed.

Mattias

Tomas Hajny
to Mattias Gaertner: On Thu, 29 Apr 2021 07:52:19 +0200 FPC does not. Delphi does:…
Mattias Gaertner wrote 15 lines
On Thu, 29 Apr 2021 07:52:19 +0200 Sven Barth via fpc-pascal wrote: > [...] > You completely ignored my first point, which in this case is the much > more significant one: Pascal does not support type inference. FPC does not. Delphi does: http://docwiki.embarcadero.com/RADStudio/Sydney/en/Inline_Variable_Declaration begin var MyDictionary := TDictionary<string, Integer>.Create; for var I:=1 to 10 do ; end.

Are you aware that Delphi limits this specifically to inline variable declaration (quoting including the typo: "compiler can now in several circumstances infer the type of a variable at ints line declaration location")? One could argue to which extent are inline variable declarations still Pascal... :/ (I have my opinion about this, but let's save everybody from a flamewar about this topic.)

Tomas

Ryan Joseph
to Sven Barth: Ryan Joseph via fpc-pascal schrieb am Mi., 28. Apr. 2021, 17:53: Nested functions simply…
On Apr 29, 2021, at 12:01 AM, Sven Barth wrote: To be precise there are two more: function/procedure variables (no special designator) and method variables ("of object"). Depending on what a anonymous function captures (or for the sake of it a nested function) it would be possible to assign it to such a type as well (for a function variable only global variables may be used, for a method variable additionally Self may be used).

As I remember the capturing code for nested functions and the new closures are not shared. That means when the parser encounters an anonymous function it needs to decide which type of capture it will use, right? In that case the user may need to state that the anon function is "nested" otherwise it will use the wrong capturing method.

Show quoted text 5 lines
So once the current work is done we have 2 kinds of closures but only one of those is compatible with anonymous functions and this is why I want the other closures (nested functions) to have the same pair of functionality. Once anonymous functions are supported we can improve their compatibility with other constructs.

Great, that's what I wanted to do. It's well worth the time to get anon functions for callbacks that return immediately, list.ForEach, list.Sort etc....

Regards,
Ryan Joseph

Michael Van Canneyt
to Bo Berglund
Bo Berglund wrote 10 lines
On Tue, 27 Apr 2021 10:52:28 -0600, Ryan Joseph via fpc-pascal wrote: > > >> On Apr 27, 2021, at 9:58 AM, Michael Van Canneyt wrote: >> >> Wait. Is this thread intentionally moved from fpc-devel?

No. Mistake, due to mailing list settings and too many recipients when you hit reply.

Michael.