Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2021-04

Nested function closures

6 messages · 3 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Ryan Joseph

Continued from our discussion at https://bugs.freepascal.org/view.php?id=24481.

if the compiler devs will allow me as soon as this is finished I want to allow the existing nested functions functionality to work with anonymous functions, so at the very least we don't need to generate the expensive interface based object which often times is not even needed. At that point we would need to make nested functions inline-able, which they are currently not. But we're not there yet so lets not complicated anything by proposing extensions to a feature that doesn't even exist yet.

Sven replies:

Getting rid of the interface only works in very narrow circumstances that are so seldom in real world code that it is not worth the effort.

I'm referring to my test I did a few years ago (https://github.com/graemeg/freepascal/compare/master...genericptr:anon_funcs) where I say we can use existing nested functions as a closure when passing is not required. As you can see I already implemented this quite easily but it is not related to the new forthcoming closures feature. I did in fact try to replace the interface with a record on the old closures branch but I ran into many problems I decided it wasn't the best route.

Indeed there are many times where we don't want a heap allocated interface you can pass around but rather a simple inline function pointer like below. Consider this loop is run 60 times a second and allocating a useless class every time for no gain. This could easily be 1000*60=60,000 constructions and allocations of a class.

  for i := 0 to entities.Count - 1 do
    begin
      value := entities[i];
      value.SortEntities(function(a, b: TEntity): integer
        begin
          // do stuff
        end
      );
    end;

So anyways what I propose is if a closure is never passed outside of scope (i.e. temporary) then use anonymous nested functions instead (like in my GitHub branch). If this is an acceptable approach I will personally do what is required to get it implemented along side the real closures.

Regards,
Ryan Joseph

Reply by email

Michael Van Canneyt
to Ryan Joseph
Ryan Joseph wrote 25 lines
Continued from our discussion at https://bugs.freepascal.org/view.php?id=24481. > if the compiler devs will allow me as soon as this is finished I want to allow the existing nested functions functionality to work with anonymous functions, so at the very least we don't need to generate the expensive interface based object which often times is not even needed. At that point we would need to make nested functions inline-able, which they are currently not. But we're not there yet so lets not complicated anything by proposing extensions to a feature that doesn't even exist yet. Sven replies: > > Getting rid of the interface only works in very narrow circumstances that are so seldom in real world code that it is not worth the effort. I'm referring to my test I did a few years ago (https://github.com/graemeg/freepascal/compare/master...genericptr:anon_funcs) where I say we can use existing nested functions as a closure when passing is not required. As you can see I already implemented this quite easily but it is not related to the new forthcoming closures feature. I did in fact try to replace the interface with a record on the old closures branch but I ran into many problems I decided it wasn't the best route. Indeed there are many times where we don't want a heap allocated interface you can pass around but rather a simple inline function pointer like below. Consider this loop is run 60 times a second and allocating a useless class every time for no gain. This could easily be 1000*60=60,000 constructions and allocations of a class. for i := 0 to entities.Count - 1 do begin value := entities[i]; value.SortEntities(function(a, b: TEntity): integer begin // do stuff end ); end; So anyways what I propose is if a closure is never passed outside of scope (i.e. temporary) then use anonymous nested functions instead (like in my GitHub branch). If this is an acceptable approach I will personally do what is required to get it implemented along side the real closures.

Wait.

I asked Sven to make sure that nested functions are under ALL circumstances usable as closures or can be used instead of anonymous functions.

Pas2js already supports this, and I want FPC and Pas2JS to be compatible in this regard.

So as Sven wrote, you would be duplicating effort, needlessly, since it has to work always... If the compiler can decide that the heap interface is not needed and optimize it away: so much the better. But I doubt this will be possible.

Michael.

Reply by email

Sven Barth
to Ryan Joseph: Continued from our discussion at https://bugs.freepascal.org/view.php?id=24481. Sven…
Ryan Joseph wrote 22 lines
Continued from our discussion at https://bugs.freepascal.org/view.php?id=24481. > if the compiler devs will allow me as soon as this is finished I want to allow the existing nested functions functionality to work with anonymous functions, so at the very least we don't need to generate the expensive interface based object which often times is not even needed. At that point we would need to make nested functions inline-able, which they are currently not. But we're not there yet so lets not complicated anything by proposing extensions to a feature that doesn't even exist yet. Sven replies: > Getting rid of the interface only works in very narrow circumstances that are so seldom in real world code that it is not worth the effort. I'm referring to my test I did a few years ago (https://github.com/graemeg/freepascal/compare/master...genericptr:anon_funcs) where I say we can use existing nested functions as a closure when passing is not required. As you can see I already implemented this quite easily but it is not related to the new forthcoming closures feature. I did in fact try to replace the interface with a record on the old closures branch but I ran into many problems I decided it wasn't the best route. Indeed there are many times where we don't want a heap allocated interface you can pass around but rather a simple inline function pointer like below. Consider this loop is run 60 times a second and allocating a useless class every time for no gain. This could easily be 1000*60=60,000 constructions and allocations of a class. for i := 0 to entities.Count - 1 do begin value := entities[i]; value.SortEntities(function(a, b: TEntity): integer begin // do stuff end ); end; So anyways what I propose is if a closure is never passed outside of scope (i.e. temporary) then use anonymous nested functions instead (like in my GitHub branch). If this is an acceptable approach I will personally do what is required to get it implemented along side the real closures.

As soon as *any* function is passed to a "reference to procedure/function" it *must* be an interface, because that's how "reference to procedure/function" is internally implemented. Anything that's calling a "reference to procedure/function" is expecting to call an interface method, thus it *must* be an interface. Also if you capture anything then that *must* be contained in a capture context, because whatever you pass that function reference to might store that for later calling and then the stack context might be long gone. The only situations where the compiler might optimize this is if it's inside the same function, maybe inside the same implementation section of the unit or possibly if WPO is involved (with a dedicated WPO pass), but those are complex optimizations.

Also your example is wrong, cause it will not create an interface for each loop iteration. Instead the pseudo code essentially looks like this:

=== code begin ===

procedure Foo;
type
   ISort = interface
     function Invoke(a, b: TEntity): Integer;
   end;
   TCaptureObject = class(TInterfacedObject, ISort)
     function Invoke(a, b: TEntity): Integer;
   end;
function TCaptureObject.Invoke(a, b: TEntity): Integer;
begin
   // do stuff
end;
var
   context: TCaptureObject;
begin
   context := TCaptureObject.Create;
   for i := 0 to entities.Count - 1 do
      begin
       value := entities[i];
       value.SortEntities(ISort(context));
      end;
end;

=== code end ===

If you capture variables they'll be part of TCaptureObject instead of the stack.

Regards,
Sven

Reply by email

Sven Barth
to Michael Van Canneyt: Wait. I asked Sven to make sure that nested functions are under ALL circumstances usable…
Michael Van Canneyt wrote 63 lines
On Tue, 27 Apr 2021, Ryan Joseph via fpc-devel wrote: > Continued from our discussion at > https://bugs.freepascal.org/view.php?id=24481. > >> if the compiler devs will allow me as soon as this is finished I >> want to allow the existing nested functions functionality to work >> with anonymous functions, so at the very least we don't need to >> generate the expensive interface based object which often times is >> not even needed. At that point we would need to make nested >> functions inline-able, which they are currently not. But we're not >> there yet so lets not complicated anything by proposing extensions >> to a feature that doesn't even exist yet. > > Sven replies: > >> >> Getting rid of the interface only works in very narrow circumstances >> that are so seldom in real world code that it is not worth the effort. > > I'm referring to my test I did a few years ago > (https://github.com/graemeg/freepascal/compare/master...genericptr:anon_funcs) > where I say we can use existing nested functions as a closure when > passing is not required. As you can see I already implemented this > quite easily but it is not related to the new forthcoming closures > feature. I did in fact try to replace the interface with a record on > the old closures branch but I ran into many problems I decided it > wasn't the best route. > > Indeed there are many times where we don't want a heap allocated > interface you can pass around  but rather a simple inline function > pointer like below. Consider this loop is run 60 times a second and > allocating a useless class every time for no gain. This could easily > be 1000*60=60,000 constructions and allocations of a class. > >  for i := 0 to entities.Count - 1 do >    begin >      value := entities[i]; >      value.SortEntities(function(a, b: TEntity): integer >        begin >          // do stuff >        end >      ); >    end; > > > So anyways what I propose is if a closure is never passed outside of > scope (i.e. temporary) then use anonymous nested functions instead > (like in my GitHub branch). If this is an acceptable approach I will > personally do what is required to get it implemented along side the > real closures. Wait. I asked Sven to make sure that nested functions are under ALL circumstances usable as closures or can be used instead of anonymous functions. Pas2js already supports this, and I want FPC and Pas2JS to be compatible in this regard.

The compiler will essentially prepare them as if they're anonymous functions.

Show quoted text 6 lines
So as Sven wrote, you would be duplicating effort, needlessly, since it has to work always... If the compiler can decide that the heap interface is not needed and optimize it away: so much the better. But I doubt this will be possible.

In nearly all cases the interface can't be optimized away.

Regards,
Sven

Reply by email

Ryan Joseph
to Sven Barth
Show quoted text 8 lines
On Apr 27, 2021, at 12:10 PM, Sven Barth via fpc-devel wrote: > So as Sven wrote, you would be duplicating effort, needlessly, since it has > to work always... If the compiler can decide that the heap interface is not > needed and optimize it away: so much the better. But I doubt this will be > possible. In nearly all cases the interface can't be optimized away.

So as you showed it's not as bad as I thought, which is good, however my point was that in my scenario, which I argue is indeed very common, it could be implemented as a "nested anonymous function". My reasons:

1) Nested functions already exist and merely need an anonymous parser to be implemented (as I already did in that branch).
2) The interface is literally 100% useless as the object is never passed outside of the receiver (SortEntities). It will be created and destroyed with absolutely no value to the program whatsoever.

Regards,
Ryan Joseph

Reply by email

Sven Barth
to Ryan Joseph
Ryan Joseph wrote 12 lines
> On Apr 27, 2021, at 12:10 PM, Sven Barth via fpc-devel wrote: > >> So as Sven wrote, you would be duplicating effort, needlessly, since it has >> to work always... If the compiler can decide that the heap interface is not >> needed and optimize it away: so much the better. But I doubt this will be >> possible. > In nearly all cases the interface can't be optimized away. So as you showed it's not as bad as I thought, which is good, however my point was that in my scenario, which I argue is indeed very common, it could be implemented as a "nested anonymous function". My reasons: 1) Nested functions already exist and merely need an anonymous parser to be implemented (as I already did in that branch). 2) The interface is literally 100% useless as the object is never passed outside of the receiver (SortEntities). It will be created and destroyed with absolutely no value to the program whatsoever.

The "is nested" in your other mail was the important part that was missing. How about providing *complete* examples in the future?

Anyway, it would in principle be possible to convert an anonymous function to a "is nested" function, but that will only come *after* the whole implementation is here so that the chance for messing that core functionality (!) up is reduced.

Regards,
Sven

Reply by email