This has been a constant problem for me with FPC and wanted to make a formal post with code examples since I've only mentioned it in passing before.

How can it be achieved to have a caller agnostic procedure variables? I've tried making some big crazy dispatch record that uses generics but because generics don't support variable templates (like some languages have TClass<T...>) it was limited and clunky to use.

The problem is that from the perspective of the receiver it shouldn't really care what the caller has provided except for that there is a procedure that needs to be called. For example if there is a "sort" function that takes a procedure variable it shouldn't care if the procedure is a global function, a method or a nested function (and eventually a closure).

It feels like the compiler needs a new type which encapsulates these different types but I'm not sure how this all works internally. Any thoughts on this?

===========================

{$mode objfpc}
program procvars;
type
  TMyAction = procedure;
  TMyClass = class
    procedure MyAction;
  end;
procedure DoThis(action: TMyAction);
  begin
    action();
  end;
procedure MyAction;
  begin
  end;
procedure Test;
  procedure MyNestedAction;
  begin
  end;
  var
    c: TMyClass;
  begin
    // of object
    c := TMyClass.Create;
    DoThis(@c.MyAction); // error: Incompatible type for arg no. 1: Got "<procedure variable type of procedure of object;Register>", expected "<procedure variable type of procedure;Register>"
    // is nested
    DoThis(@MyNestedAction); // error: Incompatible type for arg no. 1: Got "<procedure variable type of procedure is nested;Register>", expected "<procedure variable type of procedure;Register>"
    // normal
    DoThis(@MyAction);
  end;
begin
end.

Regards,
Ryan Joseph

I've answered this question before:

The "Reference to procedure" that will be part of anonymous functionswill do this for you.

Michael.

I'm sorry I forgot! This thing keeps coming up for me and driving me nuts but I knew I asked in passing before.

So "reference to procedure" are going to be the new standard? To extend my example it would look like this? I please remind me, is there a closure involved which is capturing state and adding overhead?

I'm curious what this type actually is also, maybe a dispatch table which wraps the existing types or is it something totally new?

====================

type
  TMyAction = reference to procedure;
procedure DoThis(action: TMyAction);
  begin
    action();
  end;

Regards,
Ryan Joseph

I don't know what you mean by 'new standard', but a new type, yes.

We're not going to change all procedural types to this new type, if that's what you have in mind.

To extend my example it would look like this? I please remind me, is there a closure involved which is capturing state and adding overhead?

There is not necessarily a closure; only if you use an anonymous function.

I'm curious what this type actually is also, maybe a dispatch table which wraps the existing types or is it something totally new?

I will leave it to the compiler people to answer this, because I don't know the low-level details.

Yes. The following compiles and runs in delphi:

program procdemo;
{$APPTYPE CONSOLE}
{$R *.res}
Type
   TProc = reference to procedure;
   TMyObject = Class
     Procedure DoA;
     Procedure DoTest(aTest : TProc);
     Procedure Demo;
   End;
Procedure DoPlain;
begin
   Writeln('Plain');
end;

{ TMyObject }

procedure TMyObject.Demo;
begin
   DoTest(DoA);
   DoTest(DoPlain);
   DoTest(Procedure
    begin
      Writeln('anonymous');
    end);
end;
procedure TMyObject.DoA;
begin
   Writeln(ClassName,': A');
end;
procedure TMyObject.DoTest(aTest: TProc);
begin
   aTest;
end;
begin
   With TMyObject.Create do
     try
       Demo;
     finally
       Free;
     end;
end.

I requested that this:

procedure TMyObject.Demo;
   Procedure DoSub;
   begin
     Writeln('Sub');
   end;
begin
   DoTest(DoSub);
end;

would also work. It does not work in Delphi, but Pas2js already supports that. I think it results in more readable code - the whole anonymous stuff doesn't really work for me...

Michael.

Ryan Joseph via fpc-pascal schrieb am
Di., 15. Feb. 2022, 08:34:

It contains a capture object that backs the method. If nothing is captured and the right hand side is a direct function or method pointer then the compiler could in principle create something that is essentially static to avoid allocations, but that would be a future optimization.

I'm curious what this type actually is also, maybe a dispatch table which wraps the existing types or is it something totally new?

A function reference is essentially a reference counted interface with a single method named "Invoke".

And before you ask: no, we're not going to add yet another type that combines all four when a function reference does that perfectly well already even if it might be more expensive (note: it's not the call that is that much more expensive, but the creation).

Regards,
Sven

So that means "reference to procedure" is not compatible with "is nested"?

Regards,
Ryan Joseph

In Delphi it is not. In FPC it should be :-)

Michael.

So if we do this there is nothing being captured right? In that case we get an interface which can call a function pointer? Does the interface then know about "of object" and "is nested" types at all or does it use a totally different mechanism to call those?

================

type
 TMyAction = reference to procedure;
procedure MyAction;
 begin
 end;
procedure Test;
 begin
   DoThis(@MyAction);
 end;

Regards,
Ryan Joseph

On Feb 15, 2022, at 7:10 PM, Michael Van Canneyt wrote: In Delphi it is not. In FPC it should be :-)

Indeed should be but that's what I'm trying to figure out with how this is implemented.

Why wouldn't Delphi be able to do this I wonder. The calling mechanism in this object is not clear to me so I'll wait for Sven to answer on that.

Regards,
Ryan Joseph

Ryan Joseph via fpc-pascal schrieb am
Di., 15. Feb. 2022, 13:16:

Delphi doesn't support "is nested", so there is nothing for them to support there.

Regards,
Sven

Ryan Joseph via fpc-pascal schrieb am
Di., 15. Feb. 2022, 13:06:

It's relatively "easy" to implement assigning a nested function to function references. However assigning a nested function variable to a function reference is much harder.
Assigning a function reference to a nested function variable is hard as well.

Regards,
Sven

Ryan Joseph via fpc-pascal schrieb am
Di., 15. Feb. 2022, 13:14:

A function reference is simply an interface of which the Invoke method can be called on the instance instead of manually doing "Foo.Invoke".

The real "magic" is when the compiler generates the *implementation* of said interface. So in the end what can be assigned to a function reference depends on the compiler being able to generate suitable implementations.

Anonymous functions are a given. Global functions and methods are required to work as well. Nested functions can be made to use as well. Function and method variables can be supported relatively easily as well. Only nested function variables are hard to support.

Regards,
Sven

So when you assign a global function to a function reference it has to generate a new function body? I guess that makes sense on how it can "capture" these different types of functions.

Regards,
Ryan Joseph

I'm getting lost with the terms now I think. If your function takes a function reference parameter and you assign it a nested function pointer, this is difficult? I believe this was Michaels request in that code snippet.

Regards,
Ryan Joseph

Sven will need to confirm, but as I understand it:

Procedure DoDemo(aTest : TProc);
Procedure MyTest;
   Procedure DoSub;
   begin
   end;
begin
   DoDemo(DoSub)
end;

is easy.

but

Procedure DoDemo(aTest : TProc);
Procedure MyTest;
Var
   ASub : procedure is nested;
begin
   aSub:=SomeFunctionThatReturnsANestedProc;
   DoDemo(aSub);
end;

is difficult.

Michael.

For clarity, this should be something like:

Procedure DoDemo(aTest : TProc);
begin
   // Do stuff
   aTest;
end;

So the below procedures are not local to DoDemo...

Michael.

Michael Van Canneyt via fpc-pascal
schrieb am Di., 15. Feb. 2022, 15:29:

Correct (yes, I saw your addendum).

Regards,
Sven

Ryan Joseph via fpc-pascal schrieb am
Di., 15. Feb. 2022, 15:02:

For a global function the compiler has to generate a wrapper that gets rid of the Self parameter of the interface.

Regards,
Sven

The "Reference to procedure" that will be part of anonymous functionswill do this for you.

What release are anonymous functions planed for? FPC 3.4.0?

=== code excerpt begin ===

// function/procedure/routine
function Foobar: LongInt; forward;
// function/procedure/routine variable
type
   TFoobarFunc = function: LongInt;
// method
   TFoo = class
     function Foobar: LongInt;
   end;
// method variable
   TFoobarMethod = function: LongInt of object;
// function/procedure/routine reference
   TFoobarFuncRef = reference to function: LongInt;
function Foobar: LongInt;
   // nested function/procedure/routine
   function FoobarSub: LongInt;
   begin
   end;
// nested function/procedure/routine variable
type
   TFoobarNested = function: LongInt is nested;
var
    f: TFoobarFuncRef;
begin
   // anonymous function/procedure/routine
   f := function: LongInt
         begin
         end;
end;

=== code excerpt end ===

Regards,
Sven

"However assigning a nested function variable to a function reference is much harder. Assigning a function reference to a nested function variable is hard as well. "

This means if you expanded your example with:

var
  n: TFoobarNested;
begin
  f := n;

THAT would be hard? I've never passed around nested function vars before so I don't really know the limitations of this. The important thing is the primary use case works.

Regards,
Ryan Joseph

On Feb 15, 2022, at 11:09 PM, Sven Barth via fpc-pascal wrote: For a global function the compiler has to generate a wrapper that gets rid of the Self parameter of the interface.

The compiler generates this interface at compile time right? And then when the scope of the calling function is entered the interface is allocated? I know that's how it works for the case when state is captured but I think you're saying this is happening any time a function reference is assigned to, even if there is no state captured. Just trying to get an understanding of the runtime cost to use these.

Regards,
Ryan Joseph

On Feb 16, 2022, at 12:18 AM, Thomas Kurz via fpc-pascal wrote: What release are anonymous functions planed for? FPC 3.4.0?

They aren't even in trunk yet. Could be months or years.

Regards,
Ryan Joseph

Ryan Joseph via fpc-pascal schrieb am
Mi., 16. Feb. 2022, 03:14:

The compiler is generating the *implementation* of the interface that is the function reference. The capture object with this implementation as a method is then shared by everything that's assigned to a function reference (doesn't need to be the same one). So one instance will be allocated each time the function is entered.
This is the general case. For some very specific cases there can be optimizations that can work without allocations, but that's a topic for the future.

Regards,
Sven

Ryan Joseph via fpc-pascal schrieb am
Mi., 16. Feb. 2022, 07:59:

Correct. In addition to that the general assumption for function references is that they can be called even after the function they were assigned to has been left. For a nested function itself this can hold true as well (cause the compiler simply needs to transform the nested function correctly when generating the implementation for the capture object), but for a nested function variable this assumption would be wrong.

Regards,
Sven

Sven Barth wrote

"However assigning a nested function variable to a function reference is much harder. Assigning a function reference to a nested function variable is hard as well. "

"is nested" means that the actual function passed can be either global or local. ISO Pascal style function parameters are implicitely "is nested". The formal function reference for an "is nested" function contains space for both a function address and a context pointer. The context pointer is NIL if the actual function is declared global. For local functions, the context pointer typically contains a dynamic link chain pointer, required to address variables in the actual link frame. For object methods, the context pointer is the SELF pointer.

Regards,

Adriaan van Os