Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2022-06

Are there any drawbacks to "reference to"?

24 messages · 7 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Anthony Walter

As the "reference to" feature was recently added to fpc trunk, I'd like to ask if internally there are any penalties (performance perhaps) or drawbacks to using them in place of the more traditional "procedure" and "procedure of object" types?

It's my understanding that for example using this code:

type
  TNotifyEvent = reference to procedure(Sender: TObject);
  // old version below
  // TNotifyEvent = procedure(Sender: TObject) of object;

Would allow BOTH methods and normal procedures to be assigned to fields or variables of this redefined version of TNotifyEvent. The obvious benefit would be that users could then have more options when assigning event handlers, including anonymous procedures in addition to normal procedures and methods.

If there are no drawbacks to using "reference to" (given that the compiler someone is using is up to date with this feature), would it make sense for everyone to start using it in place of l "procedure" and "procedure of object" types? Obviously, when I say "procedure" in this question, I also mean to include "function" as well.

Reply by email

Jonas Maebe
to Anthony Walter
Anthony Walter wrote
As the "reference to" feature was recently added to fpc trunk, I'd like to ask if internally there are any penalties (performance perhaps) or drawbacks to using them in place of the more traditional "procedure" and "procedure of object" types?

The downside is that a "reference to" always gets wrapped in a reference-counted interface, and hence is much more expensive than a place procedural variable ("of object" or not).

Jonas

Reply by email

Anthony Walter
to Jonas Maebe

Thanks for that, but is that true for all conditions. I remember reading and tinkering about with Delphi when it first introduced closures. At the time it made sense that the extra cruft and interface were needed to capture local state, but I was under the impression that the interface was only needed when a local state existed. In the case where the assigned method or procedure is not a closure an interface doesn't need to exist.

Consider the following bit of code:

type
  TMultiply = reference to function(Value: Integer): Integer;
function MultiplyFunc(Value: Integer): Integer;
begin
  Result := Value * 10;
end;
function MultiplyClosure(const S: string): TMultiply;
var
  I: Integer;
begin
  I := StrToInt(S);
  Exit(function(Value: Integer): Integer
  begin
    Result := I * Value;
  end);
end;

{ TForm1 }

function TForm1.MultiplyMethod(Value: Integer): Integer;
begin
  Result := Value * Tag;
end;
procedure TForm1.Button1Click(Sender: TObject);
var
  M: TMultiply;
begin
  case Tag of
    0: M := MultiplyFunc;
    1: M := MultiplyClosure('15');
  else
    M := MultiplyMethod;
  end;
  WriteLn(M(4));
  Tag := Tag + 1;
end;

In the situation of "M" being assigned a reference to MultiplyFunc no local state information needs to be captured inside MultiplyFunc. There will never be a need to create or initialize an interface for the purpose of capturing state. The underlying compiler code ought to be able to discard any code generating an interface and simply call MultiplyFunc.

In the situation of "M" being assigned a reference to MultiplyClosure the local variable "I" needs to be captured because it makes use of an anonymous function. As such, there is a need for a mechanism to capture this state. If the underlying compiler code uses a reference counted interface to accomplish this, then that is an understandable penalty because some capture mechanism is necessary.

In the situation of "M" being assigned a reference to MultiplyMethod again no local state information needs to be captured inside MultiplyMethod. The reference to Self is passed to the method as a hidden parameter each time it is invoked. "M" can a simple two pointer data structure (see TMethod), and no closure (or interface mechanism) is needed.

So in summary, some mechanism, such as a reference counted interface, is only needed when a value of type TMultiply is assigned to an anonymous function (a closure). Of course code that invokes a TMultiply would need to know when to use any of these possibilities (a function, method, or closure reference). I imagine detection of this possibility could be simply implemented as a small data structure, thereby saving a lot of overhead and or complexity in situations where TMultiply uses a closure reference infrequently:

TReferenceTo = record
  Kind: Integer; // 0 = procedure/function, 1 = method, 2 = closure
  case Integer of
    0: (Code: Pointer);
    1: (Method: TMethod);
    2: (Intf: IInterface);
  end;

Additional note: The most recent trunk revision of fpc from gitlab has a major bug with either

{$modeswitch functionreferences}
or
{$modeswitch anonymousfunctions}

I will follow up with more information about that bug in a different thread.

Reply by email

Hairy Pixels
to Jonas Maebe: The downside is that a "reference to" always gets wrapped in a reference-counted…
Show quoted text 7 lines
On Jun 6, 2022, at 2:02 AM, Jonas Maebe via fpc-pascal wrote: On 2022-06-05 18:20, Anthony Walter via fpc-pascal wrote: > As the "reference to" feature was recently added to fpc trunk, I'd like to ask if internally there are any penalties (performance perhaps) or drawbacks to using them in place of the more traditional "procedure" and "procedure of object" types? The downside is that a "reference to" always gets wrapped in a reference-counted interface, and hence is much more expensive than a place procedural variable ("of object" or not).

First off I didn’t see Anthony’s original email, just like Sven says he doesn’t see mine. What’s going on here? I’m using a Gmail account because my personal domain was getting rejected for some reason, is Anthony on Gmail also? We need to get this fixed I think.

Anytime you assign to a reference it creates this interface wrapped object, even if you don’t call it. This is of course totally wasteful and in fact makes references not reliable as a catch-all function pointer type since it comes with significant runtime overhead.

I think what the compiler needs to do is internally support a variant record type and switch to the correct function like this:

TCallback = record
  kind: byte;
  case byte of
    0: (proc: procedure);
    1: (ref: reference to procedure);
    2: (nested: procedure is nested);
    3: (obj: procedure of object);
end;

Does that sound correct? Personally what I want to see if a catch all type so I don’t need to make 4 different function aliases for each of FPCs callback types.

However as far as anonymous functions are concerned I was pleased to see that they can be can be assigned to other function pointer types so I can still use the feature in many cases.

Regards,
Ryan Joseph

Reply by email

Bo Berglund
to Hairy Pixels
Hairy Pixels wrote
First off I didn’t see Anthony’s original email, just like Sven says he doesn’t see mine. What’s going on here? I’m using a Gmail account because my personal domain was getting rejected for some reason, is Anthony on Gmail also? We need to get this fixed I think.

You can interface this discussion via GMane news mirror....

I use a dedicated news reader and it has these advantages:
- Does not pollute my regular email account with lots of messages
- Keeps a threaded local archive of the discussions for easy reference
- Any such blockades you describe are non-existent

Point your news reader to news.gmane.io and subscribe to:

gmane.comp.compilers.free-pascal.general
gmane.comp.ide.lazarus.general
gmane.comp.compilers.free-pascal.devel

Bo Berglund Developer in Sweden

Reply by email

Hairy Pixels
to Hairy Pixels: First off I didn’t see Anthony’s original email, just like Sven says he doesn’t see…
Show quoted text 12 lines
On Jun 6, 2022, at 8:26 AM, Hairy Pixels wrote: I think what the compiler needs to do is internally support a variant record type and switch to the correct function like this: TCallback = record kind: byte; case byte of 0: (proc: procedure); 1: (ref: reference to procedure); 2: (nested: procedure is nested); 3: (obj: procedure of object); end;

I found Anthony's email in the archive where he explain this optimization very nicely but he missed another aspect of it. Take for example this code:

type
  TCallback = reference to procedure;
procedure CallMe(callback: TCallback);
begin
  callback();
end;
var
  i: integer = 1;
begin
  CallMe(procedure
         begin
           writeln(‘hello ‘, i);
         end);
end;

Here state needs to be captured but the structure capturing it doesn’t need to be a heap allocated interface backed class because it never leaves the calling scope. FPC already has an existing closure type for nested functions which is far more efficient and could be implemented if there was this variant structure behind the scenes which could accept it.

So the optimization has two components:

1) If there is no anonymous function then obviously no state needs to be captured and the function reference should be either a function pointer or a “of object” pointer.

2) If the there is captured state but that closures never leaves the calling scope then it can be optimized as a “is nested” function pointer.

Unlike most of my feature requests this one actually affects performance and should seriously be considered. It’s nothing crazy like constant propagation for inlining function pointers but it does require a new dynamic type be introduced to the compiler which can wrap up these variant function pointers and dispatch the correct to say if it’s within my ability I will elect myself to implement this if it’s permitted by the compiler team.

Regards,
Ryan Joseph

Reply by email

Hairy Pixels
to Hairy Pixels: First off I didn’t see Anthony’s original email, just like Sven says he doesn’t see…
On Jun 6, 2022, at 8:26 AM, Hairy Pixels wrote: First off I didn’t see Anthony’s original email, just like Sven says he doesn’t see mine. What’s going on here? I’m using a Gmail account because my personal domain was getting rejected for some reason, is Anthony on Gmail also? We need to get this fixed I think.

Bo Berglund, I just saw your message on the mail list archive also but I’m not getting here. Ok the mail list is totally broken for me but I don’t know how news.gmane.io works (my news reader is for RSS and I have no idea what this is). I’ll research it later because I don’t think I can use this thing anymore.

Regards,
Ryan Joseph

Reply by email

Sven Barth
to Jonas Maebe: The downside is that a "reference to" always gets wrapped in a reference-counted…

Jonas Maebe via fpc-pascal schrieb am
So., 5. Juni 2022, 21:02:

Show quoted text 11 lines
On 2022-06-05 18:20, Anthony Walter via fpc-pascal wrote: > As the "reference to" feature was recently added to fpc trunk, I'd like > to ask if internally there are any penalties (performance perhaps) or > drawbacks to using them in place of the more traditional "procedure" > and "procedure of object" types? The downside is that a "reference to" always gets wrapped in a reference-counted interface, and hence is much more expensive than a place procedural variable ("of object" or not).

As I can see neither Anthony's nor Ryan's mail, but I can see them in the archive, I'll use Jonas mail for some general replies (please CC me when replying):

A function reference is *always* an interface. That's why I wrote in my announcement that it really does behave like an interface including inheritance and all that. So once you have a function reference there isn't much you can do (let's ignore some small possibilities through inlining). What *might* be done however is that for cases where no state is required the compiler could simply fake an interface (using some static methods to provide _AddRef, _Release and _QueryInterface that essentially do nothing and then the converted method providing the Invoke method). This would be more lightweight for specific use cases. But before implementing something like that I want to be sure that the feature as it should be works correctly (which it still doesn't with some optimizations enabled).

Also regarding the idea of using a variant record: you can't have a managed type in a variant record, because the compiler would not be able to know whether to touch the reference count or not. This would mean that you'd need an additional field to keep track of the type and helper methods and/or code that then handles and calls it correctly. Thus any advantage of avoiding an indirect call through the interface's VMT goes out the window again, not to mention potential branch predictor issues.

Also keep in mind that the expensive creation only happens once. Afterwards it's "only" the price of an indirect call.

Regards,
Sven

Reply by email

Hairy Pixels
to Sven Barth
On Jun 19, 2022, at 7:01 PM, Sven Barth via fpc-pascal wrote: As I can see neither Anthony's nor Ryan's mail, but I can see them in the archive, I'll use Jonas mail for some general replies (please CC me when replying):

Firstly unrelated, I posted a question about generics which you probably have the answer to and may have missed (). I just joined the Lazarus forum also and mentioned this in a related topic where another user seems to have stumbled on something similar.

Also keep in mind that the expensive creation only happens once. Afterwards it's "only" the price of an indirect call.

The performance problem will surfice if you use a reference inside a function which is called often. Lets say you decided to use a reference as a general callback type because it’s convenient not to consider “is nested” or “of object” and then call this from within a function which is called in a tight loop.

Depending on what you’re doing this extra allocation of the interface could totally blow your program out of the water but this isn’t apparent to the programer who merely sees what they’re doing a simple callback. For this reason you can’t rely on references as an all purpose callback type and if performance is important you need to make duplicate functions which take different types of functions pointers, hence our desire for a generalized function pointer that that doesn’t do the interface allocation if there’s no passing of the reference outside the calling scope.

Regards,
Ryan Joseph

Reply by email

Sven Barth
to Hairy Pixels

Hairy Pixels schrieb am So., 19. Juni 2022, 15:44:

Show quoted text 15 lines
> On Jun 19, 2022, at 7:01 PM, Sven Barth via fpc-pascal < > wrote: > > As I can see neither Anthony's nor Ryan's mail, but I can see them in the archive, I'll use Jonas mail for some general replies (please CC me when replying): Firstly unrelated, I posted a question about generics which you probably have the answer to and may have missed ( ). I just joined the Lazarus forum also and mentioned this in a related topic where another user seems to have stumbled on something similar.

I have replied on the forum. But if you want a response to your other mail please reply to your own mail with me in CC, cause I can see it only in the archive.

Show quoted text 19 lines
> Also keep in mind that the expensive creation only happens once. Afterwards it's "only" the price of an indirect call. > The performance problem will surfice if you use a reference inside a function which is called often. Lets say you decided to use a reference as a general callback type because it’s convenient not to consider “is nested” or “of object” and then call this from within a function which is called in a tight loop. Depending on what you’re doing this extra allocation of the interface could totally blow your program out of the water but this isn’t apparent to the programer who merely sees what they’re doing a simple callback. For this reason you can’t rely on references as an all purpose callback type and if performance is important you need to make duplicate functions which take different types of functions pointers, hence our desire for a generalized function pointer that that doesn’t do the interface allocation if there’s no passing of the reference outside the calling scope.

You misunderstand how function references work.

Assume the following:

=== code begin ===

function Foo: LongInt;
var
  f: reference to procedure;
  sum, i: LongInt;
begin
  f := procedure
  begin
    sum := sum + i * 2;
  end;
  sum := 0;
  for i := 0 to 100 do
    f();
  Result := sum;
end;

=== code end ===

This is essentially equivalent to the following:

=== code begin ===

function Foo: LongInt;
type
  TFunc = interface
    procedure Invoke;
  end;
  TCapturer = class(TInterfacedObject, TFunc)
    sum, i: LongInt;
    procedure TFunc.Invoke =TFunc_Invoke;
    procedure TFunc_Invoke;
  end;
  procedure TCapturer.TFunc_Invoke;
  begin
    sum := sum + i * 2;
  end;
var
  capturer: TCapturer;
  keepalive: IInterface;
  f: TFunc;
begin
  capturer := TCapturer.Create;
  keepalive := capturer;
  f := capturer;
  for capturer.i := 0 to 100 do
    f.Invoke();
  Result := capturer.sum;
end;

=== code end ===

As you can see the allocation only happens once and not all the time. What might be worse however is the optimization behavior as in this example the compiler wouldn't optimize the counter variable into a regvar with enabled optimizations (however in case of a nested function the compiler wouldn't do this either if the function isn't inlined).

And even if you do the assignment of the function variable inside the loop you "merely" get the reference counting as a side effect and not a whole allocations.

Regards,
Sven

Reply by email

Hairy Pixels
to Sven Barth
Show quoted text 5 lines
On Jun 19, 2022, at 10:04 PM, Sven Barth wrote: As you can see the allocation only happens once and not all the time. What might be worse however is the optimization behavior as in this example the compiler wouldn't optimize the counter variable into a regvar with enabled optimizations (however in case of a nested function the compiler wouldn't do this either if the function isn't inlined).

I’m saying if the OUTER method is in a tight loop (Foo in this example) you’re in trouble because it now contains an allocation which is unexpected. In fact if this happened within a class wouldn’t it capture and initialize all the classes fields also or does it know to only use the fields which are encountered in the function body?

Either way the reference never leaves the calling scope and thus is wasted and should be optimized away to something which exists on the stack, like “is nested”.

Regards,
Ryan Joseph

Reply by email

Sven Barth
to Hairy Pixels
Hairy Pixels wrote 8 lines
> On Jun 19, 2022, at 10:04 PM, Sven Barth wrote: > > As you can see the allocation only happens once and not all the time. > What might be worse however is the optimization behavior as in this example the compiler wouldn't optimize the counter variable into a regvar with enabled optimizations (however in case of a nested function the compiler wouldn't do this either if the function isn't inlined). > I’m saying if the OUTER method is in a tight loop (Foo in this example) you’re in trouble because it now contains an allocation which is unexpected.

In that case we're in the same territory as always: ensure that you know what your code does. It's the same reason why adding character by character to a managed String is slower then allocating the string once and then setting the characters. And I think it's very seldom that someone uses a function reference that does not leave the scope of the surrounding function.

In fact if this happened within a class wouldn’t it capture and initialize all the classes fields also or does it know to only use the fields which are encountered in the function body?

If you use a function reference inside a method and access one of the class' fields then it's only Self that's captured.

Either way the reference never leaves the calling scope and thus is wasted and should be optimized away to something which exists on the stack, like “is nested”.

If the compiler can proove that the function reference never leaves the scope (that means no assignment to global variables, out/var parameters, the Result variable, passing on to some other function or conversion to pointer (and I'm sure there are others)) and the function reference isn't used as an interface then and only then the compiler could reduce this to something else. But that's an optimization for another day and in my opinion such a rarlely used usecase that it's simply not worth it.

Regards,
Sven

Reply by email

Michael Van Canneyt
to Sven Barth
Sven Barth wrote 21 lines
Am 20.06.2022 um 03:37 schrieb Hairy Pixels: > >> On Jun 19, 2022, at 10:04 PM, Sven Barth wrote: >> >> As you can see the allocation only happens once and not all the time. >> What might be worse however is the optimization behavior as in this example the compiler wouldn't optimize the counter variable into a regvar with enabled optimizations (however in case of a nested function the compiler wouldn't do this either if the function isn't inlined). >> > > I’m saying if the OUTER method is in a tight loop (Foo in this example) you’re in trouble because it now contains an allocation which is unexpected. In that case we're in the same territory as always: ensure that you know what your code does. It's the same reason why adding character by character to a managed String is slower then allocating the string once and then setting the characters. And I think it's very seldom that someone uses a function reference that does not leave the scope of the surrounding function.

Does this not depend on the callback declaration ?

Type
   FuncRef = reference to procedure(a : integer);
Procedure DoSomething(f : FuncRef);
begin
end;
Procedure UseSomething;
begin
   For I:=X to SomeLimit do
     DoSomething(Whatever)
end;

Does the declaration of DoSomething (which uses reference to) not ensure that an interface is created in UseSomething ? Or is the interface only created if you use an anonymous/local function ?

I tried to find your original mail, to look for the answer myself, but I don't find it offhand.

It may be a good idea to add the explanation you gave in the announcement mail to the wiki (and link to it in the user changes in trunk page)

Michael.

Reply by email

Sven Barth
to Michael Van Canneyt

Michael Van Canneyt schrieb am Mo., 20. Juni 2022, 08:04:

Show quoted text 48 lines
On Mon, 20 Jun 2022, Sven Barth via fpc-pascal wrote: > Am 20.06.2022 um 03:37 schrieb Hairy Pixels: >> >>> On Jun 19, 2022, at 10:04 PM, Sven Barth > wrote: >>> >>> As you can see the allocation only happens once and not all the time. >>> What might be worse however is the optimization behavior as in this > example the compiler wouldn't optimize the counter variable into a regvar > with enabled optimizations (however in case of a nested function the compiler > wouldn't do this either if the function isn't inlined). >>> >> >> I’m saying if the OUTER method is in a tight loop (Foo in this example) > you’re in trouble because it now contains an allocation which is unexpected. > > In that case we're in the same territory as always: ensure that you know > what your code does. It's the same reason why adding character by > character to a managed String is slower then allocating the string once > and then setting the characters. And I think it's very seldom that > someone uses a function reference that does not leave the scope of the > surrounding function. Does this not depend on the callback declaration ? Type FuncRef = reference to procedure(a : integer); Procedure DoSomething(f : FuncRef); begin end; Procedure UseSomething; begin For I:=X to SomeLimit do DoSomething(Whatever) end; Does the declaration of DoSomething (which uses reference to) not ensure that an interface is created in UseSomething ?

Yes, in your example this is the case, but look at mine again: it's unnecessary to create the capture object here and in theory the compiler could convert it to an ordinary nested function variable. But that 's all theoretical, because right now such an optimization doesn't exist.

I tried to find your original mail, to look for the answer myself,

but I don't find it offhand.

Either in fpc-pascal or fpc-announce from end of May.

It may be a good idea to add the explanation you gave in the announcement mail to the wiki (and link to it in the user changes in trunk page)

There won't be a "User Changes Trunk" entry because nothing existing broke with this new feature.
There will however be a "New Features Trunk" entry that will point to the announcement mail. There will be no documentation by me in the wiki.

Regards,
Sven

Reply by email

Michael Van Canneyt
to Sven Barth
Sven Barth wrote 56 lines
Michael Van Canneyt schrieb am Mo., 20. Juni 2022, 08:04: > > > On Mon, 20 Jun 2022, Sven Barth via fpc-pascal wrote: > >> Am 20.06.2022 um 03:37 schrieb Hairy Pixels: >>> >>>> On Jun 19, 2022, at 10:04 PM, Sven Barth > >> wrote: >>>> >>>> As you can see the allocation only happens once and not all the time. >>>> What might be worse however is the optimization behavior as in this >> example the compiler wouldn't optimize the counter variable into a > regvar >> with enabled optimizations (however in case of a nested function the > compiler >> wouldn't do this either if the function isn't inlined). >>>> >>> >>> I’m saying if the OUTER method is in a tight loop (Foo in this example) >> you’re in trouble because it now contains an allocation which is > unexpected. >> >> In that case we're in the same territory as always: ensure that you know >> what your code does. It's the same reason why adding character by >> character to a managed String is slower then allocating the string once >> and then setting the characters. And I think it's very seldom that >> someone uses a function reference that does not leave the scope of the >> surrounding function. > > Does this not depend on the callback declaration ? > > Type > FuncRef = reference to procedure(a : integer); > > Procedure DoSomething(f : FuncRef); > begin > end; > > Procedure UseSomething; > > begin > For I:=X to SomeLimit do > DoSomething(Whatever) > end; > > Does the declaration of DoSomething (which uses reference to) not > ensure that an interface is created in UseSomething ? Yes, in your example this is the case, but look at mine again: it's unnecessary to create the capture object here and in theory the compiler could convert it to an ordinary nested function variable.

My point is that if someone creates an API where he/she uses "reference to procedure" then the consumer of the API is stuck with the creation of an interface.

So it's not always a choice whether an interface will be used or not.

My conclusion is then that 'reference to' should be used sparingly in public APIs as they are (relatively) expensive.

Unless I missed/misunderstod something of course...

Michael.

Reply by email

Hairy Pixels
to Sven Barth: In that case we're in the same territory as always: ensure that you know what your code…
On Jun 20, 2022, at 12:44 PM, Sven Barth wrote: If the compiler can proove that the function reference never leaves the scope (that means no assignment to global variables, out/var parameters, the Result variable, passing on to some other function or conversion to pointer (and I'm sure there are others)) and the function reference isn't used as an interface then and only then the compiler could reduce this to something else. But that's an optimization for another day and in my opinion such a rarlely used usecase that it's simply not worth it.

Maybe it’s off topic but it’s all very strange to me that FPC is making you think in advance what the caller of the function pointer will provide. In every other language I’ve used you simply declare a function pointer type and you can give it anything you want, a nested function, global function, method or anonymous function and the compiler figures out how to call it for you.

The function reference seems like it could almost fulfill that role but it comes with this extra baggage which isn’t always needed and hence the calls for optimizing it way.

Having said that, if nothing changes I think the only time you will ever use the reference type is if you know it needs to survive beyond the calling scope, i.e a thread in most cases.

Regards,
Ryan Joseph

Reply by email

Hairy Pixels
to Sven Barth: In that case we're in the same territory as always: ensure that you know what your code…

Can anyone explain why this works? TProc is a normal function pointer so how did it capture gVar? My understanding is that only nested proc vars could do this.

{$mode objfpc}
{$modeswitch anonymousfunctions}
program anonymous_functions;
type
  TProc = procedure;
procedure Call(proc: TProc);
begin
  proc;
end;
var
  gVar: integer;
begin
    Call(procedure
         begin
           writeln(gVar);
         end);
end.

Regards,
Ryan Joseph

Reply by email

Michael Van Canneyt
to Hairy Pixels: Maybe it’s off topic but it’s all very strange to me that FPC is making you think in…
Hairy Pixels wrote 11 lines
> On Jun 20, 2022, at 12:44 PM, Sven Barth wrote: > > If the compiler can proove that the function reference never leaves the scope (that means no assignment to global variables, out/var parameters, the Result variable, passing on to some other function or conversion to pointer (and I'm sure there are others)) and the function reference isn't used as an interface then and only then the compiler could reduce this to something else. But that's an optimization for another day and in my opinion such a rarlely used usecase that it's simply not worth it. Maybe it’s off topic but it’s all very strange to me that FPC is making you think in advance what the caller of the function pointer will provide. In every other language I’ve used you simply declare a function pointer type and you can give it anything you want, a nested function, global function, method or anonymous function and the compiler figures out how to call it for you. The function reference seems like it could almost fulfill that role but it comes with this extra baggage which isn’t always needed and hence the calls for optimizing it way. Having said that, if nothing changes I think the only time you will ever use the reference type is if you know it needs to survive beyond the calling scope, i.e a thread in most cases.

Unless I misunderstood your meaning, not really:

It allows you to run callbacks without having to declare variables globally:

Function DoSomething : String;
var
   L : TStringList;
begin
   L:=TstringList.Create;
   SomeCallBack(procedure(s : string)
    begin
      L.Add(S);
    end
   );
   Result:=L.Text;
   L.free;
end.

With a regular procedure or procedure of object, you'd need to define L outside of DoSomething.

Michael.

Reply by email

Hairy Pixels
to Michael Van Canneyt: My point is that if someone creates an API where he/she uses "reference to procedure"…
Show quoted text 7 lines
On Jun 20, 2022, at 8:30 PM, Michael Van Canneyt via fpc-pascal wrote: My point is that if someone creates an API where he/she uses "reference to procedure" then the consumer of the API is stuck with the creation of an interface. So it's not always a choice whether an interface will be used or not. My conclusion is then that 'reference to' should be used sparingly in public APIs as they are (relatively) expensive.

I just responded with basically this same point. It’s tempting to use a reference because it accepts all function types but it has this baggage. If you’re making an API and want the caller to be able to user nested, global or methods than you need to either duplicate code or use reference to and take the penalty hit.

My opinion is the compiler needs a new type which wraps these all up and lets us just think about function pointers and not what the programmer is going to supply at the time of calling.

If that won’t happen then the references should be strictly limited to instances where you need the call to survive past the calling scope and share state. If you don’t need to share state then I think you could even pass a nested proc var to a thread and it would be called later the same as a reference but the state is captured would be static.

Regards,
Ryan Joseph

Reply by email

Hairy Pixels
to Michael Van Canneyt: Unless I misunderstood your meaning, not really: It allows you to run callbacks without…
Show quoted text 22 lines
On Jun 20, 2022, at 8:39 PM, Michael Van Canneyt via fpc-pascal wrote: It allows you to run callbacks without having to declare variables globally: Function DoSomething : String; var L : TStringList; begin L:=TstringList.Create; SomeCallBack(procedure(s : string) begin L.Add(S); end ); Result:=L.Text; L.free; end. With a regular procedure or procedure of object, you'd need to define L outside of DoSomething.

I think you could use “is nested” and it would work also but see my other email with my example of a plain procedure which seems to capture for some reason.

Regards,
Ryan Joseph

Reply by email

Sven Barth
to Hairy Pixels: Maybe it’s off topic but it’s all very strange to me that FPC is making you think in…

Hairy Pixels schrieb am Mo., 20. Juni 2022, 15:30:

Show quoted text 20 lines
> On Jun 20, 2022, at 12:44 PM, Sven Barth wrote: > > If the compiler can proove that the function reference never leaves the scope (that means no assignment to global variables, out/var parameters, the Result variable, passing on to some other function or conversion to pointer (and I'm sure there are others)) and the function reference isn't used as an interface then and only then the compiler could reduce this to something else. But that's an optimization for another day and in my opinion such a rarlely used usecase that it's simply not worth it. Maybe it’s off topic but it’s all very strange to me that FPC is making you think in advance what the caller of the function pointer will provide. In every other language I’ve used you simply declare a function pointer type and you can give it anything you want, a nested function, global function, method or anonymous function and the compiler figures out how to call it for you.

Then you've never used C++...

Show quoted text 8 lines
The function reference seems like it could almost fulfill that role but it comes with this extra baggage which isn’t always needed and hence the calls for optimizing it way. Having said that, if nothing changes I think the only time you will ever use the reference type is if you know it needs to survive beyond the calling scope, i.e a thread in most cases.

You misjudge how function references are used in common Delphi code. There is nearly no case where optimizing it away would work. Thus it is simply not important to optimize this.

Regards,
Sven

Reply by email

Sven Barth
to Michael Van Canneyt: My point is that if someone creates an API where he/she uses "reference to procedure"…

Michael Van Canneyt via fpc-pascal
schrieb am Mo., 20. Juni 2022, 15:41:

Show quoted text 77 lines
On Mon, 20 Jun 2022, Sven Barth wrote: > Michael Van Canneyt schrieb am Mo., 20. Juni 2022, > 08:04: > >> >> >> On Mon, 20 Jun 2022, Sven Barth via fpc-pascal wrote: >> >>> Am 20.06.2022 um 03:37 schrieb Hairy Pixels: >>>> >>>>> On Jun 19, 2022, at 10:04 PM, Sven Barth < > >> >>> wrote: >>>>> >>>>> As you can see the allocation only happens once and not all the time. >>>>> What might be worse however is the optimization behavior as in this >>> example the compiler wouldn't optimize the counter variable into a >> regvar >>> with enabled optimizations (however in case of a nested function the >> compiler >>> wouldn't do this either if the function isn't inlined). >>>>> >>>> >>>> I’m saying if the OUTER method is in a tight loop (Foo in this example) >>> you’re in trouble because it now contains an allocation which is >> unexpected. >>> >>> In that case we're in the same territory as always: ensure that you know >>> what your code does. It's the same reason why adding character by >>> character to a managed String is slower then allocating the string once >>> and then setting the characters. And I think it's very seldom that >>> someone uses a function reference that does not leave the scope of the >>> surrounding function. >> >> Does this not depend on the callback declaration ? >> >> Type >> FuncRef = reference to procedure(a : integer); >> >> Procedure DoSomething(f : FuncRef); >> begin >> end; >> >> Procedure UseSomething; >> >> begin >> For I:=X to SomeLimit do >> DoSomething(Whatever) >> end; >> >> Does the declaration of DoSomething (which uses reference to) not >> ensure that an interface is created in UseSomething ? > > > Yes, in your example this is the case, but look at mine again: it's > unnecessary to create the capture object here and in theory the compiler > could convert it to an ordinary nested function variable. My point is that if someone creates an API where he/she uses "reference to procedure" then the consumer of the API is stuck with the creation of an interface. So it's not always a choice whether an interface will be used or not. My conclusion is then that 'reference to' should be used sparingly in public APIs as they are (relatively) expensive. Unless I missed/misunderstod something of course...

No, you're essentially right here. Though the passing onto the function itself is only an adjustment of the reference count and the creation of the capturer amortizes with the amount of different anonymous functions you have inside a function (the more the less is the cost per anonymous function).

Regards,
Sven

Reply by email