Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2022-09

Function reference doesn't capture in thread

7 messages · 2 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Hairy Pixels

Can anyone explain why this program doesn’t capture the state of the local variable “I” and execute in order? It will return something like this:

Invoked: 3 id: 00000001029D41C0
Invoked: 0 id: 00000001029D42C0
Invoked: 0 id: 00000001029D44C0
Invoked: 0 id: 00000001029D43C0

It works if you call Start directly before WaitFor, but why? I would expect the function reference to capture i, start the thread and then block the program after the sleep calls but instead the sleep calls appear to do nothing and the program exists immediately.

=================================

{$mode objfpc}
{$modeswitch anonymousfunctions}
{$modeswitch functionreferences}
{$modeswitch arrayoperators}
program test;
uses
  Cthreads, SysUtils, Classes;
type
  TProc = reference to procedure;
  TCallback = class(TThread)
    private
      proc: TProc;
    public
      constructor Create(p: TProc);
      procedure Execute; override;
end;
procedure TCallback.Execute;
begin
  proc();
  Sleep(100);
end;
constructor TCallback.Create(p: TProc);
begin
  inherited Create(true);
  proc := p;
end;
var
  i: integer;
  callback: TCallback;
  callbacks: array of TCallback = ();
begin
  for i := 1 to 4 do
    begin
      callback := TCallback.Create(procedure
                  begin
                    writeln('Invoked: ', i, ' id: ', HexStr(TThread.CurrentThread));
                  end);
      callback.Start;
      callbacks += [callback];
    end;
  for i := 0 to High(callbacks) do
    callbacks[i].WaitFor;
end.

Regards,
Ryan Joseph

Reply by email

Sven Barth
to Hairy Pixels
Hairy Pixels wrote 8 lines
Can anyone explain why this program doesn’t capture the state of the local variable “I” and execute in order? It will return something like this: Invoked: 3 id: 00000001029D41C0 Invoked: 0 id: 00000001029D42C0 Invoked: 0 id: 00000001029D44C0 Invoked: 0 id: 00000001029D43C0 It works if you call Start directly before WaitFor, but why? I would expect the function reference to capture i, start the thread and then block the program after the sleep calls but instead the sleep calls appear to do nothing and the program exists immediately.

It seems you haven't read the part about capturing variables in my announcement mail, cause you have two problems:
First the specific problem your code has: i is a global variables and global variables are *never* captured, because they don't need to. But even if you'd change your code so that the threads are initialized inside a procedure instead of the main block this would still not work, because variables are captured *by reference* which means that i would be shared between all newly created threads plus the main thread and depending on how the threads are scheduled the main thread will reach the call to WaitFor where i will again start from 0.

Regards,
Sven

Reply by email

Hairy Pixels
to Sven Barth
Show quoted text 5 lines
On Sep 17, 2022, at 7:40 PM, Sven Barth wrote: It seems you haven't read the part about capturing variables in my announcement mail, cause you have two problems: First the specific problem your code has: i is a global variables and global variables are *never* captured, because they don't need to. But even if you'd change your code so that the threads are initialized inside a procedure instead of the main block this would still not work, because variables are captured *by reference* which means that i would be shared between all newly created threads plus the main thread and depending on how the threads are scheduled the main thread will reach the call to WaitFor where i will again start from 0.

I didn’t know they were captured by reference! Does this mean they’re essentially pointers pointing the same variable?

Even so I’m trying now to modifying the example by copying to a local variable but I still see this same behavior suggesting there isn’t actually a copy. Why is this?

callback := TCallback.Create(procedure
                  var
                    index: integer;
                  begin
                    index := i;
                    writeln('Invoked: ', index, ' id: ', HexStr(TThread.CurrentThread));
                  end);

Regards,
Ryan Joseph

Reply by email

Sven Barth
to Hairy Pixels

Hairy Pixels schrieb am Sa., 17. Sep. 2022, 16:38:

Show quoted text 18 lines
> On Sep 17, 2022, at 7:40 PM, Sven Barth wrote: > > It seems you haven't read the part about capturing variables in my announcement mail, cause you have two problems: > First the specific problem your code has: i is a global variables and global variables are *never* captured, because they don't need to. > But even if you'd change your code so that the threads are initialized inside a procedure instead of the main block this would still not work, because variables are captured *by reference* which means that i would be shared between all newly created threads plus the main thread and depending on how the threads are scheduled the main thread will reach the call to WaitFor where i will again start from 0. I didn’t know they were captured by reference! Does this mean they’re essentially pointers pointing the same variable?

Then you didn't read my announcement mail deeply enough, cause it's mentioned there!
And they are not pointers, but the variable is moved to an object instance which implements the interface of the function reference.

Even so I’m trying now to modifying the example by copying to a local

variable but I still see this same behavior suggesting there isn’t actually a copy. Why is this?

When your callback is executed i might already have been changed!

Regards,
Sven

Reply by email

Hairy Pixels
to Sven Barth
On Sep 17, 2022, at 9:43 PM, Sven Barth wrote: Then you didn't read my announcement mail deeply enough, cause it's mentioned there!

Yes I just read that over and I remember it now. I feel like this not the same behavior in other languages although I’d need to test. Most recently I’m using Swift for my day job and I don’t recall ever having confusions like this come up.

And they are not pointers, but the variable is moved to an object instance which implements the interface of the function reference.

“Moved to an object” makes me think the variable is copied but the reference behavior you describe sounds like a pointer so I’m still not 100% clear on what you mean.

Show quoted text 5 lines
Even so I’m trying now to modifying the example by copying to a local variable but I still see this same behavior suggesting there isn’t actually a copy. Why is this? When your callback is executed i might already have been changed!

Ok that makes sense now since it’s just a reference. So this would have worked if I would have passed in i as a parameter and then stored it locally? I’m thinking now how I can preserve a copy of the state in the scope for each thread and I think local vars are the only way, correct? We need some more thread based examples to help people understand I think because I’m finding all sorts of edge cases updating a little thread library myself.

Regards,
Ryan Joseph

Reply by email

Sven Barth
to Hairy Pixels
Hairy Pixels wrote 5 lines
> On Sep 17, 2022, at 9:43 PM, Sven Barth wrote: > > Then you didn't read my announcement mail deeply enough, cause it's mentioned there! Yes I just read that over and I remember it now. I feel like this not the same behavior in other languages although I’d need to test. Most recently I’m using Swift for my day job and I don’t recall ever having confusions like this come up.

Complain to the developers of Delphi then.

I do however plan to extend anonymous (and nested) functions with a syntax that allows to select whether a variable should be by-reference or by-value. But that's still a bit off, cause I have other things to do first.

> And they are not pointers, but the variable is moved to an object instance which implements the interface of the function reference. “Moved to an object” makes me think the variable is copied but the reference behavior you describe sounds like a pointer so I’m still not 100% clear on what you mean.

Let's take your initial example and adjust it so that i will really be captured:

=== code begin ===

{$mode objfpc}
{$modeswitch anonymousfunctions}
{$modeswitch functionreferences}
{$modeswitch arrayoperators}
program test;
uses
   Cthreads, SysUtils, Classes;
type
   TProc = reference to procedure;
   TCallback = class(TThread)
     private
       proc: TProc;
     public
       constructor Create(p: TProc);
       procedure Execute; override;
end;
procedure TCallback.Execute;
begin
   proc();
   Sleep(100);
end;
constructor TCallback.Create(p: TProc);
begin
   inherited Create(true);
   proc := p;
end;
procedure DoTest;
var
   i: integer;
   callback: TCallback;
   callbacks: array of TCallback = ();
begin
   for i := 1 to 4 do
     begin
       callback := TCallback.Create(procedure
                   begin
                     writeln('Invoked: ', i, ' id: ',
HexStr(TThread.CurrentThread));
                   end);
       callback.Start;
       callbacks += [callback];
     end;
   for i := 0 to High(callbacks) do
     callbacks[i].WaitFor;
end;
begin
   DoTest;
end.

=== code end ===

What the compiler will make of it will be as follows (the only difference will be inside DoTest):

=== code begin ===

procedure DoTest;
type
   TCapturer = class(TInterfacedObject, TProc)
     i: integer;
     procedure Anonymous1;
     procedure TProc.Invoke = Anonymous1;
   end;
   procedure TCapturer.Anonymous1;
   begin
     writeln('Invoked: ', i, ' id: ', HexStr(TThread.CurrentThread));
   end;
var
   capturer: TCapturer;
   capturer_keepalive: IUnknown;
   callback: TCallback;
   callbacks: array of TCallback = ();
begin
   capturer := TCapturer.Create;
   capturer_keepalive := capturer;
   for capturer.i := 1 to 4 do
     begin
       callback := TCallback.Create(capturer as TProc);
       callback.Start;
       callbacks += [callback];
     end;
   for capturer.i := 0 to High(callbacks) do
     callbacks[capturer.i].WaitFor;
end;

=== code end ===

That should make it clearer what is happening behind the scenes.

Show quoted text 5 lines
> > Even so I’m trying now to modifying the example by copying to a local variable but I still see this same behavior suggesting there isn’t actually a copy. Why is this? > > When your callback is executed i might already have been changed! Ok that makes sense now since it’s just a reference. So this would have worked if I would have passed in i as a parameter and then stored it locally? I’m thinking now how I can preserve a copy of the state in the scope for each thread and I think local vars are the only way, correct? We need some more thread based examples to help people understand I think because I’m finding all sorts of edge cases updating a little thread library myself.

You need to move the creation of the thread to a separate nested function (cause each thread will then have a separate capture object; also this will work no matter if this is done in the main block or inside a procedure/function/method):

=== code begin ===

function CreateCallback(aIndex: Integer): TCallback;
begin
     Result := TCallback.Create(procedure
                 begin
                   writeln('Invoked: ', aIndex, ' id: ',
HexStr(TThread.CurrentThread));
                 end);
end;
var
   i: integer;
   callback: TCallback;
   callbacks: array of TCallback = ();
begin
   for i := 1 to 4 do
     begin
       callback := CreateCallback(i);
       callback.Start;
       callbacks += [callback];
     end;
   for i := 0 to High(callbacks) do
     callbacks[i].WaitFor;
end.

=== code end ===

Regards,
Sven

Reply by email

Hairy Pixels
to Sven Barth
Show quoted text 5 lines
On Sep 17, 2022, at 10:51 PM, Sven Barth wrote: Complain to the developers of Delphi then. I do however plan to extend anonymous (and nested) functions with a syntax that allows to select whether a variable should be by-reference or by-value. But that's still a bit off, cause I have other things to do first.

Yes that’s a very good idea and gives us more control (C++ and Swift allow explicit capture groups for example). In this example though would it even matter? The idea of capturing a loop iterator before the loops runs doesn't make sense to me. Maybe it should even be illegal or give a warning since it’s so error prone.

Speaking of that the Extended RTTI branch I did is over a year old I think now. Probably no chance of merging that without serious work. Michael got the RTL stuff done and tested everything but I’ve nearly forgotten about it since and I don’t know where it really stands. Hopefully that can be merged before it gets even more stale and decrepit. :)

Show quoted text 6 lines
> >> And they are not pointers, but the variable is moved to an object instance which implements the interface of the function reference. > “Moved to an object” makes me think the variable is copied but the reference behavior you describe sounds like a pointer so I’m still not 100% clear on what you mean. Let's take your initial example and adjust it so that i will really be captured:

Yeah I get this now. I was just being dumb with threads. The capturing isn’t even the most of my problems here.

Regards,
Ryan Joseph

Reply by email