Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2019-06

Copy/move operator

21 messages · 4 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Ryan Joseph

I just came across an efficiency issue when using management operators that I’d like to fix before the next release and we have backwards compatibility problems.

Basically the problem is that the Copy operator doesn’t know if the source value is actually mapped to memory or just a temporary value so we’re forced to always copy memory (just to be safe) even when a simple move would be good enough. C++ has a similar idea of rvalues and lvalues and a copy and move constructor which addresses this.

Here’s a little example of the problem. What I propose is that we either add a 3rd boolean parameter to the Copy operator or add a new Move operator which is preferred over the Copy operator if it exists (this is much more work obviously but maybe has some benefit).

================================

{$mode objfpc}
{$modeswitch advancedrecords}
program test;
type
  generic TList<T> = record
      data: array of T;
      constructor Create(num: integer);
      class operator Copy(constref src: TList; var dest: TList);
  end;
constructor TList.Create(num: integer);
begin
  SetLength(data, num);
end;
class operator TList.Copy(constref src: TList; var dest: TList);
begin
  // ...if we knew the “src” was a temporary would could simply swap the data...
  dest.data := system.copy(src.data);
end;
var
  a: specialize TList<integer>;
begin
  // TList.Create allocates a new array but we don’t really need to copy
  // on assignment because the source is a temporary object that doesn’t
  // actually exist at any static address.
  a := specialize TList<integer>.Create(10);
end.

Regards,
Ryan Joseph

Reply by email

Jonas Maebe
to Ryan Joseph
Ryan Joseph wrote 5 lines
Basically the problem is that the Copy operator doesn’t know if the source value is actually mapped to memory or just a temporary value so we’re forced to always copy memory (just to be safe) even when a simple move would be good enough. C++ has a similar idea of rvalues and lvalues and a copy and move constructor which addresses this.

And which also causes hard-to-find memory corruption bugs.

Here’s a little example of the problem. What I propose is that we either add a 3rd boolean parameter to the Copy operator or add a new Move operator which is preferred over the Copy operator if it exists (this is much more work obviously but maybe has some benefit).

The compiler knows that source is a temporary value that will be gone afterwards, so it could perform this optimisation itself in this case. It won't be possible to always do this and there are undoubtedly cases where a dedicated Move operator would result in more optimal code, but maintainability and safety are also important.

Jonas

Reply by email

Ryan Joseph
to Jonas Maebe
On Jun 17, 2019, at 2:19 AM, Jonas Maebe wrote: And which also causes hard-to-find memory corruption bugs.

How so? I looked into this yesterday and it’s pretty easy to tell if a node is temporary memory or not, but if there’s a chance it isn’t then calling the copy operator is still safe.

Show quoted text 7 lines
> Here’s a little example of the problem. What I propose is that we > either add a 3rd boolean parameter to the Copy operator or add a new > Move operator which is preferred over the Copy operator if it exists > (this is much more work obviously but maybe has some benefit). The compiler knows that source is a temporary value that will be gone afterwards, so it could perform this optimisation itself in this case. It won't be possible to always do this and there are undoubtedly cases where a dedicated Move operator would result in more optimal code, but maintainability and safety are also important.

If the compiler knows the source is temporary then why can’t the copy operator be told of this? This is specifically for the new management operators (which are always called) so the compiler can’t make any optimizations here anywhere. Seems like an obvious win to me to gain some performance and avoid unnecessary copies.

Regards,
Ryan Joseph

Reply by email

Sven Barth
to Ryan Joseph: I just came across an efficiency issue when using management operators that I’d like to…

Ryan Joseph schrieb am So., 16. Juni 2019, 15:29:

Show quoted text 15 lines
I just came across an efficiency issue when using management operators that I’d like to fix before the next release and we have backwards compatibility problems. Basically the problem is that the Copy operator doesn’t know if the source value is actually mapped to memory or just a temporary value so we’re forced to always copy memory (just to be safe) even when a simple move would be good enough. C++ has a similar idea of rvalues and lvalues and a copy and move constructor which addresses this. Here’s a little example of the problem. What I propose is that we either add a 3rd boolean parameter to the Copy operator or add a new Move operator which is preferred over the Copy operator if it exists (this is much more work obviously but maybe has some benefit).

If we do this at all we should use a separate operator as those two are rather different operations and the compiler would already have figured out which of the two operations is needed and thus the code inside the operator doesn't need to do that anymore.

That said this would require extending the RTL assignment helper.

And the problems that Jonas mentioned are less problems inside the compiler, but problems inside the user code when they write their Move operator incorrectly. From great power comes great responsibility after all and I'm not sure whether we should provide the users with that much power.

Regards,
Sven

Reply by email

Ben Grasset
to Ryan Joseph: I just came across an efficiency issue when using management operators that I’d like to…
AM Ryan Joseph wrote 5 lines
Here’s a little example of the problem. What I propose is that we either add a 3rd boolean parameter to the Copy operator or add a new Move operator which is preferred over the Copy operator if it exists (this is much more work obviously but maybe has some benefit).

I can't actually check the assembly at the moment, but I'm pretty sure if your example was written like the following, it would not call `Copy` for the `TList<Integer>.Create(10)` assignment:

{$mode objfpc}
{$modeswitch advancedrecords}
program test;
type
  generic TList<T> = record
      data: array of T;
      // static class function instead of actual constructor
      class function Create(num: integer): TList; static; inline;
      class operator Copy(constref src: TList; var dest: TList);
  end;
class function TList.Create(num: integer): TList;
begin
  SetLength(result.data, num);
end;
class operator TList.Copy(constref src: TList; var dest: TList);
begin
  // no reason not to use `move` here, I wouldn't say
  move(src.data, dest.data, sizeof(T) * Length(src.data));
end;
var
  a, b: specialize TList<integer>;
  i: integer;
begin
  // TList.Create allocates a new array but we don’t really need to copy
  // on assignment because the source is a temporary object that doesn’t
  // actually exist at any static address.
  a := specialize TList<integer>.Create(10);
  for i := 0 to 9 do a.data[i] := i;
  b := a;
  for i := 0 to 9 do writeln(b.data[i]);
end.

Reply by email

Ryan Joseph
to Ben Grasset
On Jun 17, 2019, at 5:16 PM, Ben Grasset wrote: I can't actually check the assembly at the moment, but I'm pretty sure if your example was written like the following, it would not call `Copy` for the `TList<Integer>.Create(10)` assignment:

The copy operator is always called on all assignments. Try actually running that and you’ll see. :)

Regards,
Ryan Joseph

Reply by email

Ryan Joseph
to Sven Barth: Ryan Joseph schrieb am So., 16. Juni 2019, 15:29: If we do this at all we should use a…
Show quoted text 7 lines
On Jun 17, 2019, at 1:00 PM, Sven Barth via fpc-devel wrote: If we do this at all we should use a separate operator as those two are rather different operations and the compiler would already have figured out which of the two operations is needed and thus the code inside the operator doesn't need to do that anymore. That said this would require extending the RTL assignment helper. And the problems that Jonas mentioned are less problems inside the compiler, but problems inside the user code when they write their Move operator incorrectly. From great power comes great responsibility after all and I'm not sure whether we should provide the users with that much power.

I see. Looking at this on Sunday I see in assignment nodes fpc_copy_proc would need to be modified or a new fpc_move_proc introduced which (for now at least) would only affect records which had a move operator. In theory the concept could be extended to dynamic arrays and other reference counted types but that’s another story.

Regards,
Ryan Joseph

Reply by email

Ben Grasset
to Ryan Joseph: The copy operator is always called on all assignments. Try actually running that and…
PM Ryan Joseph wrote
The copy operator is always called on all assignments. Try actually running that and you’ll see. :)

Yeah, I noticed that afterwards once I checked. I'm not sure how much sense that actually makes when the TList<T> is a function *result* as it is in the static class function version of Create, though (i.e, you're not assigning an existing TList<T> variable to another, you're returning a brand new one.)

The version of Copy in my last comment was missing a SetLength, also, and called `Move` incorrectly. Something like this is what I was going for:

program Test;
{$mode ObjFPC}
{$modeswitch AdvancedRecords}
type
  generic TList<T> = record
  public
    Data: array of T;
    class operator Copy(constref Src: TList; var Dest: TList); inline;
    class function Create(const Num: Integer): TList; static; inline;
  end;
  class operator TList.Copy(constref Src: TList; var Dest: TList);
  var Len: SizeInt;
  begin
    Len := Length(Src.Data);
    SetLength(Dest.Data, Len);
    Move(Src.Data[0], Dest.Data[0], SizeOf(T) * Len);
  end;
  class function TList.Create(const Num: Integer): TList;
  begin
    SetLength(Result.Data, Num);
  end;
begin
  // Do whatever here.
end.

Don't forget about your constant generics feature, also... can help in avoiding "constructorish" things altogether quite a bit.

Reply by email

Ben Grasset
to Ryan Joseph: The copy operator is always called on all assignments. Try actually running that and…
PM Ryan Joseph wrote
The copy operator is always called on all assignments. Try actually running that and you’ll see. :)

Actually, after thinking about it some more, here's what I'd actually write (in a perfect world where it was possible.)

program Test;
{$mode ObjFPC}
{$modeswitch AdvancedRecords}
type
  generic TList<T> = record
  public
    Data: array of T;
    procedure Assign(var Src: TList); inline;
    class operator Copy(constref Src: TList; var Dest: TList); //
apparently you cannot actually inline this...
    class function Create(const Num: Integer): TList; static; inline;
  end;
  procedure TList.Assign(var Src: TList);
  var Len: SizeInt;
  begin
    Len := Length(Src.Data);
    SetLength(Data, Len);
    Move(Src.Data[0], Data[0], SizeOf(T) * Len);
  end;
  class operator TList.Copy(constref Src: TList; var Dest: TList);
  begin
    // StaticAssert does not exist, unfortunately...
    StaticAssert(False, 'Don't use the := operator, call Assign instead!');
  end;
  class function TList.Create(const Num: Integer): TList;
  begin
    SetLength(Result.Data, Num);
  end;
begin
  // Do whatever here.
end.

Reply by email

Ryan Joseph
to Ben Grasset
On Jun 18, 2019, at 10:00 AM, Ben Grasset wrote: Actually, after thinking about it some more, here's what I'd actually write (in a perfect world where it was possible.)

What I was proposing is that we add another operator called “Move” which will be called instead of “Copy” if the right value (dest) is a temporary value (i.e., not mapped to static memory). Sven and Jonas hinted this could be dangerous but I personally think it’s pretty manageable (and completes the management operators).

Not sure about inline though, I don’t think the compiler inlines these seems to have lots of non-inlinable code these days which is too bad. Maybe the LLVM backend will fix that someday?

Regards,
Ryan Joseph

Reply by email

Sven Barth
to Ryan Joseph

Ryan Joseph schrieb am Di., 18. Juni 2019, 16:09:

Not sure about inline though, I don’t think the compiler inlines these seems to have lots of non-inlinable code these days which is too bad. Maybe the LLVM backend will fix that someday?

Those operators can't be inlined, because they're called from helper code, not from the compiler directly. The LLVM backend won't change anything here.

Regards,
Sven

Reply by email

Ben Grasset
to Sven Barth

On Tue, Jun 18, 2019 at 11:08 AM Sven Barth via fpc-devel <

Show quoted text 5 lines
wrote: Those operators can't be inlined, because they're called from helper code, not from the compiler directly. The LLVM backend won't change anything here.

Is there any real purpose to implementing `Copy` versus just implementing `operator :=` (or `operator Implicit` when in {$mode Delphi})? Those can be inlined, and in practice are used in essentially all the places `Copy` is. By which I mean, IIRC you never really see calls to `fpc_copy_proc` in the generated assembler unless you've specifically implemented `Copy` yourself.

Reply by email

Ryan Joseph
to Ben Grasset
On Jun 19, 2019, at 10:21 AM, Ben Grasset wrote: Is there any real purpose to implementing `Copy` versus just implementing `operator :=` (or `operator Implicit` when in {$mode Delphi})? Those can be inlined, and in practice are used in essentially all the places `Copy` is. By which I mean, IIRC you never really see calls to `fpc_copy_proc` in the generated assembler unless you've specifically implemented `Copy` yourself.

Copy operator is also called with dynamic arrays/open arrays. For example in the code below the copy operator will be called for each of the 3 records. This is why we need a Move operator because if TMyRec.Create allocates memory and then is passed directly to the array we will need to perform another copy when a simple swap would be sufficient (because the records are allocated in temporary memory).

a := [TMyRec.Create(1),TMyRec.Create(1),TMyRec.Create(1)];

Regards,
Ryan Joseph

Reply by email

Sven Barth
to Ben Grasset: On Tue, Jun 18, 2019 at 11:08 AM Sven Barth via fpc-devel < Is there any real purpose to…

Ben Grasset schrieb am Mi., 19. Juni 2019, 16:21:

Show quoted text 14 lines
On Tue, Jun 18, 2019 at 11:08 AM Sven Barth via fpc-devel < > wrote: > Those operators can't be inlined, because they're called from helper > code, not from the compiler directly. The LLVM backend won't change > anything here. > Is there any real purpose to implementing `Copy` versus just implementing `operator :=` (or `operator Implicit` when in {$mode Delphi})? Those can be inlined, and in practice are used in essentially all the places `Copy` is. By which I mean, IIRC you never really see calls to `fpc_copy_proc` in the generated assembler unless you've specifically implemented `Copy` yourself.

It wouldn't matter if the function is called Copy, operator:= or "jdjfjsje". Once a copy management operator is defined the record type is considered "managed" and thus all assignment operations are routed through fpc_copy_proc just as is the case when a record contains at least one field of a managed type (e.g. AnsiString).
Also using operator:= would go against the principle that existing, internal operators can't be overloaded.

Regards,
Sven

Reply by email

Ben Grasset
to Sven Barth

On Wed, Jun 19, 2019 at 12:01 PM Sven Barth via fpc-devel <

Show quoted text 10 lines
wrote: It wouldn't matter if the function is called Copy, operator:= or "jdjfjsje". Once a copy management operator is defined the record type is considered "managed" and thus all assignment operations are routed through fpc_copy_proc just as is the case when a record contains at least one field of a managed type (e.g. AnsiString). Also using operator:= would go against the principle that existing, internal operators can't be overloaded.

What I meant was, does `Copy` actually serve any particular purpose at all versus just overloading the assignment operator normally?

Reply by email

Ryan Joseph
to Ben Grasset
On Jun 19, 2019, at 4:48 PM, Ben Grasset wrote: What I meant was, does `Copy` actually serve any particular purpose at all versus just overloading the assignment operator normally?

It’s used for making reference counted types at the user level.

Regards,
Ryan Joseph

Reply by email

Ben Grasset
to Ryan Joseph
PM Ryan Joseph wrote
It’s used for making reference counted types at the user level.

I'm not sure why you'd necessarily need to implement Copy specifically for that kind of thing, though. Initialize and Finalize would be much more relevant, I'd say.

Reply by email

Ben Grasset
to Ben Grasset
PM Ben Grasset wrote 9 lines
On Wed, Jun 19, 2019 at 5:02 PM Ryan Joseph wrote: > It’s used for making reference counted types at the user level. > I'm not sure why you'd necessarily need to implement Copy specifically for that kind of thing, though. Initialize and Finalize would be much more relevant, I'd say.

Also, either way, the "managed record" overhead really isn't great. Try first un-commenting-out `S: AnsiString` in this Compiler Explorer example <https://godbolt.org/#g:!((g:!((g:!((h:codeEditor,i:(j:1,lang:pascal,source:'unit+Output%3B%0A%0A%7B$mode+ObjFPC%7D%0A%7B$modeswitch+AdvancedRecords%7D%0A%7B$ImplicitExceptions+Off%7D%0A%0Ainterface%0A%0Atype+TLetter+%3D+(A,+B,+C)%3B%0A%0Atype+%0A++TRecord+%3D+record%0A++++//S:+AnsiString%3B%0A++++Letter:+TLetter%3B%0A++++class+function+Make(const+ALetter:+TLetter):+TRecord%3B+static%3B+inline%3B%0A++++procedure+Proc%3B+inline%3B%0A++++class+procedure+DoIt(const+A:+array+of+TRecord)%3B+static%3B+%0A++++class+procedure+DoDoIt%3B+static%3B+inline%3B%0A++end%3B%0A%0Aimplementation%0A%0Aclass+function+TRecord.Make(const+ALetter:+TLetter):+TRecord%3B%0Abegin%0A++Result.Letter+:%3D+ALetter%3B%0Aend%3B%0A%0Aprocedure+TRecord.Proc%3B+%0Abegin%0A++WriteLn(%5BB%5D+%3C%3D+%5BA,+B,+C%5D)%3B%0Aend%3B%0A%0Aclass+procedure+TRecord.DoIt(const+A:+array+of+TRecord)%3B+%0Avar+Start,@A%5BHigh(A)%5D%3B%0A++while+Start+%3C%3D+Finish+do+begin%0A++++Start%5E.Proc()%3B%0A++++Inc(Start)%3B%0A++end%3B%0Aend%3B%0A%0Aclass+procedure+TRecord.DoDoIt%3B+%0Abegin%0A++DoIt(%5BTRecord.Make(A),+TRecord.Make(B),+TRecord.Make(C)%5D)%3B%0Aend%3B%0A%0Aend.'),l:'5',n:'0',o:'Pascal+source+%231',t:'0')),k:57.25490196078431,l:'4',n:'0',o:'',s:0,t:'0'),(g:!((h:compiler,i:(compiler:fpc304,filters:(b:'0',binary:'1',commentOnly:'0',demangle:'0',directives:'0',execute:'1',intel:'0',libraryCode:'1',trim:'0'),lang:pascal,libs:!(),options:'-Ci-+-Cr-+-g-+-CX+-XXs+-O4',source:1),l:'5',n:'0',o:'x86-64+fpc+3.0.4+(Editor+%231,+Compiler+%231)+Pascal',t:'0')),k:42.74509803921569,l:'4',n:'0',o:'',s:0,t:'0')),l:'2',n:'0',o:'',t:'0')),version:4>, and then afterwards re-commenting-out `{$ImplicitExceptions Off}`.

Reply by email

Ryan Joseph
to Ben Grasset: I'm not sure why you'd necessarily need to implement Copy specifically for that kind of…
On Jun 19, 2019, at 5:45 PM, Ben Grasset wrote: I'm not sure why you'd necessarily need to implement Copy specifically for that kind of thing, though. Initialize and Finalize would be much more relevant, I'd say.

Because a copy isn’t necessarily an assignment. See the dynamic array example again. The only way the record knows it was copied into the array is via the copy operator.

Regards,
Ryan Joseph

Reply by email

Ben Grasset
to Ryan Joseph
PM Ryan Joseph wrote
Because a copy isn’t necessarily an assignment. See the dynamic array example again. The only way the record knows it was copied into the array is via the copy operator.

I guess what I'm trying to get at is:

A) Most of the time, the default behaviour of what fpc_copy_proc does seems like it would be what you wanted anyways

B) The level of indirection between a `Copy` operator you do decide to implement yourself and how it ends up getting called seems undesirable (`fpc_copy_proc` actually itself does nothing but call a function called `fpc_copy`, for example, and obviously is not inlined away despite the fact that whoever wrote it did actually mark it as `inline.`)

Reply by email

Sven Barth
to Ben Grasset: I'm not sure why you'd necessarily need to implement Copy specifically for that kind of…

Ben Grasset schrieb am Mi., 19. Juni 2019, 23:45:

Show quoted text 9 lines
On Wed, Jun 19, 2019 at 5:02 PM Ryan Joseph wrote: > It’s used for making reference counted types at the user level. > I'm not sure why you'd necessarily need to implement Copy specifically for that kind of thing, though. Initialize and Finalize would be much more relevant, I'd say.

You need a Copy operation if you need to take some action during that operation. Main example is adjusting a reference count so that a class instance can be freed at the correct time (smart pointer).

Regards,
Sven

Reply by email