Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2016-04

Management operators AddRef and Copy vs Copy

14 messages · 5 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Maciej Izak

Hi,

Sven proposed that we need to discuss AddRef operator (important part of smart pointers implementation).

AddRef is used when record is passed as parameter to method/function by value (for records to large to copy (only the address is pushed)). AddRef is used also for dynamic array operations (temporary for SetLength operation and for Copy operation for already copied data by move).

AddRef is much faster than Copy (all data already exist, so we don't need to copy anything). AddRef is used to increment ref count for managed types.

IMO the best solution is to implement two operators:

    class operator AddRef(var aFoo: TFoo);
    class operator Copy(constref aSrc: TFoo; var aDst: TFoo);

also is possible to implement single operator:

    class operator Copy(aSrc: PFoo; var aDst: TFoo);

aSrc in that case can be nil (for AddRef operation). Anyway that is inelegant solution...

Any other solution means big performance loss for RTL operations for dynamic arrays.

Best regards, Maciej Izak

Reply by email

Florian Klämpfl
to Maciej Izak
Maciej Izak wrote 24 lines
Hi, Sven proposed that we need to discuss AddRef operator (important part of smart pointers implementation). AddRef is used when record is passed as parameter to method/function by value (for records to large to copy (only the address is pushed)). AddRef is used also for dynamic array operations (temporary for SetLength operation and for Copy operation for already copied data by move). AddRef is much faster than Copy (all data already exist, so we don't need to copy anything). AddRef is used to increment ref count for managed types. IMO the best solution is to implement two operators: class operator AddRef(var aFoo: TFoo); class operator Copy(constref aSrc: TFoo; var aDst: TFoo); also is possible to implement single operator: class operator Copy(aSrc: PFoo; var aDst: TFoo); aSrc in that case can be nil (for AddRef operation). Anyway that is inelegant solution... Any other solution means big performance loss for RTL operations for dynamic arrays.

I think this is the wrong way:
- AddRef means only to increase the ref. count of the passed data structure
- Copy is no deep copy, it means only: copy the current data structure, if it references managed types, their ref. count is increased but they are not copied

Example with dyn. arrays:
- AddRef increases the ref. count of the passed array, not its members
- Copy creates a copy of the array, if the members of the array are managed, their ref. count is increased

So I would call the operator Copy, in case of custom implemented ref. counted records, this would be an AddRef operation. For a real copy, a procedure like Clone should be declared.

Reply by email

Maciej Izak
to Florian Klämpfl

2016-04-10 14:45 GMT+02:00 Florian Klämpfl :

Show quoted text 17 lines
I think this is the wrong way: - AddRef means only to increase the ref. count of the passed data structure - Copy is no deep copy, it means only: copy the current data structure, if it references managed types, their ref. count is increased but they are not copied Example with dyn. arrays: - AddRef increases the ref. count of the passed array, not its members - Copy creates a copy of the array, if the members of the array are managed, their ref. count is increased So I would call the operator Copy, in case of custom implemented ref. counted records, this would be an AddRef operation. For a real copy, a procedure like Clone should be declared.

Proposed implementation is compatible with internal rtti.inc usage (FPC_ADDREF and FPC_COPY). In my example I mean dyn array of records and related usage of operators only in context of items of those array.

Important note:
operator Initialize is called after system int_initialize for required record fields
operator Finalize is called before system int_finalize for required record fields
operator AddRef is called after int_addref for required fields operator Copy is called after fpc_Copy_internal for required fields

note for note:
in current FPC implementation int_initialize = FPC_INITIALIZE, int_finalize = FPC_FINALIZE, int_addref = FPC_ADDREF, fpc_Copy_internal = FPC_COPY

Let me explain this with complex example:

=== code begin ===
type
  TFoo = record // record with all management operators. Existence of
management operators means that the record became managed
  public
    {... some field definitions ...  }
  private
    class operator Initialize(var aFoo: TFoo);
    class operator Finalize(var aFoo: TFoo);
    class operator AddRef(var aFoo: TFoo);
    class operator Copy(constref aSrc: TFoo; var aDst: TFoo);
  end;
procedure TestValue(Value: TFoo); begin end;
procedure TestVar(var Value: TFoo); begin end;
procedure TestConst(const Value: TFoo); begin end;
procedure TestOut(out Value: TFoo); begin end;
procedure TestConstref(constref Value: TFoo); begin end;
  TFooArray = array of TFoo;
var
  Foos: TFooArray;
  Foos2: TFooArray;
begin
  SetLength(Foos, 5); // call 5x FPC_INITIALIZE and 5x TFoo.Initialize
  SetLength(Foos, 6); // call 1x FPC_INITIALIZE and 1x TFoo.Initialize
  SetLength(Foos, 5); // call 1x FPC_FINALIZE and 1x TFoo.Finalize
  Foos2 := Copy(Foos); // call 5x FPC_ADDREF and 5x TFoo.AddRef
  Foos2[0] := Foos[1]; // call 1x FPC_COPY and 1x TFoo.Copy
  // call 1x FPC_ADDREF and 1x TFoo.AddRef
  TestValue(Foos2[1]);
  // call 1x FPC_FINALIZE and 1x TFoo.Finalize
  // ... none
  TestVar(Foos2[1]);
  // ... none
  // ... none
  TestConst(Foos2[1]);
  // ... none
  // call 1x FPC_FINALIZE and 1x TFoo.Finalize
  // call 1x FPC_INITIALIZE and 1x TFoo.Initialize
  TestOut(Foos2[1]);
  // ... none
  // ... none
  TestConstref(Foos2[1]);
  // ... none
end;  // call 10x FPC_FINALIZE and 10x TFoo.Finalize (for Foos and Foos2)
=== code end ===

As far as I understand your proposal is to use TFoo.Clone instead of TFoo.Copy and TFoo.Copy instead of TFoo.AddRef. That is correct? IMO a little confusing from system.pp point of view ;)

Best regards, Maciej Izak

Reply by email

Maciej Izak
to Maciej Izak

2016-04-10 18:53 GMT+02:00 Maciej Izak :

Show quoted text 9 lines
Important note: operator Initialize is called after system int_initialize for required record fields operator Finalize is called before system int_finalize for required record fields operator AddRef is called after int_addref for required fields operator Copy is called after fpc_Copy_internal for required fields

Small correction. Copy operator(if declared) is executed instead of fpc_Copy_internal (any other behavior has no sense). To call default fpc_Copy_internal operation is possible to call CopyArray. For example

class operator TFoo.Copy(constref aSrc: TFoo; var aDst: TFoo);
begin
  CopyArray(@aDst, @aSrc, TypeInfo(TFoo), 1);
end;

I need to fix/adjust this on branch.

Best regards, Maciej Izak

Reply by email

Florian Klämpfl
to Maciej Izak: 2016-04-10 14:45 GMT+02:00 Florian Klämpfl : Proposed implementation is compatible with…
Maciej Izak wrote 86 lines
2016-04-10 14:45 GMT+02:00 Florian Klämpfl < >: I think this is the wrong way: - AddRef means only to increase the ref. count of the passed data structure - Copy is no deep copy, it means only: copy the current data structure, if it references managed types, their ref. count is increased but they are not copied Example with dyn. arrays: - AddRef increases the ref. count of the passed array, not its members - Copy creates a copy of the array, if the members of the array are managed, their ref. count is increased So I would call the operator Copy, in case of custom implemented ref. counted records, this would be an AddRef operation. For a real copy, a procedure like Clone should be declared. Proposed implementation is compatible with internal rtti.inc usage (FPC_ADDREF and FPC_COPY). In my example I mean dyn array of records and related usage of operators only in context of items of those array. Important note: operator Initialize is called after system int_initialize for required record fields operator Finalize is called before system int_finalize for required record fields operator AddRef is called after int_addref for required fields operator Copy is called after fpc_Copy_internal for required fields note for note: in current FPC implementation int_initialize = FPC_INITIALIZE, int_finalize = FPC_FINALIZE, int_addref = FPC_ADDREF, fpc_Copy_internal = FPC_COPY Let me explain this with complex example: === code begin === type TFoo = record // record with all management operators. Existence of management operators means that the record became managed public {... some field definitions ... } private class operator Initialize(var aFoo: TFoo); class operator Finalize(var aFoo: TFoo); class operator AddRef(var aFoo: TFoo); class operator Copy(constref aSrc: TFoo; var aDst: TFoo); end; procedure TestValue(Value: TFoo); begin end; procedure TestVar(var Value: TFoo); begin end; procedure TestConst(const Value: TFoo); begin end; procedure TestOut(out Value: TFoo); begin end; procedure TestConstref(constref Value: TFoo); begin end; TFooArray = array of TFoo; var Foos: TFooArray; Foos2: TFooArray; begin SetLength(Foos, 5); // call 5x FPC_INITIALIZE and 5x TFoo.Initialize SetLength(Foos, 6); // call 1x FPC_INITIALIZE and 1x TFoo.Initialize SetLength(Foos, 5); // call 1x FPC_FINALIZE and 1x TFoo.Finalize Foos2 := Copy(Foos); // call 5x FPC_ADDREF and 5x TFoo.AddRef Foos2[0] := Foos[1]; // call 1x FPC_COPY and 1x TFoo.Copy // call 1x FPC_ADDREF and 1x TFoo.AddRef TestValue(Foos2[1]); // call 1x FPC_FINALIZE and 1x TFoo.Finalize // ... none TestVar(Foos2[1]); // ... none // ... none TestConst(Foos2[1]); // ... none // call 1x FPC_FINALIZE and 1x TFoo.Finalize // call 1x FPC_INITIALIZE and 1x TFoo.Initialize TestOut(Foos2[1]); // ... none // ... none TestConstref(Foos2[1]); // ... none end; // call 10x FPC_FINALIZE and 10x TFoo.Finalize (for Foos and Foos2) === code end ===

Besides the naming, looks good to me.

As far as I understand your proposal is to use TFoo.Clone instead of TFoo.Copy and TFoo.Copy instead of TFoo.AddRef. That is correct?

Yes. Maybe we have to do some more research what appropriate names are, the names in system.pp are historic.

IMO a little confusing from system.pp point of view ;)

Yes. Addref is IMO something which is implementation specific and should not be exposed. C++ does not know natively (!) ref. counted types. Instead the copy constructor does the ref. counting and a separate method normally does the deep copy. So the question arises imo if the deep copy deserves an own operator.

Reply by email

Maciej Izak
to Florian Klämpfl

2016-04-12 21:41 GMT+02:00 Florian Klämpfl :

Yes. Maybe we have to do some more research what appropriate names are, the names in system.pp are historic.

Maybe is good idea to keep "Copy" operator name instead of proposed Clone. Copy name is compatible with other starndard RTL functions like Copy and CopyArray. I think is more intuitive for Pascal users. Clone is more "Java story".

Instead of AddRef my proposition is: PostCopy, AfterCopy, LateCopy, LazyCopy (it is not at all "lazy copy" by definition, but I like it). All copied data already exist but programmer still can do some actions post-factum (for example proper ref. counting). IMO best choice is LateCopy.

Best regards, Maciej Izak

Reply by email

Maciej Izak
to Maciej Izak

2016-04-13 21:45 GMT+02:00 Maciej Izak :

Show quoted text 18 lines
2016-04-12 21:41 GMT+02:00 Florian Klämpfl : > Yes. Maybe we have to do some more research what appropriate names are, > the names in system.pp are > historic. Maybe is good idea to keep "Copy" operator name instead of proposed Clone. Copy name is compatible with other starndard RTL functions like Copy and CopyArray. I think is more intuitive for Pascal users. Clone is more "Java story". Instead of AddRef my proposition is: PostCopy, AfterCopy, LateCopy, LazyCopy (it is not at all "lazy copy" by definition, but I like it). All copied data already exist but programmer still can do some actions post-factum (for example proper ref. counting). IMO best choice is LateCopy.

I'd like to nominate to the list "AutoCopy". Anyone? Any additional suggestions?

Best regards, Maciej Izak

Reply by email

Anthony Walter
to Maciej Izak

Do the source and dest types need to match? How about:"CopyFrom"? As it is right now you already have type parity between types such as floats <- integer and string <- char, so why not between TBar <- TFoo?

class procedure TBar.CopyFrom(constref Source: TFoo; var Dest: TBar);

Then this begs question, what's the difference between AutoCopy/CopyFrom and:

class operator TBar.Implicit(const Value: TFoo): TBar;

Reply by email

Maciej Izak
to Anthony Walter

2016-04-18 11:42 GMT+02:00 Anthony Walter :

Do the source and dest types need to match?

Yes.

Show quoted text 11 lines
How about:"CopyFrom"? As it is right now you already have type parity between types such as floats <- integer and string <- char, so why not between TBar <- TFoo? class procedure TBar.CopyFrom(constref Source: TFoo; var Dest: TBar); Then this begs question, what's the difference between AutoCopy/CopyFrom and: class operator TBar.Implicit(const Value: TFoo): TBar;

Management operators works in different way than standard operators. Any of management operator is strictly related to RTL code (system.pp), for record with management operators is exclusively created small kind of VMT for RTL purposes. In generally set of management operators can't be changed. Existence of

- class operator Initialize(var aFoo: TFoo);
- class operator Finalize(var aFoo: TFoo);
- class operator [Copy|OtherName](constref aSrc: TFoo; var aDst: TFoo);
- class operator [AddRef|OtherName](var aFoo: TFoo);

is strictly required. That is because:

1. Performance (especially AddRef operator for dynamic arrays - see my example for Florian)
2. Cooperation with RTL functions like New/Dispose, Initialize/Finalize, InitializeArray, FinalizeArray, TObject.InitInstance/TObject.CleanupInstance
3. Related to previous point: We need to cover actions for runtime, at compile time is impossible to cover all usages of records (like dynamically created instances of records, for example for scripts purposes).

Best regards, Maciej Izak

Reply by email

Maciej Izak
to Maciej Izak

Finally I've decided to use Clone and Copy (as proposed by Florian). Should I adjust RTL inner functions to this? I mean renaming FPC_COPY to FPC_CLONE and FPC_ADDREF to FPC_COPY.

Best regards, Maciej Izak

Reply by email

Michael Van Canneyt
to Maciej Izak
Maciej Izak wrote
Finally I've decided to use Clone and Copy (as proposed by Florian). Should I adjust RTL inner functions to this? I mean renaming FPC_COPY to FPC_CLONE and FPC_ADDREF to FPC_COPY.

Are these the assembler names of existing functions ?

If so, the reason these have an assembler name is that they can be called directly from assembler code. So maybe it is better not rename them, I would think, since we don't know what assembler code calls them. If you change them, then people that call these routines will not get any warning why their code suddenly behaves differently ?

Michael.

Reply by email

Florian Klaempfl
to Maciej Izak: Finally I've decided to use Clone and Copy (as proposed by Florian). Should I adjust RTL…
Maciej Izak wrote
Finally I've decided to use Clone and Copy (as proposed by Florian).

Glad to hear :) Another thing: I wonder if those really need to be operators? In C++, "clone" is also only a special constructor.

Should I adjust RTL inner functions to this? I mean renaming FPC_COPY to FPC_CLONE and FPC_ADDREF to FPC_COPY.

I wouldn't do this, it has been this way for years. Users who dig in this code, understand it, the others are not effected.

Reply by email

Maciej Izak
to Florian Klaempfl

2016-05-01 12:14 GMT+02:00 Florian Klaempfl :

Show quoted text 6 lines
Am 25.04.2016 um 16:47 schrieb Maciej Izak: > Finally I've decided to use Clone and Copy (as proposed by Florian). Glad to hear :) Another thing: I wonder if those really need to be operators? In C++, "clone" is also only a special constructor.

I feel that "operator" is more correct than "constructor". Operators are easier to read, any operator has unique name (for constructor/destructor, unique name is not so obvious).

For new constructors/destructors (to replace class operators Initialize/Finalize/Clone/Copy) is required new modifier (my candidate for this is "default" or "managed"). Even our "special constructor" just looks like another constructor, even with modifier. Additionally those constructors (I mean management operators) are directly inaccessible for normal "create" purpose. So that is also a little confusing... Anyway we have class constructor which is also implicitly executed...

"Management operators" name is clear for me and very conclusive :), so I like to keep them as operators.

Show quoted text 6 lines
Should I adjust RTL inner functions to this? I mean renaming FPC_COPY to > FPC_CLONE and FPC_ADDREF to FPC_COPY. I wouldn't do this, it has been this way for years. Users who dig in this code, understand it, the others are not effected.

Ok I agree.

Best regards, Maciej Izak

Reply by email

Maciej Izak
to Maciej Izak: Finally I've decided to use Clone and Copy (as proposed by Florian). Should I adjust RTL…

2016-04-25 16:47 GMT+02:00 Maciej Izak:

Finally I've decided to use Clone and Copy (as proposed by Florian). Should I adjust RTL inner functions to this? I mean renaming FPC_COPY to FPC_CLONE and FPC_ADDREF to FPC_COPY.

Done in rev 33657 (AddRef -> Copy and Copy -> Clone . All tests are fine. FPC_COPY and FPC_ADDREF names are untouched.

Best regards, Maciej Izak

Reply by email