Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2017-02

Feature announcement: Management Operators

15 messages · 8 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Maciej Izak

Hi,

I'm pleased to finally announce (again but now officially ;) ) the additional record operators: Initialize, Finalize, AddRef and Copy. Available in latest FPC trunk (r35485):

1. operator Initialize - is called after memory allocation for record (called *after* FPC internal call recordrtti(data,typeinfo,@int_initialize);)

2. operator Finalize - is called when record goes out of scope (called *before* FPC internal call recordrtti(data,typeinfo,@int_finalize);)

3. operator AddRef - is called after the contents of a record has been duplicated by copying the contents byte by byte (called *after* FPC internal call recordrtti(data,typeinfo,@int_addref); )

4. operator Copy - Copy operator (if exists) is called *instead of* default copy behavior. Operator is responsible for copying everything that's needed from the source to the target.

AddRef and Copy might be at beginning a little hard for understanding but we have good explanation provided by Thorsten Engler in topic "Management operators : Copy and Clone confusion..." :

=== quote begin ===
the reason for both to exist is that there are cases (e.g. passing an interface parameter) where the callee is responsible for any lifetime management implications (and the caller might not even be implemented using fpc). The caller just passes the interface pointer on the stack (or in a register), which, while it creates a copy of the bytes, does not actually invoke the copy operator. In cases where reference counting is required (parameter not marked as const), the callee may decide to simply use the value that was passed on the stack as the backing for the parameter, so it calls AddRef on that, without performing another copy.

Given this context, I find the terms AddRef and Copy to be perfectly adequate, even when applied to types that aren't strictly speaking performing reference counting; e.g. IIRC, AddRef for a BSTR (ole string type for windows which does NOT have a reference count) actually involves making a copy of the string contents and replacing the BSTR pointer.

AddRef here means "this is now an *additional* *reference* to whatever these bytes might refer to".
=== quote end ===

my 2 cents for above: AddRef exist to speed up things.

New set of operators is unique and is called "management operators". That is because:

A. each of record (even non managed or even empty) with management operator becomes managed type.

B. is possible to implement new custom types (also thanks to other operators) with own management of memory (new string types, fast TValue implementations without hacks on RTL etc.).

C. "management operators" have no result.

D. For management operators is generated simple VMT (thanks to this is possible to work with management operators together with all RTL functions like InitializeArray/FinalizeArray).

=== example declaration begin ===
  PFoo = ^TFoo;
  TFoo = record
  private
    class operator Initialize(var a: TFoo);
    class operator Finalize(var a: TFoo);
    class operator AddRef(var a: TFoo);
    class operator Copy(constref aSrc: TFoo; var aDst: TFoo);
  end;
=== example declaration end ===

Management Operators feature can be used for many things:

* support for value types
* nullable types
* some custom ARC variations
* speed up existing types
* very fast RTTI.TValue implementation
* as replacement for manually called Init/Done record methods like in mORMot for many types (for example SynCommons.TSynLocker).
* auto init/finit for pointers/classes/simple types or whatever we have in Pascal

It works correctly in all possible ways with RTL:

* New (Initialize)
* Dispose (Finalize)
* Initialize (Initialize)
* Finalize (Finalize)
* InitializeArray (Initialize)
* FinalizeArray (Finalize)
* SetLength (Initialize/Finalize)
* Copy (AddRef)
* RTTI.IsManaged

Managements operators are often called implicitly in many cases. Few examples:

* global variables (Initialize/Finalize)
* local variables (Initialize/Finalize)
* for fields inside: records, objects, classes (Initialize/Finalize)
* variable assignment (Copy)
* for parameters for routines (AddRef/Finalize/none - depends on modifiers like var/constref/cons)

Own experiments recommended! :)

Complex example and test:
http://svn.freepascal.org/cgi-bin/viewvc.cgi/trunk/tests/tes t/tmoperator8.pp?view=co&content-type=text%2Fplain

Note: Lazarus (trunk version) has support for all new operators syntax (thanks for Mattias Gaertner).

Future directions and opened doors (where management operators can be used but not directly):

+ "default" field for better handling smart pointers / nullable types (maybe will be introduced new type called proxy record for handling "default" modifier) (eg. https://github.com/maciej-izak/PascalSmartPointers compilable with NewPascal)
+ New VMT entry "vmtManagedInit" for "Initialize operator" for record fields inside classes (optimization)
+ Base for Nullable types syntax sugar
+ Probably base in some way for ARC classes (DELPHINEXTGEN compiler mode)
+ More Oxygene compatibility
+ Aspects

Note for myself: document the feature on the wiki in New Features Trunk and the changes to the Rtti in User Changes Trunk :)

Best regards, Maciej Izak

Reply by email

Michael Van Canneyt
to Michael Van Canneyt
Michael Van Canneyt wrote 17 lines
On Tue, 28 Feb 2017, Maciej Izak wrote: > 2017-02-28 12:28 GMT+01:00 denisgolovan : > >> Thanks a lot. >> I guess http://bugs.freepascal.org/view.php?id=30687 should be closed now >> :) >> > > I don't have such powers ;) You do now. I took care of that :) But I set it to fixed for you. Seeing you were the reporter, you can set it to closed :)

By the way, thanks for the contribution.

Michael.

Reply by email

Unknown author
to Maciej Izak: I'm pleased to finally announce (again but now officially ;) ) the additional record…
Maciej Izak wrote
It works correctly in all possible ways with RTL: * New (Initialize) * Dispose (Finalize)

What happens with a stack allocated record?
(no new() required, nor dispose() so is the record ever initialized/finalized?)

Or stack records are forbidden?

One reason I love records is because you can create a stack record without any worry of freeing and creating it on heap, a huge advantage over OOP programming in delphi/fpc where everything is heap object (except old turbo pascal objects).

Sorry, I've been out of touch with what's happening with Records lately in modern pascal/delphi/etc.
(If it sounds like a silly newbie question, forgive me, I am not up on the latest and greatest!)

Reply by email

Graeme Geldenhuys
to Unknown author: What happens with a stack allocated record? (no new() required, nor dispose() so is the…
Sorry, I've been out of touch with what's happening with Records lately in modern pascal/delphi/etc.

Don't worry, I'm with you on that one. All I know is that Borland/CodeGear/EMBT broken TP Objects somewhere in Delphi. Then they decided they want that functionality back, so instead of fixing TP Objects, the decided to butcher Records to act just like TP Objects (well, pretty much)! Go figure.

I mostly code with classes, but do still us TP Objects mixed in with my code for simpler data structures where I don't really want to instantiate objects. Records - well, I use them like they were defined in Delphi 7 and earlier. I really don't see the need for Records with Methods (ie: TP Objects) and such.

Regards,
Graeme

fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp

Reply by email

Unknown author
to Graeme Geldenhuys
Graeme Geldenhuys wrote 20 lines
On 2017-02-28 20:40, wrote: > Sorry, I've been out of touch with what's happening with Records > lately > in modern pascal/delphi/etc. Don't worry, I'm with you on that one. All I know is that Borland/CodeGear/EMBT broken TP Objects somewhere in Delphi. Then they decided they want that functionality back, so instead of fixing TP Objects, the decided to butcher Records to act just like TP Objects (well, pretty much)! Go figure. I mostly code with classes, but do still us TP Objects mixed in with my code for simpler data structures where I don't really want to instantiate objects. Records - well, I use them like they were defined in Delphi 7 and earlier. I really don't see the need for Records with Methods (ie: TP Objects) and such. Regards, Graeme

There are many cases where I do not want to free and create an object and just use something on the stack instead. TP objects kind of offered a solution to this, but not really, since if you used an old TP object which was designed to be on the heap, but you used in on the stack instead - problems would occur...

I'm fond of what Oberon and Golang are doing to try and solve the issue, but, that's garbage collected rather than stack. And in those languages you have to send the self parameter in as your own (there is no hidden self, you create your own self shortform)

Reply by email

Unknown author
to Jonas Maebe: Yes, just like records that contain e.g. an ansistring field. Jonas
Jonas Maebe wrote 9 lines
On 28/02/17 21:40, wrote: > What happens with a stack allocated record? > (no new() required, nor dispose() so is the record ever > initialized/finalized?) Yes, just like records that contain e.g. an ansistring field. Jonas

do records which contain ansistring fields have some special stack initialization system (sorry, I should really read the source code in this case - still interested in theory of how this works)

Reply by email

Michael Van Canneyt
to Unknown author
Show quoted text 15 lines
On 2017-02-28 14:48, Jonas Maebe wrote: > On 28/02/17 21:40, wrote: >> What happens with a stack allocated record? >> (no new() required, nor dispose() so is the record ever >> initialized/finalized?) > > Yes, just like records that contain e.g. an ansistring field. > > > Jonas do records which contain ansistring fields have some special stack initialization system (sorry, I should really read the source code in this case - still interested in theory of how this works)

Yes. They are initialized.

Michael.

Reply by email

Benito van der Zander
to Michael Van Canneyt

Hi,

with a function call: http://bugs.freepascal.org/view.php?id=30333

I had benchmark runs where it spend more time in the initializing function than doing any calculations

Best,
Benito

Michael Van Canneyt wrote 26 lines
On Sat, 4 Mar 2017, wrote: > On 2017-02-28 14:48, Jonas Maebe wrote: >> On 28/02/17 21:40, wrote: >>> What happens with a stack allocated record? >>> (no new() required, nor dispose() so is the record ever >>> initialized/finalized?) >> >> Yes, just like records that contain e.g. an ansistring field. >> >> >> Jonas > > > do records which contain ansistring fields have some special stack > initialization system (sorry, I should really read the source code in > this case - still interested in theory of how this works) Yes. They are initialized. Michael. _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Reply by email

Hans-Peter Suter
to Graeme Geldenhuys: Don't worry, I'm with you on that one. All I know is that Borland/CodeGear/EMBT broken TP…

2017-02-28 21:50 GMT+01:00 Graeme Geldenhuys
:

Records - well, I use them like they were defined in Delphi 7 and earlier. I really don't see the need for Records with Methods (ie: TP Objects) and such.

Go with modern times! It's the 21st century now ;-)

(For inspiration have e.g. a look at Julia [1] and how they develop incl. syntax changes. - Many discussions/decisions are being done using github issues, here e.g. a quite fundamental keyword change [2, 3]. As you see in this issue, it is often not easy to find *the* good solution, but it's great, how opinions have a place to flow in and can be considered/integrated by the core or other talented devs. - There are branches: people can help but are also told to wait if the timing is not good [4]. - There is a manual, stdlib, devdoc [5] which gets you started quickly and one can easily contribute small things like typos. - A one-stop discourse based forum [6] which is a joy to use and much easier than e.g. the multitude of fpc-maillist, lazarus-forum, newpascal-/mse-<don't-know-atm>-lists. - There's a one-stop binary download location [7] and/or install master from source, just a ~ 'git clone, make install' away (llvm is not as fast as fpc though;). - In general uniform syntax (even with `end` like in pascal:) which doesn't make you guess which style/convention to choose. - Organisations which integrate people interested in a specific subject. e.g. [8]. - Package management is currently a bit 'suboptimal' but a new one is being developed, here the propositions [9] where interested people can give input in the design phase, and later while implementing. --- Of course one cannot and should not compare !!! Julia doesn't have a 25+ year long legacy with many (partly a bit obscure) platforms to conserve and maintain. (It 'stole' the compiler, à la MSElang;-) It has developers which work since ~7 years ~professionally and there is quite some funding and many many contributors. But as a maybe longterm inspiration, why not? Maybe some parts are worth stealing? And, with tool-support, maybe possible to steal?)

(Ähem, I digressed a bit from the Delphi 7 record topic... I am sorry (more so, considering I forked from you))

(I agree with what you wrote about 'polluting pascal code'. But what can one do? For me, atm, it is super-important that fpc is very compatible with delphi, e.g. generics (and that they work well). Thus, very grateful to people who contribute such things).

[1]: http://julialang.org, [2]:
https://github.com/JuliaLang/julia/issues/19157, [3]: https://github.com/JuliaLang/julia/pull/20418, [4]: https://github.com/JuliaLang/julia/pull/20885, [5]: https://github.com/JuliaLang/julia/tree/master/doc/src, [6]: http://discourse.julialang.org/ (discourse has mailling list integration nowadays...), [7]: http://julialang.org/downloads/, [8]: https://github.com/JuliaGPU, [9]:
https://github.com/JuliaLang/Juleps/blob/master/Pkg3.md

Reply by email

Graeme Geldenhuys
to Hans-Peter Suter
Hans-Peter Suter wrote
Go with modern times! It's the 21st century now ;-)

I am, that's why I am starting to use Java more and more. ;-) A very nice language, multiple choices in IDE's, fantastic debugger, fantastic contributions and ecosystem.

When I think Records, I think of trying to read a binary file off disk in a complex layout. Records were (and still are) used to read such complex binary files. Methods in records have no place there.

Now if I work with objects, I use Classes or Objects - both were designed for that purpose.

(I agree with what you wrote about 'polluting pascal code'. But what can one do?

Don't be a lemming! We can all see in what trouble Delphi has been for the last 15+ years. They are grasping at straws trying to stay above water. They f*cked up so many things during that time and became a very buggy platform to develop in.

Free Pascal and Lazarus has a large following already, and is well established and a very solid compiler. Even EMBT proved that by using FPC is one of the Delphi releases (XE2 I think).

Free Pascal doesn't need to follow Delphi over the cliff. My personal experience has proven that FPC is just as good... no wait, better that Delphi (multiple CPU support, multiple OS support, mobile support, embedded support etc). So I simply use FPC for everything! I don't need cross-compiler compatibility.

e.g. generics (and that they work well).

I'm warming up to Generics (in Object Pascal) - but that is again more so because I'm also using Java, and in Java, generics is very often used.

Regards,
Graeme

fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp

Reply by email