Hello,

In platforms with managed code (.NET, Java), objects are automatically freed by the memory manager / garbage collector.

Would not it be interesting to have a similar feature in FPC?

For example, through a root class where its objects are counted by reference, like the TInterfacedObjects. Thus, the programmer would be free from having to manually release objects.

Best regards

It's already on my radar since Delphi's NextGen compiler introduced ARC (Automatic Reference Counting) for TObject and its descendants.

Though while I'd like to implement them I don't consider them the holy grail especially since the programmer needs to take care of cycles in the objects (Delphi solves this by allowing the programmer to declare fields, paremeters and such as "weak", thus not changing the reference count).

Regards,
Sven

AFAIK some Delphi XE made TObject itself managed, by reference counting. It would be easy to introduce the same feature in FPC, so that no special base class would be required. Like with extended RTTI a decision should be made, whether managed objects should be enabled or disabled by default. Afterwards automatic management can be turned on or off for every single class or object individually.

For example, through a root class where its objects are counted by reference, like the TInterfacedObjects. Thus, the programmer would be free from having to manually release objects.

In practice it turned out that the automatic destruction of objects still requires assistance of the coder, in many cases, in all languages with garbage collection. I.e. a destructor (or finalizer) still is required to prepare an object for subsequent destruction.

IMO it's sufficient to use Interfaces for all objects that should be subject to garbage collection.

DoDi

It's basically easy, yes, but then one has to deal with code like this:

=== code begin ===

somestrlist.AddObject('Foobar', TObject(42));

=== code end ===

And also there's the problem of cycles especially considering the LCL where each TControl has an owner and a list of child components. And then there's of course the dealing with existing code:

=== code begin ===

o := TObject.Create;
try
   // something
finally
   o.Free;
end;

=== code end ===

Which could lead to some unintended side effects if "o" is passed to some other code which keeps the instance around and ".Free" merely decreases the reference count. Of course that would have been a memory leak before and now it's not, but nevertheless it changes behavior.

Regards,
Sven

Hi all,

Why would anyone want something like that? Both languages have interfaces that can be used for auto reference counting and they also provide us with a very powerful abstraction mechanism.
This also adds overhead on the performance (and locking calls) which might be important in some cases like embedded devices.

The fact that embarcadero added this feature unconditionally is a huge mistake IMHO unless they needed it for some compatibility reasons with java for android or iOS or just marketing, I don't know. If someone knows any other reason please post it on the list.

It's the same bad decision they made with the zero based strings, at least the used a switch for that.

The only gain I can see from that is that with objects someone can use reference counting and still benefit from some aggressive optimisation like inlining which is not possible for interfaces. The same with simple properties that they need getter and setter methods in interfaces that cannot be optimized but in objects it might be possible or they can access the data member directly.

In my opinion this is a very small gain compared to the huge change in the language semantics that you propose.

Chriss
Στις 20 Σεπ 2014 1:41 μ.μ., ο χρήστης "Hans-Peter Diettrich" <

Please do not reference count TObject. This is a uniquely bad and unnecessary idea. I will switch to ANSI C if you guys do it

Peter

Peter Popov wrote

Please do not reference count TObject. This is a uniquely bad and unnecessary idea. I will switch to ANSI C if you guys do it

Please enlighten me why you think it is bad. Give reasons and don't be like "a farmer doesn't eat what he doesn't know" (rough translation of a German proverb).

And just to give you a reason *for* reference counted objects: they would finally allow to use operator overloads on classes, because then temporarily created objects are created correctly (think about an expression like "a * (b + c)" with classes).

Please also note that I said that I don't consider them to be the holy grail especially considering that I know of some of the technial problems one needs to face when implementing them in an already existing language.

Regardts,
Sven

I think I have a reason.

First off, I do like refcounting: I use it a lot, *where useful*.

To make TObject refcounted would mean that everything is all of the sudden refcounted, what:
- Changes the current behavior of classes (bye bye backward compatibility)
- Adds performance overhead to every assignment between classes:

            a {TObject}:= b {TObject}; // Currently the same as a

{Pointer}:= b {Pointer};

becomes

            b.IncRefCount;
            a.DecRefCount;
            a's Pointer to instance:= b's Pointer to instance;

Throw in the handling of race conditions between threads and the entire thing becomes bloated IMO.

- Won't allow the user to cast to and fro between classes and pointers (useful to pass the classes thru an external API back to a pascal callback)

What would be handy is a seperate root class that is always refcounted, something like TRefcountedObject or something. On the other hand some directive that makes this class refcounted would also be a good solution IMO (something that works for refcounting in the same way as $M does for RTTI).

Just my 5 cents.

Simple:

For one, it will slow down your code.
There will be lots of incr/decr of reference counts, just as for ansistrings.

Secondly, there is lots of code that stores objects in lists of pointers and vice versa. Checking such existing code for side effects by the sudden introduction of TObject may take ages.

So reference counting TObject by default is IMHO out of the question.

I am not against a reference counted class (the idea exists since ages). But the solution is IMHO to invent a new attribute or whatever, to allow people to make an arbitrary class (and it's descendents) reference counted.

Michael.

Hello all,

IMO the best way to implement this is by controlling the TObject behaviour through a compiler switch, something like "enable globally", "disable globally", "enable for specific class" and in the latter case another switch should mark the class that should have this behaviour enabled.

This should allow the compiler to emit different code per case.

But the question is what should be the behaviour for genetics and also how should it work for packages (when and if they are implemented). Also should this reference count be shared for interfaced classes or not?

Chriss
Στις 20 Σεπ 2014 3:26 μ.μ., ο χρήστης "Michael Van Canneyt" <

In our previous episode, Sven Barth said:

Performance, size increase of minimal tobject. Both are killing for existing light object in memory frameworks.

And just to give you a reason *for* reference counted objects: they would finally allow to use operator overloads on classes, because then temporarily created objects are created correctly (think about an expression like "a * (b + c)" with classes).

Boring.

Please also note that I said that I don't consider them to be the holy grail especially considering that I know of some of the technial problems one needs to face when implementing them in an already existing language.

The point is that it shouldn't hurt, so IMHO should stay away from the base type. DO whatever you want for your own root object-and-down, but don't force it upon others.

[...]

Which could lead to some unintended side effects if "o" is passed to some other code which keeps the instance around and ".Free" merely decreases the reference count. Of course that would have been a memory leak before and now it's not, but nevertheless it changes behavior.

I already mentioned that destructors still are required, but will have an different purpose and usage than before. This would discourage continued use of Destroy(), BeforeDestruction() etc., which should at least be renamed to prevent unconverted legacy code from compiling. This change already will break compatibilitiy, so that consequently all libraries (in detail when dealing with lists containing objects) have to be updated. I was aware of such consequences, but I'm no more sure of the consequences of my idea of simply turning refcounting on or off for specific objects or classes.

The mere implementation of refcounting for TObject is easy, but the consequences are hell :-(

DoDi

This sort of typecast was usually used to implement associative arrays in Delphi versions that did not have generics. In later versions with generics it is always better to use a TDictionary<string, TObject>, or some other generic associative array class instead. In fact if one expects their code to compile and run with the nextgen compiler they should refrain from using TStringList like that altogether or they are certain to get AVs and leaks in mobile devices -has happened to me.

The only problem would arise where this technique has been used with TStrings properties in classes one has no control over (e.g. TListBox.Items) that cannot be converted into TDictionary<string, TObject> but this could be solved with a generic boxing class:

type
  TBox<T> = class
  private
    FItem: T:
  public
    constructor Create(const AItem: T);
    property Item: T read FItem write FItem;
    operator Implicit(const ABox: TBox<T>): T; // Result := ABox.FItem;
    operator Implicit(const AItem: T): TBox<T>; // Result :=
TBox<T>.Create(AItem);
  end;

...

  MyListBox.Items.Add('someint', TBox<Integer>.Create(42));
  MyInteger := TBox<Integer>(MyListBox.Items.Objects['someint']); // Safe
ObjRef to ObjRef hardcast, then operator overload

In a refcounted version of Object Pascal it might make sense to prohibit casting from a bare pointer or ordinal data type to TObject and vice-versa, thus turning object pointers into object reference variables, like in languages with garbage collection. This would be of great help to a person who is converting legacy code, and it is something I miss in Delphi. In cases where an object reference does indeed need to be turned into or from a non refcounted variable a helper generic class could be used to perform the typecast, something like:

  ANativeInt := TObjRefMarshal<TStream>.MarshalTo<NativeInt>(AStream);
​  AStream​ := TObjRefMarshal<TStream>.UnmarshalFrom<NativeInt>(ANativeInt);

​Regarding TObject.Free, in Delphi nextgen ​it is a nop and any calls to it are optimized away. The object will be destroyed when the variable "o" goes out of scope and its one and only reference is released.

--​Constantine​

Hi Chriss,

Personally I favor reference counted objects. While there are interfaces as you pointed, and in Delphi you can even use smart pointers now, there is still a lot of cases when you need to use objects, and have to manually free them. In single threaded environment that is not such an issue, but in parallel and heavily multithreaded environments, reference counting is a life saver. Just for that at the moment we have to declare gazillions of additional interfaces which slows the development considerably. After all even after we switched to the new RTTI, generics, and attributes, we barely succeeded to reduce our code from 4 mln lines of code to ~800K and it still is time consuming to maintain :-( . Anything that can reduce the code base further and speed up our development is important for us, as I would guess for anyone else.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Hi Peter,

I strongly disagree... I think there is a great value in a reference counted object, especially if we can enable/disable that with virtual method or even better attribute ;-) .
Then you will have it any way you want :-) .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

I would propose either a virtual method or attribute that enables disables this.
Indeed in our code we have adopted a pattern that implements both:

TMyObject = class
public
  function HasSomeFunctionality() : Boolean; virtual;
end;
function TMyObject.HasSomeFunctionality() : Boolean;
begin
  Result TypeInfo().HasAttribute( SomeFunctionalityAttribute );
end;

This way you can override the virtual method or just change/supress the attribute (In Mitov.TypeInfo advanced RTTI we have added the option to suppress attributes etc. ;-) )
Or the attribute can be boolean and use it as On/Off .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

BTW: Rumor has it that Emb. is working to introduce Ref Counting on all versions of the compiler.
At the moment all of their mobile versions use it. At the same time we have ported large portion of our code to it, and it took ~1 month of work on and off. It is actually surprisingly easy, although our code always has been designed to be well structured in terms of that, and we have been using mostly interfaces for a decade already, so other code may be a different case.
As for slowing down code that may very well be offset by other design benefits, but that is a whole different conversation ;-) . I can easily prove that it may speed up your code in some cases by factor of 100, by allowing you easily to add hash tables or other fast search methods that ware more difficult to manage without the ref counting as example ;-) .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

On Sat, Sep 20, 2014 at 10:24 AM, Chriss Kalogeropoulos <

Very nice idea, somethink like "{$ARC+}", enabled by default in FPC 2.8.0, IMHO. :-)

And it could provide an object like "TManagedObject" too, to use if "{$ARC-}".

On Sat, Sep 20, 2014 at 8:02 AM, Chriss Kalogeropoulos <

Yes. Embarcadero thinks in productivity. See the new IDE of them, it is extremely productive.

Unfortunately, who doesn't modernize, tied in old concepts, can lose your market/product/space/people to your competitors, IHMO. :-/

Hello all,

Instead of hardcoding stuff into the (amazing) fpc compiler, would it be possible to use generics and operator overloading to create a smart pointer implementation?
This might require some improvements like record constructors/destructors (or initializer/finalizer) and some way to overload the member access operator "." but it might create a more flexible and powerful implementation.

Chriss
Στις 20 Σεπ 2014 6:41 μ.μ., ο χρήστης "" <

What about adding support for destructors to (advanced) records (and/or inhertiance)?

This is actually a very good idea IMHO!
I would vote for it. Record inheritance is something I miss badly in Delphi.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Il 20/09/2014 19:20, Boian Mitov ha scritto:

Can you explain how reference counting can be safely implemented in a parallel multithreaded environment, without heavily affecting performance? A mere increment or decrement of count means to get a lock and to release a lock on that object. Application code can know what is thread safe and what is not, and use lock only when appropriate. A general mechanism to be reliable should take into account all possibilities. If it does, it will block threads even when unnecessary. If it doesn't, it will be unsafe.

What do I miss?

Giuliano

You don't miss anything, we agree. That is why I propose an explicit implementation either using a compiler switch or generic records (smart pointer). This way the developer has full control on when the feature is used or not. The smart pointer approach is more flexible I believe.

Sorry for the noise,
Chriss
Στις 20 Σεπ 2014 9:34 μ.μ., ο χρήστης "Giuliano Colla" <

as you pointed, and in Delphi you can even use smart pointers now, there is still a lot of cases when you need to use objects, and have to manually free them.

> In single threaded environment that is not such an issue, but in

parallel and heavily multithreaded environments, reference counting is a life saver.

Can you explain how reference counting can be safely implemented in a

parallel multithreaded environment, without heavily affecting performance?

A mere increment or decrement of count means to get a lock and to release

a lock on that object. Application code can know what is thread safe and what is not, and use lock only when appropriate.

A general mechanism to be reliable should take into account all

possibilities. If it does, it will block threads even when unnecessary. If it doesn't, it will be unsafe.

That would work the same way as it does in interfaces, arrays and strings: using Interlocked*-functions.

Regards,
Sven

"Boian Mitov" wrote

This is actually a very good idea IMHO! I would vote for it. Record inheritance is something I miss badly in

Delphi.

If you want inheritance, then use "object". Period. What I do think about though is initializers/finalizers or something like that.

Regards,
Sven

Hi Giuliano,

All of our libraries, are heavily multithreaded, and use exclusively reference counting with interfaces, as well as lazy evaluation nested object reference counting for performance. We have been doing this for over 11 years with great success, and we continue to expand the functionality. Not only is it doable, but it is nearly impossible to do really complex heavily distributed threading without it. Trust me, I have been doing multithreading since the days we had to implement it in interrupt cascades in 6502/6800 8 bit processors 30 years ago ;-) . I know what it takes to do it in assembler, C, C++, Delphi, with and without OOP, as well as with and without ref. canting ;-) . I have done them all...

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

PS. Forgot the mention the anonymous methods – a real revolution when you do multithreading or parallel processing. Anything else looks like joke next to them... You can see some of my examples from interview in the Blaise Pascal Magazine http://www.blaisepascal.eu/. Also did a session on it at PasCon in Holland, and will do also in Milan and in CodeRage, so you can see and judge for your self, and BTW anonymous method storage is ref counted also ;-) .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Hi,

I will address multiple previous e-mails on this subject :)

1. FPC does perform reference counting if variable type is interface, but not if variable type is class that implements interface - legacy/compatibility reasons?

If you want to do ARC for objects - just remove this limitation and allow compiler to generate reference counting code not just for for interfaces but for classes that implement interface - AFAIK any interface in FPC/Delphi extends IUnknown and has reference counting methods.

2. I agree with Sven - adding reference counting to TObject is a bad idea

Please do not look at Delphi - they just follow Apple - and both companies do not address real issues with their development tool chains
- it is all pure marketing.

In reality ARC and "weak" keyword do not magically solve the memory management issues - you either have GC or you do not have GC - there is no way around it.

I personally like TComponent approach - you have an object that owns other objects - as soon as this object dies - all children and grandchildren die too.

Combining it with ARC seems to me the way to go...

3. About "weak" attribute - current "strong" approach with interface reference has a contract - it is either nil or pointer to valid instance. If you mark interface reference as "weak" - how do you know whether this not nil reference is valid or not - I can be missing something here - please explain.

In Java weak reference is accessed through some intermediate object and you can this object whether the reference is gone or is still valid - but you need to have GC to do it...

Regards,
Gennady

The short story is that any approach has issues.
The component container approach has issue of a single ownership with the easy loss of pointers.
The ref counting has the danger of circular references. The GC has the non deterministic behavior (Actually I proposed a deterministic/semideterministic GC algorithm ~8 years ago or so, but that is a different story).
The point is that even with GC the developer is still required to carefully manage resources, and GC tends to make it even more complex.

From all the the above approaches the ARC with optional Weak pointers is the

easiest to manage and the one that tends to lead to the least problems IMHO .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Hi Boian,

I'm easily convinced that you've developed a lot of things using reference counting. Design is the art of compromise, and possibly in your class of application that's the best compromise. But we should never forget that our class of applications isn't the only possible one in the world. What is a bonus for you might be either useless or extremely harmful for someone else.

I might, for example, tell you that my company has been successfully implementing since more than 30 years a class of applications for the control of industrial processes, with hundreds of threads running simultaneously in a multi-CPU environment, with thread synchronization in the sub-millisecond range (with time it has improved down to the sub-microsecond area, with the evolution both of hardware and software tools), where reference counting simply doesn't apply, because of the specific nature of the problems. All the required resources must be created at startup, and they can only be destroyed when the process is stopped, at the end of the day in some cases, at the end of the week or for the annual vacations in others.

Or that we have another class of multithreaded applications (not so many as the other ones, and with just around 14 years of history) where objects are mainly manipulated by means of lists of pointers, and where, again, reference counting doesn't apply, because an object can't possibly be aware of what happens to one or several pointers pointing to it. In some cases it's required that destroying the list the objects are destroyed also, in other cases they must be kept. But this must be decided by program logic, not by a generic algorithm.

In any case, in our range of applications, memory leakage due to objects not destroyed, or crashes due to unduly destroyed objects has never been an issue.

For those reasons I have nothing against reference counted objects, provided I'm not forced to use them, because, in our class of applications we would only suffer the disadvantages.

Should TObject become reference counted by default, I'm afraid that for most of us, me included, switching to ANSI C would become a choice to be seriously considered, like Peter Popov suggested in this thread...

Kind regards,

Giuliano

Il 20/09/2014 21:27, Boian Mitov ha scritto:

Hi Giuliano,

Funny that you mentioned it. Industrial automation, and process control are one of our specialties ;-) . Actually this is my major. However I am somewhat unusual in that I have implemented in my career almost every class of application you can name, with a large number of different technologies. Yes, I have implemented process control applications 20 years ago without ref.counting and I implement them today with it, and I can assure the later makes it easier and actually faster at the end (less overhead). Even our current line or products covers large variety of areas – we have video, audio, DSP processing components, computer vision, Artificial Intelligence, Boolean logic/hardware simulations, and much more, and we are in about to release process control library – OPC, PID, Relay controllers etc. Aside from that I used to have my own HW company ~30 years ago, and have done a lot of HW designs many surprisingly still in use. I think I have (for good or bad) a unique overview over the full spectrum of technologies, and intimate experience with most of them.

In reality I am very dissatisfied with them all... but that is a different story, and I am working to hopefully address that in the future ;-) .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

To be more accurate..
You're talking about "closures". The difference between closures and anonymous functions is explained here:
http://en.wikipedia.org/wiki/Closure_(computer_programming)#History_and_etymology the second paragraph.

thanks,
Dmitry

Hmm.... the last time I checked this was anonymous methods... Are you sure I was talking closures?

FThread1.Process(
    procedure()
    var
      ANewText1 : String;
    begin
      Sleep( 500 ); // Simulate thread busy doing time consuming task
      ANewText1 := AText + ' + Thread1';
      FThread2.Process(
          procedure()
          var
            ANewText2 : String;
          begin
            Sleep( 500 ); // Simulate thread busy doing time consuming task
            ANewText2 := ANewText1 + ' + Thread2';
            FThread3.Process(
                procedure()
                var
                  ANewText3 : String;
                begin
                  Sleep( 500 ); // Simulate thread busy doing time consuming task
                  ANewText3 := ANewText2 + ' + Thread3';
                  FThread4.Process(
                      procedure()
                      var
                        ANewText4 : String;
                      begin
                        Sleep( 500 ); // Simulate thread busy doing time consuming task
                        ANewText4 := ANewText3 + ' + Thread4';
                        MainThreadExecute(
                            False
                          ,
                            procedure()
                            begin
                              Memo1.Lines.Add( 'Ended Processing: ' + ACounter.ToString() + ' - Result:' );
                              Memo1.Lines.Add( ANewText4 );
                              Memo1.Lines.Add( '' );
                              Dec( FInProccessingCount );
                              Label1.Caption := 'In processing pipeline = ' + FInProccessingCount.ToString();
                            end
                          );
                      end
                    );
                end
              );
          end
        );
    end
  );

This is a 4 stage pipeline with asynchronous main thread printing. I dare somebody to implement it in less than 5-6 times more code with traditional methods, and while matching the performance...

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

PS. And at least in Delphi they are implemented as interface and ref. counted...

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

I don't like the use of "GC" as a synonym for *mark-sweep* garbage collection only. Wikipedia also states Reference Counting as just another form of garbage collection.

ACK - except for Weak references. Weak references turn the conservative memory management into an aggresive/optimistic one, with unpredictable consequences.

IMO Weak references should be reserved for users who accept possible consequential problems, but should never be used in standard libraries. At least I'd suggest to make weak references subject to an compiler switch, so that every user has a chance to disable them in case of trouble.

DoDi

As I understand Interlocked Inc/Dec funtionality, it only protects the update of the reference counter against interrupts, but not the tests required before/after this update. As a precaution a RefCount should at least be incremented as soon as there exists a *chance*, that the reference is used/copied in some piece of currently active code (thread...).

DoDi

[...]

IMO realtime applications require an realtime OS, providing all required means of process synchronization and communication. Ordinary systems and developers should be happy with primitive threads, doing their work in the background and exiting when done.

It's good to know that FPC allows to implement and manage more complex parallel processing, if I understand you and Boian correctly?

DoDi

That is easy. it gets incremented when it gets assigned. The running threads have no way of accessing it if there is no reference (assignment) already in place.
Indeed that is how it works in Delphi, and BTW: that is how Strings work in Delphi and FPC the last time I checked ;-) .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Unless you implement something like the semi-deterministic GC that I proposed few years ago, weak pointers are the only solution for avoiding circular reference deadlock in ARC as far as I know. They are simply necessary evil, and a small one base on my current experience with them.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

That is easy. it gets incremented when it gets assigned. The running threads have no way of accessing it if there is no reference (assignment) already in place.

The problem arises when an object is destroyed, or even elected for destruction in _Release, while another thread starts using the same instance.

Indeed that is how it works in Delphi, and BTW: that is how Strings work in Delphi and FPC the last time I checked ;-) .

With strings it's possible to create another (unique) copy, when a string is modified, and it does no harm when that copy is destroyed later - every user will find an valid (empty) string. Not so with objects, which cannot be copied nor reused after destruction.

DoDi

I actually do not use FPC any more due to the lack of many advanced features. I use Delphi. I hope however that will change.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

I was looking at the following example in DXE7
http://delphiaball.co.uk/
procedure TFormThreading.Button1Click(Sender: TObject); var aTask: ITask; begin // not a thread safe snippet aTask := TTask.Create (procedure () begin sleep (3000); // 3 seconds ShowMessage ('Hello'); end); aTask.Start; end;

I dislike this syntax and tried to write it this way because I think it's better readable code. But it's not accepted.

procedure TFormThreading.Button1Click(Sender: TObject); var aTask: ITask;
procedure MyTask; begin sleep (3000); // 3 secondsShowMessage ('Hello');end
begin // not a thread safe snippet aTask := TTask.Create(MyTask); aTask.Start; end;
I just wonder if there is a valuable reason to do it why my example is not allowed or would it be easy to make this available in free-pascal?

Thank you.
Sam

________________________________

"Boian Mitov" wrote

Unless you implement something like the semi-deterministic GC that I

proposed few years ago, weak pointers are the only solution for avoiding circular reference deadlock in ARC as far as I know. They are simply necessary evil, and a small one base on my current experience with them.

There are cycle detection algorithms that could be used upon a decrease of a reference count. Biggest problem is that they would be quite expensive (regarding performance) in case of complex object instance "graphs" like the LCL/VCL.

Regards,
Sven

This is not possible for a correctly written program: if two threads having references to the same instance, the ref. count is 2. So no destruction is possible.

Il 21/09/2014 05:51, Hans-Peter Diettrich ha scritto:

Our class of applications takes advantage of a distributed intelligence model. IOW a number of CPU's working in parallel, each one running his own real-time multitasking environment. They're organized in a hierarchical tree and almost all of them take advantage of dedicated real-time OS's, up to the root of the tree, which governs the full process, and also provides human interface, data collection, connection with factory network and data-bases, remote connection for assistance, debugging and upgrading.

The top level, until recent times, was provided by a Linux box, with a real-time patched kernel (Adeos-Ipipe). The result was a real-time nanokernel, running the application real-time tasks, and having Linux as its idle task. On Linux side, the visual portion was provided by Kylix.

But, with the increase of CPU speed, and the the significant improvements of Linux kernel latency, it's now become possible to get rid of the foreign real-time patch, and to take advantage directly of Linux kernel thread scheduling, without a significant performance loss.

It's good to know that FPC allows to implement and manage more complex parallel processing, if I understand you and Boian correctly?

FPC comes into play with the latest evolution I described above, because all the high level functions (such as user interface, data collection, etc.) are being handled using Lazarus for the visual functions, and a number of FPC threads for the rest. While Kylix did provide a more suitable (and less buggy :-( ) widgetset, synchronization of the main thread with FPC/Lazarus is significantly better.

Giuliano

In a perfect world: yes, it would be easy to make this available in FPC.

In the real world where people write a ton of code in assembler or use pointer hacks to expose low level implementation details no one should rely on: it would probably make a few very vocal people unhappy because to be able to pass nested routines to "reference to procedure/function" parameters, the way they access variables in the parent frame (and the way they receive the parent frame) needs to be changed. And hence all these low level hacks would break.

I think that this is, possibly together with the fact that anonymous methods are a buzzword that you want to be able to add to your feature list, probably the reason Embarcadero didn't simply use nested routines instead of adding anonymous methods.

We could of course introduce yet another calling convention modification for nested routines to work around this (we already have two) so that both the old and new behaviour can coexist. The downside is that it makes the compiler more complex and hence harder to modify in the future and/or easier to break (every such modification needs to be implemented for every calling convention on every platform).

As you observed, anonymous methods and nested routines are essentially identical other than the syntax, and I agree the latter generally also result in better readable code. Therefore I'm still inclined to implement it somehow (at least for C-blocks, as already partially implemented in http://svn.freepascal.org/svn/fpc/branches/blocks/), because it seems a shame not to use an existing, perfectly suited language construct and instead force everyone to use a new one.

Jonas

Agreed.

Though I would also love to have real lambda expressions which would only use an expression, not a statement and neither a block of statements (though I'm still "fighting" with the syntax in my mind):

=== code begin ===

procedure FilterGreater(aSomeList: specialize TList<LongInt>; aValue:
LongInt);
begin
   aSomeList.Filter(lambda x: LongInt as x > aValue);
end;

=== code end ===

Regards,
Sven

No, why should it? It works pretty well with interfaces, strings and arrays (let's put aside modifying the latter two, because COW is not needed here).

Regards,
Sven

In our previous episode, Hans-Peter Diettrich said:

IMHO weak references trade one manual memory system in for a different manual memory system.

The hard part of manual memory systems, figuring out how a complex dynamic structure deallocates (that is usually tackled by having a bit of design and thought go into it), remains.

One just needs to provoke the others correctly and then the discussion starts. Great! :)

I've thought about the topic a bit more (I've already thought about it quite much in the past months) and came up with the following "RFC":

- No reference counting for classes by default
- Reference counting can be introduced to a part of the class hierarchy by declaring it as "refcounted" like this (Note: syntax is not final, but is based on other syntaxes we have (namely "sealed" and "abstract")):

=== code begin ===

type
   TARCObject = class refcounted(TObject)
   end;

=== code end ===

- a reference counted class (and its child classes) would include a reference count field that the compiler knows how to access (for automatic reference counting) and which can be accessed through RTTI (for manual reference counting); it is *not* exposed as a regular field as this might lead to identifier conflicts
- only *variables* (or parameters) that have a reference counted class as its type will be subject to ARC; in extension this means that assignments from/to a variable of a not reference counted base class (e.g. TObject) will *not* change the reference count. Take this code for example:

=== code begin ===

function CreateObject: TObject;
begin
   Result := TARCObject.Create;
end;

=== code end ===

This will most likely result in "CreateObject" returning an instance to class that is already freed (because the only reference inside "CreateObject" is a (hypothetical) temporary of type "TARCObject" which goes out of scope once "CreateObject" returns)
- to remedy this TObject is extended with non-virtual methods that allow manual reference counting and would rely on the RTTI data I mentioned (let's call the methods AddRef, Release, IsReferenceCounted and RefCount for now, which can also be used to hook up the reference counting of IUnknown interfaces); the code from above would then look like this to make it safe:

=== code begin ===

function CreateObject: TObject;
begin
   Result := TARCObject.Create;
   Result.AddRef;
end;

=== code end ===

- TObject.Free would be extended to take reference counting into account as well. If the object is reference counted (IsReferenceCounted returns true) it will call Release and otherwise it will continue to Destroy.
- there would be a TARCObject declared in System which is a direct descendant of TObject, but with reference counting enabled; same maybe also for TInterfacedObject
- all classes can now have operator overloads as well though it should be warned in the documentation that non-reference counted objects might result in memory leaks there
- this now only leaves the problems of cycles; take this code:

=== code begin ===

type
   TSomeClass = class(TARCObject)
     Children: specialize TList<TSomeClass>;
     Owner: TSomeClass;
     constructor Create(aOwner: TSomeClass);
   end;
constructor TSomeClass.Create(aOwner: TSomeClass);
begin
   Children := specialize TList<TSomeClass>.Create;
   Owner := aOwner;
   if Assigned(Owner) then
     Owner.Children.Add(Self);
end;
procedure Test;
var
   t1, t2: TSomeClass;
begin
   t1 := TSomeClass.Create(Nil);
   t2 := TSomeClass.Create(t1);
   // do something
end;

=== code end ===

Now once Test is left it would leave the instances which were assigned to t1 and t2 hanging, because they have references to each other. There are (as far as I see) three ways to solve this:
* provide a way to break the circle (in this example e.g. setting Owner to Nil before leaving Test; this is what Delphi provides with the DisposeOf virtual method)
* introduce weak references which would disable reference counting, e.g.:

=== code begin ===

type
   TSomeClass = class(TARCObject)
     // ...
     Owner: TSomeClass weak;
     // ...
   end;

=== code end ===

Now the "TSomeClass.Create(t1)" line in "Test" wouldn't increase the reference count of "t1" further and thus both class instances would be destroyed after "Test" is left.
* provide a possibilty to execute a cycle detection algorithm during the Release part of the reference counting; this has the benefit of avoiding the need for "weak", but there would be the problem that the algorithm can be potentially expensive especially with large object instance hierarchies (think LCL here) and this would also need to be executed for *each* decrement of the reference count, thus for both the automated one (which till now could have been rather efficient) and the manual one.

I think this approach would allow the best of both worlds and if someone really wants to have TObject reference counted (and face the potential consequences), then he/she should simply adjust his/her RTL. ;)

Regards,
Sven

In our previous episode, Sven Barth said:

(as sb purely interested from an academic viewpoint, I have no real need for this)

Does this mean multiple roots? That would make creating helpers for the refcounting different (since the field would be at a different offset everwhere)

One of the performance sinks with refcounting is passing parameters through a call chain. (e.g. a class calling an inner class in a field to do the actual work, inherited() etc).

How do you think to tackle that? With const?

Maybe that can

This will most likely result in "CreateObject" returning an instance to class that is already freed (because the only reference inside "CreateObject" is a (hypothetical) temporary of type "TARCObject" which goes out of scope once "CreateObject" returns)

It seems to me that implementation details of the return value handling (hidden/pseudo temp var etc) now trickle into the ARC design.

Wouldn't it be more logical to make the result temp var a bit special so that it is not released when returned (requiring a

   func(); // ignores return value

to pop the result and decrease the refcount?

Hi Sven,

This design means that old classes need to be subclassed in order to enable this feature, correct?
Would it be possible instead of declaring this on the class definition, to declare it on the actual variable? Something like

var aArcObj: TFooObj; refcounted;

Or similar.

This would allow the same classes to be used both ways.

Also how about a smart pointer record implementation with generics? Is it possible ?

Thanks,
Chriss
Στις 21 Σεπ 2014 2:12 μ.μ., ο χρήστης "Sven Barth" <

Marco van de Voort wrote

In our previous episode, Sven Barth said: (as sb purely interested from an academic viewpoint, I have no real need for this)

Understood.

No, not multiple roots. If a class is declared as refcounted (and it's parent class is not) then a hidden field is inserted into its list of fields, just as a normal one would, only that it isn't accesible using the "."-operator (because it won't have a valid Pascal identifier). The compiler of course knows the offset at compile time and at runtime the field can be accessed using the class' RTTI which would have a "RefCountOffset" field added (if that offset is < 0 then the object does not support reference counting).

In the same way it's a problem with strings, interfaces and arrays. They all react accordingly to var/const/constref by not changing the reference count. Reference counted classes would follow the same rules (consistency here). And of course as a user of reference counted classes one needs to be aware of the potential performance impact.

Did anyone till now really complain about the performance impact of reference counting on interfaces? Especially since they are used *so* much in today's OOP designs?

The function result is just a special case. This can also be achived with normal variables or parameters so it is important to have a consistent handling here. The only way to consistently handle this without making *all* object variables subject to reference counting again is to not have assignments of reference counted object instances from/to non-reference counted variables influence the reference count in any way.

Also this is compatible to how interfaces work, though the analogous interface example is highly constructed:

=== code begin ===

program tintfrefcount;
{$mode objfpc}
type
   TTestClass = class(TInterfacedObject, IInterface)
     destructor Destroy; override;
   end;
destructor TTestClass.Destroy;
begin
   Writeln('Destroy');
   inherited;
end;
function Test: TObject;
begin
   Result := (TTestClass.Create as IInterface) as TTestClass;
end;
var
   o: TObject;
begin
   o := Test;
   Writeln('After Test');
   o.Free;
end.

=== code end ===

Will result in:

=== output begin ===

Destroy
After Test

=== output end ===

Regards,
Sven

No. You would need to have a place where to store the reference count and one of the objections some people have is the size increase of TObject if it would get a RefCount field (which would be the only solution with your approach). Also you'd need to take care that all variables where you pass along your object instance are marked as refcounted. This just complicates things too much (both for programmers using this as well as for the compiler).

Also how about a smart pointer record implementation with generics? Is it possible ?

What's missing is the possiblity to hook into the initialization (unlike interfaces, strings and arrays class variables fields are not initialized with Nil) and finalization (you need to adjust the reference count) of records. And also it's currently not possible to overload the assignment operator if both types have the same record type.

Regards,
Sven

In our previous episode, Sven Barth said:

So in summary: the compiler must pass the fieldoffset to the runtime helper, instead of having one refcounted root with known offset.

Did anyone till now really complain about the performance impact of reference counting on interfaces? Especially since they are used *so* much in today's OOP designs?

I never really benchmarked it. I never use interfaces in my own abstract data structure designs, mostly because the existing classes don't, and I don't like the refcount. (since i usually also need to access the classes as normal instances)

I did benchmark interfaces in combination with variants as part of a decal evaluation. That was, euh, painful.

Hmm. This looks like a (Delphi) faq item to me, how to convert interface back to a class. Usually the answer is IComponentreference (or what it is called), if this is legal, I wonder why it didn't come up before.

Yes. That avoids having a single rooted refcounted object and thus allowing the usage of "refcounted" further down the inheritance tree.

Considering that the helper would only consist of a "InterlockedIncrement(AddressOfRefCount)" or
"InterlocedDecrement(AddressOfRecCount)" one could optimize this by declaring the helper compilerproc as "inline", thus avoiding one call.

I can imagine that benchmarking variants was painful. ;) But considering that interfaces are so "en vogue" today in Delphi communities the performance hit might not be that killer criteria...

The intf-to-class-as-operator was added with Delphi 2007 AFAIK and FPC supports it as well in 2.6.0. If you want specifics, then look at the helper functions fpc_intf_is_class (for the is operator) and fpc_intf_cast_class (for the as operator) in $fpcdir/rtl/inc/objpas.inc.

Regards,
Sven

On Sun, Sep 21, 2014 at 9:15 AM, Chriss Kalogeropoulos <

Or enabling via compiler switch, like:

http://docwiki.embarcadero.com/RADStudio/XE4/en/Automatic_Reference_Counting_in_Delphi_Mobile_Compilers

Hans,

There is no such problem. We run many hundreds of threads sharing and passing interface data for 11 years already, and in the last 2 years doing it with Anonymous methods as well.
I can assure you it works 100%. There is no problem. And since we are component vendor, it also works for all of our customers as well. So far there has not been a single bug report on anything like this.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

The biggest problem is when to initiate the detection and still maintain some form of determinism. That was the algorithm I proposed a while back, but it was not without a drawbacks. It would have affected performance. There is not really a silver bullet for this as far as I know.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Hi Sam,

You can watch my CodeRage session last year explaining the huge advantages of the first syntax. That being said, I consider it a design bug that local functions can’t be passed as anonymous methods in Delphi. There is no reason for that not to be done, except that they did not bother. I hope if FPC implements anonymous methods, that they will also allow local functions to be passed, and outdo Delphi :-) .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

I used to think the same way, until I actually started to use anonymous methods, and I realized what a full I used to be.
The simplest difference is that you can return anonymous method as a result of your function and it will preserve all the captured context. Local methods can't do that.
Furthermore local methods will not work when passed as parameter to another thread, as they lose context when the owner function exits. You can watch my session on CodeRage last year explaining most of that, and I will be doing some sessions this year on multithreading, and parallel processing where even more will be shown both at ITDevCon and hopefully CodeRage.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Absolutely! Interfaces have been used in threading for decades already in may languages. I have also never heard of anyone having issues with them.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Every memory management system has a manual part, even the M$/Java GC has element of it (the dreadful Disposable pattern to name one). There is a huge difference however. ARC requires in general less coding for the same result.
Less code in general is cheaper to develop, debug, read and maintain. It is a simple business formula actually (and as businessman I need to calculate this).
There is also the old saying "What is the only line of code where you can't have a bug?" with the obvious answer - "The one you don't write." ;-) . So on the balance sheet ARC is a cheaper and in general easier to maintain form of manual memory management, and my experience with it (Mainly interfaces - almost everything we do is interfaced nowadays), confirms that 100%.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Weak references steer/guide *automatic* memory management.

The hard part of manual memory systems, figuring out how a complex dynamic structure deallocates (that is usually tackled by having a bit of design and thought go into it), remains.

And this is what the user of such a structure (standard libraries...) does not always know. He may be unable to determine the reason for some runtime error in his code, when an object was destroyed automagically where it should still be alive. The user can debug (and fix) ordinary (owner/owned) patterns, implemented in high level code, but not table-driven (RTTI) or otherwise hidden (intrinsic) management procedures. While the user can change the owner of an object at runtime, he cannot change a weak reference into a strong one, without recompilation of at least the unit containing that declaration, and without figuring out the consequences of such an change.

DoDi

In general I would prefer to see this implemented with attributes rather than with new keywords. For god shake, we already have more keywords in this language than it is healthy. We should be extremely cautions when adding new ones.
That is one of the beauty of the attributes. They can eliminate a large number of keywords, and expand the language dynamically (trough the library not trough the compiler).
If you implement it with attributes, there is almost no need for any big changes in the compiler at least in the parsing side. Adding new keyword will probably propagate all the way to the tokenizer.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

[Jonas: anonymous methods and local functions could use the same syntax]

I'm talking about syntax, not behaviour. When assigning a local function to an anonymous method ("reference to ..."), the context will have to be captured in exactly the same way as when specifying an inline nested function (aka anonymous method) that requires context. A nested function and a "reference to..." are different types, so a type conversion would have to be inserted by the compiler anyway (a bit similar to how assigning a method to a "procedure of object" works, which also captures context, be it in a much simpler way).

You could still first keep the context on the stack for quick access by the parent function and later add indirection if necessary (in case it can escape the life of the parent function), as handled by e.g. C blocks.

Furthermore local methods will not work when passed as parameter to another thread, as they lose context when the owner function exits.

It's the same thing.

Jonas

Hi Jonas,

I actually agree that it is a nice feature, and I want to have it both in Delphi and FPC :-) .
I prefer the anonymous method syntax as it means less typing, and typing is expensive, however there are cases when I really would love to capture local function. I had few of them already.
The problem is even bigger. You can't even call a local function from anonymous method doe to the same capturing issue :-( . This really should have been addressed in Delphi.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

1. I mostly care for performance, so I will start here: They add overheads which in certain situations can be very bad. These can come in many shapes, including additional memory for the reference counter, creating cache incoherence when doing local object assignments (which would cause some function calls on various patches of memory), the actual inc/dec which would probably be in the form of virtual functions (which may cause pointer chasing), etc.
2. There is no need to force such behavior for all objects: if you want reference counting, work with interfaces, or make a special derived object or both.
3. They break the semantics of existing code.
4. You will mess up code which is tuned to work in certain ways with the heap. For example, I routinely allocate huge chunks of instances the same class to avoid the performance penalty of mallocs, to avoid heap fragmentation and to decrease cache misses in loops. These objects are managed by a custom heap manager via NewInstance/FreeInstance/Destroy. Why on earth should the compiler tell me how to manage such situations, when clearly I have something very specific in mind which does not fit the general scheme of things?
5. Reference counting does not eliminate memory leaks, the trivial situation being a loop assignements a=b, b=c, c=a.
6. You may want control on the deallocation order of classes.
7. Last but not least, you make the language inconsistent. It allows direct pointer mallocs which could be turned into classes, arrays, etc. Forcing different behavior for classes will create an inconsistent structure of the language. If you want reference counting in a consistent way you have to ban getmem/freemem. Either all memory operations are type-managed (no thanks), or you give an option for reference counting which is contained in the grammar of the language (e.g. inheriting a class). Forcing a bad mix is, as I said earlier, uniquely bad idea.

In summary, if it is already there, why force it on all, change the language and create a mess?

Engouh?

>> I just want to remind you people (whoever is working on it) that

ARC is already implemented in FPC for interface types and I am personally

>> satisfited with the way /it works now/ for interface variables, and

only wish that I could enable this feature for class instance variables.

>> Can't someone just, you know, Ctrl+C & Ctrl+V it?

I assume you are talking about ARC for variables/fields of some class that implements interface?

Yes me, too!

Coming from Java I could not even comprehend at the beginning that if variable type is interface - it is reference counted, but if variable type is a class that implements interface - it is not ?!

Regards,
Gena

So why are you not programming in Assembler?
The arguments you provide are the same that ware provided against higher level languages, adding OOP, adding anonymous methods, just to name few. I spent 2 years streamlining all of our code - 4 mln. lines of code to use the latest language features, and in the following year we had a 10x boost in development performance, so it immediately paid off, and now we have 800K code instead of 4 mln. With the old code we ware running out of steam just to maintain it. The new code just largely takes care of itself. Rewriting code is actually a good thing, and I do rewrite cycles in the company every 3-4 years, sometimes for some pars even more often. Every time the code improves as does the performance, and quality. If ARC forces you to review some code, that actually is a good thing. You will need to retest it after the changes, but if you have good unit or functional testing should not be big issue, and if you don’t will be a good chance to add it, or to at least make sure the code really behaves.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Ahh... the schizophrenic nature of Object Pascal :-D, I never understood why interface and class should be different at first place, considering that interface is practically nothing more than pure abstract class with ref. counting. Why the separated syntax etc. is beyond me... Surely helps with the language keyword pollution....

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Hi Boian
I do not mind if this is done on a derived object. As Marco said earlier, one can do anything form his base root object. Just don't do it to the root class

Personally, I operate from a base class descending from TPersistent. Down the hierarchy I do have reference counted base class (done explicitly). If I want it implicitly, I would pass interface around. I do explicitly reference count some objects. However, in other cases (I posted earlier) this is a huge problem. Someone also mentioned the issue of race conditions in multithreaded environments. I can see this happening to me. I would not mind declaring a class implicitly reference counted (an all descendants, presubably). However, not TObject.

Peter

Interfaces, among other things, are a way to bring in some features from multiple inheritance. I think the term is one-sided multiple inheritance.

If multiple inheritance is good or bad is a separate topic. With time, one begins to doubt the utility of classes in the first place and starts to appreciate ANSI C :-)

I'd add a _RefCount field to TObject, regardless of whether it's really used later; this will fix the field offset - just like the VMT reference is fixed in TObject, but not in Object types. This will eliminate problems with class helpers.

This approach also would allow to switch any object from managed to unmanaged on the fly, by setting the counter to -1, because the special value -1 already indicates an unmanaged/const memory object (like with string literals).

In my first draft I considered virtual _AddRef/_Release methods, but calling a virtual method is more expensive than calling or inlining a static method.

the code from above would then look like this to

Here the compiler would always insert _AddRef, just like with
interfaces, eventually optimized (inlined?) like:
   if Result._RefCounter  -1 then
     Result._AddRef; //or InterlockedIncrement(Result._RefCounter);

The convention, of -1 meaning unmanaged, favors managed objects by default, when InitInstance zeroes all fields of the instance just created. But when the VMT reference must be excluded or inserted afterwards afterwards, then _RefCount can be initialized at the same time (to -1 for the unmanaged default). Later on a TARCObject base class constructor/initializer will reset _RefCount to zero again.

- all classes can now have operator overloads as well though it should be warned in the documentation that non-reference counted objects might result in memory leaks there

...unless operators also test _RefCount

Here I'd prefer

   Owner.AddChild(Self);

so that the Owner can implement any decent/appropriate child management under the hood.

This depends on the implementation of TOwner.Children[] and TChild.Owner. Is a stored TChild.Owner reference really required in a managaged environment? IMO a (strong) unidirectional reference from Owner to Child will do it all. Then no child will be destroyed, as long as its owner holds a reference to it. That's the intended purpose of both owner/child and automatic memory management.

When it's desireable to definitely destroy an owned object at will, then its owner must be known, of course. In this case two different management approaches conflict with each other. In this case I'd accept a weak Owner reference, because the referenced Owner will stay alive longer than it's listed children.

More problematic are circular references without a decicated owner/child relationship.

This IMO is the preferable way to go, in a definite owner/child relationship. The lifetime of an owner can not depend on the existence of owned children, so that the owner will survive until it has destroyed/released all his children himself. A child-to-owner reference is not required in automatic management, it only is required when it must be possible to definitely destroy an owned child object.

Otherwise cyclic references *without* a definite owner/child relationship are really problematic:

That's the classic mark/sweep approach, with all its known drawbacks; in detail its asynchronous destruction of objects is a known troublemaker in OOP, see "Why a garbage collector never should call an destructor...".

In such cases IMO the developer should establish a definite owner/child relationship, by e.g. making all but one reference Weak, or by a special (intelligent) _Release method.

Even if I mentioned the use of Weak references as possible solutions of some problems, I still don't favor such solutions for the already mentioned reasons.

DoDi

Sven,
based on your comments a bizarre thought came to me. Currently, class instances are pointers. What you propose will somehow create a special pointer type (referenced class).

Should this be done, I have a request: create a 4 byte "compressed" class reference which expands into an 8 byte pointer type prior to usage:
1. Most architectures align data.
2. 64 bit comps align pointers on 8 byte boundaries, meaning that the 3 leftmost bits are 0.
3. You could store a class pointer as a 4 byte integer and, prior to usage, expand it to 8 bytes a leftshift 3.

In this way you will have 64GB of data available to your program and your pointers will occupy twice less. If, for example one uses large amount of pointers, this could be ... aesthetically pleasing ...

In our previous episode, Boian Mitov said:

I work with Delphi professionally too, and I don't really notice allocation as time costs. It's negiable. Occasionally (in the six months magnitude) I run the app through fastmm, but that time is general cleanup, so even not entirely the cost of memory management.

I can't find any basis for your allegations in my work.

What happens if such a reference (to A) is part of another object (B), known to the thread? I suspect a chance for a race condition here, when one thread clears B.A while another thread tries to acquire a reference from B.A.

But this may be an excursion into threadsafe coding, where any modification to a shared resource requires a lock in a multi-threaded environment. Then above situation should never occur...

Is it really sufficient to protect refcounter changes by Interlocked Inc/Dec, to prevent race conditions while obtaining object references?

DoDi

This is not true. Ref pointer is exactly the same size as non ref pointer. the counter is in the instance not in the pointer to the instance ;-) . You can study the interface implementations or the String implementations they are done the same way.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

I mean in terms of cost to write the lines of code. And later to review and manage it. It all adds up to a lot at the end. I have to find ways to pay for it every month :-( .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

In general you can have good and bad designs with and without ARC/weak pointers/ or sweep GC.

From all of the above, and I have done them all for years, the ARC is the

easiest to manage, and the one with the least side effects. What you was does not differ from the issues of lost pointers in non ARC. In essence you normally work with only Weak pointers, so if you let say like them so much now, I don’t see a reason to hate them in ARC ;-) .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Yes!

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
------------------------------------------------------- Is it really sufficient to protect refcounter changes by Interlocked Inc/Dec, to prevent race conditions while obtaining object references?

But not in CORBA mode. In this case no reference counting is done. I hope this behaviour will not change.

Michael

No, the idea is to compact an 8 byte pointer to a 4 byte one. Basically, save memory. This would limit the accessible memory tough. Probably for a different thread.

HOWEVER: here is another one idea. Current memory buses are 48 bit wide, so basically a pointer can be stored in 6 bytes. It is stored in 8, because of memory alignment issues.

So, for classes which are reference counted, store the reference count @ the highest two bytes of the class instance (which in practice is a pointer to the VMT). This would let you ref-count up to 2^16. You need to mask it out from the rest of the pointer when accessing VMT. No need for additional changes to the RTTI, just mask the highest two bytes when using the class.

Peter

That is not what I suggested. The idea is to compact an 8 byte pointer to a 4 byte one. Basically,
save memory. This would limit the accessible memory tough. Probably for a different thread.

HOWEVER: here is another one idea. Current memory buses are 48 bit wide, so basically a pointer can be stored in 6 bytes. It is stored in 8, because of memory alignment issues.

So, for classes which are reference counted, store the reference count @ the highest two bytes of the class instance (which in practice is a pointer to the VMT). This would let you ref-count up to 2^16. You need to mask it out from the rest of the pointer when accessing VMT. No need for additional changes to the RTTI, just mask the highest two bytes when using the class.

Peter

22.09.2014 0:28, Peter Popov:
[...]

This would look aesthetically reminds of
segment:offset ugliness of 8086 ;-)

Besides, thinking about a magnificent brand new pointer type, and as long as certain performance penalties can be tolerated, one might want to also consider providing some path for heap defragmentation, for the sake of completeness (Not that I know how to properly do it, but still my $0.02).

Regards,
Nikolai

We probably should if possible keep pointers just pointers (Ref counted pointer is same a non ref counted as we discussed so same size). Otherwise we may end up adding bits for the Moon Phases or something in it. Even if we add ARC we should do it as simple as possible and without increasing the size of the pointers (After all Delphi already has done it, and their ref counted pointers are still 4 bytes for 32 bit processes).

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

I openly *hate* the attribute syntax. It's just not Pascal and simply copied from their .Net compiler, because hey, other .Net compilers do that just like that... -.-

Also considering that the trunk compiler currently does not have support for attributes (let's put aside that branch for now) it is definitely easier to add a new keyword, because the compiler is built in a way to allow the easy addition of new keywords. Additionally using attributes would mean that one would need to add detection for special attributes. And that's more annoying stuff then a keyword is. Additionally this can be added more safely to other modes (e.g. macpas, which supports classes as well) than the attributes can.

Regards,
Sven

It's different, because interfaces were originally introduced to support interfacing with COM and the automatic reference counted was just added as a big pile of sugar to simplyfy development in context with COM. Because in C++ you have to call AddRef and Release yourself...

Regards,
Sven

I've especially written that it's not part of *every* objects, because people complained about the size increase for all instances.

And that's another thing: people complained about having that reference count overhead for *all* assignments.

It. Was. Just. An. Example. To. Illustrate. The. Problem! I would implement that differently as well, but there's *no* point to do that inside an example that's supposed to be as simple as possible!

Regards,
Sven

Peter Popov wrote

Sven, based on your comments a bizarre thought came to me. Currently, class instances are pointers. What you propose will somehow create a special pointer type (referenced class).

Huh? Why do you think that? It would just be that variables with a reference counted class type are handled differently. Isn't that exactly what you wanted? Leave normal classes untouched and add reference counting in such a way that it doesn't need to be used...

And I consider your pointer compression a really bad idea. It's bad to make assumptions about the memory layout especially when we have a general purpose compiler/RTL which would have no problems of using the complete address space.

Regards,
Sven

Definitely! :)

Regards,
Sven

AFAIR there already were several debates on ARC Objects that seem do be a "modern" language feature but in Pascal language they are considered to provide lots of pitfalls.

Why not use "interface" to add ref-counting to an object ? This seems to work nicely even though the name "interface" in not "speaking" on that behalf.

-Michael

Sven Barth wrote

"a farmer doesn't eat what he doesn't know"

Really healthy laughter from Germany,
-Michael

Boian Mitov wrote

In general I would prefer to see this implemented with attributes rather than with new keywords.

Yet another attribute to "class".

This might (re-)trigger thought about a complete rework of of record-enhancing types and allow for the "record" and "object" type definitions to be appropriately attributed (e.g. "static") "class"es.

Hence we would attributed to define classes that reside in unmodifyable memory (const), static memory (global), heap without ref-counting and heap with ref-counting (automatic).

-Michael

Giuliano Colla wrote

Can you explain how reference counting can be safely implemented in a parallel multithreaded environment, without heavily affecting performance?

*Very* good point !

IMHO a by far more advantageous feature to be added than ref-counted objects would be RTL support and Syntax candy for "parallel Loops", and the necessary support stuff like "Future variables" (or similar thingies).

All new mid-level hardware now has art least four cores and hence performance would benefit from such features in may projects.

So maybe "ARC" and "parallel" paradigms should be discussed together so that no mutual problems are introduced.

-Michael

Boian Mitov wrote

I consider it a design bug that local functions can't be passed as anonymous methods in Delphi. There is no reason for that not to be done, except that they did not bother.

???
Local functions can use local variables of the outer function. Same are logically undefined outside the outer function and technically stack based. Hence a local function can't be passed anywhere.

-Michael

Boian Mitov wrote

This is actually a very good idea IMHO! I would vote for it. Record inheritance is something I miss badly in Delphi.

As said in a previous mail: why have "Record" ad all, if you want things like inheritance (and "active" properties, ...) Maybe multiple brands of type if "class" would be more straight forward (making "record" just an alias for portability)

-Michael

On 21.09.2014 21:09, Hans-Peter Diettrich wrote: > Sven Barth schrieb:

[...]

That's why I also suggested an compiler option, useful for all people which care about program size or general I agree with your thoughts, I only wanted to add a few remarks.

[...]

See above :-)

[...]

It. Was. Just. An. Example. To. Illustrate. The. Problem! I would implement that differently as well, but there's *no* point to do that inside an example that's supposed to be as simple as possible!

Is it me, or what else makes you so angry today?
Sorry for that :-(

Regards
DoDi

"Michael Schnell" wrote

On 09/21/2014 06:36 PM, Boian Mitov wrote: > > I consider it a design bug that local functions can’t be passed as

anonymous methods in Delphi. There is no reason for that not to be done, except that they did not bother.

> ??? Local functions can use local variables of the outer function. Same are

logically undefined outside the outer function and technically stack based. Hence a local function can't be passed anywhere.

You do know that anonymous functions can use variables contained in the function they are defined in as well? Also local functions can be passed around in Delphi and recent FPC functions, but you shouldn't call them once the containing function as exited (so for local callbacks they are useable).

Regards,
Sven

"Michael Schnell" wrote

On 09/20/2014 08:33 PM, Giuliano Colla wrote: > > Can you explain how reference counting can be safely implemented in a

parallel multithreaded environment, without heavily affecting performance?

*Very* good point ! IMHO a by far more advantageous feature to be added than ref-counted

objects would be RTL support and Syntax candy for "parallel Loops", and the necessary support stuff like "Future variables" (or similar thingies).

All new mid-level hardware now has art least four cores and hence

performance would benefit from such features in may projects.

So maybe "ARC" and "parallel" paradigms should be discussed together so

that no mutual problems are introduced.

Interfaces, strings and arrays already work well in multi threaded contexts and ARC will use the same mechanisms here, so there is no need to discuss "parallel" and ARC together.

Note: ARC doesn't change that the developer him-/herself needs to make his/her classes threadsafe. ARC only "protects" the references.

Regards,
Sven

"Michael Schnell" wrote

AFAIR there already were several debates on ARC Objects that seem do be a

"modern" language feature but in Pascal language they are considered to provide lots of pitfalls.

Why not use "interface" to add ref-counting to an object ? This seems to

work nicely even though the name "interface" in not "speaking" on that behalf.

Because you'll need to declare an interface for each class you want to have reference counted so that you can access its methods, properties, etc.

Regards,
Sven

Yep. it will work.

But one point speaking against ARC is performance possible degradation, This of course is critical when trying to improve performance by using multiple cores in parallel. As the ref counting needs atomic operation, this is especially critical, because same are even more expensive when the variables are accessed by multiple cores (and hence reside in multiple caches).

OTOH ARC might be "interesting" (but supposedly not really advantageous) with parallel processing for creating thread-local objects in certain cases.

Note: ARC doesn't change that the developer him-/herself needs to make his/her classes threadsafe. ARC only "protects" the references.

Adding even more confusion to developing multithreaded projects :-( .

-Michael

Con: more typing
Pro: you know what you are doing.

-Michael

I cannot agree more.

This overhead could be eliminated by another syntax extension, like TMyARCclass = interface(TObject)
where the Compiler could allow for implementations of the declared methods just as for
TMyARCclass = class(TObject)
bridging the gap between traditional (strictly declarative) interfaces and classes (including implementations), with or without ARC.

DoDi

Another con:
Nobody will search for the term "interface" when he wants reference counting :-(

-Michael

Maybe instead of attribute appropriate base classes could be provided.

E.g. a (maybe fake) class "TRefCountedObject" that can be used instead of TObject to inherit from.

TRefCountedObject could provide the necessary properties and trigger some compiler magic that uses them (similar to when using a class derived from TInterfacedObject) and doing interface related syntax).

(Maybe in a similar line, "fake" base classes like "TRecord" and "TOldStyleObject" could trigger compiler magic to construct appropriate functionally compatible types to overcome the legacy (for me: fuzzy) paradigms "Record" and "old style Object", keeping the goodies, avoiding the quirks.)

-Michael

AFAIK *variant* records (currently) can not be replaced by something else. OTOH it should be no problem to allow for variant parts in other structured types, delimited e.g. by a visibility keyword. Variant parts should not prevent inheritance, only tricky typecasts, with "open-ended" arrays at the end of an record, then have to be handled with more care by the coder.

DoDi

copied from their .Net compiler, because hey, other .Net compilers do that just like that... -.-

> Maybe instead of attribute appropriate base classes could be provided. E.g. a (maybe fake) class "TRefCountedObject" that can be used instead of

TObject to inherit from.

TRefCountedObject could provide the necessary properties and trigger some

compiler magic that uses them (similar to when using a class derived from TInterfacedObject) and doing interface related syntax).

I already suggested the addition of a TARCObject as a general reference counted base class in my RFC ;)

Regards,
Sven

people complained about the size increase for all instances.

That's why I also suggested an compiler option, useful for all people

which care about program size or problem with the compiler option is that you'd need to rebuild the complete RTL, packages and your application so that every unit has these changes. Otherwise e.g. the RTL would still contain reference counting code and TObject would still contain a reference count field.

> > It. Was. Just. An. Example. To. Illustrate. The. Problem! > I would implement that differently as well, but there's *no* point to do

that inside an example that's supposed to be as simple as possible!

Is it me, or what else makes you so angry today? Sorry for that :-(

I'm sorry that I got aggressive, but it's quite frustrating when one writes a simple example to illustrate something and the first complain is how to make it use a better design which is completely besides the point of the example -.-

Regards,
Sven

Sven Barth wrote

I already suggested the addition of a TARCObject as a general reference counted base class in my RFC ;)

So I am proud to have independently recreated your idea :-)

-Michael

to work nicely even though the name "interface" in not "speaking" on that behalf.

> > Because you'll need to declare an interface for each class you want to

have reference counted so that you can access its methods, properties, etc.

methods just as for

TMyARCclass = class(TObject) bridging the gap between traditional (strictly declarative) interfaces

and classes (including implementations), with or without ARC.

I don't think this would be a good idea. Besides confusing newcomers there also the problem when to stop when generating an interface. Only the class declared? Also its perent classes? No, better make classes themselves reference counted and be done with it.

Regards,
Sven

contexts and ARC will use the same mechanisms here, so there is no need to discuss "parallel" and ARC together.

> Yep. it will work. But one point speaking against ARC is performance possible degradation,

This of course is critical when trying to improve performance by using multiple cores in parallel. As the ref counting needs atomic operation, this is especially critical, because same are even more expensive when the variables are accessed by multiple cores (and hence reside in multiple caches).

OTOH ARC might be "interesting" (but supposedly not really advantageous)

with parallel processing for creating thread-local objects in certain cases.

Then you should also not use interfaces, strings and arrays, because they use the same mechanisms.

> Note: ARC doesn't change that the developer him-/herself needs to make

his/her classes threadsafe. ARC only "protects" the references.

> Adding even more confusion to developing multithreaded projects :-(

What us confusing there? You already need to take care about the thread safety of your classes yourself and the refcounting will even relief you of having to think of one thing, namely when to free an object. Let's imagine this: you have two threads in addition to a mainthread that use a common class, let's say a queue. Currently you would most likely instantiate the queue before you instantiate the threads and pass tee queue along in their constructors. To free the queue you would wait until both threads have terminated.
Now with reference counting you would still pass along the queue, but you wouldn't need to care about freeing it, because it would be freed automatically once all three threads no longer hold a reference on the queue object. There's no race condition here, because even if all three threads would lose the reference at once there are only three references to lose to begin with and once the count reaches zero the object is freed. Because once the count is zero there can no longer be a reference to this object and thus no one can manipulate the reference count anymore (it's of course simplyfied, because with the interaction with non reference counted variables I mentioned you could play around this, but I'm talking about the standard case here).

Regards,
Sven

Sven Barth wrote

Then you should also not use interfaces, strings and arrays, because they use the same mechanisms.

Yep. The advanced developer should know what he is doing.

What us confusing there?

Of course I do know the pitfalls.

If fpc would provide a syntax for "parallel loop" and "future variables" e.g. similar to what Prism offers, even more inexperienced users will be inclined to use (invisible) threads. I suppose it's not possible to avoid the "confusion" that is introduced by this, but it should be limited as much as possible. (No idea how.)

-Michael

Hans-Peter Diettrich wrote

AFAIK *variant* records (currently) can not be replaced by something else.

I suppose you are right, but maybe appropriate syntax can be introduced to class definition that might be helpful in certain instances.

-Michael

Hans-Peter Diettrich wrote

TMyARCclass = interface(TObject)

IMHO, Sven's idea of introducing a "TARCObject" base class (triggering the compiler magic for ref counting) sounds a lot nicer.

-Michael

I prefer the TARCObject approach too, i don't want anything to be refcounted unless i *explicitly* ask for it (yes i know some things are already refcounted and at the past i was bitten by that which is a big reason why i don't want hidden things to do magic stuff behind my back). Also i don' want a useless field in TObject (like a refcount field), object instances are already too big IMO.

Although TBH what i'd prefer more than the above is language features that would allow me to implement refcounted objects myself. Stuff like automatic destruction when a generic variable goes out of scope, etc.

On Mon, Sep 22, 2014 at 5:45 PM, Kostas Michalopoulos <

Attributes are actually one of the biggest revolutions in computing IMHO. Pascal has one of the oldest, and lets admit it not the prettiest syntax. If we want to keep something in as un-pretty shape, we surely can, but should it be a goal?
Adding new keywords to any language pollutes the language, and is widely considered to be a bad practice. It should be avoided whenever possible IMHO.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

"Boian Mitov" wrote

Attributes are actually one of the biggest revolutions in computing IMHO. Pascal has one of the oldest, and lets admit it not the prettiest syntax.

If we want to keep something in as un-pretty shape, we surely can, but should it be a goal?

Adding new keywords to any language pollutes the language, and is widely

considered to be a bad practice. It should be avoided whenever possible IMHO.

The point is: attributes would only be enabled by default in the Delphi modes (and once I add a Delphi7 mode not even there), but classes affect at least the ObjFPC and MacPas modes as well. So there *must* be a possibility to achieve "weak"-marking (if we decide to support it) in those modes and others were a developer enables only modeswitch "class", but not attributes.

Regards,
Sven

Because it requires writing a lot more code, just to get the Ref. Count, and as I stated in another mail, you don't have direct field access.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Hi Sven,

Yes I knew the problem in the latter approach. However my thinking is that some features can be derived from others and I believe that this applies here too.

So by adding these extension to records and the operator overload the feature can be implemented as a library extension. Also those features might benefit others and also provide the groundwork for more improvements in the future.

In any case, all proposals are better than embarcaderos so I don't complain.

Thanks,
Chriss
Στις 21 Σεπ 2014 3:39 μ.μ., ο χρήστης "Sven Barth" <

If you have anonymous methods, you don’t need parallel loops, as I have demonstrated in my sessions. We use anonymous method parallel loop, and parallel recursion easily, and with greater flexibility than any language modification can achieve.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

But it can be encapsulated in the anonymous object of anonymous method as it is done with local (stack) variables already in Delphi. Anonymous methods can and do use local variables, and are regularly returned as result or passed to other threads. That is the power of anonymous methods. I covered that in great detail in my last CodeRage session on functional programming.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

In general, records and classes are inherently the same thing (and in C++ are indeed practically interchangeable). The only real difference in Delphi/FPC is that records are instantiated in the stack, the objects in the heap, and the artificial restriction on record inheritance. Now adding the option classes/objects to be instantiated in stack is a risky one, however having record hierarchies poses very little risk.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Strongly disagree...
For multithreading and parallel processing ARC is lifesaver. I have been using it trough interfaces for 10 years already, and once I rewrote the old code to use ARC it was a huge performance boost, and it keeps getting better.
Doing parallel processing without ARC is practically suicidal IMHO. I have also done a large number of sessions covering only some of the aspects of this in a number of conferences, but even those bits and pieces will give you idea of what ARC can do for parallel processing.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Then the best option is Assembler.
There you really know what you are doing.
In ARC I also know perfectly what I am doing. It may be con for some, but surely not for me ;-) .
Honestly if you don’t know what you are doing when using ARC, I doubt a bit, you really know what you are doing otherwise either ;-) .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

As I already stated, those are the same arguments that I have heard 30 years ago when people ware asking me why I would use C instead of doing it like them in Assembler. later, why I would use C++ classes, when we has sructures in C++. Why use virtual methods, in C++ instead of using the EC++ without them to “save space”? ...
I think history has at least so far always been on the side of my arguments ;-) .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

I am not big fan of the compiler handling parallel loops, as it will limit any flexibility in implementation.
A library based functionality with shared executors is a better design, and it allows not only for parallel loops but also for easy parallel recursions and many other combined(iterative/recursive) constructs.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Here is example from our implementation:

for AIndex := 0 to Length( AList ) - 1 do
AExecutionTask.Add(
FExecutor.Execute(
TDelegate.Capture<Integer>( AIndex,
procedure( ALocalIndex : Integer )

              begin
                Sleep( 1000 ); // Simulate thread busy doing time consuming

task

                AList[ ALocalIndex ] := ALocalIndex * 2;
              end
            )
        )
    );

AExecutionTask.WaitFor();

Notice the need for TDelegate :-( This is due to the fact that Delphi unlike C++11 does not allow the option to capture local context my value, and captures everything by reference :-( .
This is something to be considered when implementing anonymous methods or lambdas in FPC, especially for multithreading and parallel execution.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

BTW: Please note that since you can keep adding executions to the task, it can be done in nested loops, as well as in function calls thus doing fully fledged parallelization.
The executor also can be single thread, unlimited threads, self balancing, or limited pool depending on the specific instance, so a very fine control can be exercised over the execution, as well as it can be changed dynamically by code, thus sometime executing the code with one executor and other times with another, based on the specific needs.

Adding simple parallel loop lacks any of this power, and only serves very specific cases.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Boian Mitov wrote

In general, records and classes are inherently the same thing (and in C++ are indeed practically interchangeable). The only real difference in Delphi/FPC is that records are instantiated in the stack,

Or on the heap:
==========
Type
TMyRecord = Record

    End;

    PMyRecord = ^TMyRecord;
Var
   a: PMyRecord;
Begin
    a:= GetMem(SizeOf(TMyRecord));
End;
==========

the objects in the heap,

If you alocate them like
==========
Type
TMyObject = Object

    End;

Var
    a: TMyObject;
==========

`a` will be on the stack, not on the heap. I you allocate them using `New`, they will be on the heap.

See, that is what I love about this language and this compiler: you can do whatever you want! You want to write assembly: go ahead, wanna think higher level: you get a wonderful assortment of available primitives:
- Dynamic arrays/strings
- Static arrays/strings
- Records
- Objects
- Classes
- Interfaces

Al with their own unique flavour. It is a bit like two programmer programming assembly:
- One prefers a RISC style instruction set because it is composed of a set of primitive instructions that are easy to comprehend, the CISC instructions are complex and have many pitfalls is what he/she says.
- The other programmer says programming RISC is a drudgery: constantly writing the same instructions that do nearly nothing, have no special side effects, all the fun is gone.

As a programmer, pick the patterns you like, use them as much as you think is appropriate and know when they are not. In pascal you can do that because of the above.

So much for my little praise to pascal :-)

and the artificial restriction on record inheritance.

If you want a record with inheritance, use object.

I suggest doing one thing at a time. ARC class first, complex delegates capturing local context later

In my opinion having ARC class would be a great advantage. As far as I know this feature was proposed many times in the past. Developers have been asking for it for years. It just feels right because in FPC we already have ARC-interfaces, ARC-dynamic arrays and ARC-strings. And the Variant type, which is also ARC. In my opinion, implementing ARC for class type is both easier to implement and more important than anonymous methods. Anonymous methods save you typing, but that's it. ARC objects not only save you typing, but they also save you from many mistakes you can make while inserting .AddRef and .Release calls everywhere by hand.

I am not even sure if weak reference feature is necessary. We don't have weak references for COM interfaces, and it still works so far. Well, of course, if we had weak references as well, it would be a useful feature perhaps, but considering just *how many times* ARC class was proposed before, I lost hope that it will be ever implemented at all quite a while ago

Some people thing that GC is better than ARC. However ARC has at least one major advantage: unlike GC, it does not cause stop of the world. Complex Java programs often suffer from stop of the world because it may take considerable amount of time, and the application can't do anything during that time. For example, it can't respond to HTTP requests. If it is a game, even 200 ms freeze would be annoying. Well, you got the idea. ARC does not have to stop threads to collect garbage.

22.09.2014 22:54, Boian Mitov:

I know ;-) . I meant the default behavior.
Very much anything can be accessed and instantiated trough pointers :-) .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

hinstance wrote

I am not even sure if weak reference feature is necessary. We don't have weak references for COM interfaces, and it still works so far.

That's because you can't have circular references between interfaces, since interfaces don't have fields. Similarly, you can't have circular references between dynamic arrays and ansi/unicodestrings.

Jonas

What on earth has reference counting got to do with writing efficient multi-threaded code? You obviously confuse parallel performance, which reference counting will obviously degrade, with making some particular coding style easier. And sure, if your threads don't use shared resources they will be very parallel indeed.

If your argument is that by writing less code you have more time to refactor code, therefore you write better algorithms, and they run better
- then this would be a valid point for your situation. However, one size does not fit all. I am not going to re-profile code because suddenly assignments will not only incur a (possibly virtual) function call but also lock other threads even if for a cycle or two. Not to mention cache misses because every time you assign a pointer its value must be fetched from memory and incremented, therefore messing around with data which was supposed to be in the cache. And screw other threads' caches.

To put it differently, ARC will seriously degrade applications which fill the memory bus bandwidth, multi-threaded or otherwise. For such pieces of code it should not be used.

Peter

The thing is that we have weak references for COM – standard pointer ;-) ARC without optional weak pointers leads to guaranteed memory leak.

Simple example:

Parent calls with children, and child with pointer to parent. If the pointer to parent is not weak, the objects will never be freed. In COM we work around that with non interface pointers, (effectively weak pointers ;-) ).

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

If you are experienced in parallel processing, you should know the answer. This is a huge topic, and I have done number of sessions on some aspects of it, but here will give you a small example (again this is just a small example).
You have a data object that needs to be processed by 600 threads, and when they all have consumed(finished with it) it, it needs to be disposed. With ref counting you create the object send it to all threads, each thread has its own reference to the object.
When they all release the references the object is destroyed.

Even this very simple (again I don’t even involve pipelining or other threading nesting here) is much more difficult to implement without ref counting, and extremely error prone in traditional approaches. In our case we pass objects between hundreds of threads performing variety of processing between them. Without Ref. Counting (in this case with interfaces), we would simply would not have been able to do that... You can see our solutions here - www.mitov.com, and you can easily see the complexity of the parallel processing that takes place.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

"hinstance" wrote

I suggest doing one thing at a time. ARC class first, complex delegates

capturing local context later

Considering that those are two different features that don't have anything to do with each other their development doesn't need to be serialized, but can happen in parallel ;)

Regards,
Sven

Actually, although on first glance different, anonymous methods are done as reference counted objects in Delphi, (well technically trough interfaces.), so since the anonymous method storage by definition has to be kept alive trough reference counting or similar approach, the topic is not that detached from this one. PS: Why they call them anonymous methods when they are actually anonymous interfaced objects is something I always wondered about :-D . I guess mainly because of the syntax, not the content. i.e. name it based on the cover not the book :-D .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Right, it should be a compiler *build* option, so that everybody can create his favored flavor(s) of the compiler, and decide which one to use with which project.

Every compiler then should provide a predefined constant for every such option, in case specific handling is required by conditional compilation of user code.

Then I missed the point of the example(s). It looked to me like something final, well thought like the rest of your message.

DoDi

In general, records and classes are inherently the same thing (and in C++ are indeed practically interchangeable).

This model might have been the reason for introducing Object at all, for compatibility with CBuilder.

The only real difference in Delphi/FPC is that records are instantiated in the stack, the objects in the heap,

Like records, Objects can reside in either the stack, heap or even static memory.

and the artificial restriction on record inheritance.

Why inherit when you can't override virtual methods? I convert my Records into Objects, when I want to extend an record.

DoDi

Because TObject always has virtual table as example, and you can't use anything inherited from it to map into communication header. Structures on another hand by default start from empty hierarchy, and do not have Virtual Table pointer so I can create a structure and send it to a socket for transmission or map an received array into structure for easy access.
Unfortunately headers tend to have hierarchies ;-) , shich makes it a bit challenging to map them into Delphi records, or use the same function to process inherited record as example ;-) .
I know there are some not that pretty works around this, but that is what they are - ugly, and sometime dangerous(ahem... unsafe casting) work arounds.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

They are however only accessed through the implicitely generated interface and not through some object instance and thus the class ARC is not involved in any way, only the reference counting of interfaces and that already works. ;) Afterall anonymous functions were introduced before the class ARC :)

Regards,
Sven

In our previous episode, Boian Mitov said:

As I already stated, those are the same arguments that I have heard 30 years ago when people ware asking me why I would use C instead of doing it like them in Assembler.

OTOH the more radical people in then that said we would all be doing LISP by now weren't exactly right either :-)

In our previous episode, Boian Mitov said:

You have a data object that needs to be processed by 600 threads, and when they all have consumed(finished with it) it, it needs to be disposed.

I do this all the time, but I don't recognize one in such case has 600 objects with the per job context, and one object to capture the results with cumulatives. But since the results first need to be processed after all jobs finished, typically the result processor (a different thread) remains owner and frees.

It seems that the only results of your paralel processing is that all memory is freed.

Il 22/09/2014 23:04, Boian Mitov ha scritto:

I agree with you that parallel processing is a very huge topic, but just because of that I strongly disagree with your assumption that what can be good for a very specific kind of application is good for every application.

In addition to the observations of Marco, I can give you a small counter-example:

You have a number of threads creating thousands of data objects, which should be processed by "consumer" threads. Creating threads are unaware of how data will be used (it may depend on active options, and it follows the golden rule of "reusable code") so they can't send those objects anywhere. Therefore pointers to those objects are inserted in a list.

A "consumer" thread can either process a data object, and then dispose of it, or leave it in the list for further processing, or move it to a different list, again depending on the selected options. This scheme can apply to quite a number of applications, from data collected in a real-time process to data-base transactions, etc.

However you attempt to apply reference counting in this scheme, you'll realize that either it doesn't work at all, or it forces you to add a considerable amount of code, whose purpose would be only to fool reference counting, and artificially increase or decrease reference count.

The same applies to your example, if it happens that when creating your data object you don't know which threads will consume it, because threads are created dynamically (captors activated and deactivated, options enabled/disabled, users connecting and disconnecting, etc).

Giuliano

"Giuliano Colla" wrote

You have a number of threads creating thousands of data objects, which

should be processed by "consumer" threads. Creating threads are unaware of how data will be used (it may depend on active options, and it follows the golden rule of "reusable code") so they can't send those objects anywhere. Therefore pointers to those objects are inserted in a list.

A "consumer" thread can either process a data object, and then dispose of

it, or leave it in the list for further processing, or move it to a different list, again depending on the selected options.

This scheme can apply to quite a number of applications, from data

collected in a real-time process to data-base transactions, etc.

However you attempt to apply reference counting in this scheme, you'll

realize that either it doesn't work at all, or it forces you to add a considerable amount of code, whose purpose would be only to fool reference counting, and artificially increase or decrease reference count.

I don't see where you would need to play around with the reference count in your example. Following assumptions: the list only takes reference counted objects (currently this would be interfaces, with the class-ARC that would be object instances).
Now the producers add workitems to the list and may or may not keep an own reference to that workitem (so refcount is - simplified - either 1 or 2). A consumer now takes one object from the list, processes it, and either discards it ("obj := Nil;") or reinserts it in the list. If it discards it then either the object is now freed, if the producer did not keep a reference or it's not if it did. If the consumer puts it back in the list, then the refcount it still either 1 or 2 depending on what the producer did.

So I don't see where you'd need to manually fiddle around with the reference count here.

If you'd of course mix reference and non-reference counted workitems then you'd need to handle this, but then you should maybe overthink your implementation...

Regards,
Sven

Correct. I am only mentioning it as it is related to reference counted objects albeit trough interface.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Hi Giuliano,

I was not talking hypothetical example.
All of our products use this technology. Again, what you are telling me is some suggestion for probable solution, but even when you do it, you immediately add additional code and containers (list) just to support it due to the lack of ref counting.
This immediately allows me to argue that your solution will use more memory and will be slower as it needs additional containers and support code altogether. So you basically are reconfirming yet again the advantages of using ref. counting.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Hi Sven,

There is no one list.
We have N number o data sources of many different types. The data from each source is processed in a pipeline of threads that cane be reconfigured at any time. New sources and new data types can be added during the runtime by the customers. The data from different pipelines can be merged split, sent over network and anything in between. This is 100% universal high performance high troghput graph parallel processing of very much any type of data solution. You can learn more by looking at the OpenWire Studio as example, (free download), but all the solutions are working this way. Any other approach has limitations, and indeed more overhead (i.e. additional lists and code just to maintain the data)

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Is OpenWire compatible with FPC?

Oops, yes. :-)

http://en.wikipedia.org/wiki/OpenWire_(library)

It used to be compatible, however since we rewrote it to use anonymous methods, and attributes (And reduced the code to 1/4 of the original size in the process) ~2 years ago, the new versions are not any more Lazarus compatible :-( . I have reflected that in the Wikipedia OpenWire page. I hope one day OpenWire will again be Lazarus/FPC compatible.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

If you read the history:

June 12, 2013 | 6.0 | Dropped Lazarus support due to lack of language features.

It is not compatible any more :-( . Hope this will change.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

"Boian Mitov" wrote

There is no one list.

I was solely referring to Guiliano's example, not yours. ;)

Regards,
Sven

np :-).
The point is that no matter ref counting or a separated list+support code, you need additional overhead to maintain the objects. Arguably ref. counting is the one with the smaller overhead in this case, so even in his example ref. counting is already the better option.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Il 23/09/2014 18:00, Boian Mitov ha scritto:

I didn't claim that your example was hypothetical. I just argued that it's a very specific implementation.

Maybe I'm simple minded but I still have some pain understanding how a producer thread may manage to send an object to a one or more consumer threads which a) are unknown to it, and b) may not yet exist. Again not an hypothetical case. Just a different application frame.

Giuliano

To somebody (probably Sven):

In FPC there is concept of stack based objects (object) that behave like classes (with inheritance)?
Why not just extend this object with support for constructors and destructors, and there is no need for the ARC?

Yes, our solutions are among the most advanced that currently exist, and are on the cutting edge by any mean.
Most of the technologies are developed by us to solve problems traditionally considered unsolvable ;-) .
This is what keeps us in business ;-) .
Most people do not understand how we can do what we do, but there is no magic. It is all very well developed (I dear to say), and at the end very simple code (Took many years to get it that simple :-D ).

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

This is a valid point.
The only downside is that the object will every time do a deep copy versus a shallow (pointer only) copy. Deep copy is very, very expensive operation. Adding ref. counting to objects is probably equivalent of one interlocked assembly instruction, so a very small difference, probably not even measurable. Copy of a large object can take a large memory transfer, and will also reserve memory for each copy, thus significantly increasing memory. While both approaches have something going for them, they are not total subside for each other.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

:-(

Currently I'm programming a lot in Java (for Android) and jQuery (for web), and I miss the dynamic methods in Object Pascal of Free Pascal (Delphi already has, which are anonymous methods) too.

Nonproductive:

procedure TApp.DoMyButtonClick(Sender: TObject);
begin
  Alert('Button clicked');
end;

...

procedure TApp.Create(Sender: TObject);
begin
  MyButton.OnClick := @DoMyButtonClick;
end;

or:

procedure TApp.Create(Sender: TObject);
begin
  MyButton.AddOnClickHandler(@DoMyButtonClick);
end;

Productive:

procedure TApp.Create(Sender: TObject);
begin
  MyButton.SetOnClick(procedure Test; begin
      Alert('Button clicked');
  end);
end;

I have a lot of ideas to be implemented in two projects (Brook, dOpf) that I maintain. But unfortunately it will take a while to release the new version of the compiler (FPC 2.8), and this prevents me from continue working to release new versions of this projects.

I'm thinking to make the new version of this projects compatible with Delphi XE (the company that I work purchased the Delphi XE 2), so I can increase my productivity using new features like the new RTTI, generics and anonymous methods. But for now it will take, because I'm focused on the Java studies.

I have reflected that in the Wikipedia OpenWire page.

I hope one day OpenWire will again be Lazarus/FPC compatible.

Me too! (y)

I have a RESTful API written with Free Pascal 2.6.4 working like a charm and providing data to my clients via JSON in secure HTTPS connections, but the maintainability of this API could be improved, if the compiler decides to follow the evolution of OO languages.

I have no knowledge to help to create this improvements in the compiler, but I believe that the greatest difficulty here will be to convince the guys to break paradigms and accept that the linguangens are evolving. Our customers are increasingly demanding, so our productivity must accompany their needs, or they look for our competitors companies!

With best regards,

Boian Mitov

:-(

Sad but true.

Me too. Hope this will change! (y)

Boian,
since you keep on repeating the false argument that reference counting will somehow speed up parallel performance, I will restate the obvious: using reference counting will decrease parallel performance for the following obvious reasons:
1. Each reference assignment involves an atomic operation which, if used often enough will cause threads to wait and waste cycles.
2. Each reference assignment involves pulling the value referenced by a pointer and incrementing/decrementing it. In the process you will mess up with the cache of the current thread: data which was supposed to be in the cache for real work will be invalidated because of the ARC operation.
3. The cache of other threads will be similarly affected, even if the atomic operation does not involve other threads
4. ARC operations will likely involve virtual functions, which have overhead and do cause pointer running.

In summary, in memory intensive code, where the bus bandwidth is fully utilized, ARC is a serious problem because of 2-4. In CPU intensive code 1 is a problem.

Now, it is good that you managed to reduce your code by a factor of 4. Clearly this is not due to availability of ARC. Clearly what you do is not the only thing that programming languages are used for. So, the implication that we will all somehow benefit likewise if ARC is introduced is simply ludicrous.

Peter

Hi Pieter,

I think I explained very clear.
1. Each reference assignment involves an atomic operation which, if used often enough will cause threads to wait and waste cycles.

Correct you will lose ~3 assembly instructions (Interlocked increment/decrement-exchange, and conditional)
It will happen only when you create a new reference which happens rather rarely. The impact will be less that 0.0001% of performance and probably many orders smaller
On the other hand without this you already mentioned that you will need to create a list containing all the buffers. What is the performance impact on adding, removing from the list? What about the support code?

I think your arguments are turning against you here.

2. Each reference assignment involves pulling the value referenced by a pointer and incrementing/decrementing it. In the process you will mess up with the cache of the current thread: data which was supposed to be in the cache for real work will be invalidated because of the ARC operation.

The cache is a cache, the counter is part of the object you access, as long as you will be accessing the object (trough the pointer) it will need to be loaded into the cache, so I don't see any impact.

3. Already explained in 2.
4. Arc does not require virtual functions, however here is another misconception. Contrary to popular belief, on most processors virtual functions are executed faster than traditional functions. I can explain you the reasons for that on assembly, and cache level, but I think you can do your own research.

Now, it is good that you managed to reduce your code by a factor of 4. Clearly this is not due to availability of ARC. Clearly what you do is not the only thing that programming languages are used for. So, the implication that we will all somehow benefit likewise if ARC is introduced is simply ludicrous.

Everyone is entitled to their opinion. I share mine, and I share my experience, and I even share some of my code as open source. Others seem to prefer to just throw some imaginary arguments, not based on real experimental work.
If you have done some benchmarks between ARC and non ARC based threading as we have done, you can share them, and you can have something to back up some of the claims you make.
I have a lot of code and products to back my claims, so I am not just talking hot air, but a real concrete code, products and expertise.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

"Peter Popov" wrote

4. ARC operations will likely involve virtual functions, which have overhead and do cause pointer running.

There's no need for virtual methods for ARC. At least not in the design I envisioned for my RFC.

Regards,
Sven

Yeah.... he does not seem to understand ARC that well :-( .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

On the other hand without this you already mentioned that you will need to create a list containing all the buffers. What is the performance impact on adding, removing from the list? What about the support code?

What buffers? I destroy objects explicitly. The objects that need work create and destroy threads, not the other way around. Think of computational geometry algorithms, for example.

You pull a lot of data continuously, stored in objects with carefully aligned data fields. You execute a bunch of floating point instructions on data from several related objects. You change an object's field, then nearby fields, are invalidated from cache across threads.

Please, explain. The fundamental problem with virtual functions is pointer running. If your VMT reference is in the cache, sure, calling is fast. If object 1 execs a virtual to get object 2 who executes a virtual to get object 3 ... etc than you have a major problem. What are you going to do, keep all VMTs for all objects in the cache?

This conversation is really getting nowhere, but you are right for one thing, when more virtual functions are used, which is becoming fairly common (At least for destructors etc.), the VMT tends to stay in the cache, and since VMT functions are smaller than full address functions the execution tends to be slightly faster on average :-) . So it seems you have figured out this common misconception properly. Now you only need to learn enough to realize some other misconceptions you have, but you already have a lot of information from a lot of people in the thread, so it will probably not be long :-) .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Non productive sample is not quite accurate, since the event assignment is typically handled by RTTI during an .lfm loading. Though for dynamically created forms the assignment code indeed should be there.

About productive code. Should it look like this (note "Test" name is removed from procedure declaration)?

procedure TApp.Create(Sender: TObject);
begin
  MyButton.SetOnClick(procedure; begin
      Alert('Button clicked');
  end);
end;

thanks,
Dmitry

Maybe would be best to activate reference count only for specific classes through a compiler directive. This way we can see this new feature in production without breaking the existing code base and give the programmer the flexibility to choose which classes must be freed automatically.

Best regards

Actually the Delphi compatible code would be :

procedure TApp.Create(Sender: TObject);
begin
  MyButton.SetOnClick(
    procedure
    begin
      Alert('Button clicked');
    end );
end;

There is no comas in the procedure declaration.
We have even created multicast anonymous methods in Mitov.Runtime so you can add more than one anonymous methods to an event.

procedure TApp.Create(Sender: TObject);
begin
  MyButton.OnClick.Add(
    procedure
    begin
      Alert('Button clicked and called me');
    end )
  .Add(
    procedure
    begin
      Alert('Button clicked and called me too');
    end );
end;

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

That's exactly what I wanted to confirm - no name - for an anonymous function.

I'm sure eventually the development society, will discourage the use of anonymous functions (just like unnamed constants are considered to be a bad pattern). Thus the only thing that would remain welcomed are closures - copying the execution context for further invoke of a function.

thanks,
Dmitry

Hmm... I have serious doubt the anonymous methods (actually referenced objects as previously discussed) will be discouraged. If anything they are more and more used across the majority of the languages, as they have huge advantages. As discussed a single anonymous method can, and usually does replace a whole class declaration and implementation, so the chance of them being discouraged is very slim. We have converted the majority of our code to use them in the last 2 years, and that reduced the code to 1/4 of the original size. All new code we write in my company is 100% anonymous methods centric. So far this has worked well for us, but sure the future will show who is right on this one ;-) .

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Just like "goto" operator was available in the majority of the langauges, back in a days. And it's still there!
Don't get me wrong, I'm just sharing my thoughts, rather than trying to discourage you from using it.

We have converted the majority of our code to use them in the last 2 years, and that reduced the code to 1/4 of the original size. All new code we write in my company is 100% anonymous methods centric.

How about the maintenance cost?
Is it now easier-faster-safer to bring changes into the code without introducing regressions?

thanks,
Dmitry

Extremely easy.
We made the decision after discovering how much easier the code is to maintain and grow. We have sped up our development by a factor of more than 3 times, and we have reduced maintenance issues by approximately 6 times. As I said, the future will show who was on the right track ;-) . I have the feeling you think anonymous methods are just methods without a name, that is not the case at all (just the syntax is like that), so that is probably why you underestimate their power. If you are interested to learn what makes them so enormously powerful, you can watch my session of functional programming on the last CodeRage. But there are also plenty of other resources.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Yes, but if you have compiler support for objects with automatic calls of constructors/destructor/copy constructor/assignment operator (like in C++ classes), and if you also have generics support for objects, you can make a generic smart pointer object type that adds reference counting to any class. And then the regular class references will serve as weak references. Exactly like C++:

http://www.boost.org/doc/libs/1_56_0/libs/smart_ptr/smart_ptr.htm

What's more important is that we can extend objects this way, without breaking backward compatibility. But, unfortunately, it's a huge slippery slope from there, as anyone, who has dealt with C++ STL error messages knows:

http://pascalg.wordpress.com/2008/02/24/stl-error-messages-are-so-great/

Nikolay

100% agree, although although smart pointers also can already be implemented in Delphi with anonymous methods. I wanted to mention the possibility actually in my post. For good smart pointer however we also will need implicit casting, operator overloading and few other features, but it surely is an option. As for STL, we actually already have implemented some aspects of it in Mitov.Runtime (Free library) for Delphi, and will be doing more. I am huge fan of STL/boost and the concept based development, and I use some aspects of it even in Delphi (to the extend the language and the generics allow).

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Peter Popov wrote

, I will restate the obvious: using reference counting will decrease parallel performance

You are perfectly right.

So you should try to avoid heavy ref counting (i.e focusing / defocusing ref counted stuff, including Strings) in multithreaded code. Problem: the programmer can't easily see where focusing / defocusing is done.

OTOH, ref counted objects can reduce the programming effort when thread-local instances are necessary.

Depreciating ref counted objects for multithred-performance means to depreciate Strings as well. ;-)

-Michael

Boian Mitov wrote

Correct you will lose ~3 assembly instructions (Interlocked increment/decrement-exchange, and conditional)

With modern x86 archs, interlocked instructions are very expensive in themselves and can take thousands of clock cycles due to cache synchronization between all CPUs.

Moreover other archs use completely different (less hardware-driven) assembler constructs for interlocked operations (e.g. load locked / store conditional)

-Michael

Sounds nice, but is it (easily) possible to implement that the compiler magic for doing the ref-counting is issued exactly whenever the TARCObject is an ancestor ?

Such decision seems like unprecedented yet.

-Michael

Hmm.

I just did that (a TThreadPool class, see the other thread). Not because I wanted to use it, but to do some research on the issue.

While I do think that the thingy does work well and is decently designed, and it might be nice to provide it as a package, I think it's not "nice" to use, as you need to do and instantiate appropriate objects with "Execute" functions to make use of it.

Some compiler magic could do this on the fly for lazy programmers

Of course I do know about the possible risk if they unintentionally might access the same thingy without caring for mutexes. But I am not sure that not providing a parallel loop syntax would help much.

-Michael

As the parallel loop syntax would be nothing but syntax candy to easily make use of a thread pool library, nobody would prevent the user from directly use the library if he needs certain extended features.

-Michael

hinstance wrote

If it is a game, even 200 ms freeze would be annoying.

We do lots of embedded applications using Pascal. unavoidable "Stop of World" would be "End of World".

-Michael

Will the destructor of a reference counted object not be a kind of anonymous/closure procedure ?

-Michael

Boian Mitov wrote

Strongly disagree...

You can't deny it. Please read some technical docs about the cost of interlocked operation in multicore systems.

Doing parallel processing without ARC is practically suicidal IMHO.

That might well be, but it does not deny the cost.

-Michael

counted base class in my RFC ;)

> Sounds nice, but is it (easily) possible to implement that the compiler

magic for doing the ref-counting is issued exactly whenever the TARCObject is an ancestor ?

TARCObject would just be provided for convenience reasons. The real magic is a keyword that can be used for any class to enable ARC for that class and all it's descendant.

And even if TARCObject would be special the compiler know the full hierarchy anyway otherwise it couldn't decide which methods to call.

Such decision seems like unprecedented yet.

Nope. Normal compiler business.

Regards,
Sven

Hans-Peter Diettrich wrote

Why inherit when you can't override virtual methods?

Easily adding field.

-Michael

anything to do with each other their development doesn't need to be serialized, but can happen in parallel ;)

> Will the destructor of a reference counted object not be a kind of

anonymous/closure procedure ?

Why should it? When the reference count is decreased to zero (and thus the final reference on the instance is gone) the destructor method is executed (which is just a "obj.Destroy;"), simple as that (in reality it's a bit more complex, but that's an implementation detail).

Regards,
Sven

Boian Mitov wrote

Honestly if you don’t know what you are doing when using ARC, I doubt a bit, you really know what you are doing otherwise either ;-) .

Ref counted strings are no problem with any (normal) use of same.

In this thread you find an example describing how ARC Objects easily can impose memory leaks.

-Michael

I see. Hence, technically, there would be a kind of "attribute syntax", but it's only necessary to afflict it to the type TARCObject, that already is defined in the RTL. Hence the user does not see it.

Now the "RefCounted" attribute is inherited by it's siblings.

Nope. Normal compiler business.

Great ! So there will not be any harm with your RFC.

-Michael

Nice.

I already used "locally anonymous" Objects created by one thread and freed by another after a certain procedure had been called to implement inter-Thread events. Maybe with ref-Counted Objects this can be done in a more sexy way.

-Michael

In our previous episode, Michael Schnell said:

You can't deny it. Please read some technical docs about the cost of interlocked operation in multicore systems.

(I assume you mean multiPROCESSOR systems here?)

magic is a keyword that can be used for any class to enable ARC for that class and all it's descendant.

> I see. Hence, technically, there would be a kind of "attribute syntax",

but it's only necessary to afflict it to the type TARCObject, that already is defined in the RTL. Hence the user does not see it.

Of course the user can see it. Namely to introduce refcounting to classes where he/she can't influence the parent class (simple example: a TStrings descendant)

Regards,
Sven

"Michael Schnell" wrote

On 09/22/2014 08:37 PM, Boian Mitov wrote: > > Honestly if you don’t know what you are doing when using ARC, I doubt a

bit, you really know what you are doing otherwise either ;-) .

Ref counted strings are no problem with any (normal) use of same. In this thread you find an example describing how ARC Objects easily can

impose memory leaks.

Not really. The object instances themselves won't create memory leaks, because with strings you don't get them either. The only problems are typical "not threadsafe" problems of the classes themselves and there ARC won't help you anyway as this is the programmer's job to ensure. ARC objects will only be destroyed when the last reference to them has gone out of scope. If there is still a reference, then there's still the object.

Regards,
Sven

Hi Sven,

Can you comment on how are you going to handle the case of TInterfacedObject and descendants? Are you going to use the same Addref & Release method ? How can you guarantee that the same counter will be updated in case of IUnknown implementations that introduce their own refcount variable (possibly with different name)?

Are you going to check that the class is already destroyed (because the arc has killed the instance) on IInterface.Release so that we should not get access violations on scenarios that mix ref counted classes and interfaces ?

Thanks for your time and efforts,
Chriss
Στις 25 Σεπ 2014 4:14 μ.μ., ο χρήστης "Sven Barth" <

What I am saying is that the parallel loop handles only one specific case of parallelization, and with limited options.
A library implementation on the other hand offers full flexibility.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

> Strongly disagree... You can't deny it. Please read some technical docs about the cost of interlocked operation in multicore systems.

As compare to the cost of other locking methods such as critical sections that have to be implemented instead if the ARC is missing?

> Doing parallel processing without ARC is practically suicidal IMHO. That might well be, but it does not deny the cost.

There is always cost for parallelization (Actually same when doing it with humans - I know that as a CEO :-D )

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

Strings do not contain pioneers themselves.
Any standard tree design however usually contains circular reverences, and pose the thread of leak if Weak pointers are not used usually for the Parent pointer in the children.
Double linked list is another example.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

There are two cases considering my RFC:
- an user uses TARCObject, then maybe we can provide a TARCInterfacedObject as well, which just implements AddRef and Release of IUnknown accordingly
- an user adds the refcounted modifier to an existing class type of which one of the parents used a reference counted interface as well; the best here would be to reimplement AddRef and Release to use the reference counting methods provided by the ARC class.

Are you going to check that the class is already destroyed (because the arc has killed the instance) on IInterface.Release so that we should not get access violations on scenarios that mix ref counted classes and interfaces ?

This is not possible as in that case the reference would already be gone. But if you implemented the IUnknown reference counting using the ARC one then there shouldn't be any stray Release calls anyway, except a C++ (or another language that has no automatic reference counting for IUnknown) library has a bug regarding reference counting.

Regards,
Sven

Hi again,

How would you handle the case of a user adding an interface on a class marked as reference counted (like a tstringlist descendant, TArcStringList) and then inherits from that and implements the IUnknown TFooStringlist =class (TArcStringlist, IInterface).

Will he be able to hook on the addref and release methods of the arc object ? Or at the least use the refcount variable? Will it be done automatically by the compiler (by using the same method names maybe) ?

Thanks,
Chriss
Στις 25 Σεπ 2014 6:36 μ.μ., ο χρήστης "Sven Barth" <

Sven Barth wrote

Of course the user can see it.

But he does not _need_ to. :-)

Namely to introduce refcounting to classes where he/she can't influence the parent class (simple example: a TStrings descendant)

Sounds like a great concept.

-Michael

Boian Mitov wrote

What I am saying is that the parallel loop handles only one specific case of parallelization, and with limited options. A library implementation on the other hand offers full flexibility.

Of course I did understand that.
What I was saying that providing a "parallel loop" syntax candy to make the library functions more easily usable in standard cases is (nothing but) an additional benefit, but very helpful for many users..

-Michael

I don't see the point.

If ARC is missing you need to call "free". No Critical section is involved with that.

Yep. But with ARC you force synchronization in occasions where it would not be necessary without ref counting.

-Michael

I understood, the example showed that the object might survive even if there is no reference any more.

-Michael

A parallel loop syntax is very attractive, but the practical difficulty is that it would probably have to sit on top of threads and getting code distributed promptly over multiple worker threads, i.e. starting up within 10s of nSec rather than 10s of mSec, is non-trivial.

I still think that a prerequisite is something that can enforce sequential operation, so that a paralleled block could not be reentered:

procedure Sub (var x : array of Float); sequential;
  procedure ParallelBlock; parallel;
  begin
  end {ParallelBlock};
begin
  ParallelBlock;
end {Sub};

I suppose an interesting challenge, bearing in mind your earlier question about signals, is whether procedure or block modifiers can be declared as part of the language but implemented as replaceable runtimes.

Starting a new thread of course is too expensive for something like that.

IMHO, the RTL could be provided with a ThreadPool implementation (I just did such a thingy and it seems to work nicely).

Same could be accessible by the user for more complex stuff and be used as the infrastructure of a parallel loop syntax.

-Michael

I agree, and I don't think there's a lighter-weight alternative.

IMHO, the RTL could be provided with a ThreadPool implementation (I just did such a thingy and it seems to work nicely).

Different implementations could potentially use either local threads or OpenMPI.

Same could be accessible by the user for more complex stuff and be used as the infrastructure of a parallel loop syntax.

Agreed, but a good starting point would be working out how to map a sequential (non-reentrant) block syntax onto e.g. the existing critical section class.

If ARC is missing you need to call "free". No Critical section is involved with that.

Incorrect the pointer on which you call free needs to be guarded.

Yep. But with ARC you force synchronization in occasions where it would not be necessary without ref counting.

That is very questionable, usually ARC does it exactly in the places you have to guard, and tends to do it cheaper (in terms of code and CPU usage).

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

See... this is the point. In order to properly implement parallel loops etc. even in their limited form the compiler needs specific RTL support. So we break one of the basic principles that the compiler is the root, and the RTL is a library compiled in it. Once you start getting the compiler to depend on RTL implementation, you start to get in troubles IMHO. I think parallel processing belongs in library implementations.

With best regards,
Boian Mitov

------------------------------------------------------- Mitov Software
www.mitov.com
-------------------------------------------------------

The compiler already depends heavily on the RTL for e.g. string and dynamic array handling. If a string or a dynamic array is already a first-class construct, why shouldn't a foreach be?

I think parallel processing belongs in library implementations.

I have reservations, based in part on the fact that other language implementations are prepared to assume responsibility for parallelisation, in part on experience with e.g. APL which at the very least specifies that the user should assume that operations are parallelised, and in part on the fact that FPC already vectorises on e.g. SSE2 hardware.

What do you (both) mean by "parallel processing"?

Streaming (SSE...) does *vectoring*, i.e. multiple (floating point) operands of the same *array* are processed in parallel. Such cases can be handled by the compiler, no libraries are involved, no threads, no risk of side-effects.

When instead a general loop, possibly containing multiple statements, is broken into multiple loops, which are processed in parallel, side-effects can occur depending on the operations (virtual methods...) in the loop. Then it IMO is up to the developer to check which loop can be parallelized without side-effects, and indicates that to the compiler. In this case the compiler could turn the body of the loop into an TThread, and insert an RTL call to execute this thread split into multiple instances. The RTL then creates the threads at runtime (depending on what? cores, already active threads...?), assigns to each an subrange of the entire loop interval, starts them and waits until all of them have terminated. This would limit the types of loops to FOR loops, with a known interval, excluding REPEAT, WHILE and FOREACH loops. Right?

DoDi

If the time spent in this thread had been spent in coding, FPC would have already ARC. The list has approx. 600 members, 200 message were written. If each of the 600 members spent on average 1 min reading this thread, this is 2000 man-hour, i. e. approx. 1 MY :)

I strongly agree!

I'd suggest that http://wiki.lazarus.freepascal.org/OpenMP_support and related pages is an adequate answer; alternatively see http://www.elementswiki.com/en/Parallel_For_Loops. However the point I'm really trying to make is that IMO before anybody seriously considers adding any form of parallelisation at the language level they should add serialisation (i.e. comparable with a Brinch Hansen monitor etc.).

Streaming (SSE...) does *vectoring*, i.e. multiple (floating point) operands of the same *array* are processed in parallel. Such cases can be handled by the compiler, no libraries are involved, no threads, no risk of side-effects.

I'm thoroughly aware of the distinction.

Ehm... the coupling between the compiler and the RTL (or better the System unit and some other automatically included units) is already rather high and I don't see this reduced anywhere in the future. In fact it's more likely to get the other way round. E.g. support for the "delayed" directive will require the support currently provided by the DynLibs unit inside the System unit (though this will likely go the same way as variants, wide strings, threading, etc: using a manager record).

Regards,
Sven

When one adds an interface to a class that doesn't yet support interfaces then you need to implement the IUnknown methods which you can reroute to the refcount methods of TObject then (TObject will have non virtual methods for manual reference counting that are active if the instance is a reference counted one). If a parent class already introduced interfaces (e.g. TComponent) then it's upon the implementor to decide whether he/she wants to route the interface reference counting to the ARC one or just keep it disabled like TComponent does.

Will he be able to hook on the addref and release methods of the arc object ? Or at the least use the refcount variable? Will it be done automatically by the compiler (by using the same method names maybe) ?

The compiler won't do anything automatic here precisely for the reason that then one needs a way to prevent this automatisms.

Regards,
Sven

Florian Klämpfl wrote

If the time spent in this thread had been spent in coding, FPC would have already ARC. The list has approx. 600 members, 200 message were written. If each of the 600 members spent on average 1 min reading this thread, this is 2000 man-hour, i. e. approx. 1 MY :)

To be honest: I tried.

There are however some nasty problems inside constructors and destructors, because "Self" is reference counted as well (and should be after all as we don't want the instance to be destroyed behind our backs suddenly).

Regards,
Sven

If the time spent in this thread had been spent in coding, FPC would have already ARC. The list has approx. 600 members, 200 message were written. If each of the 600 members spent on average 1 min reading this thread, this is 2000 man-hour, i. e. approx. 1 MY :)

First understanding the compiler code will take more time, until one knows where to start coding.

Second I already supplied two proposals, which could be implemented in a few lines:
1) Use virtual _AddRef and _Release, override for ARC classes
2) Dto. non-virtual, add _RefCount to TObject, init to -1 for no ARC In either case the compiler inserts calls to _AddRef/_Release wherever it already does for interfaces.

Last not least somebody must be entitled to implement ARC, against all objections of the users ;-)

DoDi

There are however some nasty problems inside constructors and destructors, because "Self" is reference counted as well (and should be after all as we don't want the instance to be destroyed behind our backs suddenly).

IMO before the end of a constructor, and before the start of an destructor, no references to the object exist at all, so that code outside these methods has no reference that could cause trouble.

It looks to me like inside methods Self doesn't deserve refcounting, because a method can be invoked only with an existing instance, which will stay alive at least until the call returns.

DoDi

The problem is that a destructor can call other methods which might take the object reference as a parameter and each such passing will have calls to the reference counting again. So there needs to be a way to avoid infinite calling of the destructor. (I fixed this for now by simply disabling reference counting with a -1 reference count when the destructor is called by the reference count decrease helper).

It looks to me like inside methods Self doesn't deserve refcounting, because a method can be invoked only with an existing instance, which will stay alive at least until the call returns.

That's the thing I'm not yet entirely sure about. Though disabling it for Self would definitely simplyfy things. I'll simply give it a try and then my constructor problems should hopefully be solved as well...

Regards,
Sven

Sven Barth wrote

There are however some nasty problems inside constructors and destructors, because "Self" is reference counted as well (and should be after all as we don't want the instance to be destroyed behind our backs suddenly).

The parenthesis can happen in Delphi interfaced objects as well. It is perfectly possible to lose the object from under your feet if the code of one of its methods ​leads to references reaching zero (e.g. removal from a strong-ref'd list). If such a possibility exists the developer needs to manually _AddRef/_Release (e.g. store Self in a local interface variable) if they need to access Self after that point, and that is perfectly OK. The only way to avoid that would be to emit code at a compiler-generated implicit try-finally block to _AddRef/_Release, which is a bad idea IMHO as it adds overhead when it won't be required for 99.99% of cases.

As for destructors, the way refcounting is implemented in Delphi it prohibits performing _Addref/_Release (e.g. pass an interface variable of Self to an external method) in them because this will cause infinite recursion. This problem has an easy solution: Override BeforeDestruction and set RefCount to a positive value, 1 or MaxInt div 2 (not -1, consider the sequence _AddRef[0], _AddRef[1], _Release[0-bang], _Release[AV]) after calling inherited (which checks for 0). Since the object destruction process has already started due to RefCount reaching zero, refcounting is useless at this point and can be safely disabled. You could consider doing this in your implementation.

--Constantine

That's what the compiler currently does though, because it treats Self just a like a normal reference counted variable... Will need to check what I need to do do switch that of ;)

As for destructors, the way refcounting is implemented in Delphi it prohibits performing _Addref/_Release (e.g. pass an interface variable of Self to an external method) in them because this will cause infinite recursion.

Huh? Delphi really prohibits such things inside the destructor? Would you mind to share an example (possibly including error messages), please?

I've done a different approach already: the decrease helper sets refcount to -1 directly before it calls the destructor and both the increase and the decrease helper check for -1. Considering that it's currently only a proof of concept I can live with the performance impact that comparison might have.

Regards,
Sven

In our previous episode, Sven Barth said:

Methods might return SELF as function result?

Yes, and this might be a problem during and immediately following construction. When the constructor passes Self to another procedure, that procedure must not do anything that increases and later decreases the refcount. Increasing the refcount during construction might help, but then the refcount cannot be decremented at the end of the constructor. And what should happen to the result of Create?

   MyObj := TMyObj.Create;

is fine when the refcount is increased to 1 when the instance is assigned to MyObj. In contrast

   TMyObj.Create;

looks somewhat useless, and how to destroy this zombie later, without refcounting? This construct might be legal with TThread.Create? Can somebody test what Delphi does in this case?

Suggestion:
The constructor inits the refcount to 1, and decrements it on exit - *not* using _Release! This will prevent destruction by refcounting during construction.
The compiler uses (always, or only with latter syntax?) a hidden local variable, where a new reference is stored and refcounted. This will destroy an zombie on exit from the subroutine (maybe unit initialization!).

Another one - weak references:
When the compiler only handles immediate assignments to weak references, such a variable cannot be passed as VAR, because the called code cannot know about that special handling.

DoDi

This would only be a potential problem if such methods are used inside the class itself. And even then it would only be a problem if there wouldn't still be another reference to the object and if there wouldn't be on then there would be no reason for the object to be around anyway. The only case where this could really be a problem would be the constructor and there we simply would need to initialize the reference count with 1 and decrease it at the end of the constructor. Hint: TInterfacedObject does exactly that already.

Regards,
Sven

[snip]

Suggestion: The constructor inits the refcount to 1, and decrements it on exit - *not* using _Release! This will prevent destruction by refcounting during construction.

This is precisely what is already done for TInterfacedObject which faces the same problem if it's Self value is passed around as an interface.

Another one - weak references: When the compiler only handles immediate assignments to weak references, such a variable cannot be passed as VAR, because the called code cannot know about that special handling.

Would be interesting to see what the Delphi NextGen compiler is doing there. Maybe such var-parameters need to be marked with WeakRef as well...

Regards,
Sven

​Hi Sven,​

Sven Barth wrote

Huh? Delphi really prohibits such things inside the destructor? Would you mind to share an example (possibly including error messages), please?

​Sorry, I wasn't clear enough. ​
​With ​"prohibits" I didn't mean that you get a compiler error. It compiles and runs just fine but you get a stack overflow at runtime ​ when the dest​
​ructor is called​
.

I've done a different approach already: the decrease helper sets refcount

to -1 directly before it calls the destructor and both the increase and the decrease helper check for -1. Considering that it's currently only a proof of concept I can live with the performance impact that comparison might have.

​OK, but keep my proposal in mind since it has overhead only at BeforeDestruction which is once per instance lifecycle.

--Constantine

Ah, ok. That was indeed not clear enough. ;)

I can't use BeforeDestruction per se, because the user can override it. But it would be possible to insert the code at the same location where BeforeDestruction gets called... I'm doing something similar for NewInstance currently where I force the reference count to 1.

Regards,
Sven

Mark Morgan Lloyd wrote

Different implementations could potentially use either local threads or OpenMPI.

Yep.
With "Local Thread" you supposedly mean user land threads. Same would need to be implemented together with an appropriate set of synchronizing functions. Seems like a huge effort (in fact a complete preemptive multitasking OS running in user land)

I suppose there are several ways to implement OpenMP.

Hmm

As "easy to use" syntax I meant simple stuff that don't need explicit critical sections but just a means to hold the main line code until the spawned parallel processing is ready. I understand in Prism this is done with "future" variable, but I don't know how they are used.

Something like this to do a matrix multiplication

parallel for i := 0 to m-1 do begin
   parallel for j = 0 to n-1 do begin
     r := 0;
for k := 0 to n-1 do begin
       r := r + x[k,j] * y[i,k];
     end;
     z[i,j] = r
   end;
end;
future Z;
z[0,0]:=Z[0,0]+1;

(or similar :-) )

Here, obviously Z needs to be defined as a "future" that is available not before all instances of the parallel loops are done.

The compiler magic and library would need to take care of launching an appropriate count of threads (about as many as the count of processors available in hardware).

-Michael

Boian Mitov wrote

> If ARC is missing you need to call "free". No Critical section is > involved with that. Incorrect the pointer on which you call free needs to be guarded.

Only if it is used in a way that asks for this. Whether or not it is done is up to the user.

-Michael

IMHO it usually is better to first decide if you want to drink tea or bear before you run to the fridge or to the cupboard.

-Michael

Michael Schnell wrote

IMHO it usually is better to first decide if you want to drink tea or bear before you run to the fridge or to the cupboard.

That could be a fatal error, even if permitted by the parser :-)

Michael Schnell wrote

Something like this to do a matrix multiplication

At the moment the only point I am making is this. Assuming a thread pool which can be applied to a parallel activity, it might be desirable to only use that from a single app-level thread. Hence (using your syntax only for consistency):

   serial for i := 0 to m-1 do begin

parallel for j = 0 to n-1 do begin

..

end; end;

to ensure that the inner loop is parallelised to the greatest possible extent, but that the outer loop is protected from reentry.

Mark Morgan Lloyd wrote

to ensure that the inner loop is parallelised to the greatest possible extent, but that the outer loop is protected from reentry.

Of course you are right.

I did this example to show that the compiler magic should be able to handle nested parallel loops.

-Michael

I don't suppose anybody is working on a parallel loop language feature.

-Michael

Maybe; however I am asking about reference counted objects feature. I thought Svan Barth said that he considered implementing it

13.10.2014, 18:11, "Michael Schnell" :

Maybe; however I am asking about reference counted objects feature. I

thought Svan Barth said that he considered implementing it

I have a working proof of concept implementation of my RFC and am "currently' commiting it to branches/svenbarth/arc. I'll write a mail once I'm done with committing with details about the usage, etc..

Regards,
Sven

Great! Sorry for misspelling your name btw

I had worse already. "Swen" for example. Or a common one at uni was "Svn" ;)

Regards,
Sven

Better than "git" ;-)

Nikolay

Well... it's harder to miswrite my name as "git" than it is to miswrite it as "svn". But enough of the off topic :P

Regards,
Sven

Maybe; however I am asking about reference counted objects feature.

Sorry for the fuzz. (Your contribution was a direct reply to my mail about parallel loops.)

-Michael

I thought Svan Barth said that he considered implementing it

Regarding this, Maybe, "Success" in implementing should be preceded by a decision if and how this feature does make sense regarding the possible addition danger it offers (see the other contributions in this thread).

-Michael

decision if and how this feature does make sense regarding the possible addition danger it offers (see the other contributions in this thread).

It's a proof of concept implementation to have real code to work with instead of just theories. Also it's in a separate branch, so it doesn't interrupt any existing code except one explicitly uses that branch.

Regards,
Sven

Sven Barth wrote

It's a proof of concept implementation to have real code to work with instead of just theories. Also it's in a separate branch, so it doesn't interrupt any existing code except one explicitly uses that branch.

Sounds good.

-Michael (curious...)