Hi,

Is right to call Initialize operator for Default(TFoo) where TFoo is record with management operators? IMO the right direction is to keep "meta record instance" in place where TFoo is declared and call Initialize operator before "class constructors" and before initialize section (TFoo class var instance might be used inside class constructor).

"Maciej Izak" wrote

Hi, Is right to call Initialize operator for Default(TFoo) where TFoo is

record with management operators? IMO the right direction is to keep "meta record instance" in place where TFoo is declared and call Initialize operator before "class constructors" and before initialize section (TFoo class var instance might be used inside class constructor).

There are two things to differentiate:
- Default() used in global variables
- Default() used locally

The latter creates a local variable and basically FillChar()s it while the former creates a zero initialized global variable. (Thinking about this I could use the global variable approach also for local uses of Default(), with the only special case being generics, but that's another topic...)

So in the local case you'd need to add an Initialize() after the FillChar() call (if I haven't done that already).
In the global case you'd need to add the corresponding calls to the implicit initialization function which is called before the unit's initialization section. Currently class constructors are called before that, but that's not good (though the implications are worse with the finalization) and is subject to change. Please don't work around it although right now it would work incorrectly for such variables used in class constructors.

Note: and don't forget the finalization ;)

Regards,
Sven

record with management operators? IMO the right direction is to keep "meta record instance" in place where TFoo is declared and call Initialize operator before "class constructors" and before initialize section (TFoo class var instance might be used inside class constructor).

the former creates a zero initialized global variable.

(Thinking about this I could use the global variable approach also for

local uses of Default(), with the only special case being generics, but that's another topic...)

So in the local case you'd need to add an Initialize() after the

FillChar() call (if I haven't done that already).

In the global case you'd need to add the corresponding calls to the

implicit initialization function which is called before the unit's initialization section. Currently class constructors are called before that, but that's not good (though the implications are worse with the finalization) and is subject to change. Please don't work around it although right now it would work incorrectly for such variables used in class constructors.

Note: and don't forget the finalization ;)

Of course these remarks apply to *any* global variable as well.

Regards,
Sven

Maciej,

I've done a little testing on your smart pointer branch but haven't found any issues recently. Can you tell me if there is a tag for this feature in mantis yet and/or if there are any known issues?

Thanks

"Anthony Walter" wrote

Maciej, I've done a little testing on your smart pointer branch but haven't found

any issues recently. Can you tell me if there is a tag for this feature in mantis yet and/or if there are any known issues?

There is no tag for this, because Maciej is no core developer. Until this feature is part of trunk you should report issues here.

Regards,
Sven

2016-04-10 23:54 GMT+02:00 Sven Barth :

order of class constructor/destructor and variable implicit initialization/finalization (as mentioned in
http://bugs.freepascal.org/view.php?id=29245 ). Anyway I do research in this direction to cover all possible usage of new operators (seems like Default intrinsic is the last one).

Sven, could you look at topic "Management operators AddRef and Copy vs Copy" thread? New operators are ready for test in branch, anyway we can adjust the naming of AddRef and Copy. Florian proposed "Clone" (instead of Copy) and "Copy" (instead of AddRef).

Note: and don't forget the finalization ;)

"Finalize" is mentioned in thread name so it will not be forgotten ;)

2016-04-11 6:55 GMT+02:00 Anthony Walter :

I've done a little testing on your smart pointer branch but haven't found any issues recently. Can you tell me if there is a tag for this feature in mantis yet and/or if there are any known issues?

As Sven stated I am not core developer and reporting here, seems right :).

Could you test new operators AddRef and Copy? Florian Klämpfl does not like at all naming AddRef and Copy, anyway the functional will be the same even with new names (more info in thread "Management operators AddRef and Copy vs Copy").

To see how to use and how works AddRef and Copy you can open new test:

http://svn.freepascal.org/cgi-bin/viewvc.cgi/branches/maciej/smart_pointers/tests/test/toperator96.pp?revision=33478&view=co&pathrev=33478

"Maciej Izak" wrote

Thanks for introduction. Workaround is not a case, eventually patch to

fix order of class constructor/destructor and variable implicit initialization/finalization (as mentioned in
http://bugs.freepascal.org/view.php?id=29245 ). Anyway I do research in this direction to cover all possible usage of new operators (seems like Default intrinsic is the last one).

There is another problem that occurred to.me after my previous mail. Have you ever heard of the "static initialization order fiasco"? I've stumbled upon that a few weeks ago at work with C++ and was glad that we don't have that problem in Pascal, but with the new management operators and the need to have global variables be initialized correctly as well this would change
- which would be very bad as this is a hard to debug problem. I'll provide you with an example when I'm back home again in the evening.

Sven, could you look at topic "Management operators AddRef and Copy vs

Copy" thread? New operators are ready for test in branch, anyway we can adjust the naming of AddRef and Copy. Florian proposed "Clone" (instead of Copy) and "Copy" (instead of AddRef).

I've yet to find a quiet minute to digest that one :P

Regards,
Sven

2016-04-11 14:38 GMT+02:00 Sven Barth :

Even worse, currently is possible to achieve some similar... I've experienced from another side - "static finalization order fiasco" for Generics.Collections (fixed 24 Dec 2015
https://github.com/dathox/generics.collections/commit/fda586932bd80ef58c08f8ebf5a24316ca4ccca5 ).

I can confirm that "static initialization/finalization order fiasco" bugs are really hard to debug. In my case for Generics.Collections was something like this (maybe not the same but very similar):

=== X ===
program X;
uses
  UnitA, ..., UnitZ, GC, Generics.Collections, Unit1;
...
=== GC ===
...
interface
var
  G: Contnrs.TObjectList;
implementation
...
initialization
  G := Contnrs.TObjectList.Create(True);
finalization
  G.Free; // !!!
end.
=== Unit1 ===
... somewhere in code ...
var
  d: TDictionary<string, string>;
begin
  d := TDictionary<string, string>.Create;
  G.Add(d);
=== END CODE ===

at line marked by "!!!" was memory corruption, because any of manual interface used by dictionary was already destroyed in static/class destructor in Generics.Defaults... Really hard to find that kind of bug. What was funny the corruption was visible only on android... Anyway bug is fixed :)

Ok! Thanks. I'm looking forward :P

My example would be this:

=== unit1 begin ===

unit unit1;
interface
function GetSomething: PSomeType;
implementation

var

  gSomething: TSomeType;
  { assume TSomeType is a record that contains an initializer
     method that simply zeros the record and it also has a
     TObject field called "inst" }

function GetSomething: PSomeType;
begin
  if not Assigned(gSomething.inst) then
    gSomething.inst := TObject.Create;
  Result := @gSomething;
end;
{ gSomething will implicitely be initialized here }
end.

=== unit1 end ===

=== unit2 begin ===

unit unit2;
interface
implementation
uses
  unit1;
initialization
  Writeln(GetSomething^.inst.ToString);
end.

=== unit2 end ===

Now depending on the order of the execution of the two initialization sections we'll either have correct behavior (unit1 before unit2) or we'll have a memory leak (unit2 before unit1), because gSomething's initializer in unit1 will fill the instance variable that got assigned during unit2's initialization (the GetSomething() call) with zeros. In a simple program the order can be controlled easily, but in a more complex scenario (think the compiler itself or Lazarus) this might no longer be the case and it's essentially random for the user.

I know this is a rather constructed example, but it's similar to the C++ code we had at work, so it's code that can happen in the real world. If we don't find a way to solve this problem then I'm afraid that I won't include your changes in trunk, cause I don't want to open that can of worms.

Regards,
Sven

2016-04-11 23:36 GMT+02:00 Sven Barth :

Eeeee? I am a little shocked by your arguments because that kind of bug is possible since we have initialization/finalization section and uses section in interface/implementation section. *The programmer must be aware of*. The mentioned bugs are here a looong time. For example fixed by me for Lazarus: http://svn.freepascal.org/cgi-bin/viewvc.cgi?view=revision&root=lazarus&sortby=date&revision=49547

The reason of memory leak in mentioned rev was the same as presented by your example but in context of initialization/finalization. It was raised in Lazarus in random cases (installed packages combination).

Pascal was never safe language because we have ASM, Pointers and other stuff. The pros is much more than cons for management operators (SmartPointer + Generics.Collections is awesome and big step forward). Management operators is powerfully syntax and I don't want to drop them because it is base for my next improvements. No one is forced to using management operators. You are killing the whole tree. It is like atom power. You can create nuclear weapon to kill, but you can use that power to build nuclear power plant.

Seems like my existence here on mailing list and branches has no sense. I will drop my branches here and move my work to github. What is the sense working here if all work is rejected?

My github branch will be often synchronized with core trunk. Anyway good luck and thanks for "maciej" branch! :)

is possible since we have initialization/finalization section and uses section in interface/implementation section. *The programmer must be aware of*. The mentioned bugs are here a looong time. For example fixed by me for Lazarus:
http://svn.freepascal.org/cgi-bin/viewvc.cgi?view=revision&root=lazarus&sortby=date&revision=49547

That might be, but this just adds a new category of possibilities for such bugs and that is something that we should not do, especially one as subtle as this.

One possible solution would be to disallow records with management operators for global variables (variables created for Default() might be an exception here, but I'd need to think that through).

Regards,
Sven

C++

>> code we had at work, so it's code that can happen in the real world. >> If we don't find a way to solve this problem then I'm afraid that I >> won't include your changes in trunk, cause I don't want to open that

can

>> of worms. > > > Eeeee? I am a little shocked by your arguments because that kind of bug

is possible since we have initialization/finalization section and uses section in interface/implementation section. *The programmer must be aware of*. The mentioned bugs are here a looong time. For example fixed by me for Lazarus:
http://svn.freepascal.org/cgi-bin/viewvc.cgi?view=revision&root=lazarus&sortby=date&revision=49547

> That might be, but this just adds a new category of possibilities for

such bugs and that is something that we should not do, especially one as subtle as this.

One possible solution would be to disallow records with management

operators for global variables (variables created for Default() might be an exception here, but I'd need to think that through).

Another possibility I just thought of: have the compiler analyse such inter unit dependencies in the initialization sections (yes, I know, easier said than done) and print a warning in case something might depend on the order of execution.
If the user is at least aware that there might be something fishy going on then I'm more lenient.

Regards,
Sven

2016-04-12 11:21 GMT+02:00 Sven Barth :

another way is to introduce attributes for selected language elements in Delphi compatible way. For example in Delphi we have:

  TCustomAttribute = class(TObject)
  end;
  WeakAttribute = class(TCustomAttribute);
  UnsafeAttribute = class(TCustomAttribute);
  RefAttribute = class(TCustomAttribute);
  VolatileAttribute = class(TCustomAttribute);

which is used like this:

procedure Foo([ref] const X: TFoo); // FPC equivalent is procedure
Foo(constref X: TFoo);

or like this for TObject field for ref. count for ARC:

[Volatile] FRefCount: Integer;

Note: these attributes are on my TODO list.

To protect our management operators we should declare in System.pp:

  FixedAttribute = class(TCustomAttribute); // or SystemInitializedAttribute

Now the gSomething from example is declared as:

var
  [Fixed] gSomething: TSomeType; // fixed can be used only for global
variables and for class var/{$J+} const (aka static var)

... that means "don't' [Initialize|Finalize|FillChar|Whatever] this variable from module but from System.pp (in special table in initialization section)". Any module may contain special hidden section (if [Fixed] attribute is detected); that will register all global variables declared with [Fixed] and any variable with [Fixed] will be excluded from standard initialization/finalization process. That special all "Fixed Sections" from all modules should be called before any other initialize/finalize to register all variables (and before system initialize section ofc, where all registered fixed variables shall be initialized, and after all ofc finalized).

That approach is also usefully to fix existing problems with initialize/finalize (!)

2016-04-12 12:01 GMT+02:00 Maciej Izak :

Of course is still possible to obtain the error, but to achieve this you need to really crave this.

2016-04-12 12:12 GMT+02:00 Maciej Izak :

To to enable the complicated dependencies is possible to allow priority parameter for Fixed attribute:

  FixedAttribute = class(TCustomAttribute)
    constructor Create(aPriority: Integer = 2000); // can be default set to
2000
  end;

... some module A ...

 var
  [Fixed] gFoo: TSomeType; // default priority is 2000
  [Fixed(10)] gSomething: TSomeType; // lower number means higher priority,
for example, first 1000 can be reserved for internal FPC usage

... somewhere in other module B, gSomething2 has Initialize operator, where gSomething is used for something...

var
  [Fixed(100)] gSomething2: TSomeType; // has lower priority and will be
initialized after gSomething and finalized before gSomething

2016-04-12 11:17 GMT+02:00 Sven Barth :

One possible solution would be to disallow records with management operators for global variables (variables created for Default() might be an exception here, but I'd need to think that through).

Maybe for global variables with management operators is good to force usage of [Fixed] / [NonFixed] attribute. NonFixed is not described in my previous messages. It can be used only to force programmer attention (has no real effect, it only pass these kind of variable in compilation process).

"Maciej Izak" wrote

another way is to introduce attributes for selected language elements in

Delphi compatible way. For example in Delphi we have:

What I don't like about such a usage of attributes (aside from the syntax, but that's an orthogonal problem) is that the compiler needs to be made aware of each and every special attribute... (yes, it also needs to support each keyword, but in the case of keywords it doesn't have to be somehow defined in some units as well)
[Note: this is less a "I don't want it", but a "I don't like it"]

Note: these attributes are on my TODO list.

Please don't do anything attribute related until I've merged Joost's class attributes branch which lays the foundations for the attributes. I've planned this for after the reintegration of my dynamic packages branch which is currently WIP (the merge, not the branch ;) )

To protect our management operators we should declare in System.pp: FixedAttribute = class(TCustomAttribute); // or

SystemInitializedAttribute

variables and for class var/{$J+} const (aka static var)

That together with your suggestion to force this attribute for records with management operators might indeed be useful, but there are some problems... (see below)

... that means "don't' [Initialize|Finalize|FillChar|Whatever] this

variable from module but from System.pp (in special table in initialization section)". Any module may contain special hidden section (if [Fixed] attribute is detected); that will register all global variables declared with [Fixed] and any variable with [Fixed] will be excluded from standard initialization/finalization process. That special all "Fixed Sections" from all modules should be called before any other initialize/finalize to register all variables (and before system initialize section ofc, where all registered fixed variables shall be initialized, and after all ofc finalized).

The system unit's initialization *must* be run first, otherwise you won't have a heap or synchronisation primitives or whatever you might need. That then also extends to units like heaptrc, cmem, cthreads, etc.

Maybe it would be better to initialize such records always *after* *all* initialization sections have been run... :/ (and deinitialized first)

Regards,
Sven

I really don't think you should delegate such things to attributes. You make 2 completely unrelated language constructs suddenly related. A bad design decision.

Michael.

2016-04-12 14:33 GMT+02:00 Sven Barth :

IIRC the branch is incomplete (for example RTTI in early stage and for example TValue need to be improved). Extending these branch is also on my TODO. :) Maybe the only target of mentioned branch is to allow attributes of classes (which has no much sense without Invoke method and extended RTTI
- and that is absolutely unimplemented).

Maybe "run" for my idea is a bad word. We don't need to run any section before initialization section from system. What we need is the table/list with "fixed" variables which needs to be initialized in system.pp initialization (and finalized in finalization). I think that is doable.

Attributes do not need Invoke ?

Michael.

2016-04-12 15:06 GMT+02:00 Michael Van Canneyt :

Attributes do not need Invoke ?

Of course they need Invoke. When
TRttiContext.GetType(TMyClassWithAttributes) is executed, deep inside GetType is executed constructor for each attribute by Invoke. IMO mentioned branch is far far from "ready to merge".

Maybe you thinking about attributes just as simple flags/marks without constructors (not parameterized attributes), then you are right.

2016-04-12 14:48 GMT+02:00 Michael Van Canneyt :

I really don't think you should delegate such things to attributes. You make 2 completely unrelated language constructs suddenly related. A bad design decision.

I think you are wrong. Attributes are not only dedicated to RTTI just look at other languages: Delphi, C#, Java. Instead of extending language into infinity you can use simple attribute with parameter. I don't see any reason why don't use attributes to describe some compiler/RTL behavior? Any attribute can be easier placed in many language structures without breaking language syntax. That is much harder with new keywords.

Maybe that is not ideal but works excellent.

Quite strange, Joost claimed the exact opposite :-)

Maybe you thinking about attributes just as simple flags/marks without constructors (not parameterized attributes), then you are right.

Yes. AFAIK this is what Joost had in mind. Maybe he used some trick to parametrize, because AFAIK he needed/used it to implement a ORM architecture; He used the attributes to specify in which DB field a particular property needed to be stored.

But better ask Joost directly.

Michael.

Nono.

You deviate from an important design principle: orthogonality.

By linking 2 concepts, any change in 1 concept risks to influence the other. This is ALWAYS bad. If you implement things orthogonally, a change in 1 concept does not risk influencing the other.

So you will really, really have to provide better arguments than 'but works excellent'.

Michael.

2016-04-12 15:33 GMT+02:00 Michael Van Canneyt :

Quite strange, Joost claimed the exact opposite :-)

Maybe different approach than in Delphi, or I miss something. But as far as I can see, there is something like that:

http://svn.freepascal.org/cgi-bin/viewvc.cgi/branches/joost/classattributes/tests/test/tclassattribute4.pp?view=markup&sortby=date

I don't think Joost planned to support the same low-level API as delphi.

I remember discussions about that, but this is meanwhile some years ago. The best source of information is still Joost.

Michael.

Στις 2016-04-12 16:31, Maciej Izak έγραψε:

Just my 2 ...

AFAIU, attributes in Delphi are ignorable ( ... Attributes do not modify the behavior of types or members by themselves. The consumer code must specifically query for their existence and take appropriate actions when this is required ... ).

IMO attributes that aren't ignorable aren't really attributes in the narrow sense, they're just keywords in disguise.

regards,

"Maciej Izak" wrote

> > > Note: these attributes are on my TODO list. > > Please don't do anything attribute related until I've merged Joost's

class attributes branch which lays the foundations for the attributes. I've planned this for after the reintegration of my dynamic packages branch which is currently WIP (the merge, not the branch ;) )

IIRC the branch is incomplete (for example RTTI in early stage and for

example TValue need to be improved). Extending these branch is also on my TODO. :) Maybe the only target of mentioned branch is to allow attributes of classes (which has no much sense without Invoke method and extended RTTI
- and that is absolutely unimplemented).

Attributes are implemented differentlybin FPC than in Delphi. In FPC a constructor routine is created which calls the attribute class' constructor. This in turn is what's provided in the RTTI (don't know whether the parameters are currently provided as well, but they shouldn't be needed anyway with that approach). This has the added benefit that attributes don't need a working Invoke() implementation on each and every platform (which is an important point for a cross platform compiler like FPC).
Also I'm aware that the branch currently only allows class attributes, but that should be the least problematic part to extend. So just let me reintegrate the code and we can work from there.

> That together with your suggestion to force this attribute for records

with management operators might indeed be useful, but there are some problems... (see below)

> > > ... > > The system unit's initialization *must* be run first, otherwise you

won't have a heap or synchronisation primitives or whatever you might need. That then also extends to units like heaptrc, cmem, cthreads, etc.

> > Maybe it would be better to initialize such records always *after* *all*

initialization sections have been run... :/ (and deinitialized first)

Maybe "run" for my idea is a bad word. We don't need to run any section

before initialization section from system. What we need is the table/list with "fixed" variables which needs to be initialized in system.pp initialization (and finalized in finalization). I think that is doable.

It doesn't matter. You need to run the initializer operators and these might rely on functionality that is not initialized yet. So the safest approach would definitely be to init these after all initialization sections and to document this and the reasons behind it. This also alleviates the need to use attributes for this (or a keyword or whatever).

Regards,
Sven

Am 12.04.2016 16:12 schrieb "Dimitrios Chr. Ioannidis" <

the behavior of types or members by themselves. The consumer code must specifically query for their existence and take appropriate actions when this is required ... ).

IMO attributes that aren't ignorable aren't really attributes in the

narrow sense, they're just keywords in disguise.

I agree, that's why I also don't like the use of attributes for this. The question is how Delphi compatible do we want to be? (I personally wouldn't mind not achieving compatibility for this)

Regards,
Sven

I agree with Maciej here: this kind of stuff can also happen with normal initialization code and uses of units in the implementation part. In this case the programmer must simply be very careful. Though in your example I think unit1 is always initialized first as it does not depend on unit2. The problem might pop up though if you use unit2 in the implementation part of unit1.

In that specific example you would be right, but units are seldomly used in isolation and once both unit1 and unit2 are used in more units it isn't that clear (to the user) anymore which unit is initialized first. As I wrote in another mail a good solution would probably be for these kind of global variables to be initialized after all initialization sections have run and to have that documented like that. This way we avoid adding *another* potential pitfall with unit initializations. Functions that use such global variables could have a flag added which gets added recursively and thus we could warn in the initialization sections if such routines are used nevertheless.
The latter could be also done in case we indeed don't do anything special with these records. This way the user would at least be warned...

Regards,
Sven

Sven Barth wrote

Functions that use such global variables could have a flag added which gets added recursively and thus we could warn in the initialization sections if such routines are used nevertheless.

This is not desirable, because then you need to modify the interface information for such routines after their implementation gets parsed. Even if you do that, it's not possible, because
a) in case of circular dependencies, the implementation of unit1, which uses such a routine from the interface of unit2, may already be compiled by the time you set that flag
b) procedure variables

Jonas

Sven Barth wrote

In that specific example you would be right, but units are seldomly used

in

isolation and once both unit1 and unit2 are used in more units it isn't

that

clear (to the user) anymore which unit is initialized first.

I've always found the rules about this very clear in Delphi (and I assumed so far it applies to FPC as well):

If your unit depends on another unit being initialized before and finalized after your unit, then that other unit must be listed in the interface uses clause.

Units in the implementation uses clause may initialize before or after your unit, and the order may change in the future depending on the contents of other units in the project. But all units in the interface section are guaranteed to have been initialized before your unit and are guaranteed to finalize after your unit.

So if you depend on another unit being initialized first, list it in the interface section. If you can't because of circular references, then your code is broken and what you want to do is impossible.

Delphi ensures this is true, even when dynamically loaded packages come into play, by essentially creating the following code for each unit behind the scenes:

Unit ThisUnit;
Interface

Uses
InterfaceUsedUnit1,

  InterfaceUsedUnit2;

//...
Procedure AddRef;
Procedure Release;
Implementation
Uses
  ImplementationUsedUnit;
Procedure Init;
Begin
  InterfaceUsedUnit1.AddRef;
  InterfaceUsedUnit2.AddRef;
  ImplementationUsedUnit.AddRef;
  //.. code from original initialization section goes here
End;
Procedure Done;
Begin
  //.. code from original finalization section goes here
  ImplementationUsedUnit.Release;
  InterfaceUsedUnit2.Release;
  InterfaceUsedUnit1.Release;
End;
Var
  RefCount: Integer;
Procedure AddRef;
Begin
  If LockedInc(RefCount) = 1 then
    Init;
End;
Procedure Release;
Begin
  If LockedDec(RefCount) = 0 then
    Done;
End;
Initialization
  AddRef;
Finalization
  Release;
End.

information for such routines after their implementation gets parsed. Even if you do that, it's not possible, because

a) in case of circular dependencies, the implementation of unit1, which

uses such a routine from the interface of unit2, may already be compiled by the time you set that flag

b) procedure variables

Right... -.-

Then I go back to the point to initialize those types afterwards. :) Or the suggestion by Thorsten might help...

Regards,
Sven

finalized

after your unit, then that other unit must be listed in the interface uses clause. Units in the implementation uses clause may initialize before or after

your

into

play, by essentially creating the following code for each unit behind the scenes:

I had thought about something like this for dynamic packages as well. Seems like this would be useful for other things as well then.

Regards,
Sven

2016-04-12 17:15 GMT+02:00 Sven Barth :

Running the initializer operators after initialization is IMO bad idea... Very "pascal incompatible" and confusing even for me ;). I see many problems for that approach...

I think I have other solution (that also can help to solve old problems). What is needed is new section like "managementinitialization" and "managementfinalization" (to complement "initialization"/"finalization" with a little different logic). managementinitialization should contains only module related stuff (memory allocation etc.) mostly used/required by Initialize management operators without dependencies nor complex logic. How it supposed to work? Below is attached modified example presented by Thorsten Engler and modified example presented by Sven + my invention ;).

Note: "$" prefix means -> invisible for programmer/created by compiler.

=== unit1 begin ===
unit unit1;
interface

uses
InterfaceUsedUnit1,

  InterfaceUsedUnit2;

//...
Procedure $AddRef;
Procedure $Release;
type
  PSomeType = ^TSomeType;
  TSomeType = record
    ... with management operators
  end;
function GetSomething: PSomeType;
implementation
uses
  ImplementationUsedUnit;
var
  BufferForTSomeTypeManagementOperators: PByte;
procedure $Init;
begin
  InterfaceUsedUnit1.$AddRef;
  InterfaceUsedUnit2.$AddRef;
  ImplementationUsedUnit.$AddRef;
  //.. code from original initialization section goes here
end;
procedure $Finit;
begin
  //.. code from original finalization section goes here
  ImplementationUsedUnit.$Release;
  InterfaceUsedUnit2.$Release;
  InterfaceUsedUnit1.$Release;
end;
var
  $RefCount: Integer;
var
  gSomething: TSomeType;
  gSomething2: array[0..2] of TSomeType;
function GetSomething: PSomeType;
begin
  if not Assigned(gSomething.inst) then
    gSomething.inst := TObject.Create;
  Result := @gSomething;
end;
procedure $ManagementInit;
begin
  $managementinitialization(); // execute our new section for operators
purpose
  // genereted for all global variables with management operators
  InitializeArray(@gSomething, TypeInfo(TSomeType), 1); // call the
Initialize operator
  InitializeArray(@gSomething2[0], TypeInfo(TSomeType), 3);
end;
procedure $ManagementFinit;
begin
  // genereted for all global variables with management operators
  FinalizeArray(@gSomething, TypeInfo(TSomeType), 1); // call the Finalize
operator
  FinalizeArray(@gSomething2[0], TypeInfo(TSomeType), 3);
  $managementfinalization(); // execute our new section for operators
purpose
end;
procedure $AddRef;
begin
  If LockedInc($RefCount) = 1 then
  begin
    $ManagementInit;
    $Init;
  end;
end;
procedure $Release;
begin
  If LockedDec($RefCount) = 0 then
  begin
    $Finit;
    $ManagementFinit;
  end;
end;
managementinitialization // in the fact it is procedure with symbol
$managementinitialization
  BufferForInitializeOperator := GetMem(100);
initialization
  $AddRef;
finalization
  $Release;
managementfinalization // in the fact it is procedure with symbol
$managementfinalization
  FreeMem(BufferForInitializeOperator);
end.
=== unit1 end ===

simplified than unit1, but behind the scene is the same mechanism

=== unit2 begin ===
unit unit2;
interface
implementation
uses
  unit1;
//
initialization
  Writeln(GetSomething^.inst.ToString); // no memory leak anymore even if
inst is filled inside TSomeType.Initialize operator
end.
=== unit2 end ===

2016-04-13 15:24 GMT+02:00 Maciej Izak :

Even more. In Delphi lack of
managementinitialization/managementfinalization is serious problem when package is unloaded. (Is hard to clean up things in right order for complicated dependencies... sometimes is almost impossible to do it right...). Using new sections can fix many issues!

In case of usage through the uses in the interfaces, it is clear. A used unit is initialized before the unit which uses it.

For uses by the implementation section the initialization order is undefined especially in case of
circular dependencies. There is no way round.

Just make a simple rule: custom managed types are initialized directly before or after (to be defined but I prefer before) the initialization part of the unit is run.

2016-04-13 19:50 GMT+02:00 Florian Klämpfl :

Just make a simple rule: custom managed types are initialized directly before (...) initialization

Current implementation (smart_pointers branch) works in that way... custom managed types are initialized directly before initialization

I see no need to change this. And IMO the comparison with C++ is not valid: if the units have no circular dependencies, the initialization order is well defined.

Fine.

Regards,
Sven

Fine.

Regards,
Sven

The code provided by Thorsten is sufficient (and needed for dynamic packages anyway); an additional section is not needed, that just adds unnecessary complexity for users.

Regards,
Sven

I doubt that.

Regards,
Sven

2016-04-13 22:00 GMT+02:00 Sven Barth :

The code provided by Thorsten is sufficient (and needed for dynamic packages anyway); an additional section is not needed, that just adds unnecessary complexity for users.

Yes I agree with that :) you should know why it was proposed:

"If we don't find a way to solve this problem then I'm afraid that I won't include your changes in trunk, cause I don't want to open that can of worms."

2016-04-13 22:01 GMT+02:00 Sven Barth :

I doubt that.

Eeeee? Sparta project was initially game engine created as plugin for Delphi XE2 with a lot of packages and with complicated dependencies system (330 modules + 15 packages). I can say that proposed sections can potentially solve many things (and can complicate other things).

Maciej,

A game engine? What happened to it? I've moved my game engine to OpenGL ES 2.0 for all platforms and am looking to create a FPC Android starter template.

2016-04-14 16:26 GMT+02:00 Anthony Walter :

A game engine? What happened to it? I've moved my game engine to OpenGL ES 2.0 for all platforms and am looking to create a FPC Android starter template.

Yes, but strictly integrated with XE2 (WYSIWYG + events like OnClick but for game development), it was created before Firemonkey idea. The project was developed as "Delphied Unity3D" ;). Project is abandoned. Without Embarcadero support/cooperation was impossible to finish the project.

few memories:

https://www.youtube.com/watch?v=za_JtzO6QGI
https://www.youtube.com/watch?v=Q19K1-Pcafk
http://mort.dathox.com/index.php/pl/media/screenshots http://www.freesparta.com/dathox/dtx_tp_en.pdf

Looks really good. Why did you feel the need to have IDE integration? From my perspective (having written a game toolkit) the only point I needed IDE integration was a project template to add a reference to my game engine package and create a source code file. All the tools which can be used to work with a game engine, such as an asset manager to add/remove textures, models, sound clips, scripts, and animation storyboard are best written as tools external to the IDE. That is you can slice up an image into sprites textures without needing to open the IDE.

Anyhow if your still interested in this stuff I'm using SDL 2 and GL ES 2 which works across all platforms. Mostly I like the idea of using this stuff for 2D games, with 3D capabilities.

https://www.youtube.com/watch?v=lOUB4pqRLjY

"Anthony Walter" wrote

Looks really good. Why did you feel the need to have IDE integration?

He wrote that he intended that as a "Delphied Unity3D" and Unity3D is a game engine together with an IDE. And I personally agree as that makes things much easier approachable for beginners (Unity3D isn't popular for nothing after all).

Regards,
Sven

"Maciej Izak" wrote

2016-04-14 16:26 GMT+02:00 Anthony Walter : > > A game engine? What happened to it? I've moved my game engine to OpenGL

ES 2.0 for all platforms and am looking to create a FPC Android starter template.

Yes, but strictly integrated with XE2 (WYSIWYG + events like OnClick but

for game development), it was created before Firemonkey idea. The project was developed as "Delphied Unity3D" ;). Project is abandoned. Without Embarcadero support/cooperation was impossible to finish the project.

My hope is to do some more IDE integration for the Castle Game Engine as well sometime(TM) in the future. Once we support runtime packages that should indeed be possible :)

Regards,
Sven

I am really looking forward to runtime package support. :)

As far as game engine design, I wanted/prefer a KISS style similar to XNA. That is you just derive a game class and override two methods: Logic() and Render() and each is passed a Stopwatch class representing time. One you fill in with game state (Logic), the other you fill in with drawing stuff (Render). Finally the engine provides a lot of functionality to help you with both such as Sprite classes for both collision detection/movement/storyboarding or a canvas class for vector graphics in a 3D world.

I think the popularity of Unity is due to it providing a single toolkit which can target all devices including render in a webpage canvas using a common core. Sure the tooling helps, but I think it's more along the lines of everything for all platforms works without a whole lot of finangling, rather than having built in visual editors for stuff. I mean if anyoen is at all serious their going to need external programs like Gimp/Photoshop for creating sprite sheets and textures, a 3D modeler like modo for creating models and UV maps, a bone editor like Spine to create animations, something like Audactity for editing sound, and if their like me they'll probably end up writing their own external tools like a level editor:

http://cache.getlazarus.org/images/showcase/windows.mushroom.editor.jpg

Plus a whole lot more external tools. So from my perspective (and maybe a lot of others as well) a game engine IDE should primarily be focused on making compilation and targetting of many platforms transparent. In that regard Lazarus/FPC still needs some work. Setting up the IDE and compiler to get a GL context working on Android/iOS or every transpiling to Javascript for web ... yeah we're not there yet (I am working on distributing the Android part).

"Anthony Walter" wrote

Plus a whole lot more external tools. So from my perspective (and maybe a

lot of others as well) a game engine IDE should primarily be focused on making compilation and targetting of many platforms transparent.

But that's not what Unity3D is about anyway. The thing where they shine is basically a RAD approach to their engine. Easily changeable settings and scripts at runtime and such little tidbits. For some stuff you always need external programs, but not for dealing with the rendering and the game logic. And that's the intriguing part.

In that regard Lazarus/FPC still needs some work. Setting up the IDE and

compiler to get a GL context working on Android/iOS or every transpiling to Javascript for web ... yeah we're not there yet (I am working on distributing the Android part).

Castle Game Engine already has the Android part working... (I don't know about iOS right now though)

Regards,
Sven

2016-04-15 0:19 GMT+02:00 Sven Barth :

My hope is to do some more IDE integration for the Castle Game Engine as well sometime(TM) in the future

!!! that is also my target. To start my work on this I need anonymous methods, Generics.Collections, extended RTTI (with Invoke), and +/- ToolsAPI. Few parts of original Sparta game engine can be used as strong background for integration of castle game engine.

2016-04-15 1:36 GMT+02:00 Anthony Walter :

I'm too lazy to do things in that way. I need object inspector with properties/events tabs + WYSIWYG. ;)

That is also the reason of introducing smart pointers branch. I'd like to omit any Free/FreeMems.

2016-04-14 23:16 GMT+02:00 Anthony Walter :

I'd like to integrate somehow your easing system. Anyway I'am opened to any cooperation in that matter. The work not started yet. Did you think that Design tab for Lazarus/FreeSparta was developed for LCL stuff? Noooo! That is for future gamedevelopment and integrated tools!

The biggest problem I ran into has to do with asset lifetime management. That is, allowing for ad-hoc creation or loading and unloading of (list to follow):

Textures
Models
Storyboards
Skeletons
Scripts
Audio Samples
Fonts
Skyboxes
Shaders/Shader programs

... and anything else which a game developer might want to create/destroy, or load/unload.

To cope with this generics can be quite handy. A master asset manager class can hold these types of things for you so that you don't need to declare instances of each type in a class. The asset manager can also hold copies for you so that you don't need to recreate then when they are needed. Some examples.

// Change the texture of a sprite to a ghost
Sprite.Texture := Assets.Textures[SGhostTexture];
Sprite.Shader := Assets.ShaderPrograms[SEtherealShader];
// Add a scary sound sample to an audio bank and loop it
AudioBank := Audio.Banks.Add(Assets.Sounds[SScaryMusic]);
AudioBank.Looping := True;
AudioBank.Play;
// Load up a slimy looking font
FontWriter.Font  := Assets.Fonts[SSlimeFont];

Where Assets is a global TAssetManager containing all possible assets

type
  TAssetManager = class(TManaged)
  public
    property Sprites: TManagedCollection<TSprite> read GetSprites;
    property Textures: TManagedCollection<TTexture> read GetTextures;
    property Sounds: TManagedCollection<TAudioSamples> read GetSounds;
    // and so on
  end;

The problem then comes in where you have certain sprites/sounds allocated at different times, for example changing/advancing game levels. One solution is just to load every possible asset before the game starts, but this is kind of wasteful and also possibly not feasible due to hardware resource constraints. Perhaps you have a game selections screen. There is no reason the assets specific to that screen should be taking up audio banks and texture memory after you start the game. Okay then you need a clean way to group all the assets for different stages together such that they can all be loaded and unloaded en mass when preparing to enter or exit a scene. Then things can get a little messy from there.

Now this where managed types can help. If the asset manager can hold a reference count to asset items (via a managed collection class), and a class can also hold the same references directly, as in not inside a container but declared directly, then grouping assets can be simplified a bit. Perhaps a TGameSelectScene can hold direct copies of the assets it needs and use Assets (TAssetManager) to request them, The scene can then hold references to these assets and remove them from the asset manager, but when the scene is released so are its very specific assets.

I really haven't come up with an acceptable solution to this problem yet. I also need better integration with a compression library to store assets before they are uncompressed and loaded from some kind of WAD file (like how Doom used to handle this stuff). For now assets are compressed and linked directly into the exe/application, but I feel they should be in separate compressed files. Perhaps grouped by scene (TGameSelectScene would have it's own WAD).

https://en.wikipedia.org/wiki/Doom_WAD

Anyhow the point is to make managing game of assets as transparent as possible to the game programmer, where he can request textures, shaders, fonts, 3D models, or whatever, without the burden of remembering when to create and destroy these things.

to me.
Their own IDE. their own scripting, fulling working resource management. Coveras all modern gaming platforms, including web and game consoles.

thanks,
Dmitry