Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2012-01

potential bug, generics vs helpers

15 messages · 4 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Kornel Kisielewicz

Hello all,

This my first mail to the list, and while this message might fit fpc-pascal, I wasn't sure about it, due to the fact that it has questions only a dev can answer, and possibly hints some bugs. Using FPC 2.6.0 / Win32 on Windows 7/64.

So first of all, there's either a mistake in the manual, or FPC bug:

( following Ref 8.3 Generic class specialization )

{$MODE OBJFPC}
program generic_bug1;
type generic TList<T> = class  x : T; end;
// Note that 2 specializations of a generic type with
// the same types in a placeholder are not assignment
// compatible. In the following example:
type
  TA = specialize TList<Pointer>;
  TB = specialize TList<Pointer>;
// variables of types TA and TB cannot be assigned to each
// other, i.e the following assignment will be invalid:
Var
  A : TA;
  B : TB;
begin
  A:=B;
end.

Well it turns out it's not compiles without
warnings, and if added surrounding sensible code, works all well. Seems as the compiler treats TA and TB as the same types. However if we define TA and TB in a different units it doesn't work anymore -- which isn't surprising. So either the manual should be updated, or the "reusage" of the generic should be removed.

While this may seem very cosmetic, I found a case where it really makes a difference:

{$MODE OBJFPC}
{$MODESWITCH ADVANCEDRECORDS}
program generic_bug2;
type
generic TList<T> = record
  x : T;
end;
TA = specialize TList<Single>;
TB = specialize TList<Single>;
type TAHelper = record helper for TA
  procedure Write;
end;
procedure TAHelper.Write;
begin
  Write('A');
end;
type TBHelper = record helper for TB
  procedure Write;
end;
procedure TBHelper.Write;
begin
  Write('B');
end;
var  A : TA;
    B : TB;
begin
  A.Write;
  B.Write;
end.

So the above program outputs "AA" -- because type TA and TB are the same. However, if we split out the generic and TA into another unit:

// Unit
{$MODE OBJFPC}
{$MODESWITCH ADVANCEDRECORDS}
unit generic_bug3_unit;
interface
type generic TList<T> = record
  x : T;
end;
type TA = specialize TList<Single>;
type TAHelper = record helper for TA
  procedure Write;
end;
implementation
procedure TAHelper.Write;
begin
  Writeln('A');
end;
end.
// Program
{$MODE OBJFPC}
{$MODESWITCH ADVANCEDRECORDS}
program generic_bug3;
uses generic_bug3_unit;
type
  TB = specialize TList<Single>;
type TBHelper = record helper for TB
  procedure Write;
end;
procedure TBHelper.Write;
begin
  Writeln('B');
end;
var
  A : TA;
  B : TB;
begin
  A.Write;
  B.Write;
end.

... then TA and TB are treated as separate types, hence each has it's own helper -- program outputs "AB".

Now there are several questions I have in regards to this (way to long) introduction:

1) In case of the first program and the A := B assignment - is the manual wrong (and it will stay working this way) or the compiler should generate separate instances?

2) In the second program, is it OK that the compiler silently discards the first Helper class, or maybe should it produce an error? Note that this also works for non-generic records (what does Delphi do?)

3) (most important for me) this all was discovered because I wanted to find a way to have two different "flavors" of a base record, extended by different helpers (eg. Vector4D base class extended to a Quaternion, and a Color) -- using generics provided a easy way to make a base record "evolve" into two distinct types, so in one unit I have TGLColor4 = specialize TVector4< Single >; with a TGGColor4Helper and in another I have TQuaternion = specialize TVector4<Single> with a TQuaternionHelper). I really like this solution, but depending on the way the above may change, I don't know if it won't be "fixed" not to work anymore in a later version of FPC?

regards,

regards, Kornel Kisielewicz http://chaosforge.org/

Reply by email

Sven Barth
to Kornel Kisielewicz
Kornel Kisielewicz wrote 6 lines
Hello all, This my first mail to the list, and while this message might fit fpc-pascal, I wasn't sure about it, due to the fact that it has questions only a dev can answer, and possibly hints some bugs. Using FPC 2.6.0 / Win32 on Windows 7/64.

fpc-pascal would have been ok as well (maybe even better, because it's about usage of FPC...). Nevertheless the devs are normally subscribed to at least fpc-pascal and fpc-devel.

Show quoted text 33 lines
So first of all, there's either a mistake in the manual, or FPC bug: ( following Ref 8.3 Generic class specialization ) {$MODE OBJFPC} program generic_bug1; type generic TList<T> = class x : T; end; // Note that 2 specializations of a generic type with // the same types in a placeholder are not assignment // compatible. In the following example: type TA = specialize TList<Pointer>; TB = specialize TList<Pointer>; // variables of types TA and TB cannot be assigned to each // other, i.e the following assignment will be invalid: Var A : TA; B : TB; begin A:=B; end. Well it turns out it's not compiles without warnings, and if added surrounding sensible code, works all well. Seems as the compiler treats TA and TB as the same types. However if we define TA and TB in a different units it doesn't work anymore -- which isn't surprising. So either the manual should be updated, or the "reusage" of the generic should be removed.

In my opinion the documentation should be updated, because in Delphi (and FPC's mode Delphi in trunk) you use code like the following:

procedure Foo;
var
   A: TList<Pointer>;
   B: TList<Pointer>;
begin
   A := TList<Pointer>;
   B := A;
end;

So it would not make sense for A and B to be of different types as you can also write

var
   A, B: TList<Pointer>;

One might argue that

type
   TA = specialize TList<Pointer>;
   TB = specialize TList<Pointer>;

is something different, but I say it is not.

Show quoted text 97 lines
While this may seem very cosmetic, I found a case where it really makes a difference: {$MODE OBJFPC} {$MODESWITCH ADVANCEDRECORDS} program generic_bug2; type generic TList<T> = record x : T; end; TA = specialize TList<Single>; TB = specialize TList<Single>; type TAHelper = record helper for TA procedure Write; end; procedure TAHelper.Write; begin Write('A'); end; type TBHelper = record helper for TB procedure Write; end; procedure TBHelper.Write; begin Write('B'); end; var A : TA; B : TB; begin A.Write; B.Write; end. So the above program outputs "AA" -- because type TA and TB are the same. However, if we split out the generic and TA into another unit: // Unit {$MODE OBJFPC} {$MODESWITCH ADVANCEDRECORDS} unit generic_bug3_unit; interface type generic TList<T> = record x : T; end; type TA = specialize TList<Single>; type TAHelper = record helper for TA procedure Write; end; implementation procedure TAHelper.Write; begin Writeln('A'); end; end. // Program {$MODE OBJFPC} {$MODESWITCH ADVANCEDRECORDS} program generic_bug3; uses generic_bug3_unit; type TB = specialize TList<Single>; type TBHelper = record helper for TB procedure Write; end; procedure TBHelper.Write; begin Writeln('B'); end; var A : TA; B : TB; begin A.Write; B.Write; end. ... then TA and TB are treated as separate types, hence each has it's own helper -- program outputs "AB".

This is interesting. In my opinion (under the aspect that TA = TB currently) it should print "BB" in both cases. I'll need to investigate this.

Also the case that both spezializations in the same unit are treated the same, but aren't in the other one. Also something I'll need to investigate (also in FPC 2.7.1).

2) In the second program, is it OK that the compiler silently discards the first Helper class, or maybe should it produce an error? Note that this also works for non-generic records (what does Delphi do?)

As I said above: given that TA is considered equal to TB it should print the same letter in both cases.

Show quoted text 11 lines
3) (most important for me) this all was discovered because I wanted to find a way to have two different "flavors" of a base record, extended by different helpers (eg. Vector4D base class extended to a Quaternion, and a Color) -- using generics provided a easy way to make a base record "evolve" into two distinct types, so in one unit I have TGLColor4 = specialize TVector4< Single>; with a TGGColor4Helper and in another I have TQuaternion = specialize TVector4<Single> with a TQuaternionHelper). I really like this solution, but depending on the way the above may change, I don't know if it won't be "fixed" not to work anymore in a later version of FPC?

Currently they will sometimes be the same and sometimes not. This would be ok if we wouldn't have helpers, but as we have helpers now as well a solution for this needs to be found.

Regards,
Sven

Reply by email

Kornel Kisielewicz
to Sven Barth
Sven Barth wrote 16 lines
Am 27.01.2012 16:23, schrieb Kornel Kisielewicz: > we define TA and TB in a different units it doesn't work anymore -- > which isn't surprising. So either the manual should be updated, or the > "reusage" of the generic should be removed. In my opinion the documentation should be updated, because in Delphi (and FPC's mode Delphi in trunk) you use code like the following: procedure Foo; var  A: TList<Pointer>;  B: TList<Pointer>; begin  A := TList<Pointer>;  B := A; end;

Does it also work (in Delphi and in FPC Delphi mode) if A and B would be defined in different units?

If I understand generics correctly, this is not so simple -- in which object file does TList<Pointer> if it is used in two different units? It shouldn't go to the object file of the unit with TList, because it might not be used in a different compilation. It should also not go into either unit because that would make them depend on each other...

Show quoted text 10 lines
> ... then TA and TB are treated as separate types, hence each has it's > own helper -- program outputs "AB". > This is interesting. In my opinion (under the aspect that TA = TB currently) it should print "BB" in both cases. I'll need to investigate this. Also the case that both spezializations in the same unit are treated the same, but aren't in the other one. Also something I'll need to investigate (also in FPC 2.7.1).

Well but this would introduce the need to either duplicate the generated code for each unit, or keeping in the TList unit object, or do I understand the compilation process incorrectly?

Show quoted text 6 lines
> 2) In the second program, is it OK that the compiler silently discards > the first Helper class, or maybe should it produce an error? Note that > this also works for non-generic records (what does Delphi do?) As I said above: given that TA is considered equal to TB it should print the same letter in both cases.

So multiple helpers for a single class are allowed? And the order in which they appear defines which overloaded names are used?

Show quoted text 15 lines
> 3) (most important for me) this all was discovered because I wanted to > find a way to have two different "flavors" of a base record, extended > by different helpers (eg. Vector4D base class extended to a > Quaternion, and a Color) -- using generics provided a easy way to make > a base record "evolve" into two distinct types, so in one unit I have > TGLColor4 = specialize TVector4<  Single>; with a TGGColor4Helper and > in another I have TQuaternion = specialize TVector4<Single>  with a > TQuaternionHelper). I really like this solution, but depending on the > way the above may change, I don't know if it won't be "fixed" not to > work anymore in a later version of FPC? Currently they will sometimes be the same and sometimes not. This would be ok if we wouldn't have helpers, but as we have helpers now as well a solution for this needs to be found.

In this case, would there be another possibility to differentiate base objects for helpers? As in manually force the compiler to treat types differently? This would be extremely useful in a case like mine. I tried "type TBRecord = type TARecord;" but that also didn't work.

regards, Kornel Kisielewicz

Reply by email

Marco van de Voort
to Kornel Kisielewicz

In our previous episode, Kornel Kisielewicz said:

Show quoted text 5 lines
> ?B := A; > end; Does it also work (in Delphi and in FPC Delphi mode) if A and B would be defined in different units?

This compiles in Delphi:

{$apptype console}
Type
    TLIST<T>=class
     end;
type
    Ta= TLIST<POINTER>;
    TB= TLIST<POINTER>;
Type
    TC=TLIST<POINTER>;
var
   A: TList<Pointer>;
   B: TList<Pointer>;
   aa : TA;
   bb : TB;
   cc : TC;
begin
   B := A;
   aa:=a;
   bb:=aa;
   cc:=aa;
end.

I tried to use "x = type a", syntax but that is not allowed for generics.

Reply by email

Thaddy
to Marco van de Voort
Marco van de Voort wrote 10 lines
In our previous episode, Kornel Kisielewicz said: > {$apptype console} > > Type > TLIST<T>=class > end; > > type > Ta= TLIST<POINTER>; > TB= TLIST<POINTER>;

This may very well be a (semantics) bug in Delphi. Did you put this to Embarcadero?
IMO a type Ta<..> should never be assignable to a type Tb<..> and reverse when using generics. (unless with operator overloading or similar defined)
I think this may be an unintentional "feature" in Delphi.

Reply by email

Sven Barth
to Kornel Kisielewicz: Does it also work (in Delphi and in FPC Delphi mode) if A and B would be defined in…
Kornel Kisielewicz wrote 20 lines
On Fri, Jan 27, 2012 at 4:54 PM, Sven Barth wrote: > Am 27.01.2012 16:23, schrieb Kornel Kisielewicz: >> we define TA and TB in a different units it doesn't work anymore -- >> which isn't surprising. So either the manual should be updated, or the >> "reusage" of the generic should be removed. > > In my opinion the documentation should be updated, because in Delphi (and > FPC's mode Delphi in trunk) you use code like the following: > > procedure Foo; > var > A: TList<Pointer>; > B: TList<Pointer>; > begin > A := TList<Pointer>; > B := A; > end; Does it also work (in Delphi and in FPC Delphi mode) if A and B would be defined in different units?

I'll need to try that in Delphi (which is the reference implementation after all for this).

Show quoted text 7 lines
If I understand generics correctly, this is not so simple -- in which object file does TList<Pointer> if it is used in two different units? It shouldn't go to the object file of the unit with TList, because it might not be used in a different compilation. It should also not go into either unit because that would make them depend on each other...

I'll go into details a bit for your question below.

Show quoted text 15 lines
>> ... then TA and TB are treated as separate types, hence each has it's >> own helper -- program outputs "AB". >> > > This is interesting. In my opinion (under the aspect that TA = TB currently) > it should print "BB" in both cases. I'll need to investigate this. > > Also the case that both spezializations in the same unit are treated the > same, but aren't in the other one. Also something I'll need to investigate > (also in FPC 2.7.1). Well but this would introduce the need to either duplicate the generated code for each unit, or keeping in the TList unit object, or do I understand the compilation process incorrectly?

First there is the following differentiation in the compiler: you have symbols and definitions. For example

type
   TFoo = class
   end;

creates a definition TFoo to which a symbol TFoo points to. If you now do a

type
   TBar = TFoo

then you'll have a symbol TBar which also points to the definition TFoo.

For generics you'll need to know this: while parsing them a definition and a symbol (more correctly: often two) are created. E.g.

type
   generic TGeneric<T> = class
   end;

will generate a definition TGeneric$1 and the symbols TGeneric (which will point to TGeneric$1 in non-Delphi modes and to a special "unassigned definition" in mode Delphi) and TGeneric$1 which will point to TGeneric$1 (the logic behind this is related to the fact that Delphi allows you to overload generics; thus you can easily define TGeneric<T> and TGeneric<T, S> and also a non generic TGeneric in one single unit - this only works in mode Delphi in FPC's trunk). But no code will be generated for the generic only the "tokens" will be saved into the compiled unit (the PPU).

If you now specialize a generic the saved tokenstream will be loaded and passed to the parser which simply "reparses" the tokenstream with the only difference that the definition will now be e.g. for TGeneric<LongInt> TGeneric$1$LongInt and the symbol will be named whatever you called the type name.

E.g.

type
   TLongIntGeneric = specialize TGeneric<LongInt>;

Here the symbol will be named TLongIntGeneric and point to TGeneric$1$LongInt.

Another example (mode Delphi):

var
   longintgeneric: TGeneric<LongInt>;

Here no typesymbol will be created, but a variable symbol called longintgeneric of which the definiton will point to TGeneric$1$LongInt.

Now the point is that the compiler tries to optimize the code generation a bit and thus tries to generate the definition as few times as possible. If you thus put both examples in the same unit e.g. in mode Delphi like the following:

type
   TLongIntGeneric = TGeneric<LongInt>;
var
   longintgeneric: TGeneric<LongInt>;

then the definiton TGeneric$1$LongInt will only be generated once (otherwise you'd get symbol duplication errors, because of members of TGeneric).

The specialization process does not look in other units though. So if you have already specialized a generic in another unit with a certain type than it will (currently) be different from the one in the current unit (if it is specialized with the same type).

So currently the situation is like follows:
* specializations of generics with a type are considered the same if they happen in the same unit
* specializations of generics with a type are considered different if they happen in different units.
Note: If Delphi does not handle point 2 the same way then it's very likely that this will be changed in the future

Show quoted text 9 lines
>> 2) In the second program, is it OK that the compiler silently discards >> the first Helper class, or maybe should it produce an error? Note that >> this also works for non-generic records (what does Delphi do?) > > As I said above: given that TA is considered equal to TB it should print the > same letter in both cases. So multiple helpers for a single class are allowed? And the order in which they appear defines which overloaded names are used?

You can write multiple helpers for a class, but only one helper will be completely active which is defined by the last helper that has been added to the scope (though I still need to check whether it was correct that in your example "A" was printed).

E.g.:

unit A defines:
MyHelperA = class helper for TObject
   procedure Foo;
   procedure Bar;
end;
unit B defines:
MyHelperB = class helper for TObject
   procedure Foo;
   procedure Blubb;
end;

main program:

uses
   MyHelperA, MyHelperB;
var
   o: TObject;
begin
   o.Foo; // => MyHelperB.Foo
   o.Bar; // => Error, identifer not found
   o.Blubb; // => MyHelperB.Blubb
end.

main program 2:

uses
   MyHelperB, MyHelperA;
var
   o: TObject;
begin
   o.Foo; // => MyHelperA.Foo
   o.Bar; // => MyHelperA.Bar
   o.Blubb; // => Error, identifer not found
end.
Show quoted text 20 lines
>> 3) (most important for me) this all was discovered because I wanted to >> find a way to have two different "flavors" of a base record, extended >> by different helpers (eg. Vector4D base class extended to a >> Quaternion, and a Color) -- using generics provided a easy way to make >> a base record "evolve" into two distinct types, so in one unit I have >> TGLColor4 = specialize TVector4< Single>; with a TGGColor4Helper and >> in another I have TQuaternion = specialize TVector4<Single> with a >> TQuaternionHelper). I really like this solution, but depending on the >> way the above may change, I don't know if it won't be "fixed" not to >> work anymore in a later version of FPC? > > > Currently they will sometimes be the same and sometimes not. This would be > ok if we wouldn't have helpers, but as we have helpers now as well a > solution for this needs to be found. In this case, would there be another possibility to differentiate base objects for helpers? As in manually force the compiler to treat types differently? This would be extremely useful in a case like mine. I tried "type TBRecord = type TARecord;" but that also didn't work.

I don't know a solution for this currently... sorry.

As a sidenote to my lengthy explanation above: helpers are differentiated based on definitions not symbols (which is Delphi compatible)

Regards,
Sven

Reply by email

Sven Barth
to Marco van de Voort: In our previous episode, Kornel Kisielewicz said: This compiles in Delphi: {$apptype…
Marco van de Voort wrote 35 lines
In our previous episode, Kornel Kisielewicz said: >> ?B := A; >> end; > > Does it also work (in Delphi and in FPC Delphi mode) if A and B would > be defined in different units? This compiles in Delphi: {$apptype console} Type TLIST<T>=class end; type Ta= TLIST<POINTER>; TB= TLIST<POINTER>; Type TC=TLIST<POINTER>; var A: TList<Pointer>; B: TList<Pointer>; aa : TA; bb : TB; cc : TC; begin B := A; aa:=a; bb:=aa; cc:=aa; end.

This should compile also in FPC (in mode Delphi). The question is about different units though (I will test that myself on my ride home in a few minutes, no worries :) )

Regards,
Sven

Reply by email

Sven Barth
to Thaddy: This may very well be a (semantics) bug in Delphi. Did you put this to Embarcadero? IMO a…
Thaddy wrote 17 lines
On 27-1-2012 17:25, Marco van de Voort wrote: > In our previous episode, Kornel Kisielewicz said: >> {$apptype console} >> >> Type >> TLIST<T>=class >> end; >> >> type >> Ta= TLIST<POINTER>; >> TB= TLIST<POINTER>; This may very well be a (semantics) bug in Delphi. Did you put this to Embarcadero? IMO a type Ta<..> should never be assignable to a type Tb<..> and reverse when using generics. (unless with operator overloading or similar defined) I think this may be an unintentional "feature" in Delphi.

I believe this is intentional as generics are then specialized as (using FPC's notation): TList$1$Pointer. There can not be another TList$1$Pointer in the current unit, so they must be assignment compatible if the Delphi developers didn't purposedly introduce checks for the type symbol they belong to (and at least in FPC it can be sometimes a bit hard to find the correct symbol one needs).

Regards,
Sven

Reply by email

Sven Barth
to Thaddy: This may very well be a (semantics) bug in Delphi. Did you put this to Embarcadero? IMO a…
Thaddy wrote 17 lines
On 27-1-2012 17:25, Marco van de Voort wrote: > In our previous episode, Kornel Kisielewicz said: >> {$apptype console} >> >> Type >> TLIST<T>=class >> end; >> >> type >> Ta= TLIST<POINTER>; >> TB= TLIST<POINTER>; This may very well be a (semantics) bug in Delphi. Did you put this to Embarcadero? IMO a type Ta<..> should never be assignable to a type Tb<..> and reverse when using generics. (unless with operator overloading or similar defined) I think this may be an unintentional "feature" in Delphi.

I now tested a "generic diamond" in Delphi XE.

Following setup:

===

unit ugendiamonddecl;
interface
type
   TGeneric<T> = class
     f: T;
   end;
implementation
end.

===

unit ugendiamond1;
interface
uses
   ugendiamonddecl;
type
   TSpez1 = TGeneric<LongInt>;
implementation
end.

===

unit ugendiamond2;
interface
uses
   ugendiamonddecl;
type
   TSpez2 = TGeneric<LongInt>;
implementation
end.

===

program tgendiamond;
{$APPTYPE CONSOLE}
uses
//  ugendiamonddecl in 'ugendiamonddecl.pas',
   ugendiamond1 in 'ugendiamond1.pas',
   ugendiamond2 in 'ugendiamond2.pas';
var
   s1: TSpez1;
   s2: TSpez2;
begin
   s1 := s2;
end.

===

This compiles!

So there MUST be specific code in place in the Delphi compiler to allow the assignment from ugendiamond2.TSpez2 to ugendiamond1.TSpez1 as those are different definitions. Thus this can not be a bug, but must be an intentional feature. Allowing this in FPC as well would be doable.

Now regarding Kornel's problem: we could(!) change the way the compiler works in non-Delphi modes to what is described in the documentation (by adding the type symbol's name to the generic's name as well), but this will mean that we would have to define how units in Delphi and non-Delphi modes can interact here. After that the corresponding code in the compiler can be implemented to ensure this.

Regards,
Sven

Reply by email

Kornel Kisielewicz
to Sven Barth
Sven Barth wrote
Am 27.01.2012 17:46, schrieb Thaddy: > On 27-1-2012 17:25, Marco van de Voort wrote: >> In our previous episode, Kornel Kisielewicz said:

(snip)

Show quoted text 15 lines
=== This compiles! So there MUST be specific code in place in the Delphi compiler to allow the assignment from ugendiamond2.TSpez2 to ugendiamond1.TSpez1 as those are different definitions. Thus this can not be a bug, but must be an intentional feature. Allowing this in FPC as well would be doable. Now regarding Kornel's problem: we could(!) change the way the compiler works in non-Delphi modes to what is described in the documentation (by adding the type symbol's name to the generic's name as well), but this will mean that we would have to define how units in Delphi and non-Delphi modes can interact here. After that the corresponding code in the compiler can be implemented to ensure this.

What I still don't understand is the way the code is compiled in the diamond case. Let's assume that TGeneric in this example has a lot of code. So after compilation we have:

ugendiamonddecl.ppu
ugendiamond1.ppu
ugendiamond2.ppu
ugendiamonddecl.o
ugendiamond1.o
ugendiamond2.o

As you stated, the ugendiamonddecl.ppu holds the tokens for generation of TGeneric, so the code is "kept" in ugendiamonddecl.ppu. What about the object files? The only reasonable general solution I see is that both ugendiamond1.o and ugendiamond2.o hold the generated code -- Delphi may do it otherwise because it may use different intermediate file formats (true?)

In this case FPC's specialize could be the mode to be "smarter" requiring a explicit instantiation that at the same time shows in which object should the code be stored.

It's kind of like the problem with the instantiation and template export in C++...

Also, I'd be a big fan of extending the FPC version of templates in the direction allowing more powerful generic metaprogramming (without the broken C++ heritage), and I think that explicit instantiation would simplify that quite a lot.

regards, Kornel Kisielewicz

Reply by email

Kornel Kisielewicz
to Sven Barth: I now tested a "generic diamond" in Delphi XE. Following setup: === unit ugendiamonddecl;…
Sven Barth wrote 6 lines
Now regarding Kornel's problem: we could(!) change the way the compiler works in non-Delphi modes to what is described in the documentation (by adding the type symbol's name to the generic's name as well), but this will mean that we would have to define how units in Delphi and non-Delphi modes can interact here. After that the corresponding code in the compiler can be implemented to ensure this.

Okay, after some thoughts about it, changing the implementation to the way the documentation is written would also be bad, because sometimes we need to access a generic from another generic, eg:

generic TGMatrix4<T> = record
type TVector4 = specialize TGVector4<T>;
...
end;

Obviously we want that TGMatrix4 to be compatible with any compatible TGVector2 defined elsewhere... Same problem would be in the case of implementation of some data structures.

regards, Kornel Kisielewicz

Reply by email

Sven Barth
to Kornel Kisielewicz: (snip) What I still don't understand is the way the code is compiled in the diamond case.…
Kornel Kisielewicz wrote 38 lines
On Fri, Jan 27, 2012 at 6:37 PM, Sven Barth wrote: > Am 27.01.2012 17:46, schrieb Thaddy: >> On 27-1-2012 17:25, Marco van de Voort wrote: >>> In our previous episode, Kornel Kisielewicz said: (snip) > === > > This compiles! > > So there MUST be specific code in place in the Delphi compiler to allow the > assignment from ugendiamond2.TSpez2 to ugendiamond1.TSpez1 as those are > different definitions. Thus this can not be a bug, but must be an > intentional feature. Allowing this in FPC as well would be doable. > > Now regarding Kornel's problem: we could(!) change the way the compiler > works in non-Delphi modes to what is described in the documentation (by > adding the type symbol's name to the generic's name as well), but this will > mean that we would have to define how units in Delphi and non-Delphi modes > can interact here. After that the corresponding code in the compiler can be > implemented to ensure this. What I still don't understand is the way the code is compiled in the diamond case. Let's assume that TGeneric in this example has a lot of code. So after compilation we have: ugendiamonddecl.ppu ugendiamond1.ppu ugendiamond2.ppu ugendiamonddecl.o ugendiamond1.o ugendiamond2.o As you stated, the ugendiamonddecl.ppu holds the tokens for generation of TGeneric, so the code is "kept" in ugendiamonddecl.ppu. What about the object files? The only reasonable general solution I see is that both ugendiamond1.o and ugendiamond2.o hold the generated code -- Delphi may do it otherwise because it may use different intermediate file formats (true?)

The ugendiamonddecl.ppu does not contain any code of the TGeneric<T> class while both ugendiamond1.o and ugendiamond2.o contain both their own version of the specialization. And there is no other way to solve it without much hazzle, because if ugendiamond1 and ugendiamond2 don't depend on each other then there is nothing that stops you from using only ugendiamond1 or only ugendiamond2 and then the generated code needs to be available. Delphi uses the same way.

Show quoted text 5 lines
In this case FPC's specialize could be the mode to be "smarter" requiring a explicit instantiation that at the same time shows in which object should the code be stored.

No. A definition should be stored where it is defined. I can't just use ugendiamond1 for storage, because then ugendiamond2 will depend on ugendiamond1 which might not be the intention. Also what about units that were definitely compiled by different people, but by "accident" have the same specializations? You can't store them in the ugendiamonddecl.ppu/.o either, because that file must be considered fix.

It's already good like it is. We just need to decide whether we want to support assignments like Delphi does or not. Then we can fix the compiler, because currently it's only working correctly in half of the matter.

Also, I'd be a big fan of extending the FPC version of templates in the direction allowing more powerful generic metaprogramming (without the broken C++ heritage), and I think that explicit instantiation would simplify that quite a lot.

The only thing I would add to FPC generics that's a bit in the direction of C++ templates (besides constraints which are also a Delphi feature and thus need to be supported sooner or later) would be the ability to "specialize" a generic with constants (and that only if I have too much time and no other feature to attend to). But that's it! No more C++ template ways. I don't want a turing complete compile time construct in FPC!

Regards,
Sven

Reply by email

Sven Barth
to Kornel Kisielewicz: Okay, after some thoughts about it, changing the implementation to the way the…
Kornel Kisielewicz wrote 20 lines
On Fri, Jan 27, 2012 at 6:37 PM, Sven Barth wrote: > Now regarding Kornel's problem: we could(!) change the way the compiler > works in non-Delphi modes to what is described in the documentation (by > adding the type symbol's name to the generic's name as well), but this will > mean that we would have to define how units in Delphi and non-Delphi modes > can interact here. After that the corresponding code in the compiler can be > implemented to ensure this. Okay, after some thoughts about it, changing the implementation to the way the documentation is written would also be bad, because sometimes we need to access a generic from another generic, eg: generic TGMatrix4<T> = record type TVector4 = specialize TGVector4<T>; ... end; Obviously we want that TGMatrix4 to be compatible with any compatible TGVector2 defined elsewhere... Same problem would be in the case of implementation of some data structures.

Language design is not as easy as one might think ;) There are always corner cases lurking around the corner (pun intended) that you didn't thought about before.

But don't worry. I sometimes have the problem as well that I don't see the problems with my suggestions.

Regards,
Sven

Reply by email

Kornel Kisielewicz
to Sven Barth: The ugendiamonddecl.ppu does not contain any code of the TGeneric<T> class while both…
Sven Barth wrote 11 lines
On 27.01.2012 19:05, Kornel Kisielewicz wrote: > In this case FPC's specialize could be the mode to be "smarter" > requiring a explicit instantiation that at the same time shows in > which object should the code be stored. No. A definition should be stored where it is defined. I can't just use ugendiamond1 for storage, because then ugendiamond2 will depend on ugendiamond1 which might not be the intention. Also what about units that were definitely compiled by different people, but by "accident" have the same specializations? You can't store them in the ugendiamonddecl.ppu/.o either, because that file must be considered fix.

What I meant was a solution (only for FPC mode) that only allowed compatibility between specializations.

So if ugen1 and ugen2 would both have specialization TGeneric then the situation would be as now (two instances), only no assignment would be possible. To have a possible assignment you'd have to have them both include a common unit that has this particual specialization. Basically restricting template assignment only to the same instantiation as stated in the manual, which would allow to choose between one or several instantiations.

However, this suggestion is void, due to the Matrix/Vector case I mentioned before.

Show quoted text 10 lines
> Also, I'd be a big fan of extending the FPC version of templates in > the direction allowing more powerful generic metaprogramming (without > the broken C++ heritage), and I think that explicit instantiation > would simplify that quite a lot. The only thing I would add to FPC generics that's a bit in the direction of C++ templates (besides constraints which are also a Delphi feature and thus need to be supported sooner or later) would be the ability to "specialize" a generic with constants (and that only if I have too much time and no other feature to attend to).

I actually faked a small part of that in one of my programs by using SizeOf ;).

type GNumeric1 = byte;
type GNumeric2 = packed array[2] of byte;
type GNumeric3 = packed array[3] of byte;
...
// and use the "constant":
generic GenericArray<T,Size> = record
  Data : array[0..SizeOf(Size)-1] of T;
end;
type IntArraySize2 = specialize GenericArray<Integer, GNumeric2>;

Booleans can be similary faked.

But that's it! No more C++ template ways. I don't want a turing complete compile time construct in FPC!

I guess it's too late :P.

Let's start with something simple...

generic GNumericInc<N> = packed record
  unused1 : Byte;
  unused2 : N;
end;
generic GNumericAdd<N> = packed record
  unused1 : Size;
  unused2 : Size;
end;
generic GNumericSquare<N> = packed record
  unused : array[0..SizeOf(Size)-1] of Size;
end;
generic GNumericMult<A,B> = packed record
  unused : array[0..SizeOf(A)-1] of B;
end;

However, I have yet to find a way to stop a recursive loop :P (yes, I've got a Boost background)

On a more serious note however, we don't want a C++ templates-like monster, however any enhancement that allows more powerful compile-time code generation that is clean and doesn't affect normal code isn't something to ditch. Although I'd need to seriously rethink that, to come up with something powerful, yet clean and simple enough not to provoke code bloat and exponential explosion of compile time.

regards, Kornel Kisielewicz

Reply by email

Sven Barth
to Kornel Kisielewicz
Kornel Kisielewicz wrote 24 lines
On Fri, Jan 27, 2012 at 10:25 PM, Sven Barth wrote: > On 27.01.2012 19:05, Kornel Kisielewicz wrote: >> In this case FPC's specialize could be the mode to be "smarter" >> requiring a explicit instantiation that at the same time shows in >> which object should the code be stored. > > No. A definition should be stored where it is defined. I can't just use > ugendiamond1 for storage, because then ugendiamond2 will depend on > ugendiamond1 which might not be the intention. Also what about units that > were definitely compiled by different people, but by "accident" have the > same specializations? You can't store them in the ugendiamonddecl.ppu/.o > either, because that file must be considered fix. What I meant was a solution (only for FPC mode) that only allowed compatibility between specializations. So if ugen1 and ugen2 would both have specialization TGeneric then the situation would be as now (two instances), only no assignment would be possible. To have a possible assignment you'd have to have them both include a common unit that has this particual specialization. Basically restricting template assignment only to the same instantiation as stated in the manual, which would allow to choose between one or several instantiations.

I believe that having an overview over which types are assignable to which other types in a bigger application will be rather difficult.

However, this suggestion is void, due to the Matrix/Vector case I mentioned before.

Maybe it's definitely better if we go the Delphi route even if we won't be able to define helpers for two "different" specializations using the same type...

Show quoted text 27 lines
>> Also, I'd be a big fan of extending the FPC version of templates in >> the direction allowing more powerful generic metaprogramming (without >> the broken C++ heritage), and I think that explicit instantiation >> would simplify that quite a lot. > > The only thing I would add to FPC generics that's a bit in the direction of > C++ templates (besides constraints which are also a Delphi feature and thus > need to be supported sooner or later) would be the ability to "specialize" a > generic with constants (and that only if I have too much time and no other > feature to attend to). I actually faked a small part of that in one of my programs by using SizeOf ;). type GNumeric1 = byte; type GNumeric2 = packed array[2] of byte; type GNumeric3 = packed array[3] of byte; ... // and use the "constant": generic GenericArray<T,Size> = record Data : array[0..SizeOf(Size)-1] of T; end; type IntArraySize2 = specialize GenericArray<Integer, GNumeric2>; Booleans can be similary faked.

A have to admit, that is neat O.o

Show quoted text 30 lines
> But that's it! No more C++ template ways. > I don't > want a turing complete compile time construct in FPC! I guess it's too late :P. Let's start with something simple... generic GNumericInc<N> = packed record unused1 : Byte; unused2 : N; end; generic GNumericAdd<N> = packed record unused1 : Size; unused2 : Size; end; generic GNumericSquare<N> = packed record unused : array[0..SizeOf(Size)-1] of Size; end; generic GNumericMult<A,B> = packed record unused : array[0..SizeOf(A)-1] of B; end; However, I have yet to find a way to stop a recursive loop :P (yes, I've got a Boost background)

Somehow I fail to see what you exactly want to show me here. Would you please explain?

Show quoted text 6 lines
On a more serious note however, we don't want a C++ templates-like monster, however any enhancement that allows more powerful compile-time code generation that is clean and doesn't affect normal code isn't something to ditch. Although I'd need to seriously rethink that, to come up with something powerful, yet clean and simple enough not to provoke code bloat and exponential explosion of compile time.

The biggest problems are:
* backwards compatibility (I did quite some thinking around many corners to implement Delphi compatible generics in a way that allowed old non-Delphi-mode generics to still be compileable as is)
* Delphi compatibility (regarding new features at least so far that generics can still be used in a Delphi compatible way)
* not too much complication of the parser (it already got a bit more complicate by me implementing support for "mylist := TFPGList<LongInt>.Create" and I don't have yet implemented generic functions which will allow things like this in the end: "if myfunc1<LongInt>(42) < myfunc2<LongInt>(21) then ..." [I hope you can spot where the parser will have problems ^^])

Regards,
Sven

Reply by email