The Free Pascal team is happy to announce the addition of a new language feature: constant parameters for generics.

This allows to declare generic types or routines that aside from type parameters can also take constant parameters that can be used inside the generic as if one would use untyped constants. Together with inlining and optimizations this allows to generate optimal code that is yet flexible.

This feature was developed by Ryan Joseph. Thank you very much Ryan for your contribution and your patience until this feature was included.

A generic constant parameter is declared like this:

CONSTPARAM::=const <IDLIST>: <TYPE>
IDLIST::=<ID>[, <ID>]
TYPE::=<ID>

The following types are supported for constant parameters:
- integer types (including range types)
- floating point types
- string types
- set types
- enum types
- Boolean types
- Pointer types

While the type declaration might look like a typed constant it is in fact an untyped constant. The type is used to allow the author of a generic to restrict the range of the type. This has some implications:

- the constant parameter can be used inside the generic whereever an untyped constant can be used (e.g. variable initializers, constant initializers, default parameters, array indices, compile time intrinsics and operations)
- types that can't be used for untyped constants can't be used for generic constant parameters either

Example:

=== code begin ===

{$mode objfpc}
type
   generic TStaticArray<T; const N: Integer> = array[0..N-1] of T;
generic function TimesX<const N: Integer>(aArg: Integer): Integer;
begin
   Result := aArg * N;
end;
var
   myArray: specialize TStaticArray<LongInt, 2>;
   i: LongInt;
begin
   i := specialize TimesX<2>(21);
end.

=== code end ===

Important: Unlike C++ FPC does not support default specializations, thus doing recursive specializations together with operations on constants will result in out of stack exceptions or infinite loops:

=== code begin ===

{$mode objfpc}
type
   generic TTest<const N: Integer> = class
   const
     N1 = N - 1;
   type
     TSubTest = specialize TTest<N1>;
     // the following does not work currently for a different reason:
     // TSubTest = specialize TTest<N - 1>;
   end;
begin
end.

=== code end ===

Delphi compatibility: this feature is NOT Delphi compatible. However to not inconvience users that prefer mode Delphi this feature is also available in that mode as our stance usually is that Delphi code should compile with FPC while the inverse is not necessarily true.

Also this feature will NOT be part of 3.2.

Please give the generic constant parameters a try and report bugs in the bugtracker.

Regards,
Sven

On Apr 26, 2020, at 5:13 AM, Sven Barth via fpc-devel wrote: The Free Pascal team is happy to announce the addition of a new language feature: constant parameters for generics.

Excellent! Thanks for getting this merged. It was a long battle but it's finally over. ;)

Regards,
Ryan Joseph

As the original author, can you say something about the intended use of this feature ?

Sven gave some examples, and they show how it works, but from his examples I don't see the point of this feature.

Michael.

Thank you Ryan and Sven. Your work is much appreciated as usual.

However, Michael beat me to it in asking how this feature is useful. I am sure there might be use cases, but for right now I am coming up without any real advantages. Could anyone with better insight than me please explain?

On Apr 26, 2020, at 2:38 PM, Michael Van Canneyt wrote: As the original author, can you say something about the intended use of this feature ?

It was meant for a pretty narrow use of array types. If I knew how much work it would be to implement I probably would have not done it. :P I personally wanted it for static array lists that had methods like Add/Delete.

Here's an example of something I came across in a C++ project for decoding some data files old DOS games.

==================

{$mode objfpc}
program Block;
type
  generic TBlock<T, const Alpha, Depth: Integer> = record
    bits: array[0..(Alpha + Depth) - 1] of T;
  end;
type
  TBlock32 = specialize TBlock<UInt32, 4, 1>;
  { ... other block types ... }
begin
end.

Here's an example from the bug tracker by another user (Akira) that helped with testing. More arrays you'll note.

==================

program ConstMatrixExampleObjFPC;
{$mode ObjFPC}
{$modeswitch AdvancedRecords}
type
  String3 = String[3];
  generic TRawMatrix<T; const N: SizeUInt> = array[0..N-1] of array[0..N-1] of T;
  generic TMatrix<T; const N: SizeUInt> = record
  private type
    ArrayType = specialize TRawMatrix<T, N>;
  private
    Data: ArrayType;
  public
    class operator :=(constref Arr: ArrayType): TMatrix; inline;
    procedure Display;
  end;
  class operator TMatrix.:=(constref Arr: ArrayType): TMatrix;
  begin
    Result.Data := Arr;
  end;
  procedure TMatrix.Display;
  var I, J: SizeInt;
  begin
    WriteLn('[');
    for I := 0 to N - 1 do begin
      Write(' [');
      for J := 0 to N - 2 do
        Write(Data[I, J], ', ');
      Write(Data[I, N - 1]);
      Writeln('] ');
    end;
    Write(']');
  end;

const RawMat: specialize TRawMatrix<String3, 4> = ( ('AAA', 'BBB', 'CCC', 'DDD'),
('EEE', 'FFF', 'GGG', 'HHH'),
('III', 'JJJ', 'KKK', 'LLL'),
('MMM', 'NNN', 'OOO', 'PPP')
);

var Mat: specialize TMatrix<String3, 4>;
begin
  Mat := RawMat;
  Mat.Display();
end.

And another example from him that gets pretty clever. ;)

==================

program Example;
// using Delphi-mode here as a matter of preference...
{$mode Delphi}
type
  Meters<const Value: Double> = record
  public const
    ToKilometers = Value / 1000;
    Unconverted = Value;
    ToCentimeters = Value * 100;
    ToMillimeters = Value * 1000;
    ToFeet = Value * 3.28084;
  end;
  procedure Test;
  type
    TenMeters = Meters<10>;
  const
    KILOMETER_VALUE = TenMeters.ToKilometers;
    UNCONVERTED_VALUE = TenMeters.Unconverted;
    CENTIMETER_VALUE = TenMeters.ToCentimeters;
    MILLIMETER_VALUE = TenMeters.ToMillimeters;
    FOOT_VALUE = TenMeters.ToFeet;
  begin
    WriteLn(KILOMETER_VALUE : 0 : 4);
    WriteLn(UNCONVERTED_VALUE);
    WriteLn(CENTIMETER_VALUE);
    WriteLn(MILLIMETER_VALUE);
    WriteLn(FOOT_VALUE : 0 : 4);
  end;
begin
  Test();
end.

Regards,
Ryan Joseph

W dniu 2020-04-26 o 09:38, Michael Van Canneyt pisze:

It can be quite useful. You can do something similar (I hope): https://sourceforge.net/p/ultravnc/code/HEAD/tree/UltraVNC%20Project%20Root/UltraVNC/vncviewer/ClientConnectionHextile.cpp

Michał.

Fixed-length arrays are the only practical use I can think of.

The record with constants example is too convoluted to my taste. As a reader, I find such code hard to understand.

Thanks for the examples !

Michael.

Another example might be headers for embedded systems.

On Apr 26, 2020, at 4:02 PM, Michael Van Canneyt wrote: Fixed-length arrays are the only practical use I can think of.

I think you're right. Here's what I'll say next time: They're for composing types of dynamic size at compile time. :)

Regards,
Ryan Joseph

Jeppe had provided a potential usecase on the core mailing list in October '18. His example is not useable as-is, but to give you an idea:

=== code begin ===

program tgpio;
{$mode objfpc}
{$modeswitch advancedrecords}
type
  generic TSomeMicroGPIO<const Base: PtrUInt> = record
  private
    procedure SetPin(aIndex: SizeInt; aEnable: Boolean); inline;
    function GetPin(aIndex: SizeInt): Boolean; inline;
  public
    property Pin[Index: SizeInt]: Boolean read GetPin write SetPin;
  end;
procedure TSomeMicroGPIO.SetPin(aIndex: SizeInt; aEnable: Boolean);
begin
   if aEnable then
     PLongWord(Base)[2] := PLongWord(Base)[2] or (1 shl aIndex)
   else
     PLongWord(Base)[2] := PLongWord(Base)[2] and not (1 shl aIndex);
end;
function TSomeMicroGPIO.GetPin(aIndex: SizeInt): Boolean;
begin
   Result := (PLongWord(Base)[2] and (1 shl aIndex))  0
end;
var
   GPIOA: specialize TSomeMicroGPIO<$8000F000>;
   GPIOB: specialize TSomeMicroGPIO<$8000F100>;
   GPIOC: specialize TSomeMicroGPIO<$8000F200>;
begin
   GPIOA.Pin[2] := True;
end.

=== code end ===

As the compiler can inline all this, the writing of maintainable, hardware agnostic frameworks for embedded controllers becomes easier.

In general I agree with you that the use of constants as generic parameters is less wide. But there cases where one might want them.

A further example could be to determine the size of a hash table: Determining that at compile time instead of runtime might allow for better code. At the same time the user of that code would still be able to influence it.

In the bug report there was a further example by Akira1364. At its core it's about static arrays again, but it shows what can be done with this:

=== code begin ===

program ConstMatrixExampleObjFPC;
{$mode ObjFPC}
{$modeswitch AdvancedRecords}
type
   String3 = String[3];
   generic TRawMatrix<T; const N: SizeUInt> = array[0..N-1] of
array[0..N-1] of T;
   generic TMatrix<T; const N: SizeUInt> = record
   private type
     ArrayType = specialize TRawMatrix<T, N>;
   private
     Data: ArrayType;
   public
     class operator :=(constref Arr: ArrayType): TMatrix; inline;
     procedure Display;
   end;
   class operator TMatrix.:=(constref Arr: ArrayType): TMatrix;
   begin
     Result.Data := Arr;
   end;
   procedure TMatrix.Display;
   var I, J: SizeInt;
   begin
     WriteLn('[');
     for I := 0 to N - 1 do begin
       Write(' [');
       for J := 0 to N - 2 do
         Write(Data[I, J], ', ');
       Write(Data[I, N - 1]);
       Writeln('] ');
     end;
     Write(']');
   end;

const RawMat: specialize TRawMatrix<String3, 4> = (   ('AAA', 'BBB', 'CCC', 'DDD'),
  ('EEE', 'FFF', 'GGG', 'HHH'),
  ('III', 'JJJ', 'KKK', 'LLL'),
  ('MMM', 'NNN', 'OOO', 'PPP')
);

var Mat: specialize TMatrix<String3, 4>;
begin
   Mat := RawMat;
   Mat.Display();
end.

=== code end ===

I'm sure the future will show more potential examples. Or one could look at C++ examples that allow constants as well.

Regards,
Sven

Hi,

perhaps it could be used to merge specializations (if fpc cannot do that on its own):

Like when you have a hashmap THashMap<Key,Value>, and need three specializations:

THashMap<string, pointer>

THashMap<string, TObject>

THashMap<string, sizeint>

It is basically three times the same hashmap, but if fpc does not detect that, it might generate three times the same assembly code, which waste a lot of space.

But with constants it can be merged to TBaseHashMap<Key, ValueSize: integer> and then you only have one map in the assembly code, and three wrappers to remove the casting:

THashMap<string, pointer> = TBaseHashMap<string, sizeof(pointer) > = TBaseHashMap<string, 8 >

THashMap<string, TObject> = TBaseHashMap<string, sizeof(TObject) > = TBaseHashMap<string, 8 >

THashMap<string, sizeint> = TBaseHashMap<string, sizeof(sizeint) > = TBaseHashMap<string, 8 >

Cheers,
Benito

It's not that simple. In principle you're right that the compiler could try to merge more implementations, but this does not depend on the declaration of the generic, but the use of the parameter types.

Take the following example:

=== code begin ===

{$mode objfpc}
type
   generic TTest<T> = class
     procedure DoSomething;
   end;
   TMyClass1 = class
      procedure Foobar;
   end;
   TMyClass2 = class
     procedure Foobar; virtual;
   end;
procedure TTest.DoSomething;
var
   o: T;
begin
   o.Foobar;
end;
procedure TMyClass1.Foobar;
begin
   Writeln('TMyClass1.Foobar');
end;
procedure TMyClass2.Foobar;
begin
   Writeln('TMyClass2.Foobar');
end;
type
   TTestMyClass1 = specialize TTest<TMyClass1>;
   TTestMyClass2 = specialize TTest<TMyClass2>;
begin
end.

=== code end ===

In case of TMyClass1 this will result in a static call to TMyClass1.Foobar, however in case of TMyClass2 this will result in an indirect call through the VMT.

The type information needs to be correct as well, even more so once we have support for Extended RTTI where one can enumerate non-published fields, properties and methods in addition to published.

Not to mention that your TBaseHashMap would not work with managed types...

Regards,
Sven

On Sat, Apr 25, 2020 at 6:14 PM Sven Barth via fpc-devel <

YES! This is fantastic news.

On Sun, Apr 26, 2020 at 4:08 AM Anthony Walter via fpc-devel <

A simple, but IMO exciting use case is just that it can provide impossible-to-achieve-otherwise optimizations via const propagation.

For example, some functions and procedures that used to have to take numerical values as regular parameters can now instead take them as generic constant parameters, which allows the compiler to "carry them farther" than it ever would have previously.

Op 2020-04-26 om 11:02 schreef Michael Van Canneyt:

Valuetype sets of enum>256 elements.  Type T is the enum, const is ord(high(enum) div 32)+1

De const is then a high bound of the array, type T is only used to implement the operators and methods.

Hi Sven,

It's not that simple. In principle you're right that the compiler could try to merge more implementations, but this does not depend on the declaration of the generic, but the use of the parameter types.

I mostly use generics for containers, especially hashmaps. They only stores the values and never calls specific methods on them

Not to mention that your TBaseHashMap would not work with managed types...

That would need be handled separately. There are probably also people who want a container that calls .free on TObject descendants.

Best,
Benito

Hey folks!

Ryan Joseph wrote

It was meant for a pretty narrow use of array types. [...]

If so, can I finally have my Extended Pascal schemata?

https://wiki.freepascal.org/Extended_Pascal#Schemata_.28not_yet_implemented.29

It's basically the same, isn't it? Just a different syntax.

[For reference: http://www.pascal-central.com/docs/iso10206.pdf (§ 6.4.7, PDF page 53)]

Op 2020-04-26 om 11:48 schreef Sven Barth via fpc-devel:

(Actually you'd probably want to use atomic RMW operations, because otherwise a write from a thread or interrupt routine could be overwritten, even if it was not on bits that clash)

A further example could be to determine the size of a hash table: Determining that at compile time instead of runtime might allow for better code. At the same time the user of that code would still be able to influence it.

Yup. My Genlight sorted stringlist could also use it for tuning. To workaround the classic tstringlist ordered insertion limit, it uses an (dyn) array of small arrays to store the strings, which postpones the inevitable with about a magnitude 10-20.

The constant would be the size of the deeper array, now a weighted value 1024.

Larger (e.g. 4096) means less memory overhead, faster iteration, but with slower (random) insertion performance, which can be perfectly fine if your datasource is mostly in order + few mutations.

Well, nice and well for /you/ then, but there are many more uses for generics. And as a compiler developer I /must/ think about these cases as well.

The user should not need to care about that, especially for general base classes.

Also assume you have a record and a hashmap that stores them. Now you add a string to the record which turns it into a managed one and suddenly your code will no longer work, probably resulting in subtle bugs or memory leaks. That's /not/ what should happen for merely adding a string. It's already bad enough that this happens with the Pascal style I/O we don't need to have this with base container types.

For calling Free we have explicit classes. But the difference between managed types vs. not-managed types is subtle and can change - as mentioned above - by merely adding a field.

Regards,
Sven

I think constants in generics may be the only way to create dynamically sized types at compile time (via static arrays). That seems like a really narrow usage but it opens a lot of doors most of us have probably never thought about. I suspect in time more good usages will be discovered.

Regards,
Ryan Joseph

Ryan Joseph via fpc-devel schrieb am Mo.,
27. Apr. 2020, 18:28:

That's what I think as well.

Regards,
Sven

Hi,

That is why FreePascal should merge non-managed generic classes that have the same size when it can.

But as long as FreePascal does not do it, the user needs to do the merging manually, with all the resulting problems. And constants generics can be used to store the size

Best,
Benito