Hi,

What is the state of published record in fpc trunk?

The structures in typinfo.pp seems to be pretty complete, but fpc complains:

unit1.pas(42,24) Error: This kind of property cannot be published
{$mode delphi}
interface
uses Classes, TypInfo;
type
   TMyRec = record
     x,y: integer;
   end;
   TTest = class(TPersistent)
   private
     FRec: TMyRec;
   published
     property Rec: TMyRec read FRec;
   end;

Mattias

Mattias Gaertner wrote

Hi, What is the state of published record in fpc trunk?

You cannot publish records, and this is not planned.

This is because things like

MyClass.MyRecord.MyField:=12;

do not work as one would naively expect. It does work if you use a TPersistent instead of a record (the very reason the TPersistent class was invented)

However, you can use extended RTTI to access this typeinfo and manipulate the record. That works.

Michael.

Do you mean the trap when "MyField" is actually a property (with possible setter) and "MyClass.MyRecord.MyField:=123" effectively (may) modify a temporary value? I feel this is something a bit independent to being able to publish the records.

It is indeed a trap, and can be solved by designing a record to not have any properties/methods that change it's state. The only allowed changes to record state should be directly through the fields. Then the compiler already correctly prevents you from doing "MyClass.MyRecord.MyField:=12;" . We dealt with this in CGE too, see https://castle-engine.io/coding_traps ,
https://github.com/castle-engine/castle-engine/blob/master/src/base/castlevectors_generic_float_record.inc#L44 .

However, you can use extended RTTI to access this typeinfo and manipulate the record. That works.

Last time I checked, the RTTI information was not generated for records in only the "public" section, and (as per above) the records could not be placed in the "published" section. Is now one of the above possible? :) This would be cool, and it is very desired by Castle Game Engine.

To explain more our use-case in CGE:

We do use a few records (like for vectors and matrices, TVector3, TCastleColor etc.) and we like to have properties that expose them on object (e.g. "TCastleTransform.Translation:TVector3"). Right now, being able to see this in the object inspector (in Lazarus or in CGE editor) requires a bit of a hack -- we need a class, like TCastleVector3Persistent (
https://github.com/castle-engine/castle-engine/blob/master/src/base/castlevectors_persistent.inc#L92 ), that exposes the record, effectively is synchronized 2-way with TVector3 record. And every time someone wants to expose TVector3 property to the editor, they need to

- make the TVector3 property public (and this is what should be used by Pascal developers),

- and put the corresponding TCastleVector3Persistent in published. It is possible, but requires some boiler-plate code each time: https://castle-engine.io/custom_components#_publishing_vectors_and_colors .

If we could instead just put TVector3 in the published section, and make it exposed in the object inspector (maybe adding some additional code to use RTTI, but that is made *once*, not every time we need to expose TVector3) this would be great :)

Regards,
Michalis

Michalis Kamburelis wrote

If we could instead just put TVector3 in the published section, and make it exposed in the object inspector (maybe adding some additional code to use RTTI, but that is made *once*, not every time we need to expose TVector3) this would be great :)

Seconding this, for pretty much the same use case: my 3D game engine also uses TVector, etc records (and objects! - stuff being allocated in stack or "inline" other objects/classes is incredibly important) and needs to expose them via RTTI/published and i have to use various kludgy hacks to work around a language limitation that IMO was already questionable back in the 90s when Delphi 2 was released (i might kinda accept it for Delphi 1 having to work in 16bit mode :-P).

Instead of having a proxy object, i "flatten" the fields using getters and setters so, e.g., a "property Position: TVector read FPosition write SetPosition" in the public section also has "property PositionX: Single read GetPositionX write SetPositionX" and similar for PositionY and PositionZ. Obviously with something like "property Transformation: TTransformation read FTransformation write SetTransformation" which has Position, Rotation and Scale properties (as TVectors) that get flattened to nine published properties so they can be accessed via RTTI (and thus have automatic serialization, deserialization, property editing UI, clipboard copy/paste, undo/redo support via property diffs, etc without having to do all that manually via code) it becomes kinda unwieldy fast.

Sadly i also do have to use proxy objects in some cases, e.g. i have a generic object "TDynArray" for dynamic arrays which is wrapped via a "TSerializableCollection" class so that the
serialization/deserialization code can automatically write/construct/read published collections of objects.

It'd be very useful if the Run-Time Type Information could describe all the Type Information the compiler already knows about :-P

Kostas

Partly, yes.

You need "published" when you want to stream them (in a .lfm file).

But for streaming the above behaviour is prohibitive.

When using a TPersistent, it is a class which behaves 'normally' and you get the streaming "for free": streaming a TPersistent is not different from a TComponent.

To be able to stream published records one would need to create an empty record, fill it with data and then assign the record (possibly recursively) because of the above behaviour.

Doing this correctly would simply lead too far, since basically you need to duplicate the whole mechanism that exists for classes.

A more funddamental reason is of course that records can have variant parts and streaming that in a correct way is impossible: you do not know what actual data is present.
Which of the variants should then be streamed ?

Last time I checked, the RTTI information was not generated for records in only the "public" section, and (as per above) the records could not be placed in the "published" section. Is now one of the above possible? :)

It is since some time, but you must explicitly enable it. The basic TObject does not have extended RTTI enabled as it is in Delphi, except when you compile the unicode rtl.

Enabling it in TObject doubles the binary size of any project I tested it with, so I don't plan to enable it in the default RTL.

Michael.

See my answer to Michalis: It is possible, but no streaming, so no published.

Michael.

Thanks Michael!

OK, that's partially good news :)

I understand that streaming records by default to LFM would lead to too many questions (and I'm unsure whether Delphi does it too, for records and DFM). If this means they are not allowed in the "published" section, *but* the RTTI is capable of telling me what records are in the "public" section, and read / write them (using proper getters / setters), this is something we can work with.

I mean, I can 1. add my own serialization / deserialization of them, and 2. make them exposed in the object inspector in CGE editor.

(
Details:
AD 1 - For serialization / deserialization, in Castle Game Engine, we use CastleComponentSerialize on top of FpJsonRtti. One of the reasons I like JSON and FpJsonRtti over LFM is that this gives us more control -- I know I can add code to CastleComponentSerialize to deal with records, if only RTTI tells me about them.
AD 2 - For object inspector, we use Lazarus "ObjectInspector" unit. I'm less sure about it, but hopefully it is possible to add there "fake" subcomponents too.
)

Can you expand your statement "It is since some time, but you must explicitly enable it."? What compiler switch enables it? What API should I use to access it -- Rtti unit in FPC has all we need? (I guess TypeInfo unit API does not expose it, or it does?)

To make my question more concrete, imagine a code like this:

"""
type
  TVector3 = record X, Y, Z: Single end;
  TCastleTransform = class
  public
    property Translation: TVector3 read ... write ...;
    property Scale: TVector3 read ... write ...;
  end;
"""

I'm looking for a way to tell, at runtime, using RTTI, having any instance of TObject (maybe of TCastleTransform):

- what public properties of type TVector3 are there (like "Translation" and "Scale"),
- what public fields does TVector3 have (like X, Y, Z) (this is nice-to-have, we could work without this info but then limiting our record serialization / deserialization to only specific record types),
- read / write these record properties using the proper getter / setter (whether it's direct field access or a method).

Regards,
Michalis

sob., 17 sie 2024 o 16:44 Michael Van Canneyt via fpc-devel napisał(a):

It does not.
Streaming has not changed in an essential way since D7, I think.

With the coming of extended rtti, they could have changed it, but they didn't, I believe since it would be difficult to maintain backwards compatibility.

If this means they are not allowed in the "published" section, *but* the RTTI is capable of telling me what records are in the "public" section, and read / write them (using proper getters / setters), this is something we can work with.

You should be fine then, as this is supported.

I prefer JSON too.

AD 2 - For object inspector, we use Lazarus "ObjectInspector" unit. I'm less sure about it, but hopefully it is possible to add there "fake" subcomponents too. )

Actually, it is!

see the work the FP/Lazarus foundation sponsored in

lazarus/components/jitclasses/

This is all perfectly possible.

Simply add

   {$RTTI EXPLICIT
      PROPERTIES([vcPublic])
      FIELDS([vcPublic])
      METHODS([vcPublic])}

in your units and you're all set.

Michael.

Thank you -- this is excellent information, both RTTI for records and lazarus/components/jitclasses/ . So I really have all the puzzle pieces I need :)

I have recorded a TODO to handle it all in
https://castle-engine.io/roadmap#records_rtti .

Regards,
Michalis

niedz., 18 sie 2024 o 10:49 Michael Van Canneyt
napisał(a):

you may try this:

TSomeObjToSerial = class(TPersistent)
private
  FPoint: TPoint;
public
  property Point: TPoint read FPoint write FPoint;
published
  property PointX: Integer read FPoint.x write FPoint.x;
  property PointY: Integer read FPoint.y write FPoint.y;
end;

not tested by compiler, but as I remeber it's work.