Hello to all,

While enjoying variant late binding for a couple of month now, I would like to ask for an update on the status of RTTI.

The mORMot developer has designed a work-around for the missing RTTI. This works well.
But it would be (much) nicer to be able to use RTTI without workaround.

Does anybody have some info about the status of this (feature) request ?

Thanks !

No more than: http://bugs.freepascal.org/view.php?id=26774

I am working on "Invoke" method from RTTI module and TValue. "Invoke" is important for attributes, and for ObjAuto module.

Some related work to RTTI you can find in this branch:

http://svn.freepascal.org/cgi-bin/viewvc.cgi/branches/joost/?view=log

2015-02-05 11:23 GMT+01:00 Alfred :

Regards,

Maciej Izak

If you are referring to http://bugs.freepascal.org/view.php?id=26774

Work on it was started by Steve Hildebrandt, but I got the impression that due to the cross-platform nature of FPC he ran into some problems. I may be wrong, though.
I do not know enough about the compiler internals to assess the situation.

Michael.

Your assesment is correct.
I have a version that is nearly complete for Windows(for other platforms I have't done anything). But to have RTTI on all platforms, that is still delphi compatible is not possible(at least for me), due to the nature of some architetures / calling conventions. When I have some free time(after the tests for this semester) I will probably be able to post a patch.

Steve

Would it be possible that you post the current state of the patch somewhere? So people can look/work on it.

You don't need to implement Invoke for all platforms for now. Just raise an exception and hopefully platform maintainers will take it from there. The most important part is that the parameter description is flexible enough to potentially allow for all platforms to be supported.

Regards,
Sven

And here is the problem, depending on the calling convention it may not fit into the delphi compatible RTTI layout.
If more information is needed than a stack offset or a single register index, you will need to acomodate that somehow.

Steve

How does the information record look currently? Our RTTIs aren't 100% compatible anyway, so I wouldn't mind if we add needed fields for other platforms. Of course we'd first need to know whether any other platform has such needs...
At least for m68k I can say that register index and stack offset should be enough.

Regards,
Sven

x86-64 (PowerPC too) is pretty compilcated in this regard: single parameters can span across multiple registers. IIRC some ABIs even require mixed integer/fpu/stack locations for one parameter.

Just an idea: What about serializing TCGPara/TCGParalocation?

Ehm. It's unfair. I arleady posted some time ago that "Invoke" is done for windows by me... Then you posted that we need version for all platforms... Double standards in community? :\
7 lut 2015 11:28 "Sven Barth" napisał(a):

Florian Klämpfl wrote

Just an idea: What about serializing TCGPara/TCGParalocation?

The main issue I see with that is that this format changes from time to time. I'll probably have to change it again to fully support the ppc64le ABI.

To me, the optimal solution would be to simply use libffi (you would only need logic to translate the rtti for the parameter types to libffi format, and it would automagically work for all platforms), but since that's an external dependency I guess that's not acceptable. I'm however definitely not looking forward to reimplementing parameter passing/loading and calling routines for all platforms again inside the rtl.

Jonas

Maciej Izak wrote

Ehm. It's unfair. I arleady posted some time ago that "Invoke" is done for windows by me... Then you posted that we need version for all platforms... Double standards in community? :\

It's a bit hard to keep the overview when volunteers from outside are working without communicating with us. That was the first update I read from Steve since he said that he's going to work on extended RTTI a few months back and to be frankly honest: I simply forgot your mail from the fifth (or do you mean a different one?) among my normal daily work. So I'm sorry if it came of as having double standards we're just human beings after all...

Would you please elaborate what exactly you are working on? Maybe we can find synergies between your work and Steve's.

And of course we need a cross platform solution. It won't work for us to have it just for Windows. At least the main platforms i386-win32, x86_64-win64, i386-linux, x86_64-linux, i386-darwin and x86_64-darwin should be covered. Immediately followed by arm-linux and arm-darwin with aarch64-darwin not so far behind I suspect... Though most work should be per CPU-platform and not per-target anyway...

Regards,
Sven

Well, the technical aspect is one thing, but its license is another issue. Violating it happens too quick.

To clarify what I have done:
I implemented RTTI in a delphi compatible way that could be used for "invoke" or other shenanigans, while this works nice if we need to 1 offset / 1 register index,
it will not work as it is for each and every convention that is supported by fpc. What I haven't done is build TValue.invoke or somthing similar.

The calling code for x86_64 invoke or stdcall should be somwhat different. Consequence is someone will need to implement invoke for each convention so it works with the provided information.

The questions I have is, how is the information you use for invoke organised? like delphi?

Since I have some project and tests going on for university I have no problem to stop working on it.
I started the work quite a while back and only becouse I needed some of it for my own projects.

mfg Steve

arm-winCE and arm-android?

Stephano

That's why I propose that people do they work in a branch of the central svn :)

That was the first update I read from Steve since he said that he's going to work on

extended RTTI a few months back and to be frankly honest: I simply forgot your mail from the fifth (or do you mean a different one?) among my normal daily work. So I'm sorry if it came of as having double standards we're just human beings after all...

Just to be clear, because it sounds like there is some confusion in the discussion.

About http://bugs.freepascal.org/view.php?id=26774

1. We do NOT need Invoke() implemented.
2. We do NOT need extended RTTI as available since Delphi 2010, nor features like attributes.
3. We do NOT need 100% compatibility with Delphi - it is fine to have some diverse data layout.
4. There should not be huge platform specific work.

Because:

1. We have our own low-level Invoke, for x86 and x64 platforms, working on both Windows and Linux.
Note: you can take a look at how we defined the asm stubs, in mORMot.pas
- much better design than the RTTI.pas or RIO.pas code IMHO - see http://blog.synopse.info/post/2012/03/07/Interface-based-services-implementation-details

2. We need only the RTTI as available since Delphi 6 for IInvokable methods arguments.

3. We already introduced some FPC-specific code, e.g. to follow RTTI data alignment on platforms which demand it.

4. By design, point 2 is mostly platform-independent, and some details (like calling conventions) do already exist for other part of the existing FPC RTTI.
What is needed by this bug report is to have
objectdef_rtti_interface_full(def:tobjectdef) implemented in ncgrtti.pas.

Any new input is welcome!

Arnaud

The libffi license is basically a BSD license, that's no problem at all: https://github.com/atgreen/libffi/blob/master/LICENSE

Jonas

This is already the issue :)

that's no problem at all:

There is imo a big one: just compiling a small program containing libffi and give it to another person without a libffi copyright notice is a license violation.

Using libffi in core parts of the rtl would mean even the smallest fpc program would need to come with a libffi copyright notice either by a readme or some command line option to display copyrights.

https://github.com/atgreen/libffi/blob/master/LICENSE

Considering calling conventions arm-android and arm-linux are the same. And if arm-wince doesn't have any special differences to other arm targets like e.g. x86_64-win64 does in comparison to other x86_64 targets then this should be solved together with arm-linux as well. As I said: it's mostly CPU specific stuff. So if one target of the platform works the others should work as well as long as they use the same calling conventions.

Regards,
Sven

Agreed.

Regards,
Sven

I am not a proponent of third-party dependencies.

Then, there is a practical issue:
I doubt libffi knows the Delphi calling conventions and the pascal calling convention where the callee clears the stack (or was it vice versa)? Since that is our main target group, I consider this a serious problem.

Thirdly, from looking at the use of this library, you will have to implement some kind of layer in between anyway that describes the function, so I do not really see the gain.

Michael.

Nevertheless, Florian's proposal seems to me like the optimal solution.

As far as I can see the compiler just needs to simply write a small block with the needed info in the generated executable. The RTL just needs to match this.

That this changes from time to time, I consider much less of a problem. The .ppu files also change regularly, that is just how things are...

As soon as a new platform appears you need to wait for libffi etc; the correct version needs to be installed and whatnot. I doubt this will cause less problems.

Michael.

Then I would like at least a "manager-style" solution (like with the widestring/thread manager). I really do not want to be forced to duplicate this work for every single platform I support. The mess with fcl-res is already bad enough.

Jonas

I am not a proponent of reimplementing everything but the kitchen sink because of other people's adversity to reusing existing solutions.

It does support those calling conventions.

Thirdly, from looking at the use of this library, you will have to implement some kind of layer in between anyway that describes the function, so I do not really see the gain.

The gain is that you implement this once and it will work for all platforms, rather than that you have to reimplement it for every single architecture (and depending on the implementation: every single calling convention too) combination that we support.

Jonas

Indeed, but even leaving the very unpleasant license aside, depending on third party libraries depending on libc is an issue on linux and an even bigger issue on windows not to mention rare platforms like amiga or msdos. While using libffi might work for the platforms you are mainly using, for most other platforms third party dependencies are a pain.

I don't mind reusing existing solutions, I often do so.

But keeping dependencies on external libs to a minimum is also a concern.

Well.
Prior to posting my statement, I checked, and it is not in the list of supported architectures.

But diving in the source I see it is indeed supported. OK, that alleviates this problem.

At least I suggest that if this path is taken, then it is done using a 'manager' approach such as for the rest of functionality that introduces a dependency. This allows to re-use the work of Steve without external dependencies on common (read: x86) platforms, and maybe implement native solutions for some common platforms, for people that want it.

Michael.

My point too.

Michael.

Considering that we aren't the fastest to implement new platforms I don't think that this will be much of a problem.

Looking at their site I noticed however that there are a few platforms that we support that they don't. For example m68k-amiga, i8086-msdos, arm-wince and some of the more exotic targets that we have (GBA, NDS, Wii for example).

So in the end a manager approach as Jonas suggested might be the best approach. Then we can for now implement the main platforms using libffi and implement full FPC ones one at a time.
Of course we need to assure that all necessary information can be provided to the manager and if that should mean that we'll need to divert from Delphi's RTTI format more then so be it.

Regards,
Sven

In our previous episode, Michael Van Canneyt said:

I am not a proponent of third-party dependencies.

(and specially not since we would depend on their RTTI format for this. This is more than something like mysql where a new header can be fixed with a header fix, as soon as the new ffi is default, functionality is broken in old FPC versions)

Actually, FPC had win64 support before gcc/binutils.

As long as we have no realiable approach to prevent people to violate the libffi license, I wouldn't recommend to do so. Besides this I do not want either that every FPC compiled binary needs to come with a libffi license text.

Of course we need to assure that all necessary information can be provided to the manager and if that should mean that we'll need to divert from Delphi's RTTI format more then so be it.

"Delphi" is not an architecture. The "register" calling convention is part of the x86 support. If you look at $ffisrc/src/x86/ffitarget.h you'll notice a FFI_REGISTER variable which is the default Delphi calling convention.

Regards,
Sven

Yes, that was one of the glorious exception that I still like to quote when talking about Free Pascal :D

But other then that we aren't normally the first to implement a platform...

Yes, the license might be the biggest problem here...

Regards,
Sven

Yes, I had done so, that is what I meant with 'diving in the sources'.

Michael.

Forgive me my ignorance:
Where does it say that you need to ship the license with the binary ?

Michael.

https://github.com/atgreen/libffi/blob/master/LICENSE

"software" means probably binary as well as source code.

Ah, sorry. Somehow I had skipped the paragraph where you said that *blush*

Regards,
Sven

In our previous episode, Florian Kl?mpfl said:

Yes, and that makes it GPL incompatible.

To be honest I'm a bit afraid of the situation where some Debian devel sees Lazarus (that has used .invoke somewhere in the LCL in the future) generating GPL incompatible binaries, including the lazarus binary itself.

I think it is better to simply not support the functionality on some targets/archs then rashly use libffi.

Well, quiet logical if we depend on other utils.

But not the only one, libgdb is a good example what the issue with external libraries in core parts of FPC. Typical procedure before release building for win32: try randomly mingw and gdb versions to find a pair which compiles a libgdb *and* results in an IDE with a working debugger. Unfortunaly, even if everything runs at a first glance, there is no guarantee that things work flawless as before because something in the C header could have changed causing random crashes. 700 lines defines and ifdefs (!) at the start of gdbint.pp tell the story.

Then these rare platforms can use something else, that's why I proposed a manager-style solution. And in spite of all the comments in the previous thread regarding cthreads/cwstring, I did not see a single point that argued that simply having a dependency on libc (especially via something as basic and portable as libffi) would cause problems other than for cross-compiling, even on Linux. It does not depend on particular versioned libc functionality, and this is not 199x anymore where even the name of the Linux (g)libc library was changed every full moon.

Besides, many FPC programs for Linux already depend directly or indirectly on libc anyway via the LCL, cthreads, cwstring, clocale, database units and whatnot. I really think the fear of problems due to this is way overblown (and note that I'm not proposing to have a complete RTL always based on libc — which was another misunderstanding raised in that previous thread; I'm just talking about making use of certain extra functionality used via C libraries).

While using libffi might work for the platforms you are mainly using, for most other platforms third party dependencies are a pain.

What would be a pain is that every time I want to modify something to the implementation of the parameter manager, I would have to go through the RTTI dispatch code of all of our platforms to fix them up (because I know how frustrating it is to have your platform broken all the time if other people change things; that's also why I implemented the codepage-aware functionality for all platforms at the same time, or at least tried to make sure that those changes didn't break them).

What would also be a pain is that every time I add support for a new platform, I would have to implement and debug the parameter loading/dispatching code twice: once in the compiler and once in the RTL.

Regarding some of the other remarks:
* Florian
- libgdb is an officially *unsupported* and extremely low level and very OS-dependent interface to GDB. The problems with using that library are related to specifically depending on that, library not on depending on C libraries in general. I fully agree it's a mess, which is one of the reasons I never even tried to get it working on Mac OS X.
* Marco
- the libffi license is not GPL-incompatible. The requirement to ship the license with binaries is also in the modified BSD license, which is GPL-compatible per the FSF.
- Their "rtti" consists basically of integers of various sizes, arrays, structs (which are plain arrays of elements) and pointers. You can't get more basic than that, and we already have to convert everything down to that level inside the compiler anyway to map Pascal types to the official platform ABIs. It is nevertheless true that an API version change in libffi could break compatibility with existing programs, so you would have to ship libffi together with your program if you'd want to avoid that.
* Michael:
- support for "register" and "pascal" calling conventions is listed in libffi's readme (https://github.com/atgreen/libffi/blob/master/README )
- "As far as I can see the compiler just needs to simply write a small block with the needed info in the generated executable. The RTL just needs to match this. .. That this changes from time to time, I consider much less of a problem. The .ppu files also change regularly, that is just how things are..." -- I think that is quite an insensitive statement, because you won't be the one that will need to make such a change or fix things as a consequence. And it's by no means comparable to a ppu file change. It's only "a fact of life" that such a change would have huge consequences if we choose so.

Look at the code of libffi (or better, don't — see below): we would basically have to duplicate all of that code for the platforms we support into the RTL, since all it does is dispatching and that's exactly what we need. Well, not duplicate, since then we have the same licensing issue as using that library — we have to implement it from scratch without looking at its source code. And in case of x86-64, you won't even be able to base yourself on the compiler source code, because I translated the libffi code largely into Pascal to implement its parameter manager. Seriously: it would mean I can't use libffi anymore for my compiler work, or not work on the dispatching code in the RTL for the same platform. Which means that I either cannot do complete ports by myself anymore, or I cannot make use of the knowledge in libffi when doing so (I also looked a lot at libffi for AArch64, so I won't be able to work on RTL dispatching code for that platform either, even if I wanted to).

Just like I may not be sensitive enough to people's issues with external dependencies on some platforms, I'd like to ask other people to be more sensitive to the work required to add and maintain support for a particular platform in FPC (both compiler and RTL).

Jonas

In our previous episode, Jonas Maebe said:

That is not a solution, it is moving the problem from you to target maintainers who need to deal with the versioning problems and getting it into distributions.

Just like you don't like maintaining multiple params, I'm not interesting in gettting saddled with an perpetual pain because of dependencies. Getting recent versions to the user, by distribution or own compilation means is hard enough.

So IMHO native solution for invoke, or just don't support it for the target is good enough for me.

If the policy is abandonned, it should be done because sb invested time in FPC_USE_LIBC solutions and guided the relevant targets through the change.

Then, with real feedback on issues a decision could be taken.

Such decision of a long standing policy should not be done because of some immediate need of ffi.

Moreover, my ffi weariness is not just because it links to libc, but because of the dependency and breakage of that package too. I don't feel like managing this for such a little feature.

Besides, many FPC programs for Linux already depend directly or indirectly on libc anyway via the LCL, cthreads, cwstring, clocale, database units and whatnot.

But do they distribute cross binary releases (like our .tar) that need to be good for a while?

I'm not against that, but it shouldn't be done rashly, and is unrelated to the ffi yes or no.

which was another misunderstanding raised in that previous thread; I'm just talking about making use of certain extra functionality used via C libraries).

No, you are also talking about having to manage those dependencies in the release process. The release cycles are slow and complicated enough, I don't need anything making it worse.

If the FFI metadata changes you must massively update everything too, and worse, existing releases are suddenly void for new distributions.

- the libffi license is not GPL-incompatible. The requirement to ship the license with binaries is also in the modified BSD license, which is GPL-compatible per the FSF.

True.

Pushing it on the user is IMHO a nono. They can barely manage building now.

Just like I may not be sensitive enough to people's issues with external dependencies on some platforms, I'd like to ask other people to be more sensitive to the work required to add and maintain support for a particular platform in FPC (both compiler and RTL).

So don't do it. You effectively offer FFI as a crutch. Don't offer the crutch, and keep it native or not on a per target basis.

If you can't afford to add a target just leave it and mark it as such in the documentation.

It is not "moving the problem", it's at most changing one problem into a completely different one. And libffi is probably already in all distributions, because it's used by a.o. CPython and OpenJDK.

So IMHO native solution for invoke, or just don't support it for the target is good enough for me.

That is however unacceptable to me. If it is considered basic functionality (which RTTI is, afaik), then it must be implemented for at least all main targets.

I explicitly mentioned later on that I did not argue in favour of making the entire RTL libc-based. I'm talking about the rule that nothing at all can depend on libc whatsoever.

Such decision of a long standing policy should not be done because of some immediate need of ffi.

There have been cthreads, cwstring and clocale before. And there will undoubtedly be more in the future. And as mentioned, a manager approach is fine by me.

Moreover, my ffi weariness is not just because it links to libc, but because of the dependency and breakage of that package too. I don't feel like managing this for such a little feature.

I don't feel like reimplementing and mainting the parameter manager in the RTL for "such a little feature" either.

The compiler wouldn't depend on libffi if it's implemented via a manager, because the compiler itself won't make use of that functionality. Should it ever do so anyway, we'll cross that bridge when we get there (although I can't immediately think of a practical scenario where we would want to do that). Someone could even implement an "fplibffi" then if they'd want to, like fpwidestring.

No, that's my main point: even if that would happen, then you only have to update things in one place (the mapping from our generic RTTI to libffi's generic parameter description). While if things would change in the compiler when you have a Pascal implementation for every platform, then you possibly have to change this Pascal implementation for every platform.

"The user" already has to deal with this for every single Lazarus program.

Using a library as intended is not a crutch.

If you can't afford to add a target just leave it and mark it as such in the documentation.

Which won't confuse or annoy users at all...

Jonas

That may be so, but if we implement a solution, we don't want people to have to redo everything you did.

So we prefer to implement invoke as part of the process.

Michael.

For me, that is good enough. It leaves open the possibility of native implementations. Given that Arnaud has an implementation for x86 and x64 (windows/linux), that can be used to implement a manager implementation for these platforms, this covers the main platforms. Most likely some other people will implement the calls for other platforms.

I don't see how this case differs from cthreads, clocale or cwstring: Like always, we offer the choice.

Michael.

2015-02-07 12:30 GMT+01:00 Sven Barth :

I checked my old code from 2006 (!) : my invoke implementation is affected by PascalScript BSD like license :\ ... So my implementation is useless.

My current work is probably useless too - now I am working on new operators to make this fast TValue implementation multiplatform:

http://www.cromis.net/blog/2013/02/tanyvalue-an-attempt-to-make-the-best-variable-data-container/

To achieve this I need to create new special record operators:

====== code begin ======
{$MODESWITCH MANAGEMENTOPERATORS}
  TValue = record
    ...
    class operator Initialize(Data, TypeInfo: Pointer): pointer;
    class operator Finalize(Data, TypeInfo: Pointer): pointer;
    class operator Copy(Source, Dest, TypeInfo: Pointer): pointer;
  end;
====== code end ======

Regards,

Maciej Izak ()

I think that you should continue your work. My implementation is affected by BSD like license :\ . My Invoke version is based on some extracted parts of PascalScript code.

Regards,
Maciej Izak

This is /really/ bad design. If you want such operators they would need to be defined like this:

  TValue = record
    ...
    class operator Initialize(var aRec: TValue);
   class operator Finalize(var aRec: TValue);
    class operator Copy(constref aSource: TValue; var aDest: TValue);
  end;
====== code end ======

Regards,
Sven

Here ist the current state of affairs for i386 only right now.

The Patches are currently based on

mfg Steve

Hello Steve, all,

I would like to report 100% success with the latest RTTI patches !!!!

The mORMot framework is now able to use RTTI the same way as under Delphi.
No trickery needed anymore !!

I would like to express my gratitude for the work done by the development-team.
Hopefully this patch will be included soon in trunk.

One (first) request:
Would it be ok to add pfConstRef to the TParamFlag in typinfo.pp ?

One (second) request:
The same RTTI on ARM would also be very welcome.
As stated before, my interest is running mORMot on ARM (e.g. Raspberry Pi[2]).

Many thanks, Alfred.

Alfred wrote

Hello Steve, all, I would like to report 100% success with the latest RTTI patches !!!!

Nice to hear.
If anything is not compatible with the information delphi provides please report back to me.

One (first) request: Would it be ok to add pfConstRef to the TParamFlag in typinfo.pp ?

The RTTI should already be generated the enum value was simply missing. See attached Patch.

One (second) request: The same RTTI on ARM would also be very welcome. As stated before, my interest is running mORMot on ARM (e.g. Raspberry Pi[2]).

Since I'm unable to test it myself right now, I could only try to write this blind and let somebody else do the testing.
It would also help to know what calling conventions are supported and how they work if they are not compareable to the i386/x64 ones. (Link to a good documentation or a short write up) The other way this could work is, if somebody with an ARM system takes over and I explain where compiler changes need to be made.

mfg Steve

Well. About the non compatible. I can only detect these:

1)
RTTICount: Word; //$FFFF if there is no further info, or the value of Count
I did not see it set to $FFFF in case of missing RTTI. As Delphi reportedly does.
But perhaps this has already been done somewhere else.

But, in my opinion, this is not of much importance

2)
Properties are in front of methods.
This is different from Delphi.
But I know that this cannot be changed.
That would break a whole lot of code !
And again, if you are aware of this: no , for me, this is all ok now !!

And I am willing to test on simple ARM (Pi[2], BBB). If you provide me with code !

Thanks again, Alfred.

------ Origineel bericht ------
Van: "Steve Hildebrandt"
Aan:
Verzonden: 25-2-2015 19:16:17
Onderwerp: Re: [fpc-devel] RTTI interface & variant late binding issue (mORMot)

In my opinion this is never the case the way the RTTI is currently generated so this shouldn't be an issue.

Here the opinion of others would be nice, since the RTTI was partly
generated but not added to "typinfo.pas".
Original:
tkInterface:
               (
                IntfParent: PTypeInfo;
                IntfFlags : TIntfFlagsBase;
                GUID: TGUID;
                IntfUnit: ShortString;
                {
                IntfProps: array of TPropInfo;
                }
               );
//useless since you would not know how many properties there are
Current:
tkInterface:
               (
                IntfParent: PTypeInfo;
                IntfFlags : TIntfFlagsBase;
                GUID: TGUID;
                IntfUnit: ShortString;
                {
                IntfPropCount: Word;
                IntfProps: array[0..IntfPropCount-1] of TPropInfo;
                IntfMethTable : TIntfMethodTable;
                }
               );
//I simply assumed this to be the easier change. Still would break some
code, if property RTTI for interfaces was ever used anywhere.

How would the layout in Delphi look like?

And I am willing to test on simple ARM (Pi[2], BBB). If you provide me with code !

It seemed simple enough see attached Patch.
ParReg: r0-r3=0..3

mfg Steve

Forgot to commit some parts of it sorry.
Also if reg < 255 and off > 0 it is the offset for the register. For example reg=0 and off=4 means upper 32 bit of the mm register 0.

Since this is all experimetal and there is no convention to uphold for the values ParaReg and ParaOff
I'm open to suggestions.

mfg Steve

mfg Steve

Hello Steve,

After much testing on ARM (RPi2), included a test-program that works on both i386 and ARM, with your RTTI patches.

As you can see, a lot of alignment issues. But its working 100% !!!

(My) Conclusion:
New RTTI info also available now on ARM. And working. Thanks !!

Some alignment problems have to be solved yet.
I hope that the included file gives you some clues.

Thanks, Alfred.

Hello again to all,

I am enjoying variant late binding AND interface RTTI for a while now ! Thanks again !

As far as I know, the patch for RTTI has not yet been included in trunk. But correct me if I am wrong.

I would like to ask to include the RTTI patch for i386 in trunk. With i386 interface RTTI, FPC is on level with Kylix for the RTTI part.

To ease introduction, I would like to suggest to include a new HASINTERFACERTTI flag, which will let the FPC user know, the new RTTI is available for use.

At this moment, it will only be i386.
But I hope also ARM will be included in the trunk. On my Raspberry Pi2, the new RTTI works flawless.

Thanks in advance.

I can't remember when I last used i386 code. Must have been more than 10 years ago.

But I hope also ARM will be included in the trunk. On my Raspberry Pi2, the new RTTI works flawless.

The core developers are divided on the approach to take. Hence it is not yet included. The code is highly non-portable, which means that each platform must be maintained separately. This means additional pressure on people who maintain these other platforms.

The idea is to use a manager approach (as for instance for threads, locale, widestrings) using a libffi wrapper as fallback for platforms for which no native manager is available. (see https://sourceware.org/libffi/)

However this means the RTTI info must be written in a format that can be fed directly to libffi. (or using a conversion routine). In each case, additional work is needed.

My - and Florian Klaempfl's, if I understood correctly - idea is to include what exists, and then modify the code as needed so it can work with libffi. Others fear that this will mean the ffi wrapper will not get written anytime soon.

Michael.

Michael Van Canneyt wrote

My - and Florian Klaempfl's, if I understood correctly - idea is to include what exists, and then modify the code as needed so it can work with libffi. Others fear that this will mean the ffi wrapper will not get written anytime soon.

The entire code will have to be rewritten to add support for any other platform regardless of whether libffi is used or not. I'm not even sure if the current code can work for ARMHF and floating point parameters (which have to be passed in vector registers).

Again: it has nothing to do with libffi in particular, and everything with not being generic (although if it's generic, that will obviously also help a manager/libffi implementation).

Jonas

On Sun, Mar 22, 2015 at 6:38 AM, Michael Van Canneyt <

It seems that somebody implemented that in FPC:

https://code.google.com/p/la-pe/

Not tested yet.

True. I just used libffi as one way out using a "generic approach".

Michael.

As I see it there is no generall problem with my approach. While the location(paramgr), the interface(get_para_regoff) and the stored data(reg: Byte; off: LongInt) are worth some discussion, I think whatever kind of manager like approach will be taken, will turn out to be not realy different from what I already did.

procedure get_para_regoff(proccalloption: tproccalloption; paraloc:
pcgparalocation; out reg: Byte; out off: LongInt);

Currently the default implementation does nothing but zero the reg and off. This would be the place for a potential libffi implementation or any "generic" solution chosen.
While leaving the possibility of specific corrections if need be by overriding get_para_regoff.

Maybe it is just me missing the bigger picture, so please correct me if I am wrong.

mfg Steve

Steve Hildebrandt wrote on Mon, 23 Mar 2015:

The encoded information cannot be specific to a particular kind of consumer. The encoding format should be sufficiently generic to be able to represent the data required for all platforms, and to provide all the information required by any implementation that will use this information to call routines. The purpose of "manager" units is plug & play: you just add one to the uses clause of your main program, and it works. It cannot require compiling the entire FPC source tree and all used units with a particular command line option.

One example: on some PowerPC64 ABIs, if the last bytes of a record do not fill an entire (64 bit) register (e.g. because 5 bytes are left at the end), they must be passed in most significant bits of that register. In the compiler this is encoded as a "shift value" for that parameter. So you need to be able to encode these shift values in the RTTI.

Custom-encoding everything in a platform-specific way into these reg and offset fields (which then wouldn't have any relation to actual registers or offset values anymore in many cases) would lead to requiring platform-specific decoders in the RTL, which I very much would like to avoid.

I'm also not certain whether over time 255 registers will be enough if you also consider vector registers and all kinds of sub registers that are possible there (single precision scalar, double precision scalar, quad precision scaler (?), 32 bit vector (?), 64 bit vector, 128 bit vector, 256 bit vector -- all times the number of vector parameter registers).

Jonas

I agree that we need a better format as said above this is worth discussing, but unless someone knows all the edge cases we are unable to design the data format.
The only thing that comes to my mind is either have different RTTI data per platform and generic access, or store all information that is neccessary for all supported platforms.
Adding the manager to the main program, does only one thing it moves the information generation from compile to run time(not so good), while you gain no advantage?
An arm abi manager will be of little use in a win64 program, so there are only 2 implementations of importance fallback and win64. Also there is no aditional cmd option we realy need, since we know what platform we target while generating the RTTI.
Or are you reffering to compilerswitches related to how much RTTI data is generated?

In order to support this you would simply need to open up the interface to suit your needs, e.g. add an additional parameter that defaults to some value if it is of no use.
If another field shift is needed, we can either encode it somhow, store it, or generate it at runtime.

if we know we may have more than 255 registers nothings stops us from using a QWord or whatever we need.
This kind of RTTI will need to be interpreted no matter what, all we can do is make this a simple process.
Since supporting things like TValue.invoke, will need an assembler call frame that is written once per platform(if you are lucky you may be able to reuse parts).

As I see it:
we have platform specific data
we need a method for writing this data into the generated RTTI (encoded or raw)
we need a gracefull way of handling not generated data this should happen at compile time

either :
TABIData = record
   {$IF defined(i386) or defined(x64)}
   reg : Byte;
   off : Integer;
   {$ELSE} {$IF defined(ppc64)}
   reg : Byte;
   shift : Byte;
   {$IFEND}
   {$IFEND}
   ....
end;
or:
TABIData = record
case platform of
   i386, arm, x64: (
     reg : Byte;
     off : Integer;
   );
   ppc64: (
     reg : Byte;
     shift : Byte;
   );
end;

mfg Steve

I'll create a branch and commit the supplied patches so far, we can use this branch to bring things into a reasonable shape.

Steve Hildebrandt wrote

Here ist the current state of affairs for i386 only right now. The Patches are currently based on

I've created an svn branch for the patches: http://svn.freepascal.org/svn/fpc/branches/interfacertti (if anybody is interested to contribute, just drop me a note so I can create an account with write access to it). I plan to commit the patches Steve submitted so far to this branch. However, I am missing the 0004-... .patch. Did I overlook something or is there simply no patch with number 0004?

Here are all changes I made again.
I would like to have an account.

mfg Steve

Please do include me !

I have included the rtti-patch (diff) based on the current (today :
30566) trunk.

Its nothing else than the patch-set from Steve, but with one change:

for k:=0 to def.paras.count-1 do
begin

     para:=tparavarsym(def.paras[k]);
     {note from Alf:}
     {this was in the first patch set by Steve}
     {left out by Steve in the current patch}
     {?}
     {
     if (vo_is_hidden_para in para.varoptions) and not
        (vo_is_self in para.varoptions) then
        continue;
     }

Thanks, Alf.

This was made, when I started actually using the generated RTTI. It was kind of problematic to invoke a method, if you have no information how to pass the self parameter along.

mfg Steve

I think something got mixed up here.
I got confused and didn't look into things properly, sorry. The Patches are alright as they are, but the unified diff seems incorrect.

+ if (vo_is_hidden_para in para.varoptions) and not
+ (vo_is_self in para.varoptions) then
+ continue;

should be included see "[PATCH 2/6] Added base for Reg/Off calculation to paramgr"

I've also attached a simple test.
If you would rather have a unified diff, I can provide it.

mfg Steve

Everything works as expected now (my programs, as well as your recent TestInterfaceRTTI.lpr), if
+ if (vo_is_hidden_para in para.varoptions) and not
+ (vo_is_self in para.varoptions) then
+ continue;
is included in the ncgrtti.pas patch !!
Without it: no success.

I think it would be good to provide a working/correct unified diff against trunk.

With this message, I have included a diff of your patches. Could you please check ?
If correct, this could be the starting point for
http://svn.freepascal.org/svn/fpc/branches/interfacertti

Greetings.

The RTTI branch is included with fpcup !

https://github.com/LongDirtyAnimAlf/Reiniero-fpcup/

See fpcup.ini. Getting FPC with the new RTTI is now as simple as:

fpcup --fpcURL="RTTI" --only="fpc"