I can only assume that you simply have no idea about how and for what Attributes are used.

He's not alone :-) .

I used RTTI to avoid defining an additional constant array of strings to have a unit output debug information for enumerated types. Otherwise I never did see any use of this.

Could you give a real world example ?

-Michael

Sorry, I don't understand anything of this.

(I did not see any use for interfaces in my work yet, either.)

This seems to be a completely different world from mine (as a programmer for mainly "embedded" projects).

I once used variants for a project that accessed a database.

-Michael

I also think you don't recognize sarcasm/scepticism when you encounter it.

IMHO, sarcasm should not be used in a forum, especially if non native language members are to be considered. They are easily fooled involuntarily.

-Michael

Hi,

I'm wondering if anybody has sufficient knowledge in this (I don't). Many developers that moved over to the dark side (.NET) using Delphi.Net or C# always rave about .Net Reflection and how much better it is to RTTI.

Anybody actually know what is better? What limits are they talking about in RTTI?
I have used RTTI extensively in my work and I don't seem to have any issues with it, but clearly I am missing something!

I've also heard of people like Joanna Carter (well know in Borlands OODesign newsgroup) that has used Delphi (win32) and wrote her own meta-framework instead of using RTTI to overcome some of the issues with RTTI?
Would it be beneficial to investigate this and maybe implement such a new meta-framework directly into FPC if the need is there?

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

RTTI in native Delphi / FPC is just the pure data structures that the compiler generated and by and large you are on your own to decipher them once you got a pointer to it. Reflection in .NET on the other hand is a higher level object oriented Framework which sits on top of the actual metadata and gives relatively easy and comfortable access to that information.

As far as the information itself that's available through RTTI / Reflection is concerned, the type information available in .NET is truly complete, down to every field and every method of every type. Also, the ability to associate any number of custom attributes with any of the entities that make up the type information in .NET is a very powerful tool.

Another huge advantage of Reflection in .NET over RTTI as provided by Delphi and FPC is that given an instance of an object and a MethodInfo object describing any of the accessible methods of that object it is trivial to call that method using MethodInfo.Invoke(Instance, [Array, Of, Parameters]). (Access to fields, properties and event is similar easy using the respective *Info objects).

The combination of this (truly complete type information, easy access, custom attributes) makes it rather trivial to e.g. tag private fields of a class with the [Serializable] attribute and use Reflection to serialize the class to/from a stream.

Cheers,
Thorsten

You forgot to complete: (...) but 99% of time the it just makes for useless bloat ;-)

Cheers,
Flávio

You forgot to complete: (...) but 99% of time the it just makes for useless bloat ;-)

The type information required to make this work is required anyway to a) support the compilation of IL into machine code b) to allow verification of type safety and c) support the garbage collector. As such there is no bloat whatsoefer in the context of .NET from supporting Reflection.

Cheers,
Thorsten

Apart from the custom attributes and the Invoke, I don't see what there is extra ?

The 'easy access' can be coded in a single day, and could be made available in TPersistent. It's on my agenda since years.

And I highly doubt the usefullness of the custom attributes. In their generality, it's not usable. If you want to make it usable, you'll have to write code anyway, so it's no different from custom written code. The only use for this is maybe storing DB persistence information.

So if Graeme writes the 'Invoke' then we're all set.

Michael.

Michael Van Canneyt wrote

to write code anyway, so it's no different from custom written code. The only use for this is maybe storing DB persistence information.

Ok, maybe I should have made the context more clear as well. My primary use for RTTI is in a Object Persistence Framework and a Presentation Framework for my business objects.

So if Graeme writes the 'Invoke' then we're all set.

If only I knew how and where to begin. :)

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Somehow I guessed :-)
Why do you think I mentioned it in 1 mail together with a task for you ? :-)

Easy: look up the method in RTTI - you'll get the address. Push all arguments on the stack (see Remobjects/Innerfuse PascalScript on how to do this, you need some assembler for it), and return result value.

2 days, tops.

Lots easier than writing a MGM or MVM (heck, don't remember all these acronyms ;)) for tiOPF :-)

Michael.

Apart from the custom attributes and the Invoke, I don't see what there is extra ?

No usable RTTI for records
No usable RTTI for arrays
No RTTI for private, protected and public(!) members (only published) No usable RTTI for published methods (param and return types are just given as strings, not as references to other type infos, no information about calling convention)
No usable type information for published fields (only a name to offset mapping is given, no type information at all)
And so on...

The 'easy access' can be coded in a single day, and could be made available in TPersistent. It's on my agenda since years.

Implementing easy and comfortable access to at least the information that is available is going to take a lot longer then "a single day". And I never said it couldn't be done. But it's a simple fact that with the reflection API in .NET you can get absolute any information you could get from reading the interface section of the relevant source code with just a couple of lines of code, whereas with Delphi/FPC RTTI and and the helper functions and record definitions for it that you have to search together all over the RTL source you can only get a fraction of that information and only with significant research and writing of code.

And I highly doubt the usefullness of the custom attributes. In their generality, it's not usable. If you want to make it usable, you'll have to write code anyway, so it's no different from custom written code.

No different?? Custom attributes allow me to associate pretty much any form of custom data with types and their members and discover this data at runtime without having to write code which has to get executed at some time to add entries into some form of list.

Instead of having to put code into unit initialization sections to build up some form of list which associates some type with some additional information which has to run during application startup and is almost guaranteed to have to allocate memory from the heap to put that information with Custom Attributes the compiler can directly bake your custom information into the type information it already generates during compile.

Much cleaner approach. Much less overhead both in code execution and memory.

The only use for this is maybe storing DB persistence information.

I can only assume that you simply have no idea about how and for what Attributes are used.
Take a look here to get some idea about the uses for attributes in just the core framework: http://msdn2.microsoft.com/en-us/library/2e39z096.aspx

Cheers,
Thorsten

> Apart from the custom attributes and the Invoke, I don't see what > there is extra ?

Sorry, forgot: No RTTI whatsoever for pointer types.

Cheers,
Thorsten

Note that all this can be remedied (and better) by a simple code generator (e.g. codesmith), as long as you alone control the project.

The .NET framework however also needs these attributes to interact with Windows server components as IIS, and I can also imagine this, when used sparingly applies to other server component vendors (hi Thorsten) too.

Then the codegeneration approach fails because you can't recompile certain parts. (though you could generate listener info in many cases)

And breaks every other FPC release to due bug fixes/changes in calling conventions, and needs to be ported to at least the default calling convention for every supported target.

2 days, tops.

You basically have to re-implement 80% of compiler/<arch>/cpupara. There's no way you can do that in two days. Not to mention that it's a horrible and completely unsupported hack.

Jonas

Op Tue, 20 Nov 2007, schreef Marco van de Voort:

Ok, but the question to be asked is why would you want to write a code generator if the compiler can generate usable RTTI for you?

If you want you can say the whole concept of RTTI is nonsense, since you can make those data structures yourself. However, the fact that the compiler generates them, saves you the work of writing and maintaining those RTTI data structures.

The .NET framework however also needs these attributes to interact with Windows server components as IIS, and I can also imagine this, when used sparingly applies to other server component vendors (hi Thorsten) too.

Attributes are IMO syntactic sugar for:

const datatype1_attributes:Tdatatype1_attributes=( rtti:typinfo(datatype1),
attribute1:value1,
attribute2:value2,
attribute3:value3

      );

I don't see why you would need dynamically allocated data structures, after all, if you put them into the language, they are static as well. The extra work and maintenance argument for normal RTTI doesn't hold for attributes since attributes as language feature need to be written and maintained as well.

Syntactic sugar might still be a good thing, but then it has to be a common annoyance for developers; it doesn't make sense to enhance the feature with features that are seldomly used. This criterium IMO isn't the case here.

Daniël

You are funny.

First, the RTTI can only give you the address of published methods. And then, have you actually ever looked at PascalScript?? What do you think why PSUnitImporter.exe is needed to parse all units that contain types which you want to access from PascalScript and build import units which build up all the additional type information that's missing in RTTI and create wrapper code to make this work?

Have you ever tried to write the code that's required to dynamically call a method given reasonable complete type information? I have and it's not exactly trivial.

From the lack of anything compareable to InvokeRegistry in the FPC RTL

source I assume that FPC currently doesn't implement interface RTTI which is the closest Delphi comes to actually offering usable RTTI for implementing something like MethodInfo.Invoke. But even in that case where you DO have PTypeInfo references for the params and result value and you know the correct calling convention you are severly restricted by the general limitations of RTTI, no usable info for record parameters, static arrays, open arrays or pointers of any type. Also, the Delphi compiler is broken when it comes to forward declared types so that you won't get any usable type information if you have 2 interfaces which use each other in their members so that you have to forward declare on of them.

Even ignoring custom attributes there are at a guess at least a couple of weeks work on the compiler and RTL between what Delphi/FPC currently offer and anything that comes even close to what the .NET reflection API currently does.

I'm not saying any of this is impossible, but it's a far cry from the "single day" or "2 days, tops" you are talking about. And especially implementing and maintaining something like a dynamic invoke for all supported platforms is a huge undertaking.

Cheers,
Thorsten

One could prostitute the dwarf debuginfo for this ;)

Marc

RTTI is only half (you need to write the RTTI inspecting code anyway), the generated code is well integrated, and because there is an higher abstraction level you can perform checks.

If you want you can say the whole concept of RTTI is nonsense, since you can make those data structures yourself. However, the fact that the compiler generates them, saves you the work of writing and maintaining those RTTI data structures.

I'm not against RTTI, but it simply must not be overrated.

In a statically compiled language, yes. .NET probably pulls them out of the JIT context for the relevant assembly when needed. Can do an extra JIT pass even if it needs them.

I don't know what you are getting to with the rest of your msgs.

Granted.

No RTTI for private, protected and public(!) members (only published)

Private and protected are not your business in the first place, exposing it through RTTI or whatever is bad design. Public, OK.

No usable RTTI for published methods (param and return types are just given as strings, not as references to other type infos, no information about calling convention)

As far as I know, this is generated ?

No usable type information for published fields (only a name to offset mapping is given, no type information at all) And so on...

Huh ? of course you know the type, how else can we stream it ?

So: they needed this information for their CLR or whatever, and they decided to make their own API available through some system calls.

You don't need this with RTTI either, if coded correctly.

I think you code wrong, because all this is not needed :-)

I have seen it and didn't see anything of practical value for me.

I'm sorry, I am not convinced. To me, they just provided the info they need to make their runtime engine tick. Big deal; I don't use their engine in the first place.

But I still fail to see any practical value of this for me as an application programmer.

Unless I am intimately acquainted with the use of all these properties and types and whatnot, there is nothing I can do with this information. (DB storage excepted, and for this you need more info than available through introspection anyway.)

It's the same as the SOAP story, whole pages of "introspection" specs: Absolutely useless. You must know the exact meaning of each type/parameter, and when to set it and what the effect is this action has. You'll always need knowledge which is not available through introspection, thus rendering the very idea pretty much useless...

Ockham's razor leaves nothing from all this. The benefit of this is - in my eyes - marginal for the application programmer.

Michael.

Maybe, but I also think you don't recognize sarcasm/scepticism when you encounter it. Next time I'll put it in big letters in front of it, so it's more clear.
(Obviously I'm the only one seeing it, and that is a serious remark...)

Let's make something clear:

I'm not trying to convince anyone here. I just am absolutely not convinced that all this has any practical use whatsoever other than for people writing a runtime engine.

Hence the scepticism.

Oh well, I must be more clear. Lesson learned.

Michael.

Private and protected are not your business in the first place, exposing it through RTTI or whatever is bad design. Public, OK.

The class itself should have access to that information. It can then use it to e.g. stream the contents of all/some of it's private fields to/from a stream.

If yes then it's not documented in typinfo.pp:

            tkMethod:
              (MethodKind : TMethodKind;
               ParamCount : Byte;
               ParamList : array[0..1023] of Char
             {in reality ParamList is a array[1..ParamCount] of:
                  record
                    Flags : TParamFlags;
                    ParamName : ShortString;
                    TypeName : ShortString;
                  end;
              followed by
                  ResultType : ShortString}
              );

The streaming system itself doesn't touch the published fields at all (it streams published properties and it streams all child Tcomponents). The published fields are used like this:

Procedure TComponent.SetReference(Enable: Boolean);
var
  Field: ^TComponent;
begin
  if Assigned(Owner) then
  begin
    Field := Owner.FieldAddress(Name);
    if Assigned(Field) then
      if Enable then
        Field^ := Self
      else
        Field^ := nil;
  end;
end;

As you can see, it doesn't care at all about the actual type of the published field, it simply assumes that it's a Tcomponent or derived and that it matches. It's trivial to mislead this into assigning a TComponent instance to string variable if you want by simply having one component with a published string field with a specific name and then creating another component with that first component as owner and assigning the Name property of that component with the name of the string field...

I think you code wrong, because all this is not needed :-)

That's great. Why don't we just all go back to coding with a hex editor? Or maybe just 2 switches? The first one to select 0 and 1 and the second as a clock signal to commit the current bit value of the first one?

Thorsten Engler wrote

Take a look here to get some idea about the uses for attributes in just the core framework: http://msdn2.microsoft.com/en-us/library/2e39z096.aspx

What? Most sounds like poor hacks to get around compiler/linker limitations.

<paste>
Microsoft.VisualC.DebugInfoInPDBAttribute: An attribute applied to native classes that tells the debugger to look up field information in the pdb rather than in metadata.

Microsoft.VisualC.DecoratedNameAttribute: An attribute used by the compiler to pass the decorated name of a method to the linker.

System.CLSCompliantAttribute: Indicates whether a program element is compliant with the Common Language Specification (CLS). This class cannot be inherited.
</paste>

Half of them have "This class cannot be inherited.".

There are also useful ones though:

<paste>
System.ComponentModel.DefaultPropertyAttribute: Specifies the default property for a component.
</paste>

No, wait! We already have that! :-)

Perhaps there are also useful ones, that do reduce application and framework programmers code significantly?

Micha

Serious:

This is exactly my point: You will have to code this yourself anyway, making the whole idea of introspection superfluous because the introspection misses the semantical information which is needed to be able to code that.

Only the programmer knows which private fields must be stored and which not. You cou could store that in additional attributes, but then you would need custom code to check those attributes anyway, since no framework exists to interpret the meaning of your attributes: again error prone.

And to be able to store it optimally, he needs more semantical knowledge. You can of course do the brute force attack, but that is so hideously slow that it is not usable in practice. Again, this cannot be stored in your attributes. (probably you could, but then you're coding again, just metadata this time, for which additional custom code is needed - again.)

Scepticism/Sarcasm:
So, my personal conclusion:
Introspection just saves a bit of typing because of the easy access, but that is about it. It doesn't make code safer, because if the programmer specifies the wrong attribute it fails anyway.

Serious again:
And the easy access: I'm all for it, I have it in my agenda, as I said. That it should include maybe records and other types: No problem. (this is the RTTI scope, after all)

Metaphysical:

I'm not proposing we go back to the stone age, and use hammer and chisel to carve messages in rock tablets. But I remain sceptical of many "innovating" things. IT is very prone to hypes, so I judge each "new" thing by it's merit. For introspection and custom attributes this happens to be rather low, becuse I think it's totally missing the point since it cannot include semantical data needed to make it really useful.

And that's the last thing I will say about it, the next thing will be simply showing the easy access code. The 'better' RTTI is for the compiler people to implement.

Thorsten, if you want to reply, please in private. Sorry if I offended someone, no offense was meant.

Michael.

Do you want a real world example for what better RTTI in general is useful? Or custom attributes specifically?

For just one of many examples for attributes, just have a look here:

http://msdn2.microsoft.com/en-us/library/Aa480190.aspx

You don't need to read all of it, just search for "[" and have a look at the code snippets.

Yes, sure there are other ways how you could possibly provide that information. But custom attributes provide a simple and generic mechanism to associate runtime dicoverable information with specific types/members of type.

As for better runtime information in general, one example of what I've done myself in Delphi using Interface RTTI (which as I mentioned in another post is unfortunately rather limited) is something like this:

    function CreateInvokeWrapper(const aInterface   : InxInterface;
                                 const aInterceptor : InxInvokeInterceptor;
                                   out aWrappedIntf : InxInterface)
                                                    : HRESULT;
  InxInvokeInterceptor = interface(InxInterface)
    ['{7DC415F5-B252-457E-A57C-393CD326275C}']
    function InterceptInvoke(aMethod    : PnxIntfMethEntry;
                             aParams    : PnxPointerArray;
                       const aCallFrame : InxCallFrame)
                                        : HRESULT;
  end;
  InxCallFrame = interface(InxInterface)
    ['{6DABF6D0-4419-4074-9CF4-75145AC8A34B}']
    function CopyFrom(aParams : PnxPointerArray)
                              : HRESULT;
    function MakeCall: HRESULT;
  end;

You call CreateInvokeWrapper, passing in an existing interface and a reference to an interceptor class you implement and get back a new interface. You can call QueryInterface on that new interface as required to get other interfaces implemented by the class of the interface you originally passed in. Any call on a method of any of these wrapped interfaces ends up in a call to InterceptInvoke where you aMethod gives you exact information which interface, which method, how many params, param and result types... aParams then gives you access to the params and result which allows you to inspect and/or change them. If you want to forward the call (with possibly changed parameter values) to the inner interface you can call aCallFrame.MakeCall (but you don't have to) and can then inspect and/or change the params (might be var and out params among them) and result before returning.

Naturally, if I can do that I can also do the 2 half (proxy/stub) in isolation on different machines which I'm using to implement a remoting framework: http://nexusdbng.blogspot.com/2007/09/remoting-introduction.html

Cheers,
Thorsten

Op Tue, 20 Nov 2007, schreef Marco van de Voort:

I don't understand what you mean here. If you want to inspect types at runtime, of course you need to write inspection codes. Seems a tautology to me :)

It should also not be underrated. It can save people mountains of work.

Pascal is a statically typed language. Therefore things like record1.array1[0].field3 only work for data structures known at compile time.

Often, the data expected is known at compile time, but it is exposed by dynamic datastructures. I.e. retrieve a record from a database, it is known at compile time which fields you will get and what type they are. Yet you retrieve them using a dynamic interface.

RTTI allows one to convert dynamic data structures into static ones, without writing code for each individual stituation, here you have the mountains of work saved. I.e. you could write a procedure that converts any SQL record to any Pascal record, then use such a record with all comforts static data structures give.

Requests for RTTI on arrays and records are IMO completely logical.

Yes.

I don't know what you are getting to with the rest of your msgs.

If people are continuously writing such attribute data structures like above, it could make sense to give them some syntactic sugar. However, as I never encountered attribute data structures like above, it makes no sense to enhance the language with a feature no one need.

Daniël

<snip>

Perhaps there are also useful ones, that do reduce application and framework programmers code significantly?

In c# code I use attributes for the unit test frame work. Unittest are compiled into a dll. The unit test runner loads the Dll and considers all classes with the [TestFixture] attribute to be classes that contain test methods. All test methods have the [Test] attribute. Test methods can also be given an [Ignore] attribrute to disable testing it. Or an [Category("LongTest")] attribute with "LongTest" a free string parameter to create categories in your test suite, so that you tell the test runner to run only a selection of the tests or to skip running them.

The current fpc solution is that you need to register classes with test methods. This is an extra action.

For detecting the test methods, the name of published methods is used: if they start with 'Test' then it is considered to be a test method. Not more work, but less freedom in method name and they tend to be 4 chars longer ...

Creating categories could be done by construction different test suites, but this is more awkward then the attributes approach.

The Ignore attribute can be simulated by adding Assert.Ignore as first line in the testmethod, but this has as drawback that test setup and test teardown will be executed anyway.

All in all, I like the flexibility and the seperation of execution logic (how to execute a test in program code) and its meta (what test to execute in the attribute declaration) in c#. I think it is more productive.

Vincent

You must type the 'TestFixture' attribute also, so no extras here. Really...

Michael.

I know what is easier to write: textfixture at the class declaration or some registertest(MyTestClass) in some intialization section. It is a change in two locations. Really...

Vincent

The difference being that the compiler checks the former, the latter is unchecked. Since you wrote textfixture, you're already wrong without noticing it, and you would wonder why your test is not picked up... :-)

(just teasing, please don't take it serious)

Michael.

Better not take you seriously, because you are wrong ;-)

A. Attributes are a kind of type too, so the compile would have errored saying that TextFixture is an unknown attribute.

B: Intellisense would have helped me and I would not have type text.

Vincent

As far as I have seen from the discussion so far, attributes were free texts;
if what you say is true, then you can disregard my remark...

Michael.

IMHO it would be very nice if the FPC RTTI system could be extended to support some of these things indeed.

My applications:
- streaming: for a custom input file format (that is more or less like JSON, but has a stronger typing mechanism etc.)
- scripting: easily expose parts of my program to a scripting engine, e.g. to Lua (see also pLua, http://eonclash.com/ViewProduct.php?ProductID=26)

My wishlist:
- support for dynamic/static arrays, e.g. property Items:array of string
- support for array properties, e.g. property Items[i:integer]:string (though perhaps less useful for streaming since the range of the index(es) is not known, it is very useful in scripting)
- support for records and pointers would also be nice
- support for method invocation (scripting)
- support for calling a (parameterless) constructor instead of having to stick to NewInstance or some virtual constructor in a known base class (such as TComponent's Create)
- RTTI for public members (private and protected are internal to objects anyway; note also that fields are usually private in OOP designs)

Just my 2 cents,

Bram