Hi all,

The safecall calling convention is now implemented for i386/linux in fpc-trunk. The next step to get XPCom working now is to update the QueryInterface (and AddRef and ReleaseRef) definitions in IInterface.

Problem is that they are defined as 'stdcall' but to make them work on Linux/i386 that should be 'cdecl'. So that will change in the near future. (using an ifdef, so it's stdcall on windows, cdecl on all others)
This will break backwards compatibility on non-windows systems for users who implement their own QueryInterface, AddRef and ReleaseRef methods! But it's easy fixable and the compiler will complain.

The problem that then still remains is that the IID parameter of QueryInterface has to be passed by reference. Now it's defined as 'const' so it won't be passed by reference in Linux (cdecl).

Now I see three options: pass all records with a size bigger then the register-size by reference for all cdecl-params which are defined as 'const'. The same happens for stdcall parameters on Windows and this is suggested before by some people. This should not lead to any backwards compatibility problems, because when 'const' is used right, it leaves the decision to pass by reference or not to the compiler. So when this changes, there shoudn't be any problem.
But this is also the weak part of this solution: with this change the developer will use 'const' to make the compiler pass the variable by reference, which should be free for the compiler to decide.

(In other words: it will make the usage of 'const' parameters for connecting with external (c) programs very difficult, because the meaning of const will be difference than how c-compilers will handle it)

Second option is to change the 'iid' parameter of QueryInterface from 'const' to 'var'. This is not a really option, though, because then you can not pass constant values to QueryInterface anymore. Which means that you also can not pass an IInterface (and thus it's GUID) to it directly.

This leads to the third option: introduce a new parameter-type which will always pass the variable by reference, and will be handled as being constant by the compiler. We could name it 'in', for example.

First and third option are possible. I would choose for the third option because this will also settle these kind of problems for the future. (This idea is also mentioned on this list before by others)

Or are there any considerations/arguments that I missed?

Joost

In our previous episode, Joost van der Sluis said:

The weak part of this kind of reasoning is that first the application crashes, and only a week the developer finds the reason. If he is skilled enough. Nearly all problems I had when porting big Delphi packages to FPC turned to be hidden assumptions like this, or string conversion assumptions.

Of course, most of those are solved now, but this could produce similar problems on systems with multiple compilers that adhere to cdecl semantics.

(In other words: it will make the usage of 'const' parameters for connecting with external (c) programs very difficult, because the meaning of const will be difference than how c-compilers will handle it)

The problem is that currently we use the calling convention to decide all aspects of the ABI, not just calling convention. This is specially a problem on systems where the ABI is not rigidly followed by everybody like on Windows.

The only solution is to fully parameterise the calling process (which registers to use, how and what const means const, how to use the stack, what to do with floating point (skip int regs for them or not) etc. This might also allow to kill som assembler from the RTL, since you can do much more that defies the standard procedural regime in Pascal.

I've seen this in compilers before, but they didn't have the mass of parameters that FPC has. I don't know if it is doable. (and even if, it is probably a _major_ rewrite)

Second option is to change the 'iid' parameter of QueryInterface from 'const' to 'var'. This is not a really option, though, because then you can not pass constant values to QueryInterface anymore. Which means that you also can not pass an IInterface (and thus it's GUID) to it directly.

Getting back to the practical, if you really need to force byref, we need a way to force byref, period. Delphi compatible or not. A way to keep it delphi compat might be using a directive. (e.g. to specify the minimal size of byref passing)

Because now cdecl is tuned to this application, but potentially hampering or breaking compatibility with others. And tomorrow a bugreport will be filed for another.

Yes, you are completely right. That's why changing the queryinterface definition from stdcall to cdecl isn't really a problem, because the compiler will complain. This change will lead to hidden results.

And only usable for professionals who know what they are doing. And aren't you exaggerate it a bit? (schieten met een kanon op een mug) I only need a forced pass-by-reference.

Hmm... could be an idea. But then you can't manually decide which paremeter should be passed by reference and which one not. And isn't the suggested 'in' parameter type much cleaner in it's use?

Because now cdecl is tuned to this application, but potentially hampering or breaking compatibility with others. And tomorrow a bugreport will be filed for another.

For example? I don't see where adding 'in' will lead to problems, except that the code won't compile on Delphi anymore?

Joost

That's not true for cdecl. For cdecl, "const" means "pass the same way as if it was declared as 'const' in C".

This leads to the third option: introduce a new parameter-type which will always pass the variable by reference, and will be handled as being constant by the compiler. We could name it 'in', for example.

"in" is not a good name, it does not convey any information to the reader about how the parameter is passed (and since the whole point of introducing such a modifier is changing the way the parameter is passed...). Past suggestions have included "const var" (http://bugs.freepascal.org/view.php?id=7612), "constvar" and "constref".

Another option is to declare it as "mwpascal" instead of "cdecl". The mwpascal calling convention is identical to cdecl, except that all structured const parameters are passed by reference (for compatibility with the MetroWerks Pascal compiler).

Jonas

I would say 'constref'. But what do you thing of the idea from Marco to use a directive? It will avoid problems with Delphi-compatibility.

http://www.hu.freepascal.org/fpcircbot/cgifpcbot?channel=fpc&fromdate=2010-08-28&todate=2010-08-28&linecount=50&fromtime=14%3A20&totime=15%3A00&sender=&msg=

Hey, that's the easiest for me, now. No need to change the compiler. But i'm not sure that this is the best solution for the long run.

Joost.

Joost van der Sluis wrote

I would say 'constref'. But what do you thing of the idea from Marco to use a directive? It will avoid problems with Delphi-compatibility. http://www.hu.freepascal.org/fpcircbot/cgifpcbot?channel=fpc&fromdate=2010-08-28&todate=2010-08-28&linecount=50&fromtime=14%3A20&totime=15%3A00&sender=&msg=

Probably being predictable again: I don't like the ability to change the meaning of keywords using switches like that. It makes code harder to read, because you have to know which switches are active before you know what the code actually means. In case of mwpascal, the modifier that influences the behaviour is right next to the declaration, so the context is clear (unless you use macros, but it's possible to obfuscate everything using that feature).

> Another option is to declare it as "mwpascal" instead of "cdecl". The mwpascal calling convention is identical to cdecl, except that all structured const parameters are passed by reference (for compatibility with the MetroWerks Pascal compiler). Hey, that's the easiest for me, now. No need to change the compiler. But i'm not sure that this is the best solution for the long run.

My second preference would be something like constref.

Jonas

In our previous episode, Jonas Maebe said:

> http://www.hu.freepascal.org/fpcircbot/cgifpcbot?channel=fpc&fromdate=2010-08-28&todate=2010-08-28&linecount=50&fromtime=14%3A20&totime=15%3A00&sender=&msg= Probably being predictable again:

Don't read too much into that.

I don't like the ability to change the meaning of keywords using switches like that. It makes code harder to read, because you have to know which switches are active before you know what the code actually means.

It is meant to avoid a heavy ifdefed solution which is IMHO even worse.

It is also meant to avoid the combinatorial aspect. For calling conventions you need a separate one for each detail deviation. Or you need to introduce separate modifiers for separate aspects of the calling convention.

This includes an ability to keep code depending on current cdecl semantics running.

Needs ifdef in every prototype that needs to remain Delphi compat, and that IMHO makes it an hard sell.

That should not be required here. Afaik, only AddRef() and ReleaseRef() need it. All other routines are safecall, which can be used in FPC (after Joost's patch) and on Delphi.

It is also meant to avoid the combinatorial aspect. For calling conventions you need a separate one for each detail deviation. Or you need to introduce separate modifiers for separate aspects of the calling convention.

A constref parameter is a different kind of parameter, just like a value, var and out parameter (basically the equivalent of a "typename const *" parameter in C). A header can perfectly contain both plain "const" and "constref" parameters of the same type passed to routines using the same calling convention.

Jonas

In our previous episode, Jonas Maebe said:

The main reason is that I doubted using a new syntax element to describe interfaces that need to be implementable in a Delphi compatible way without ifdefs.

The alternative, a $directive to set the constref situation can be also used globally , and can be added to the ifdef FPC section in some global section, and also usable for other cases than specifically this case.

I was thinking more of cases where a bit of control over structure ref/value passing might avoid changing heaps of headers of Delphi origin to (slightly incompatible) pointer syntax if some discrepancy in how structures are passed is found on this or that platform.

To fix problems where value passing is expected, but not done, there even is no workaround in FPC.

More related to the other side of changing what calling convention modifiers mean every so and so often, and want to at least provide the possibility (even if it costs work like global directives) for users to keep the old situation running.

These were my two main concerns in suggesting a directive, and also try to have a way to allow users to fix/workaround such issues without having to migrate to trunk all the time (if a such a directive reach mainstream in the future)

Anyway, it was my opinion on the matter, and I still think it is the better one over const ref. (or how it is named)

What does stdcall calling convention currently do on non-windows platforms ?

Because if it is something we decided, it simply means that we should change it's meaning to be identical to mwpascal: cdecl calling convention, and const structs passed by reference. If I'm right, 'stdcall' currently has no meaning on non-windows platforms other than the one we put on it.

It solves all your problems in one go, plus it makes writing header translations a lot easier. (Pending the question of what the C compiler does with const structured C parameters)

If that is not possible, then I think Marco's idea of a switch is best. I see it as similar to the {$packrecords } directive: it doesn't change the syntax or meaning (as Jonas fears), but does instruct the compiler how to handle things on a more low-level (in casu: assembler) scale.

I think that having to write

   {$ifdef windows}const{$else}constref{$endif} myarg: mytype

for possibly thousands of arguments in an interface description is simply not an option.

Michael.

Michael Van Canneyt wrote

What does stdcall calling convention currently do on non-windows platforms ?

The same as on Windows.

I think that having to write {$ifdef windows}const{$else}constref{$endif} myarg: mytype for possibly thousands of arguments in an interface description is simply not an option.

That would not be necessary:
a) constref would be correct both for Windows and non-Windows b) I'm not aware of a single routine that uses stdcall on non-Windows, other than the AddRef and ReleaseRef routines mentioned at the start of this discussion

If you want to apply a custom calling convention to an entire file, that is also already supported using {$calling xxx} (which I'm not very fond of either).

Jonas

In our previous episode, Michael Van Canneyt said:

Because if it is something we decided, it simply means that we should change it's meaning to be identical to mwpascal: cdecl calling convention, and const structs passed by reference. If I'm right, 'stdcall' currently has no meaning on non-windows platforms other than the one we put on it.

mwpascal is still per declaration and Delphi incompatible. And keep in mind that any implementation of a IUnknown must repeat any modifiers in the original source (including const ref and calling conventions) regardless of what option is chosen.

That also goes for directives, but, if selective enough (e.g. for calling convention) modifier, they can be unit global, or even in practice application global (includefile)

If that is not possible, then I think Marco's idea of a switch is best. I see it as similar to the {$packrecords } directive: it doesn't change the syntax or meaning (as Jonas fears), but does instruct the compiler how to handle things on a more low-level (in casu: assembler) scale.

Good analogy, exactly what I meant:
1. more variants possible
2. not per declaration individually.

I think that having to write {$ifdef windows}const{$else}constref{$endif} myarg: mytype for possibly thousands of arguments in an interface description is simply not an option.

Note that I don't 100% surely know if this will happen in this particular case, specially if safecall meaning is modified. (IOW Jonas wants to do that to avoid this)

It is just that a $switch will give some leeway in the future for similar cases.

I made a mistake here: there in fact some differences between stdcall on win32 and on other platforms. But as mentioned before, I'm not aware of anyone wanting to use stdcall on multiple platforms other than possibly for the addref/releaseref methods of iunknown.

Jonas

But we are under no obligation to have this so ?

Does a C compiler (gcc) under linux understand 'stdcall' ?

Yes, but is unusable in the cases where source code is shared between Delphi and FPC, such as the Gecko sources.

I simplified the example, maybe the following would have been more clear:

{$ifndef fpc}const{$else}constref{$endif} myarg: mytype

I implicitly assumed that FPC would be used for any non-windows platform.

b) I'm not aware of a single routine that uses stdcall on non-Windows, other than the AddRef and ReleaseRef routines mentioned at the start of this discussion

Well, there are in the Gecko interface:

sEnumerators.pas:    function HasMoreElements(out _retval: PRBool): nsresult; stdcall;
nsEnumerators.pas:    function GetNext(out _retval: nsISupports_std19): nsresult; stdcall;
nsErrorUtils.pas:                aStringBundleURL: PAnsiChar): nsresult; stdcall;
nsErrorUtils.pas:                aErrorModule: PRInt16): nsresult; stdcall;
nsErrorUtils.pas:                out aResult: PAnsiChar): nsresult; stdcall;
nsErrorUtils.pas:                aStringBundleKey: PAnsiChar): nsresult; stdcall;
nsErrorUtils.pas:                aError: nsresult): nsresult; stdcall;
nsErrorUtils.pas:                out aResult: PAnsiChar): nsresult; stdcall;
nsInit.pas:    function CallMethod(aMethodIndex: PRUint16; const aInfo: XPTMethodDescriptor; aParams: PXPTCMiniVariantArray): nsresult; stdcall;
nsInit.pas:  XPCOMExitRoutine = function : Longword; stdcall;
nsInit.pas:  nsIDirectoryServiceProvider_stdcall = interface(nsISupports)
nsInit.pas:    function GetFile(const prop: PAnsiChar; out persistent: PRBool; out AFile: nsIFile): nsresult; stdcall;
nsInit.pas:                                  nsIDirectoryServiceProvider_stdcall)
nsInit.pas:    function GetFile(const prop: PAnsiChar; out persistent: PRBool; out AFile: nsIFile): nsresult; stdcall;
nsThreadUtils.pas:    function acceptEvent(aEvent: nsIRunnable): PRBool; stdcall;
nsXPCOM.pas:                                out aWriteCount: PRUint32): nsresult; stdcall;
nsXPCOM.pas:                               out aReadCount: PRUint32): nsresult; stdcall;
nsXPCOM.pas:    function Alloc(size: size_t): Pointer; stdcall;
nsXPCOM.pas:    function Realloc(ptr: Pointer; newSize: size_t): Pointer; stdcall;
nsXPCOM.pas:

If you want to apply a custom calling convention to an entire file, that is also already supported using {$calling xxx} (which I'm not very fond of either).

But that is only if none is specified explicitly ?

In each case, a question to Joost is: Why does he need to change IInterface calling convention ? I see no code in the Gecko/XPCom units that would require this ? Because if it doesn't need to be changed, the whole discussion is academic.

Michael.

I meant people write 'stdcall', but the same semantics as 'mwpascal' are used by the compiler, since stdcall is a windows calling convention construct, invented only by Microsoft for backwards compatibility with early versions of Windows. (to my knowledge, that is)

_if_ stdcall is not used by any compiler on non-windows, then we can assign whatever meaning to it that we please, which would in this case be the same meaning as mwpascal.

But I still fail to see why Joost needs all this.

His change would mean that the GCC compiler does things which we currently know nothing about, namely it pushes const parameters by reference in some cases. In casu: What is the C declaration for the QueryInterface call ?

It is defined in FPC as:
function QueryInterface(const iid : tguid;out obj) : longint;stdcall;

But what is QueryInterface as defined in C ? Does C have a tguid type ?

These questions should be answered prior to changing anything in the compiler.

Michael.

Michael Van Canneyt wrote

His change would mean that the GCC compiler does things which we currently know nothing about, namely it pushes const parameters by reference in some cases.

No, the problem is many people (wrongly) use "const" in Pascal as an equivalent for "type const *" (= the unclean "const type *") of C, mostly for lack of a better alternative.

Jonas

Michael Van Canneyt wrote

In each case, a question to Joost is: Why does he need to change IInterface calling convention ? I see no code in the Gecko/XPCom units that would require this ? Because if it doesn't need to be changed, the whole discussion is academic.

If I understood that right, he wants to change it because XPCom seems to use cdecl on non Windows systems (I can neither confirm nor deny it).

Regards,
Sven

In our previous episode, Michael Van Canneyt said:

Afaik in the SDK there is some macro REFIID, even on IDL level.

which is define in the MS SDK as

#ifndef _REFIID_DEFINED
#define _REFIID_DEFINED
#ifdef __cplusplus
#define REFIID const IID &
#else
#define REFIID const IID * __MIDL_CONST
#endif
#endif

and mingw seems to follow that more or less, if I understand the murky code a bit.

In other words, by value semantics (const TGUID and not PGUID) is a borland invention.

So looking to GCC or even MSVC won't help us here. At least not in the obvious place (the iunknown declaration, I checked it is same in SDK and mingw w32api)

When in fact they should declare it as

const P : ^type

? (knowing fully that this is semantically not quite the same)

This of course means that they must always provide the @ when calling this method.

In that case, to be more clean, I think it is better to simply correct the declarations and not to mess with the argument difficult problem in the case of the interface, since it would require adapting existing pascal code to match C semantics. Maybe your proposal is not so bad after all, using "constref"; at least that would make it very explicit.

Michael.

I've read the other mails but respond to this one.

That was the first thing I though about, changing the meaning of stdcall on non-windows platforms. But I find that kinda messy. Now 'stdcall' is mainly defined as being callee-clean-up, parameters pushed right-to-left. To make this dependent on the used OS is quite messy imho.

Otoh, this is exactly what I did for 'safecall'. But 'safecall' is slightly 'defined' as the COM/OLE-calling convention from Borland. Then it is not that strange to make it XPCom compatible on non-Windows. (Even if that changes the calling convention from callee-cleanup to caller-cleanup)

Btw: I think I know why there are those differences between stdcall on Windows and Linux that Jonas found. My guess is that Borland changed the stdcall convention on some places to make live easier for them. (For example: pass some records by value, because the WinApi demands that.)

There is no need to add an {$ifdef windows} to the definition of QueryInterface. Because on windows/stdcall record-parameters with a size>pointersize are already passed by value. (Borland did this probably just for this case) So when we use 'constref' effectively nothing changes on Windows.

The problem with the {$ifdef}'s does only exist for other header conversions. (For example the Gecko headers)

And Michael, this is all important for Gecko because all objects are passed as interfaces. And Gecko uses QueryInterface to 'cast' one interface to the other, so QueryInterface is called by Gecko a lot. Then these calls have to be binary compatible.

Changing the definition of QueryInterface to something completely different is also hard, because now you can do:

  Object.QueryInterface(IInterface, AInterfaceInstance);

This is used a lot...

Conclusion(?)

I think that adding 'constref' is the best way to go. It's more 'pascalish', imho. And it has a clear meaning and use, namely converting 'typename const *' definitions from C-headers.

If we want to have flexible ways of determine what to do exactly on a function call, we have to design something different. (Which is maybe not worth the trouble.)

Only disadvantage is that users have to add {$ifdef} statements if they want to keep their header-files compilable on Delphi. In my opinion this is a problem from Embarcadero. If we add new features to Pascal and they don't, there's just nothing we can do about that. And not a reason to come up with some syntax to avoid this incompatibility.

Did I now answer all questions to me?

Joost

Like I wrote in my last reply to Jonas: I agree that this is the best way to go.

It just means that the gecko interface will need some {$ifdefs} because Delphi does not support constref.

Michael.