Hello FPC,

In fpc 2.5.1 trunk today the QueryInterface in rtl\inc\objpash.inc is defined as:

IUnknown = interface
  ['{00000000-0000-0000-C000-000000000046}']
  function QueryInterface({$IFDEF FPC_HAS_CONSTREF}constref{$ELSE}const{$ENDIF} iid : tguid;out obj) : longint;{$IFNDEF WINDOWS}cdecl{$ELSE}stdcall{$ENDIF};
  function _AddRef : longint;{$IFNDEF WINDOWS}cdecl{$ELSE}stdcall{$ENDIF};
  function _Release : longint;{$IFNDEF WINDOWS}cdecl{$ELSE}stdcall{$ENDIF};
end;
IInterface = IUnknown;

Which after the resolved ifdefs it is compiled in Windows as:

IUnknown = interface
  ['{00000000-0000-0000-C000-000000000046}']
  function QueryInterface(constref iid : tguid;out obj) : longint; stdcall;
  function _AddRef : longint; stdcall;
  function _Release : longint; stdcall;
end;
IInterface = IUnknown;

Which breaks the compilation of source code like Zeos which expect the "iid: tguid" to be const, not constref.

I'm aware about the changes done to be XPCOM compatible, but at the price of "breaking" all existing COM code ? OK, the break is quite easy to solve, just replace const by contref, but the question is... Is needed to change it to constref in Windows platform too ?

I know Joost do the changes, but I think direct email is not a good choice unless no answer in list of course.

Iirc, in ZEOS svn this is already solved.

Well, this issue leaded to a large discussion elsewhere. I was wondering when someone else would even notice this incompatibility. A lot of alternatives have been discussed, but I coud not find any which is less intrusive.

In an nutshell: the behavior of 'const' parameters is undefined. When using cdecl the 'const' keyword is simply ignored. But we do need to pass the GUID by reference.

There are two options:

1: Change the behavior of const-parameters. That can lead to unpredictable problems which are not noticed by the compiler. It could be an option to use some setting to define the behavior of 'const', but in that case that setting have to be set for all units that do a call to QueryInterface. (ie: all code using interfaces, so it means this setting has to be set everywhere where COM code is used)

2: Change the definition to constref, so the compiler will complain if a user override the QueryInterface's definition, and can adapt the code.

Notice that until now I've only heard of code which overrides QueryInterface in the Free Pascal code itself, in the Lazarus code, ZEOS and the code from one commercial developer who also works on Lazarus/Free Pascal.

It is an incompatibility, yes. But I think it's not too intrusive and I see no alternatives...

See also:
http://wiki.freepascal.org/User_Changes_Trunk#IInterface.QueryInterface.2C_._AddRef_and_._Release_definitions_have_been_changed

Joost.

See http://zeos.firmos.at/viewtopic.php?p=12307&sid=0ac36e5e5c88d2e740abd551df7b4699#12307

Marcos Douglas

In our previous episode, Joost van der Sluis said:

Only code that actually overrides those methods, not all interface code.

No, also the places where QueryInterface is called. Because the caller need to know that it has to pass the GUID by reference, not value. So the caller need to know that 'const' actually means 'constref' in this case.

Joost.

Hello Joost,

Friday, November 19, 2010, 1:02:59 PM, you wrote:

JvdS> Well, this issue leaded to a large discussion elsewhere. I was wondering JvdS> when someone else would even notice this incompatibility. A lot of JvdS> alternatives have been discussed, but I coud not find any which is less JvdS> intrusive.

I had followed the discussion in the public lists but not in the development internal lists.

JvdS> In an nutshell: the behavior of 'const' parameters is undefined. When JvdS> using cdecl the 'const' keyword is simply ignored. But we do need to JvdS> pass the GUID by reference.

I'm aware...

JvdS> There are two options:
JvdS> 1: Change the behavior of const-parameters. That can lead to JvdS> unpredictable problems which are not noticed by the compiler. It could JvdS> be an option to use some setting to define the behavior of 'const', but JvdS> in that case that setting have to be set for all units that do a call to JvdS> QueryInterface. (ie: all code using interfaces, so it means this setting JvdS> has to be set everywhere where COM code is used)

Currently Win32 (only win32) could work with const in that interface, taking in account that any other interface functions in IUnknown descendents will be mostly typed with const instead constref, so people with interfaces will inadvertly have const everywhere but QueryInterface. Anyway, continue reading...

JvdS> 2: Change the definition to constref, so the compiler will complain if a JvdS> user override the QueryInterface's definition, and can adapt the code.

This is the best, of course, but is there any possibility to fine tune the message error in that case to offer the solution to the programmer directly ? Can a special error message be emited for this error in QueryInterface ?

JvdS> Notice that until now I've only heard of code which overrides JvdS> QueryInterface in the Free Pascal code itself, in the Lazarus code, ZEOS JvdS> and the code from one commercial developer who also works on JvdS> Lazarus/Free Pascal.
JvdS> It is an incompatibility, yes. But I think it's not too intrusive and I JvdS> see no alternatives...

No, not intrusive, but very hard to find the solution for average joe, in fact I had solved it in my Zeos just after seen the "error" but most people are not aware of the change. I'm looking for a help in solution, not an interface change.

In our previous episode, Joost van der Sluis said:

No, since const+directive can be put in the tree in the same way as a constref syntax element.

Hmm.. that's probaby true. If it can be made a local define, the intrusion is as small as the constref approach.

Joost.

Joost van der Sluis wrote

No, also the places where QueryInterface is called. Because the caller need to know that it has to pass the GUID by reference, not value. So the caller need to know that 'const' actually means 'constref' in this case.

Can const = constref on stdcall convention? And can compiler skip error reporting in this case?

Best regards,
Paul Ishenin

Paul Ishenin wrote

Can const = constref on stdcall convention?

Only on Windows, because on other platforms that would change the meaning of existing stdcall routines (and on all non-i386 platforms, the default calling convention is pocall_stdcall).

Jonas

Well, it could be done since "const" is implementation-dependent, but it would force all non-x86 platforms to pass all const parameters by reference by default, which may not be desirable from an efficiency point of view.

Jonas

As I understand not all const parameters are passed by reference in delphi stdcall implementation but only records? Or say integer/byte will be passed by reference too?

Best regards,
Paul Ishenin.

AFAIK references are used for data types, that exceed the size of an pointer.

I'd suggest that "constref" works like const, except on platforms or for external libraries, that expect only references for const arguments. The exceptions then can be handled by an according mode switch.

DoDi

Paul Ishenin wrote

19.11.2010 23:47, Jonas Maebe wrote: > > Well, it could be done since "const" is implementation-dependent, but it would force all non-x86 platforms to pass all const parameters by reference by default, which may not be desirable from an efficiency point of view. As I understand not all const parameters are passed by reference in delphi stdcall implementation but only records? Or say integer/byte will be passed by reference too?

You proposed to make "const = constref on stdcall convention". constref passes everything by reference (that's the literal meaning of the keyword "const argument by reference").

Also, the default calling convention on non-x86 platforms is supposed to be compliant with the default ABI of the platform. Many of these platforms pass small records all by value (const or not).

Jonas

Jonas Maebe wrote

Also, the default calling convention on non-x86 platforms is supposed to be compliant with the default ABI of the platform. Many of these platforms pass small records all by value (const or not).

Const is a pascal keyword which can mean for pascal "pass records by reference". If you need to pass a record by value to some system C routine then don't mark that argument as const during the header convertion. What is wrong?

Best regards,
Paul Ishenin.

Paul Ishenin wrote

20.11.2010 17:53, Jonas Maebe wrote: > Also, the default calling convention on non-x86 platforms is supposed to be compliant with the default ABI of the platform. Many of these platforms pass small records all by value (const or not). Const is a pascal keyword which can mean for pascal "pass records by reference".

Making it mean that would make a lot of code less efficient. It doesn't have that meaning for the i386 "register" calling convention either (and it cannot be made to mean that there, because that would break Delphi compatibility).

If you need to pass a record by value to some system C routine then don't mark that argument as const during the header convertion. What is wrong?

A few things are being mixed here:
1) all C header translations have to mark the imported routines as "cdecl", and cdecl, cppdecl and mwpascal are the only calling convention where "const" has a defined meaning in FPC as far as how a parameter is passed (namely "by value" for cdecl and cppdecl, because in C/C++ a "const" parameter is also passed by value; and for mwpascal it's "by value, except for records which are passed "by reference", because that's how MetroWerks Pascal does it -- i.e., another "solution" could be to define the interface helper methods as "mwpascal", but that would still require ifdefs).
2) because of all the troubles we had on i386 with a different default calling convention than the one used by code generated by other compilers (because of Delphi compatibility), we decided that on other platforms we would stick as close as possible to the ABI even for non-cdecl/cppdecl code. "const" is not part of the ABI (it's part of languages), so there we indeed sort of "do what we want" and what we do is different on different platforms (sometimes more closely sticking to the C "const" behaviour, sometimes a little less). Since the default calling convention is called "stdcall" on non-i386 platforms, changing the meaning of "const" for stdcall there would change this behaviour for almost all code. While we are free to do so (since "const" indeed has no defined meaning for stdcall on non-win32 platforms), a) the change you propose would result in slowdowns on several platforms b) like with any such change, it will also break some people's assembler code. We can do that, but we should try to minimise that and in combination with a), I don't think it's worth it in this case

Jonas

Jonas Maebe wrote

A few things are being mixed here: 1) all C header translations have to mark the imported routines as "cdecl", and cdecl, cppdecl and mwpascal are the only calling convention where "const" has a defined meaning in FPC as far as how a parameter is passed (namely "by value" for cdecl and cppdecl, because in C/C++ a "const" parameter is also passed by value; and for mwpascal it's "by value, except for records which are passed "by reference", because that's how MetroWerks Pascal does it -- i.e., another "solution" could be to define the interface helper methods as "mwpascal", but that would still require ifdefs). 2) because of all the troubles we had on i386 with a different default calling convention than the one used by code generated by other compilers (because of Delphi compatibility), we decided that on other platforms we would stick as close as possible to the ABI even for non-cdecl/cppdecl code. "const" is not part of the ABI (it's part of languages), so there we indeed sort of "do what we want" and what we do is different on different platforms (sometimes more closely sticking to the C "const" behaviour, sometimes a little less). Since the default calling convention is called "stdcall" on non-i386 platforms, changing the meaning of "const" for stdcall there would change this behaviour for almost all code. While we are free to do so (since "const" indeed has no defined meaning for stdcall on non-win32 platforms), a) the change you propose would result in slowdowns on several platforms

don't use const and it will be passed by value -> no slowdown

b) like with any such change, it will also break some people's assembler code. We can do that, but we should try to minimise that and in combination with a), I don't think it's worth it in this case

why assembler outweights interfaces? Moreover those who need to port their projects to FPC will need to review their assember anyway because as I know assember is not compatible. And now code wich uses interfaces is not compatible too.

Best regards,
Paul Ishenin

Lots of existing Pascal code uses "const", and generally "const" is used to give a hint to the compiler so it can make code faster.

> b) like with any such change, it will also break some people's assembler code. We can do that, but we should try to minimise that and in combination with a), I don't think it's worth it in this case why assembler outweights interfaces?

It's not only assembler, it's assembler + the fact that the generated code for const parameters will become worse on most platforms.

Moreover those who need to port their projects to FPC will need to review their assember anyway because as I know assember is not compatible.

It's not about porting code, it's about existing FPC code.

Jonas

Paul Ishenin wrote

Moreover those who need to port their projects to FPC will need to review their assember anyway because as I know assember is not compatible.

Delphi assembler is supposed to work?

Jonas Maebe wrote

It's not about porting code, it's about existing FPC code.

Changes in interfaces also breaks existing FPC code. And plus adds more problems to delphi code porting.

Best regards,
Paul Ishenin.

Paul Ishenin wrote

20.11.2010 22:48, Jonas Maebe wrote: > It's not about porting code, it's about existing FPC code. Changes in interfaces also breaks existing FPC code. And plus adds more problems to delphi code porting.

Yes, but it's only part of the reason why I don't think it's a good idea, as I've tried to explain in my previous answers.

Jonas

AFAIR Delphi treats the byval/byref issue as "compiler specific", so nothing can be broken - apart from code that is bound to an specific compiler.

> If you need to pass a record by value to some system C routine then > don't mark that argument as const during the header convertion. > What is wrong?

A const record cannot be passed as a non-const parameter. This is the only difference between constref and var. Thus "const var" could have been used instead of a new "constref" keyword.

DoDi

Maybe an alternative solution could be to create a new calling convention that has the same meaning stdcall has currently on non-win32 platforms, make that the default calling conventino on non-i386, and then change the behaviour of stdcall to what it is on win32 as far as what "const" means.

Jonas

Paul Ishenin wrote

20.11.2010 22:48, Jonas Maebe wrote: > It's not about porting code, it's about existing FPC code. Changes in interfaces also breaks existing FPC code. And plus adds more problems to delphi code porting.

Again, these changes are detected by the compiler, and easily fixable. The problems that may arise if we change the behavior of 'const' are very difficult to detect. (Not only in assembly, but also in header-conversion units) It will simple lead to random AV's.

Joost.