Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2010-06

Mozilla XPCOM

40 messages · 6 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

José Mejuto

Hello FPC-Pascal,

I'm trying to make the Gecko engine port compile in Ubuntu. It currently works in Win32 without problems, but in Ubuntu I had found a strange problem that could be related to fpc or not.

Mozilla XPCOM is more or less based in COM interfaces, so in simple words, the code loads the libxpcom.so file, get some function pointers from it and call a function that returns an interface and later this interface is used to create a new interface. Example:

var
  ServiceManager: nsIServiceManager;
  WindowWatcher: nsIWindowWatcher;
begin
  xpcomfuncGetServicemanager(ServiceManager);
  ServiceManager.GetServiceByContractID('@mozilla/...;1',nsIWindowWatcher,WindowWatcher);
end;

Everything seems to work up to this point, where something strange happends, I get the ServiceManager interface and it looks fine, up to where gdb let me see (non nil) and when calling
.GetServiceByContractID execution stops, but it stops "clean", no SIGSGV, no exception, nothing. My first idea was that the function due some kind of error calls halt or any processor instruction that cancels execution, but when compiled with heaptrc I get the heaptrc dump, which should not happend if the stop were abnormal.

All interfaces are safecall and I'm using up to date fpc 2.5.1.

Does anybody have a remote idea of what could be happening ? Any idea about how to debug the reason of the sudden stop ? Maybe COM style interfaces can not be used in Linux ?

If anybody wants to try it in the own computer I can upload the current version to the Lazarus CCR "gecko port", the current one does not compile in Linux, only win32.

Best regards, José
Michael Van Canneyt
to José Mejuto

Hi,

Are you cooperating with Phil on this ?

When I was debugging it (linux 64 bit) , the whole application stopped on a floating point error somewhere in mozilla's Javascript engine; I never got around to debugging that.

Have you seen this error too, or does the .GetServiceByContractID stop already before this ?

Michael.

José Mejuto wrote 46 lines
Hello FPC-Pascal, I'm trying to make the Gecko engine port compile in Ubuntu. It currently works in Win32 without problems, but in Ubuntu I had found a strange problem that could be related to fpc or not. Mozilla XPCOM is more or less based in COM interfaces, so in simple words, the code loads the libxpcom.so file, get some function pointers from it and call a function that returns an interface and later this interface is used to create a new interface. Example: var ServiceManager: nsIServiceManager; WindowWatcher: nsIWindowWatcher; begin xpcomfuncGetServicemanager(ServiceManager); ServiceManager.GetServiceByContractID('@mozilla/...;1',nsIWindowWatcher,WindowWatcher); end; Everything seems to work up to this point, where something strange happends, I get the ServiceManager interface and it looks fine, up to where gdb let me see (non nil) and when calling .GetServiceByContractID execution stops, but it stops "clean", no SIGSGV, no exception, nothing. My first idea was that the function due some kind of error calls halt or any processor instruction that cancels execution, but when compiled with heaptrc I get the heaptrc dump, which should not happend if the stop were abnormal. All interfaces are safecall and I'm using up to date fpc 2.5.1. Does anybody have a remote idea of what could be happening ? Any idea about how to debug the reason of the sudden stop ? Maybe COM style interfaces can not be used in Linux ? If anybody wants to try it in the own computer I can upload the current version to the Lazarus CCR "gecko port", the current one does not compile in Linux, only win32. -- Best regards, José _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Jonas Maebe
to Michael Van Canneyt
Michael Van Canneyt wrote
When I was debugging it (linux 64 bit) , the whole application stopped on a floating point error somewhere in mozilla's Javascript engine; I never got around to debugging that.

Did you try disabling floating point exceptions? A lot of C code assumes that floating point exceptions are disabled (since that's what the C library does on startup).

Jonas

Michael Van Canneyt
to Jonas Maebe
Jonas Maebe wrote 10 lines
On 03 Jun 2010, at 08:55, Michael Van Canneyt wrote: > When I was debugging it (linux 64 bit) , the whole application stopped on a > floating point error somewhere in mozilla's Javascript engine; I never got > around to debugging that. Did you try disabling floating point exceptions? A lot of C code assumes that floating point exceptions are disabled (since that's what the C library does on startup).

Aha. And how do I do that ?

Michael.

José Mejuto
to Michael Van Canneyt: Are you cooperating with Phil on this ? When I was debugging it (linux 64 bit) , the…

Hello FPC-Pascal,

Thursday, June 3, 2010, 8:55:47 AM, you wrote:

MVC> Are you cooperating with Phil on this ?

Yes. Phil is not supporting it now so I had added some improvements.

MVC> When I was debugging it (linux 64 bit) , the whole application stopped on a MVC> floating point error somewhere in mozilla's Javascript engine; I never got MVC> around to debugging that.

That happends with Gecko 1.9.2.x where Javascript wngine was changed, and there are two kind of errors, the floating point ones and SSE ones. Most floating point problems are solved by simply setting the Set8087CW but the SSE ones are more serious, I think they are somehow related to memory alignement but as I do not know how get the register values in gdb I left this problem by now as Gecko < 1.9.2.0 works.

MVC> Have you seen this error too, or does the .GetServiceByContractID stop MVC> already before this ?

In Linux it stops just before creating the window container, at the CreateHwnd level, where the nsIWindowWatcher is to be created. In Windows it works just fine, at least up to where I had tested (I'm writting a mini navigator in order to test facilities).

Best regards, José
José Mejuto
to Jonas Maebe: Did you try disabling floating point exceptions? A lot of C code assumes that floating…

Hello FPC-Pascal,

you wrote
> When I was debugging it (linux 64 bit) , the whole application > stopped on a floating point error somewhere in mozilla's Javascript > engine; I never got > around to debugging that.

JM> Did you try disabling floating point exceptions? A lot of C code JM> assumes that floating point exceptions are disabled (since that's what JM> the C library does on startup).

Yes, and also it should raise a SIGFPE and leave a "note" in the gdb output, even when non visual feedback happends, do not ?

Best regards, José
José Mejuto
to Michael Van Canneyt: Aha. And how do I do that ? Michael.

Hello FPC-Pascal,

you wrote
> Did you try disabling floating point exceptions? A lot of C code assumes that > floating point exceptions are disabled (since that's what the C library does > on startup).

MVC> Aha. And how do I do that ?

Set8087CW($133F); ? Or maybe could I need signal(SIGFPE,SIG_IGN); ? I'll check.

Best regards, José
Jonas Maebe
to Michael Van Canneyt: Aha. And how do I do that ? Michael.
Michael Van Canneyt wrote 14 lines
On Thu, 3 Jun 2010, Jonas Maebe wrote: > On 03 Jun 2010, at 08:55, Michael Van Canneyt wrote: > >> When I was debugging it (linux 64 bit) , the whole application >> stopped on a floating point error somewhere in mozilla's >> Javascript engine; I never got >> around to debugging that. > > Did you try disabling floating point exceptions? A lot of C code > assumes that floating point exceptions are disabled (since that's > what the C library does on startup). Aha. And how do I do that ?
uses
   math;

SetExceptionMask([exInvalidOp, exDenormalized, exZeroDivide, exOverflow, exUnderflow, exPrecision])

Jonas

José Mejuto
to Michael Van Canneyt: Are you cooperating with Phil on this ? When I was debugging it (linux 64 bit) , the…

Hello FPC-Pascal,

Thursday, June 3, 2010, 8:55:47 AM, you wrote:

MVC> Are you cooperating with Phil on this ?

Just to keep the message in thread, I re-answer this message.

After a bit more debugging I found something strange (again it could be fpc related or not, I'm not sure). This is the firs lines of definition of nsIFile interface:

---------------------------------------
  nsIFile = interface(nsISupports)
  ['{c8c0a080-0868-11d3-915f-d9d889d48e3c}']
    procedure Append(const node: nsAString); safecall;
    procedure AppendNative(const node: nsACString); safecall;
    procedure Normalize(); safecall;
    procedure Create(_type: PRUint32; permissions: PRUint32); safecall;
    procedure GetLeafName(aLeafName: nsAString); safecall;
    procedure SetLeafName(const aLeafName: nsAString); safecall;
    procedure GetNativeLeafName(aNativeLeafName: nsACString); safecall;
    procedure SetNativeLeafName(const aNativeLeafName: nsACString); safecall;
    procedure CopyTo(newParentDir: nsIFile; const newName: nsAString); safecall;
    procedure CopyToNative(newParentDir: nsIFile; const newName: nsACString); safecall;
    procedure CopyToFollowingLinks(newParentDir: nsIFile; const newName: nsAString); safecall;
    procedure CopyToFollowingLinksNative(newParentDir: nsIFile; const newName: nsACString); safecall;
    procedure MoveTo(newParentDir: nsIFile; const newName: nsAString); safecall;
    procedure MoveToNative(newParentDir: nsIFile; const newName: nsACString); safecall;
    procedure Remove(recursive: PRBool); safecall;
    function GetPermissions(): PRUint32; safecall;
---------------------------------------

All "procedure" works like "GetLeafName" or "Remove", but functions, all of them return some kind of "COM error" like "$80004003" or "$80070057" in example in the "GetPermissions" or "GetFileSize". Does this could be related to a different handling of procedure and function ? Is there any kind of manual test that I can perform to detect if the problem is located in fpc ?

Best regards, José
Jonas Maebe
to Jonas Maebe: Did you try disabling floating point exceptions? A lot of C code assumes that floating…
Jonas Maebe wrote 10 lines
On 03 Jun 2010, at 08:55, Michael Van Canneyt wrote: > When I was debugging it (linux 64 bit) , the whole application > stopped on a floating point error somewhere in mozilla's Javascript > engine; I never got > around to debugging that. Did you try disabling floating point exceptions? A lot of C code assumes that floating point exceptions are disabled (since that's what the C library does on startup).

Another possible problem can be the fact that FPC currently is not fully compliant with the x86-64 ABIs. In particular, the following data structures are wrongly handled when passed to cdecl/stdcall routines *as value or const parameter*, and when they are returned from a cdecl/stdcall routine:
* arrays <= 16 bytes of integer or floating point numbers (handled wrongly on all x86-64 platforms)
* records <= 16 bytes containing floating point numbers or arrays of floating point numbers (handled wrongly on non-Win64 platforms under most, but not all, circumstances )

I'm working on a fix for that, and it's finally almost finished.

Jonas

José Mejuto
to Jonas Maebe

Hello FPC-Pascal,

you wrote
> Did you try disabling floating point exceptions? A lot of C code > assumes that floating point exceptions are disabled (since that's > what the C library does on startup).

JM> Another possible problem can be the fact that FPC currently is not JM> fully compliant with the x86-64 ABIs. In particular, the following

I'm aware about that problems, but this is Ubuntu 32 bits. Also structures seems fine as the nsAString which is in fact being passed to interfaces is a 4 fields record and this record is passed in the creation and then used by the interface when called any of the methods. All methods that are procedures work, functions do not :-? My next test is to confirm that procedures in the interface after any function definition works or not.

Is there any manual ASM that I can write to call the interface function manually ? No parameter is being passed and only returns a 32 bits integer, but I do not know at all how safecall works and even less how call an interface at all in ASM :( Check ASM generated code is a possibility too, but the project is big and the setup code is also quite big and the ASM output could be giant :(

JM> I'm working on a fix for that, and it's finally almost finished.

That's great! :)

Best regards, José
José Mejuto
to Jonas Maebe

Hello FPC-Pascal,

Thursday, June 3, 2010, 3:31:31 PM, you wrote:

JM> You can try putting breakpoints on FPEXIT, _haltproc and _exit (the JM> last one only exists if libc is linked, but that's definitely the case JM> here; note that all of them are case-sensitive). That should catch JM> most exits.

I was only able to set a break over _haltproc and it catches the exit with this backtrace:

#0 0x082da470 in SI_C21__FPC_LIBC21_HALTPROC ()
#1 0x080779d4 in SYSTEM_SYSTEM_EXIT ()
#2 0x080700da in SYSTEM_DO_EXIT ()
#3 0x080700fa in SYSTEM_HALT$LONGINT ()
#4 0x0806d364 in fpc_catches ()
#5 0x08419fd8 in TC_NSXPCOMGLUE_SCOMPMGR ()
#6 0xb6b5d7c0 in ?? ()
#7 0x082bf858 in INITWINDOWCREATOR () at CallbackInterfaces.pas:108 #8 0x082b8747 in TCUSTOMGECKOBROWSER__INITWEBBROWSER (this=0xb6b8bd30) at GeckoBrowser.pas:1096

I'm unable to determine which is TC_NSXPCOMGLUE_SCOMPMGR. fpc_catches appears just after the procedure that raises the exception or it could appear later ?

Best regards, José
Jonas Maebe
to José Mejuto
José Mejuto wrote 17 lines
I was only able to set a break over _haltproc and it catches the exit with this backtrace: #0 0x082da470 in SI_C21__FPC_LIBC21_HALTPROC () #1 0x080779d4 in SYSTEM_SYSTEM_EXIT () #2 0x080700da in SYSTEM_DO_EXIT () #3 0x080700fa in SYSTEM_HALT$LONGINT () #4 0x0806d364 in fpc_catches () #5 0x08419fd8 in TC_NSXPCOMGLUE_SCOMPMGR () #6 0xb6b5d7c0 in ?? () #7 0x082bf858 in INITWINDOWCREATOR () at CallbackInterfaces.pas:108 #8 0x082b8747 in TCUSTOMGECKOBROWSER__INITWEBBROWSER (this=0xb6b8bd30) at GeckoBrowser.pas:1096 I'm unable to determine which is TC_NSXPCOMGLUE_SCOMPMGR. fpc_catches appears just after the procedure that raises the exception or it could appear later ?

Since the RTL is compiled with optimisation enabled, you may be missing intermediate stack frames. You will have to recompile it without optimisations to get a full backtrace.

Jonas

José Mejuto
to Jonas Maebe

Hello FPC-Pascal,

Thursday, June 3, 2010, 6:51:22 PM, you wrote:

JM> Since the RTL is compiled with optimisation enabled, you may JM> be missing intermediate stack frames. You will have to recompile JM> it without optimisations to get a full backtrace.

Done, the halt is in fpc code:

---------------------------------------
Function fpc_Catches(Objtype : TClass) : TObject;[Public, Alias : 'FPC_CATCHES']; compilerproc;
var
  _Objtype : TExceptObjectClass;
  _ExceptObjectStack : PExceptObject;
begin
  _ExceptObjectStack:=ExceptObjectStack;
  If _ExceptObjectStack=Nil then
   begin
{$ifdef excdebug}
     Writeln ('Internal error.');
{$endif}
     halt (255);
   end;
[...]
---------------------------------------

And this is the backtrace. Any idea ?
---------------------------------------
#0 _FPC_LIBC21_HALTPROC () at ./i386/si_c21.inc:146 #1 0x08077fb4 in SYSTEM_EXIT () at system.pp:96
#2 0x080706ba in DO_EXIT () at /home/joshy/pascal/fpc/rtl/inc/system.inc:916 #3 0x080706da in HALT (ERRNUM=0)
at /home/joshy/pascal/fpc/rtl/inc/system.inc:929
#4 0x0806d944 in fpc_catches (OBJTYPE=<incomplete type>) at /home/joshy/pascal/fpc/rtl/inc/except.inc:321
#5 0x0841cf88 in TC_NSXPCOMGLUE_SCOMPMGR ()
#6 0xb6c099c0 in ?? ()
#7 0x082c0978 in INITWINDOWCREATOR () at CallbackInterfaces.pas:108 ---------------------------------------

Is there any way to known the exception type ?

Best regards, José
Jonas Maebe
to José Mejuto
José Mejuto wrote 27 lines
Thursday, June 3, 2010, 6:51:22 PM, you wrote: JM> Since the RTL is compiled with optimisation enabled, you may JM> be missing intermediate stack frames. You will have to recompile JM> it without optimisations to get a full backtrace. Done, the halt is in fpc code: --------------------------------------- Function fpc_Catches(Objtype : TClass) : TObject;[Public, Alias : 'FPC_CATCHES']; compilerproc; var _Objtype : TExceptObjectClass; _ExceptObjectStack : PExceptObject; begin _ExceptObjectStack:=ExceptObjectStack; If _ExceptObjectStack=Nil then begin {$ifdef excdebug} Writeln ('Internal error.'); {$endif} halt (255); end; [...] --------------------------------------- And this is the backtrace. Any idea ?

Maybe you are executing Pascal code in threads that have not been started via the FPC rtl? (i.e., not via beginthread nor via tthread.create) That is not supported on Unix platforms.

Jonas

José Mejuto
to Jonas Maebe

Hello FPC-Pascal,

you wrote
> And this is the backtrace. Any idea ?

JM> Maybe you are executing Pascal code in threads that have not been JM> started via the FPC rtl? (i.e., not via beginthread nor via JM> tthread.create) That is not supported on Unix platforms.

Not, well, not at least on intention but maybe is the XPCOM which is calling pascal code from a different thread :-? XPCOM tells me across the documentation to call most of its functions from the main thread, but maybe the callback is happening from a different one :-?

If that's the case is any kind of workaround ? Callbacks are only 4 or 5 functions +/- and maybe I can create another thread (in pascal) and inquiry this thread to process the data and put result in some kind of shared memory block ?

Best regards, José
Michael Van Canneyt
to José Mejuto
José Mejuto wrote 18 lines
Hello FPC-Pascal, Friday, June 4, 2010, 10:37:42 AM, you wrote: >> And this is the backtrace. Any idea ? JM> Maybe you are executing Pascal code in threads that have not been JM> started via the FPC rtl? (i.e., not via beginthread nor via JM> tthread.create) That is not supported on Unix platforms. Not, well, not at least on intention but maybe is the XPCOM which is calling pascal code from a different thread :-? XPCOM tells me across the documentation to call most of its functions from the main thread, but maybe the callback is happening from a different one :-? If that's the case is any kind of workaround ? Callbacks are only 4 or 5 functions +/- and maybe I can create another thread (in pascal) and inquiry this thread to process the data and put result in some kind of shared memory block ?

I've been studying the thread problem for years and never found a workable solution on Unix.

Michael.

José Mejuto
to Michael Van Canneyt

Hello FPC-Pascal,

you wrote
> If that's the case is any kind of workaround ? Callbacks are only 4 or > 5 functions +/- and maybe I can create another thread (in pascal) and > inquiry this thread to process the data and put result in some kind of > shared memory block ?

MVC> I've been studying the thread problem for years and never found a workable MVC> solution on Unix.

So as no "real solution" maybe I can find one shoot solution. When Pascal code is being called from an unknown thread (not aware of it) which is not possible to do ? Can I access local variables non refcounted ? Can I access global variables ? I'm quite sure the real problem is a "lost" of memory pools, refcounters and other internal managed structures but also I'm quite sure others can be accessed safelly. Can you give me a little note about which must not be used at all ? Thank you.

Best regards, José
Jonas Maebe
to José Mejuto: JM> Maybe you are executing Pascal code in threads that have not been JM> started via the…
José Mejuto wrote 12 lines
Hello FPC-Pascal, Friday, June 4, 2010, 10:37:42 AM, you wrote: JM> Maybe you are executing Pascal code in threads that have not been JM> started via the FPC rtl? (i.e., not via beginthread nor via JM> tthread.create) That is not supported on Unix platforms. Not, well, not at least on intention but maybe is the XPCOM which is calling pascal code from a different thread :-? XPCOM tells me across the documentation to call most of its functions from the main thread, but maybe the callback is happening from a different one :-?

It doesn't matter that it's called from a different thread, as long as this different thread is also created via the FPC RTL.

If that's the case is any kind of workaround ?

No, it cannot be worked around (other than adding functions to the FPC rtl that you can call *once* from externally created threads to initialise and finalise the FPC thread support for that specific thread, but that has not yet been done).

Jonas

José Mejuto
to Jonas Maebe

Hello FPC-Pascal,

you wrote
> Not, well, not at least on intention but maybe is the XPCOM which is > calling pascal code from a different thread :-? XPCOM tells me across > the documentation to call most of its functions from the main thread, > but maybe the callback is happening from a different one :-?

JM> It doesn't matter that it's called from a different thread, as long as JM> this different thread is also created via the FPC RTL.

It is being calledback from the same thread, at least the ThreadID var and the function GetCurrentThreadID returns the same value in the main thread and in the callback functions :-? I think it is time to get windows ASM and Linux ASM and compare them :( or the project will die by my side in non Windows platform :(

Best regards, José
José Mejuto
to Jonas Maebe: It doesn't matter that it's called from a different thread, as long as this different…

Hello FPC-Pascal,

you wrote
> JM> Maybe you are executing Pascal code in threads that have not been

[...]

> Not, well, not at least on intention but maybe is the XPCOM which is > If that's the case is any kind of workaround ?

JM> No, it cannot be worked around (other than adding functions to the FPC [...]

I had found the problem, after a lot of debuging and trying to understand the underlayed COM interface the bug is in the mozilla XPCOM or in the fpc undertanding of COM interfaces in Linux. Aside the fact that under safecall in linux fpc does not call "fpc_safecallcheck" which I think it should be done to test the EAX register for errors, fpc uses the same exact code under Windows and Linux but mozilla's XPCOM and I think that finally GCC uses a hybird calling convention, it is the same as in Linux but the stack must be cleaned by the caller, not the callee. So the same code to work in Linux (XPCOM) and in Windows:

asm
    mov eax, [TheCOMInterface];
    mov eax,[eax]
    push Intf;
    push IID;
    push ContractID;
    push TheCOMInterface;
    Call [eax]+16;
{$IFDEF LINUX}
    add esp,16;
{$ENDIF}
    call fpc_safecallcheck;
end;

Which in Linux renders to a "cdecl", but tagging the function as cdecl crashes because the const parameter TGUID is passed as a 16 bytes instead one pointer in this declaration:

Original one (Windows):
-----------------------
procedure GetServiceByContractID(const aContractID: PAnsiChar;
                                 const aIID: TGUID;
                                 out _result); safecall;

cdecl that fails (Linux):
-----------------

procedure GetServiceByContractID(const aContractID: PAnsiChar;
                                 const aIID: TGUID;
                                 out _result); cdecl;

cdecl that works as expected (Linux):
-------------------------------------

procedure GetServiceByContractID(const aContractID: PAnsiChar;
                                 const aIID: PGUID;
                                 out _result); cdecl;

Changing the TGUID to PGUID and in the call changing the IID by @IID it works fine.

Could this be considered a bug in fpc interfaces ? It is expected and is the XPCOM/GCC the wrong one ? Could or would it be changed in fpc in a near future ?

Best regards, José
Jonas Maebe
to José Mejuto
José Mejuto wrote 6 lines
I had found the problem, after a lot of debuging and trying to understand the underlayed COM interface the bug is in the mozilla XPCOM or in the fpc undertanding of COM interfaces in Linux. Aside the fact that under safecall in linux fpc does not call "fpc_safecallcheck" which I think it should be done to test the EAX register for errors,

safecall is not supported (= ignored) on non-Windows platforms, because it is completely Windows-specific. Afaik, GCC does not support any safecall calling convention on Linux either.

fpc uses the same exact code under Windows and Linux but mozilla's XPCOM and I think that finally GCC uses a hybird calling convention, it is the same as in Linux but the stack must be cleaned by the caller, not the callee.

That's actually the standard C calling convention, not a hybrid calling convention (= cdecl, as you mention below).

Show quoted text 16 lines
So the same code to work in Linux (XPCOM) and in Windows: asm mov eax, [TheCOMInterface]; mov eax,[eax] push Intf; push IID; push ContractID; push TheCOMInterface; Call [eax]+16; {$IFDEF LINUX} add esp,16; {$ENDIF} call fpc_safecallcheck; end;

Are you certain when using XPCOM with GCC, it automatically transforms every function so that
a) the result is passed as a hidden parameter by reference b) the actual function results becomes an error code c) a check is inserted after every call that checks this error code

?

Show quoted text 18 lines
cdecl that fails (Linux): ----------------- procedure GetServiceByContractID(const aContractID: PAnsiChar; const aIID: TGUID; out _result); cdecl; cdecl that works as expected (Linux): ------------------------------------- procedure GetServiceByContractID(const aContractID: PAnsiChar; const aIID: PGUID; out _result); cdecl; Changing the TGUID to PGUID and in the call changing the IID by @IID it works fine. Could this be considered a bug in fpc interfaces ?

Yes, it is always wrong to assume that "const" enforces passing by reference (except when using the mwpascal calling convention).

Jonas

José Mejuto
to Jonas Maebe

Hello FPC-Pascal,

Monday, June 7, 2010, 10:34:26 AM, you wrote:

JM> safecall is not supported (= ignored) on non-Windows platforms, JM> because it is completely Windows-specific. Afaik, GCC does not support JM> any safecall calling convention on Linux either.

It is "silently" changed by stdcall do not ?

> Linux but mozilla's XPCOM and I think that finally GCC uses a hybird > calling convention, it is the same as in Linux but the stack must be

JM> That's actually the standard C calling convention, not a hybrid JM> calling convention (= cdecl, as you mention below).

Yes, at the beginning it looks to me strange, because cdecl generates a totally different code (the TGUID problem) and my first idea was a non 100% standard calling convention, which is obviously wrong.

Show quoted text 16 lines
> So the same code to work in > Linux (XPCOM) and in Windows: > > asm > mov eax, [TheCOMInterface]; > mov eax,[eax] > push Intf; > push IID; > push ContractID; > push TheCOMInterface; > Call [eax]+16; > {$IFDEF LINUX} > add esp,16; > {$ENDIF} > call fpc_safecallcheck; > end;

JM> Are you certain when using XPCOM with GCC, it automatically transforms JM> every function so that

First I must conclude that XPCOM in Linux is compiled with GCC and which version is being used, but I'm quite sure it is GCC ;)

JM> a) the result is passed as a hidden parameter by reference JM> b) the actual function results becomes an error code

Plain function result is in EAX, if I call (in ASM) QueryInterface with garbaged IIDs I get "$80004002" in the EAX (NOT_INTERFACE). Just the same as in Windows using safecall except the stack clean.

JM> c) a check is inserted after every call that checks this error code

To check this I must compile something with GCC and I do not have it installed and I do not have a source to compile with it. I should investigate following other paths.

> Could this be considered a bug in fpc interfaces ?

JM> Yes, it is always wrong to assume that "const" enforces passing by JM> reference (except when using the mwpascal calling convention).

So a "const x: TGUID" will be passed by reference in safecall and as 16 bytes in cdecl ? Is this OK ? Is this how GCC excepts a const parameter TGUID to be passed ? I'm asking because now I have changed my interface definition to use PGUID instead (both windows and linux) and it works more or less, but now I have a problem in the TInterfacedObject which seems to be the cause of some crashes, but I can not say by now if it is related to XPCOM exposing a different "IUnknown" (maybe cdecl instead stdcall) or TGUID related. I'll investigate further.

One question, is there any way in fpc to "define" the calling convention as a C define ? Just something like:

{$IFDEF UNIX}
  {$DEFINE THECALLING cdecl}
{$ELSE}
  {$DEFINE THECALLING safecall}
{$ENDIF}
function x: integer; THECALLING;

I had replaced all safecalls with:

{$IFDEF xx}safecall{$ENDIF}{$IFDEF y}cdecl{$ENDIF}

But it is quite "awful" :)

Best regards, José
Jonas Maebe
to José Mejuto
José Mejuto wrote 8 lines
Monday, June 7, 2010, 10:34:26 AM, you wrote: JM> safecall is not supported (= ignored) on non-Windows platforms, JM> because it is completely Windows-specific. Afaik, GCC does not support JM> any safecall calling convention on Linux either. It is "silently" changed by stdcall do not ?

It is completely ignored. So it is the same as the default calling convention (which is register on i386, and stdcall=register=cdecl=... on other platforms).

Show quoted text 13 lines
JM> Are you certain when using XPCOM with GCC, it automatically transforms JM> every function so that First I must conclude that XPCOM in Linux is compiled with GCC and which version is being used, but I'm quite sure it is GCC ;) JM> a) the result is passed as a hidden parameter by reference JM> b) the actual function results becomes an error code Plain function result is in EAX, if I call (in ASM) QueryInterface with garbaged IIDs I get "$80004002" in the EAX (NOT_INTERFACE). Just the same as in Windows using safecall except the stack clean.

Ok. In the mean time I found some more info about how XPCOM works:
* http://benjamin.smedbergs.us/blog/2008-09-26/allocated-memory-and-shared-library-boundaries/ : apparently, all code that makes use of XPCOM via C/C++ is supposed to be written in IDL, which is then transformed into C++ code that conforms to the safecall conventions (apart from the caller rather than the callee cleaning up).
* https://developer.mozilla.org/En/Xptcall_Porting_Guide : a description of how the dispatching works (but that's not really relevant in case of FPC, since FPC natively supports calling COM interfaces on Linux).

Show quoted text 6 lines
JM> c) a check is inserted after every call that checks this error code To check this I must compile something with GCC and I do not have it installed and I do not have a source to compile with it. I should investigate following other paths.

At first sight that does not happen in XPCOM, even on Windows (but since we do it on Windows, maybe we should also do it elsewhere for consistency reasons).

Show quoted text 8 lines
> Could this be considered a bug in fpc interfaces ? > JM> Yes, it is always wrong to assume that "const" enforces passing by JM> reference (except when using the mwpascal calling convention). So a "const x: TGUID" will be passed by reference in safecall and as 16 bytes in cdecl ? Is this OK ? Is this how GCC excepts a const parameter TGUID to be passed ?

In C, "const" does not modify how a parameter is passed in any way. If a parameter is passed by value without "const", then it's also passed by value with "const". The problems are
a) in Pascal, the behaviour of const is completely undefined (the compiler can do whatever it wants)
b) a lot of people use Pascal "const" as translation for C's "const *" for record parameters, which is wrong (but which sometimes works by accident)

Moreover, in FPC not all platforms consistently treat const record parameters the same as a regular parameters for cdecl (at least ARM currently always passes const records by reference for some reason). In short: "const" is a mess as far as predicting how it affects passing record parameters, and hence it should never be used in record parameter declarations for external C functions.

It's possible that it has a special meaning for safecall, I don't know.

Show quoted text 7 lines
I'm asking because now I have changed my interface definition to use PGUID instead (both windows and linux) and it works more or less, but now I have a problem in the TInterfacedObject which seems to be the cause of some crashes, but I can not say by now if it is related to XPCOM exposing a different "IUnknown" (maybe cdecl instead stdcall) or TGUID related. I'll investigate further.

I think that nobody who understands both on the code generator and the RTL side of COM interfaces ever worked on XPCOM interoperability on non-Windows.

Show quoted text 15 lines
One question, is there any way in fpc to "define" the calling convention as a C define ? Just something like: {$IFDEF UNIX} {$DEFINE THECALLING cdecl} {$ELSE} {$DEFINE THECALLING safecall} {$ENDIF} function x: integer; THECALLING; I had replaced all safecalls with: {$IFDEF xx}safecall{$ENDIF}{$IFDEF y}cdecl{$ENDIF} But it is quite "awful" :)

Yes, it is possible with {$macro on}. Example from the opengl unit:

{$MODE Delphi}
{$MACRO ON}
{$IFDEF Windows}
   {$DEFINE extdecl := stdcall}
{$ELSE}
   {$DEFINE extdecl := cdecl}
   {$IFDEF MorphOS}
     {$INLINE ON}
     {$DEFINE GL_UNIT}
   {$ELSE}
     {$LINKLIB c}
   {$ENDIF}
{$ENDIF}
...
var
   glAccum: procedure(op: GLenum; value: GLfloat); extdecl;

I'm personally not really interested in fixing this whole mess though. I spend a lot of time in the past disabling safecall in the compiler for all non-Windows platforms because it always generated wrong code elsewhere. Now it seems it should be
a) enabled everywhere, and then
b) fixed for all platforms, which requires modification to the parameter managers of most architectures (since most don't contain any support for safecall)

Jonas

Marcos Douglas
to Jonas Maebe

2010/6/7 Jonas Maebe :

Show quoted text 8 lines
(...) In C, "const" does not modify how a parameter is passed in any way. If a parameter is passed by value without "const", then it's also passed by value with "const". The problems are a) in Pascal, the behaviour of const is completely undefined (the compiler can do whatever it wants)

The compiler can do whatever is wants?!

Marcos Douglas

Jonas Maebe
to Marcos Douglas
Marcos Douglas wrote 7 lines
2010/6/7 Jonas Maebe : > > a) in Pascal, the behaviour of const is completely undefined (the > compiler > can do whatever it wants) The compiler can do whatever is wants?!

At the implementation level, yes.

Jonas

Marcos Douglas
to Jonas Maebe
Jonas Maebe wrote 12 lines
On 07 Jun 2010, at 16:30, Marcos Douglas wrote: > 2010/6/7 Jonas Maebe : >> >> a) in Pascal, the behaviour of const is completely undefined (the >> compiler >> can do whatever it wants) > > The compiler can do whatever is wants?! At the implementation level, yes.

Well... just to I will know. A long time ago (Delphi 4) I learned: Always use 'const' for 'strings' because this is faster. Strings will be passed by reference. This remains true?

eg:

var str: string;
procedure Foo1(const s: string);
procedure Foo2(s: string);

I know in Foo1 the 's' parameter is readonly and in Foo2 not. But Foo1 and Foo2 are different for the compiler at the level otimization, even I not change the 's' value inside the procedure?

Thanks and sorry for the newbie question ;-)

Marcos Douglas

Jonas Maebe
to Marcos Douglas
Marcos Douglas wrote
Well... just to I will know. A long time ago (Delphi 4) I learned: Always use 'const' for 'strings' because this is faster. Strings will be passed by reference.

It can be faster to pass very short strings by value instead of by reference. That's why "const" does not specify how a parameter is passed.

This remains true?

What you can count on is that a "const" parameter should never be slower than a regular value parameter. How exactly it is implemented is however something you have no control over.

Jonas

José Mejuto
to Jonas Maebe: It is completely ignored. So it is the same as the default calling convention (which is…

Hello FPC-Pascal,

you wrote
> It is "silently" changed by stdcall do not ?

JM> It is completely ignored. So it is the same as the default calling JM> convention (which is register on i386, and stdcall=register=cdecl=... JM> on other platforms).

Wow, that looks "dangerous" at least it should produce a warning when used in interfaces, or as safecall is a form of stdcall (with return values check) replace by stdcall.

Show quoted text 5 lines
> JM> a) the result is passed as a hidden parameter by reference > JM> b) the actual function results becomes an error code > Plain function result is in EAX, if I call (in ASM) QueryInterface > with garbaged IIDs I get "$80004002" in the EAX (NOT_INTERFACE). Just > the same as in Windows using safecall except the stack clean.

The functions declared in the interfaces now with "cdecl" crashes the program as the stack remains unbalanced after call due extra 4 bytes in it, which i think it is the "hidden parameter result".

JM> Ok. In the mean time I found some more info about how XPCOM works: JM> http://benjamin.smedbergs.us/blog/2008-09-26/allocated-memory-and-shared-library-boundaries/ JM> : apparently, all code that makes use of XPCOM via C/C++ is supposed JM> to be written in IDL, which is then transformed into C++ code that JM> conforms to the safecall conventions (apart from the caller rather JM> than the callee cleaning up).

Yes, that's the "whole picture".

> To check this I must compile something with GCC and I do not have it > installed and I do not have a source to compile with it. I should > investigate following other paths.

JM> At first sight that does not happen in XPCOM, even on Windows (but JM> since we do it on Windows, maybe we should also do it elsewhere for JM> consistency reasons).

Maybe, but that's not a great problem, maybe do it only in the case of
"Verify method calls" ?

JM> In C, "const" does not modify how a parameter is passed in any way. If JM> a parameter is passed by value without "const", then it's also passed JM> by value with "const". The problems are
JM> a) in Pascal, the behaviour of const is completely undefined (the JM> compiler can do whatever it wants)
JM> b) a lot of people use Pascal "const" as translation for C's "const *" JM> for record parameters, which is wrong (but which sometimes works by JM> accident)

Yes, but removing the "const" still passes the parameter as 16 bytes, and defining it as "var" creates problems when passing an interface to extract the GUID instead passing the interface reference. Anyway most of this problems are quite easy to be bypassed with some source changes which still remain compatible between windows and linux.

The great problem now is the extra parameter in the stack which imposes me to wrap all interfaces in some kind of assembler functions to compensate the stack.

JM> Moreover, in FPC not all platforms consistently treat const record JM> parameters the same as a regular parameters for cdecl (at least ARM JM> currently always passes const records by reference for some reason). JM> In short: "const" is a mess as far as predicting how it affects JM> passing record parameters, and hence it should never be used in record JM> parameter declarations for external C functions.

I understand, but now the question is which one to use instead ? const passes 16 bytes, var does not allow to pass a constant and no definition passes also 16 bytes. Maybe use a formal parameter ?

Show quoted text 7 lines
> I'm asking because now I have changed > my interface definition to use PGUID instead (both windows and linux) > and it works more or less, but now I have a problem in the > TInterfacedObject which seems to be the cause of some crashes, but I > can not say by now if it is related to XPCOM exposing a different > "IUnknown" (maybe cdecl instead stdcall) or TGUID related. I'll > investigate further.

JM> I think that nobody who understands both on the code generator and the JM> RTL side of COM interfaces ever worked on XPCOM interoperability on JM> non-Windows.

This has been solved the "QueryReferent" function must be stdcall and pass TGUID because it is being called by fpc.

> One question, is there any way in fpc to "define" the calling > convention as a C define ? Just something like:

JM> Yes, it is possible with {$macro on}. Example from the opengl unit: JM> {$MODE Delphi}
JM> {$MACRO ON}
JM> {$IFDEF Windows}
JM> {$DEFINE extdecl := stdcall}
JM> {$ELSE}
JM> {$DEFINE extdecl := cdecl}
JM> {$IFDEF MorphOS}
JM> {$INLINE ON}
JM> {$DEFINE GL_UNIT}
JM> {$ELSE}
JM> {$LINKLIB c}
JM> {$ENDIF}
JM> {$ENDIF}
JM> ...
JM> var
JM> glAccum: procedure(op: GLenum; value: GLfloat); extdecl;

Ok, I'll use it. I hate macros, but this time they seems quite useful.

JM> I'm personally not really interested in fixing this whole mess though. JM> I spend a lot of time in the past disabling safecall in the compiler JM> for all non-Windows platforms because it always generated wrong code JM> elsewhere. Now it seems it should be
JM> a) enabled everywhere, and then
JM> b) fixed for all platforms, which requires modification to the JM> parameter managers of most architectures (since most don't contain any JM> support for safecall)

That looks as a hard job :( I can only help testing XPCOM in both windows and linux, but I doubt that I can help in the compiler side :(

Maybe it is possible in some way to define some kind of plugable calling convention generator ? Just like a script to define the calling convention when it is not any of the standard ? Thinking again, it does not look like a great idea :( Now I must think in how to solve the extra integer in the stack when calling functions :(

Best regards, José
Jonas Maebe
to José Mejuto
José Mejuto wrote
Wow, that looks "dangerous" at least it should produce a warning when used in interfaces, or as safecall is a form of stdcall (with return values check) replace by stdcall.

Yes, probably. The reason was that nobody knew that this calling convention made any sense at all on non-Windows platforms, so its appearance was assumed to be a porting artefact.

Show quoted text 6 lines
JM> At first sight that does not happen in XPCOM, even on Windows (but JM> since we do it on Windows, maybe we should also do it elsewhere for JM> consistency reasons). Maybe, but that's not a great problem, maybe do it only in the case of "Verify method calls" ?

Only if it's only done on Windows in that case as well.

Show quoted text 9 lines
JM> b) a lot of people use Pascal "const" as translation for C's "const *" JM> for record parameters, which is wrong (but which sometimes works by JM> accident) Yes, but removing the "const" still passes the parameter as 16 bytes, and defining it as "var" creates problems when passing an interface to extract the GUID instead passing the interface reference. Anyway most of this problems are quite easy to be bypassed with some source changes which still remain compatible between windows and linux.

In such cases, the parameter should be a pointer. It's probably "const" for Delphi compatibility (which probably guarantees passing const records by reference for safecall, and hence FPC probably also does on platforms that support it).

Show quoted text 11 lines
JM> I'm personally not really interested in fixing this whole mess though. JM> I spend a lot of time in the past disabling safecall in the compiler JM> for all non-Windows platforms because it always generated wrong code JM> elsewhere. Now it seems it should be JM> a) enabled everywhere, and then JM> b) fixed for all platforms, which requires modification to the JM> parameter managers of most architectures (since most don't contain any JM> support for safecall) That looks as a hard job :( I can only help testing XPCOM in both windows and linux, but I doubt that I can help in the compiler side :(

You should file a bug report.

Jonas

Marcos Douglas
to Jonas Maebe: It can be faster to pass very short strings by value instead of by reference. That's why…
Jonas Maebe wrote 9 lines
On 07 Jun 2010, at 16:59, Marcos Douglas wrote: > Well... just to I will know. A long time ago (Delphi 4) I learned: > Always use 'const' for 'strings' because this is faster. Strings will > be passed by reference. It can be faster to pass very short strings by value instead of by reference. That's why "const" does not specify how a parameter is passed.

In others words, the compiler choice, right?

Show quoted text 5 lines
> This remains true? What you can count on is that a "const" parameter should never be slower than a regular value parameter. How exactly it is implemented is however something you have no control over.

Even variable different of string?

Marcos Douglas

José Mejuto
to Jonas Maebe: Yes, probably. The reason was that nobody knew that this calling convention made any…

Hello FPC-Pascal,

you wrote 5 lines
> Yes, but removing the "const" still passes the parameter as 16 bytes, > and defining it as "var" creates problems when passing an interface to > extract the GUID instead passing the interface reference. Anyway most > of this problems are quite easy to be bypassed with some source > changes which still remain compatible between windows and linux.

JM> In such cases, the parameter should be a pointer. It's JM> probably "const" for Delphi compatibility (which probably JM> guarantees passing const records by reference for safecall, and JM> hence FPC probably also does on platforms that support it).

The interfaces has a TGUID special handling when passing an interface definition as parameter:

interface MyInterfaceDefinition
  procedure Something(const X: TGUID);

when calling this method:

var
  MyIntf: MyInterfaceDefinition;
begin
  MyInft.Something(MyIntf);
end;

This code is replaced by the MyInterfaceDefinition GUID, so in this case PGUID is not valid because @MyInft does not point to the same PGUID information. I think it is an "on the fly" compiler trick to simplify GUID operations, which are "easy" to solve using the consts with TGUID values usually present in headers, but as they are not a must some problems when porting could happend.

> That looks as a hard job :( I can only help testing XPCOM in both > windows and linux, but I doubt that I can help in the compiler side :(

JM> You should file a bug report.

I'll file one ASAP. Thank you.

Best regards, José
Luiz Americo Pereira Camara
to Marcos Douglas: Well... just to I will know. A long time ago (Delphi 4) I learned: Always use 'const' for…
Marcos Douglas wrote 12 lines
Well... just to I will know. A long time ago (Delphi 4) I learned: Always use 'const' for 'strings' because this is faster. Strings will be passed by reference. This remains true? eg: var str: string; procedure Foo1(const s: string); procedure Foo2(s: string);

Take a look at
http://lazarusroad.blogspot.com/2008/11/effect-of-using-constant-parameter-for.html

Luiz

Luiz Americo Pereira Camara
to Marcos Douglas
Marcos Douglas wrote 20 lines
Hi Luiz, On Mon, Jun 7, 2010 at 10:35 PM, Luiz Americo Pereira Camara wrote: > Take a look at > http://lazarusroad.blogspot.com/2008/11/effect-of-using-constant-parameter-for.html > > Then, this continue true how I said in other mail: On Mon, Jun 7, 2010 at 11:59 AM, Marcos Douglas wrote: > Always use 'const' for 'strings' because this is faster. Strings will > be passed by reference. > And about not string types, is more faster too? eg: foo (const i: Integer)

No. I did this test too. Double should also benefit but the generated assembler is equal with or without const.

Luiz

Marcos Douglas
to Luiz Americo Pereira Camara
Luiz Americo Pereira Camara wrote 28 lines
Marcos Douglas escreveu: > > Hi Luiz, > > On Mon, Jun 7, 2010 at 10:35 PM, Luiz Americo Pereira Camara > wrote: > >> >> Take a look at >> >> http://lazarusroad.blogspot.com/2008/11/effect-of-using-constant-parameter-for.html >> >> > > Then, this continue true how I said in other mail: > > On Mon, Jun 7, 2010 at 11:59 AM, Marcos Douglas wrote: > >> >> Always use 'const' for 'strings' because this is faster. Strings will >> be passed by reference. >> > > And about not string types, is more faster too? eg: foo (const i: Integer) > No. I did this test too. Double should also benefit but the generated assembler is equal with or without const.

Exactly.
Therefore I always used 'const' only at 'strings' params. I thought it had changed, but not. ;-)

Marcos Dougla

Luiz Americo Pereira Camara
to Marcos Douglas
Marcos Douglas wrote 36 lines
On Tue, Jun 8, 2010 at 12:55 PM, Luiz Americo Pereira Camara wrote: > Marcos Douglas escreveu: > >> Hi Luiz, >> >> On Mon, Jun 7, 2010 at 10:35 PM, Luiz Americo Pereira Camara >> wrote: >> >> >>> Take a look at >>> >>> http://lazarusroad.blogspot.com/2008/11/effect-of-using-constant-parameter-for.html >>> >>> >>> >> Then, this continue true how I said in other mail: >> >> On Mon, Jun 7, 2010 at 11:59 AM, Marcos Douglas wrote: >> >> >>> Always use 'const' for 'strings' because this is faster. Strings will >>> be passed by reference. >>> >>> >> And about not string types, is more faster too? eg: foo (const i: Integer) >> >> > No. I did this test too. Double should also benefit but the generated > assembler is equal with or without const. > Exactly. Therefore I always used 'const' only at 'strings' params. I thought it had changed, but not. ;-)

It can be useful for record parameters also

Luiz

Marcos Douglas
to Luiz Americo Pereira Camara
Luiz Americo Pereira Camara wrote 7 lines
Marcos Douglas escreveu: > Exactly. > Therefore I always used 'const' only at 'strings' params. I thought it > had changed, but not.  ;-) It can be useful for record parameters also

If the record have strings, I think.

Marcos Douglas

Frank Peelo
to Marcos Douglas
Marcos Douglas wrote 15 lines
On Tue, Jun 8, 2010 at 3:44 PM, Luiz Americo Pereira Camara wrote: > Marcos Douglas escreveu: > >> Exactly. >> Therefore I always used 'const' only at 'strings' params. I thought it >> had changed, but not. ;-) >> > It can be useful for record parameters also > > If the record have strings, I think.

Surely it's for any parameter which the called procedure does not change, which would be more efficiently passed as a reference? That would include any record larger than a few bytes, and arrays.

FP