Hi,

to interface with c libraries containing SSE code, the stack must be aligned to 16 bytes.

Currently the default -mpreferred-stack-boundary=num of gcc provides for this.

However current fpc 2.4.4 does not align the stack as such. I do not know if this correct or not when reading things like: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=40838#c8 http://groups.google.com/group/ia32-abi/browse_thread/thread/4f9b3e5069943bf1

I tried to change alignment with {$CODEALIGN 16} but this didn't work.

kind regards,

Den Jean

That's because it changes the alignment of procedures (code), rather than that of the stack. You cannot change this setting via a switch, it's hardcoded in the compiler as part of the ABI.

Changing this will also require quite a few changes in the RTL as well to be safe (e.g., in case cmem is used and if the glibc memory manager one day would use sse), because the RTL contains lots of assembler code that doesn't care at all about stack alignment. The places where there's an {$ifndef darwin} in i386-specific files can probably help, since Darwin/i386 also requires a 16-byte aligned stack. The assembler startup code for Linux/i386 probably also needs to be adapted.

Overall, it's not a trivial change to make properly.

Jonas

I'm just curious, where can I find more information about SSE alignment requirements? What exactly should be aligned - the stack segment, ESP or EBP?

A requirement like
"so try that "test $15, %esp; jnz abort" at every function" doesn't look sane to me, because every following push will result in a differently aligned ESP.

I could imagine that the addresses of SSE *operands* have to be aligned on 16 byte boundaries, what can be satisfied almost only by using accordingly aligned (static) variables, instead of pushing such *values* onto the stack. Then even local (stack based) SSE variables could be allocated in blocks 15 or 16 bytes bigger than requested, so that the references into such an block can be adjusted dynamically to a 16 bit boundary.

DoDi

Jonas Maebe wrote

That's because it changes the alignment of procedures (code), rather than that of the stack. You cannot change this setting via a switch, it's hardcoded in the compiler as part of the ABI.

The problem is that gcc already changed this long ago, and to interface with library binaries of a distribution (compiled by gcc), fpc does not provide an option. For the Qt binding I hope I solved it, so I am not in need of anything.

Thanks for the information.

regards,

Den Jean

Den Jean wrote

The problem is that gcc already changed this long ago, and to interface with library binaries of a distribution (compiled by gcc), fpc does not provide an option. For the Qt binding I hope I solved it, so I am not in need of anything.

How ?

zeljko

zeljko wrote

How ?

The binding now aligns the stack before calling the Qt libraries

Ow, we hit this problem. We are calling a library that uses opencv (where SSE instructions are used). Luckily we can add -mstackrealign to the CFLAGS of our library. People without an "interface" lib will be in trouble, though.

Henry

And now I finally know why OpenCV is so hard to handle when used from within FPC... -.-

Regards,
Sven

Den Jean wrote

On Monday 19 December 2011 08:04:30 zeljko wrote: > How ? The binding now aligns the stack before calling the Qt libraries

Would you please post an example how exactly you aligned the stack?

Regards,
Sven

He added -mincoming-stack-boundary into
-mstackrealign -mincoming-stack-boundary=2

I've tested it yesterday with various gcc versions from various distributions (gcc 4.5.1, 4.5.2, 4.6.1).
At first time Den added only -mstackrealign and that helped at that time with that gcc version (4.4.SOMETHING or 4.5.0), but decent versions crashes at various places. With mincoming-stack-boundary qt bindings doesn't crash anymore.

zeljko

Ok, that doesn't help me then, because I'm directly using OpenCV which I didn't (and don't want to) compile myself.

Regards,
Sven

Sven Barth wrote

Ok, that doesn't help me then, because I'm directly using OpenCV which I didn't (and don't want to) compile myself.

that is why I wrote that to interface with library binaries of a distribution (compiled by gcc),fpc does not provide an option.

Jonas Maebe wrote

since Darwin/i386 also requires a 16-byte aligned stack

Does this mean I can drop these stack alignment options for Osx 32 and 64 bit? Anyhow my old OsX has gcc 4.0.1 and does not have all these options. It has mstackrealign but not the -mincoming-stack-boundary=num

regards,

Den Jean

Den Jean wrote

On Sunday 18 December 2011 14:05:00 Jonas Maebe wrote: > since Darwin/i386 also requires a 16-byte aligned stack Does this mean I can drop these stack alignment options for Osx 32 and 64 bit?

Yes. Any program that does not have a 16 byte alliged stack on Mac OS X/i386 will crash almost immediately, because the dynamic linker enforces it. You also don't need it on any x86-64 platform, because they all have always required a 16 byte aligned stack.

Jonas

Jonas Maebe wrote

Yes. Any program that does not have a 16 byte alliged stack on Mac OS X/i386 will crash almost immediately, because the dynamic linker enforces it. You also don't need it on any x86-64 platform, because they all have always required a 16 byte aligned stack.

thx for the information, I will remove these options on Darwin and x86-64.

I'll test with Snow Leopard (gcc-4.2) today.
Anyway -mstackrealign means nothing on 64bit platforms afaik (will test once more again under Fedora 64bit), and it works there w//o problems.

zeljko

Hmm... I have thought a bit. Do you (or someone else) think it's feasible (and useful) to add a "procedure modifier" that allows you to specify the stack alignment that should be used when calling that procedure? (of course this modifier should be considered as "from great power comes great responsibilty" ^^)

Regards,
Sven

I think that the i386/Linux compiler and RTL should be modified to guarantee 16 byte alignment at all times, since that's how they apparently redefined their ABI. The generated code will also be backwards compatible.

Jonas

i386/Win32 should be adjusted then as well, because I suspect very much that this is the reason for my problems with OpenCV... (on Windows as well as on Linux)

Regards,
Sven

Sven Barth wrote

i386/Win32 should be adjusted then as well, because I suspect very much that this is the reason for my problems with OpenCV... (on Windows as well as on Linux)

From what I understand from http://msdn.microsoft.com/en-us/library/aa290049(v=vs.71).aspx , a function is required to align the stack to 8 or 16 bytes itself if it needs better alignment on win32. These links also all say the required alignment for win32 is 4 bytes:
* http://msdn.microsoft.com/en-us/library/k1a8ss06.aspx : Some SSE types require eight-byte stack alignment, forcing the compiler to emit dynamic stack-alignment code.
* http://old.nabble.com/ATLAS-on-win32,-pthreads,-SSE-and-stack-alignment-p19828585.html : in accordance win the Win32 ABI, only guarantees a 4-byte aligned stack
* http://eigen.tuxfamily.org/dox/TopicWrongStackAlignment.html : this assumption can be wrong on Windows, where the stack is only guaranteed to have 4-byte alignment

You should probably add the flag -mpreferred-stack-boundary=2 when compiling OpenCV for win32 using gcc: http://sourceforge.net/tracker/index.php?func=detail&aid=2170667&group_id=23725&atid=379483

Also keep in mind that switching to 16 byte aligned stacks for existing targets
a) requires many assembler routines in the RTL to be disabled (or rewritten, but that was not done for Darwin/i386, in part because in many cases it can use faster libc equivalents anyway) b) will probably break quite a bit assembler code from existing programs, both written for FPC and for Delphi (unless "nostackframe;" is added, the compiler will add stack alignment code to assembler routines which may mess up hardcoded offsets or other assumptions)

Jonas

While with OpenCV this issue might be solved by recompiling it (which I'd yet need to look into, because I'm simply using the provided binaries), what should be done about closed source code?

Somehow this motivates me more to add a "stackalignment" modifier...

Regards,
Sven

In our previous episode, Sven Barth said:

While with OpenCV this issue might be solved by recompiling it (which I'd yet need to look into, because I'm simply using the provided binaries), what should be done about closed source code?

In such case, it is better to let the vendor recompile it :-)

Sven Barth wrote

While with OpenCV this issue might be solved by recompiling it (which I'd yet need to look into, because I'm simply using the provided binaries), what should be done about closed source code?

Closed source code should be compiled with a compiler that respects the ABI by default. I guess most of it is compiled with Microsoft Visual Studio, which does presumably does that. I don't think that any compiler other than GCC can create code for Win32 that assumes a 16- byte aligned stack.

Somehow this motivates me more to add a "stackalignment" modifier...

Then you would have to add this to all routines that may (directly or indirectly) a routine from a broken library.

Jonas

My mysterious OpenCV problems are with the MSVC version of the libraries... with the GCC variant I don't even reach the point of displaying a OpenCV window ^^

> Somehow this motivates me more to add a "stackalignment" modifier... Then you would have to add this to all routines that may (directly or indirectly) a routine from a broken library.

Wouldn't it be sufficient to align the stack when calling a routine from such a library? (sorry for sounding naive here, but the requirements for stack alignment still puzzle me a bit)

Regards,
Sven

Then the problem is probably something different.

> Then you would have to add this to all routines that may (directly or > indirectly)

*call*

Keeping an aligned stack requires completely different code generation inside the calling routine, you can't implement that on a call-by-call basis.

Jonas

Jonas Maebe wrote

Keeping an aligned stack requires completely different code generation inside the calling routine, you can't implement that on a call-by-call basis.

Intel defined once how procedure entry/exit code should look like for procedures doing 16 byte alignment by themself.

Of course, main problem is, this requires another register on i386.