Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2009-11

Stack checking in dynamic libraries

5 messages · 3 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

cobines

Hello,

I have found out the following during development of my project and wanted to ask for some explanation.

I have a dynamic library which is dynamically loaded from the main thread of my program. I then create a worker thread in the main program and call the functions in the library from this worker thread. If I enable stack overflow checking in the library then whenever I call any function in that library I get a stack overflow error. This is because the StackBottom variable in fpc_stackcheck function is taken from the main thread instead of the worker thread. Is this correct or not?

My system is i386 Debian linux. I am using FPC revision 13700. I build the library and the main program with "cthreads" unit included.

I made a sample program to show some info (attached with the mail). It outputs several values describing current thread and the thread-specific stack info (from rtl/inc/systemh.inc). The worker thread has default stack size 4MB, main thread stack size is 8MB. The main thread stack address begins with BF..... , worker thread with B7..... . This is the output of the program (checkthreadinfo.lpr):

MainThreadId : 3077936832
< main thread, main program >
CurrentThreadId : 3077936832
Thread ID : 3077479680
Stack ptr : BFDBFF70
Stack top : 00000000
Stack bottom : BF5C008C
Stack length : 8388608
< main thread, library >
Libr. CurrentThreadId: 3077936832
Libr. ThreadId : 3077479680
Libr. Stack ptr : BFDBFF70
Libr. Stack top : 00000000
Libr. Stack bottom : BF5BFC94
Libr. Stack length : 8388608
< worker thread, main program >
CurrentThreadId : 3076807536
Thread ID : 3076807536
Stack ptr : B7645188
Stack top : 00000000
Stack bottom : B7245300
Stack length : 4194304
< worker thread, library >
Libr. CurrentThreadId: 3076807536
Libr. ThreadId : 3077479680
Libr. Stack ptr : B7645188
Libr. Stack top : 00000000
Libr. Stack bottom : BF5BFC94
Libr. Stack length : 8388608

The stack check will fail in < worker thread, library > case, because the StackBottom is greater than current stack pointer (it is the same value as in < main thread, library >).

If I change the program so that the worker thread loads its own instance of the library from itself (attached as
checkthreadinfoV2.lpr) then I get the following:

MainThreadId : 3076835008
< main thread, main program >
CurrentThreadId : 3076835008
Thread ID : 3076377856
Stack ptr : BFFEAA40
Stack top : 00000000
Stack bottom : BF7EAB5C
Stack length : 8388608
< main thread, library > (this library is loaded from main thread) Libr. CurrentThreadId: 3076835008
Libr. ThreadId : 3076377856
Libr. Stack ptr : BFFEAA40
Libr. Stack top : 00000000
Libr. Stack bottom : BF7EA764
Libr. Stack length : 8388608
< worker thread, main program >
CurrentThreadId : 3076283248
Thread ID : 3076283248
Stack ptr : B75C5188
Stack top : 00000000
Stack bottom : B71C5300
Stack length : 4194304
< worker thread, library > (this library is loaded from worker thread) Libr. CurrentThreadId: 3076283248
Libr. ThreadId : 3076377856
Libr. Stack ptr : B75C5188
Libr. Stack top : 00000000
Libr. Stack bottom : B6DC4EAC
Libr. Stack length : 8388608

Stack check will now not fail.

But stack length in < worker thread, library > shows 8MB, not 4MB (which should be the thread's stack size). Does code executed in dynamic libraries have a different stack? It seems not, because the current stack pointer is the same in both < worker thread, main program > and < worker thread, library >. According to the numbers the stack for < worker thread, main program > is at B71C5300-B75C5300 and the stack for < worker thread, library >
the addresses overlap. Is this correct? Moreover, in < worker thread, library > the current stack pointer is outside the stack range.

An unrelated question: why there are two thread identifiers? There's GetCurrentThreadId in rtl/inc/thread.inc and ThreadId in rtl/inc/systemh.inc. The CurrentThreadId is consistent - the same in both < main thread, ... > cases, and the same in both <worker thread , ... > cases. But ThreadId in < worker thread, library > shows the same value as in < main thread, main program > and in < main thread, library >.

I'd be grateful for any explanations, or any pointers if I'm doing anything wrong.

cobines

Reply by email

cobines
to cobines

A followup from the previous mail. I have expanded the program to query pthreads library directly for thread's stack info. Here are the results (the lines containing "Pthr" string have values reported by the pthreads library):

MainThreadId : 3076675264
< main thread, main program >
CurrentThreadId : 3076675264
Thread ID : 3078065408
Stack ptr : BFC4B964
Stack top : 00000000
Stack bottom : BF44BA8C
Stack length : 8388608
Pthr ThreadId : 3076675264
Pthr Stack ptr : BFC4B928
Pthr Stack top : BFC4D000
Pthr Stack bot : BF44D000
Pthr Stack Len : 8388608
< main thread, library >
Libr. CurrentThreadId: 3076675264
Libr. ThreadId : 3078065408
Libr. Stack ptr : BFC4B964
Libr. Stack top : 00000000
Libr. Stack bottom : BF44B694
Libr. Stack length : 8388608
Libr. Pthr ThreadId : 3076675264
Libr. Pthr Stack ptr : BFC4B928
Libr. Pthr Stack top : BFC4D000
Libr. Pthr Stack bot : BF44D000
Libr. Pthr Stack Len : 8388608
< worker thread, main program >
CurrentThreadId : 3075664752
Thread ID : 3075664752
Stack ptr : B752E17C
Stack top : 00000000
Stack bottom : B712E300
Stack length : 4194304
Pthr ThreadId : 3075664752
Pthr Stack ptr : B752E140
Pthr Stack top : B752F000
Pthr Stack bot : B712E000
Pthr Stack Len : 4198400
< worker thread, library >
Libr. CurrentThreadId: 3075664752
Libr. ThreadId : 3078065408
Libr. Stack ptr : B752E17C
Libr. Stack top : 00000000
Libr. Stack bottom : BF44B694
Libr. Stack length : 8388608
Libr. Pthr ThreadId : 3075664752
Libr. Pthr Stack ptr : B752E140
Libr. Pthr Stack top : B752F000
Libr. Pthr Stack bot : B712E000
Libr. Pthr Stack Len : 4198400

The values for < worker thread, library > reported by pthreads appear to be correct.
First, the stack size is ~ 4MB, which is the worker thread's stack size, although I'm don't know why the discrepancy: 4194304 vs 4198400 bytes.
Second, the stack address begins with B7.... (address of the main thread's stack begins with BF.....).
Third, the stack pointer is well within the stack limits: B712E000 < B752E140 < B752F000.

I have to conclude that the values in rtl/inc/systemh.inc: StackTop, StackBottom, StackLength are not behaving as thread-specific variables in the library, even though they are declared using ThreadVar. The IsMultiThread value in < worker thread, main program > is TRUE, but in < worker thread, library > it is FALSE. Do I have to initialize multithreading in the dynamic library somehow?

Reply by email

Jonas Maebe
to cobines

cobines wrote on Sun, 08 Nov 2009:

I have to conclude that the values in rtl/inc/systemh.inc: StackTop, StackBottom, StackLength are not behaving as thread-specific variables in the library, even though they are declared using ThreadVar.

Your problem is that every FPC-compiled library contains its own copy of the RTL. At this time you cannot create an FPC library whose RTL state is shared between a library and the main program. On Linux this will be possible after the patches attached to
http://bugs.freepascal.org/view.php?id=12492 have been fully applied (they've only been applied partially for 2.4.0).

Furthermore, stack checking is broken for multi-threaded programs under *nix in 2.2.x. It has been fixed in 2.4.0. See http://bugs.freepascal.org/view.php?id=12942 for more info. And keep in mind that it's only an approximation, and can only be used to get a rough estimate of the required stack space (as explained in the comments to that bug).

Jonas

---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

Reply by email

Joost van der Sluis
to Jonas Maebe: cobines wrote on Sun, 08 Nov 2009: Your problem is that every FPC-compiled library…
Jonas Maebe wrote 12 lines
cobines wrote on Sun, 08 Nov 2009: > I have to conclude that the values in rtl/inc/systemh.inc: StackTop, > StackBottom, StackLength are not behaving as thread-specific variables > in the library, even though they are declared using ThreadVar. Your problem is that every FPC-compiled library contains its own copy of the RTL. At this time you cannot create an FPC library whose RTL state is shared between a library and the main program. On Linux this will be possible after the patches attached to http://bugs.freepascal.org/view.php?id=12492 have been fully applied (they've only been applied partially for 2.4.0).

Wow.... I've never seen that bug-report before. This is great! And offcourse an important part of the 'packages-support' people are always asking for. Great thanks to the reporter Pierre Pede...

If we can make this stable, it would be great for the packages included by several linux-distributions/repositories...

Joost.

Reply by email