Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2010-08

Mutlithreaded Dll Callback crashes my Application

64 messages · 7 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Maik Wojcieszak

Hi,

I'm working on a port of a Delphi component to Lazarus. The component wraps a dll for Network Communication
based on ASPLs Vortex Library but implementing a full set of "easy to use" mechanisms for "daily work problems".

The dll itself is written in C++ with a C like interface. It is multithreaded and uses callbacks for a lot of purposes. The callback function/procedure is defined like

//#---------------------------------------------------------------------
//                     CALLBACK PROCEDURES
//#---------------------------------------------------------------------
procedure ExecuteCommandClb(tmlhandle : TML_COMMAND_HANDLE; pCBData :
Pointer); cdecl;
begin
  //...
end;

No matter what I do inside this procedure if I reach the end the assembler window is popping up in the debugger.
The Parameters seem to be correct.

pCBData is a Pointer to an object stored with the pointer to the callback. The procedure is used as a proxy to call the real worker method of the object.

tmlhandle is also a Pointer, but to an object that is implemented in the dll.

The callback procedure is set like

tml_Profile_Register_Cmd(FTMLCoreHandle, PCHAR(profile), cmd.CommandId,
@ExecuteCommandClb, Pointer(cmd));

This command stores the inside the dlls for callbacks.

I've set the user flag -dUseCThreads in the package and it is inherited to my project.

No matter what I do inside the ExecuteCommandClb my application will crash after some calls. The Parameters seem to be correct because for some calls they will be used correctly.

Alle this code and the dlls are tested and work stable with Delphi. That's why I hope somebody can tell me
if it is a Lazarus/FPC problem and how to solve it.

Thanks in advance
Connor

My Versions:

Lazarus IDE v0.9.28.2 Beta
MS Windows XP Prof DE sp3
FPC Version 2.2.4

Reply by email

Alexander Grau
to Maik Wojcieszak

Hello Maik,

My assumption is that you have discovered a (Win32 specific) FreePascal bug!

I had a similar issue when writing FreePascal in a *callback* of a Mac OS X libary that is called in an *non-FreePascal* thread - the reason for that crash is that the FPC runtime is not correctly initialized at the time of the callback. Jonas fixed this for Mac OS X in FreePascal revision 15557 (SVN).

At the end of that thread I asked the question there if this issue does exist for the Windows (Win32/64) target too (and if so, maybe there is already a fix for this?) - Maybe we both get an answer here of the Win32 maintainer now :-)

Here's the old thread:
http:/msg21101.html

There's a FPC test for the non-FPC callback bug (for non-Win32 targets only):
http://svn.freepascal.org/svn/fpc/trunk/tests/test/textthr.pp

Regards,
Alexander

Show quoted text 67 lines
Hi, I'm working on a port of a Delphi component to Lazarus. The component wraps a dll for Network Communication based on ASPLs Vortex Library but implementing a full set of "easy to use" mechanisms for "daily work problems". The dll itself is written in C++ with a C like interface. It is multithreaded and uses callbacks for a lot of purposes. The callback function/procedure is defined like //#--------------------------------------------------------------------- // CALLBACK PROCEDURES //#--------------------------------------------------------------------- procedure ExecuteCommandClb(tmlhandle : TML_COMMAND_HANDLE; pCBData : Pointer); cdecl; begin //... end; No matter what I do inside this procedure if I reach the end the assembler window is popping up in the debugger. The Parameters seem to be correct. pCBData is a Pointer to an object stored with the pointer to the callback. The procedure is used as a proxy to call the real worker method of the object. tmlhandle is also a Pointer, but to an object that is implemented in the dll. The callback procedure is set like tml_Profile_Register_Cmd(FTMLCoreHandle, PCHAR(profile), cmd.CommandId, @ExecuteCommandClb, Pointer(cmd)); This command stores the inside the dlls for callbacks. I've set the user flag -dUseCThreads in the package and it is inherited to my project. No matter what I do inside the ExecuteCommandClb my application will crash after some calls. The Parameters seem to be correct because for some calls they will be used correctly. Alle this code and the dlls are tested and work stable with Delphi. That's why I hope somebody can tell me if it is a Lazarus/FPC problem and how to solve it. Thanks in advance Connor My Versions: Lazarus IDE v0.9.28.2 Beta MS Windows XP Prof DE sp3 FPC Version 2.2.4 -- _______________________________________________ Lazarus mailing list http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Reply by email

Michael Schnell
to Maik Wojcieszak: I'm working on a port of a Delphi component to Lazarus. The component wraps a dll for…
Maik Wojcieszak wrote 10 lines
Hi, I'm working on a port of a Delphi component to Lazarus. The component wraps a dll for Network Communication based on ASPLs Vortex Library but implementing a full set of "easy to use" mechanisms for "daily work problems". The dll itself is written in C++ with a C like interface. It is multithreaded and uses callbacks for a lot of purposes. The callback function/procedure is defined like

OK a "flattened" interface is necessary.

Show quoted text 9 lines
//#--------------------------------------------------------------------- // CALLBACK PROCEDURES //#--------------------------------------------------------------------- procedure ExecuteCommandClb(tmlhandle : TML_COMMAND_HANDLE; pCBData : Pointer); cdecl; begin //... end;

Did you define cdecl specification both in C and in Pascal ? AFAIK, with M$ Studio this is "__declspec(dllexport)", I don't know what the GNU notation for this is.

You also can use STDCALL instead of cdecl on both sites. I seem to remember the GNU C notation is __STDCALL, but this might be wrong.

Show quoted text 8 lines
No matter what I do inside this procedure if I reach the end the assembler window is popping up in the debugger. The Parameters seem to be correct. pCBData is a Pointer to an object stored with the pointer to the callback. The procedure is used as a proxy to call the real worker method of the object.

You can't use C++ objects in Pascal. You need to flatten the interface completely by explicitly communicating addresses of flat functions to do a callback. This can be quite complicated :(.

tmlhandle is also a Pointer, but to an object that is implemented in the dll.

Also pointers need to be "flat" and not point to C++ (or FPC) objects.

Of course a multithreaded DLL offers some even more advanced problems. I would do a test with a singlethreaded DLL just to test all aspects of the interface to your FPC project and activate the threads later.

It should be no problem if a thread in a DLL calls a callback to your Pascal project. (We do this with Delphi and a PJ-SIP DLL.) But of course such a callback can't do any "visual" LCL stuff (like writing to the screen). Here you need to store the information (e.g. in a FIFO done with a TThreadList) and use the mainthread to do the LCL based stuff.

-Michael

Reply by email

Maik Wojcieszak
to Michael Schnell

Michael,

Show quoted text 12 lines
On 08/03/2010 10:40 AM, Maik Wojcieszak wrote: > Hi, > > I'm working on a port of a Delphi component to Lazarus. The component > wraps a dll for Network Communication > based on ASPLs Vortex Library but implementing a full set of "easy to > use" mechanisms for "daily work problems". > > The dll itself is written in C++ with a C like interface. It is > multithreaded and uses callbacks for a lot of purposes. > The callback function/procedure is defined like OK a "flattened" interface is necessary.

The interface is as flat as possible.

Show quoted text 15 lines
> > //#--------------------------------------------------------------------- > // CALLBACK PROCEDURES > //#--------------------------------------------------------------------- > procedure ExecuteCommandClb(tmlhandle : TML_COMMAND_HANDLE; pCBData : > Pointer); cdecl; > begin > //... > end; Did you define cdecl specification both in C and in Pascal ? AFAIK, with M$ Studio this is "__declspec(dllexport)", I don't know what the GNU notation for this is. You also can use STDCALL instead of cdecl on both sites. I seem to remember the GNU C notation is __STDCALL, but this might be wrong.

I am using __declspec(dllexport) on windows with Visual Studio 2005. I know that I can use stdcall instead but currently I saw now reason to do so.

Show quoted text 12 lines
> > No matter what I do inside this procedure if I reach the end the > assembler window is popping up in the debugger. > The Parameters seem to be correct. > > pCBData is a Pointer to an object stored with the pointer to the > callback. The procedure is used as a proxy to call > the real worker method of the object. You can't use C++ objects in Pascal. You need to flatten the interface completely by explicitly communicating addresses of flat functions to do a callback. This can be quite complicated :(.

That's the problem with most language bindings. The object I've mentioned is implented in Pascal. It's only the pointer which is stored and passesd to the callback to access it.
This is quite common to achieve a binding to an object instance with a flat interface. A pointer or integer is saved with the callback pointer and passed to it if it is called.

> > tmlhandle is also a Pointer, but to an object that is implemented in > the dll. Also pointers need to be "flat" and not point to C++ (or FPC) objects.

From the viewpoint of the application the pointer is a number. There is no structure behind it. Only the dll knows what to do with it when it is passed to it. In fact it doesn't matter
what it is pointing to as long as there is no need to access the data directly.

Of course a multithreaded DLL offers some even more advanced problems. I would do a test with a singlethreaded DLL just to test all aspects of the interface to your FPC project and activate the threads later.

Great, let me know if there is some more information I can provide.

Show quoted text 8 lines
It should be no problem if a thread in a DLL calls a callback to your Pascal project. (We do this with Delphi and a PJ-SIP DLL.) But of course such a callback can't do any "visual" LCL stuff (like writing to the screen). Here you need to store the information (e.g. in a FIFO done with a TThreadList) and use the mainthread to do the LCL based stuff.

That's what I thought too.
I know about the LCL limitations regarding multithreading. It is the same with VCL. There should be now problem to use objects dreived from TComponent as long as they
don't do any visual stuff.

Maik

Show quoted text 9 lines
-Michael -- _______________________________________________ Lazarus mailing list http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Reply by email

Alexander Grau
to Maik Wojcieszak
Show quoted text 6 lines
> That's what I thought too. I know about the LCL limitations regarding multithreading. It is the same with VCL. There should be now problem to use objects dreived from TComponent as long as they don't do any visual stuff.

Try the following to limit the problem: in your callback do not call *ANY* FPC runtime related code - Just a "Win32 MessagBox(0, "hi", "text", MB_OK)". Add the process id (GetProcessId) to that dialog output and also in your main program (to see if you are really in an external thread).

If that works, call a simple global FPC function (no object) and again just that callback there.

After that works, just a simple FPC object call and again that message box.

I still assume in the compiler version you are using, you simply can't call FPC objects inside external threads.

Also have a look at the mantis - e.g. see if 0012987 is related to your problem? (http://bugs.freepascal.org/view.php?id=12987)

I might be fully wrong with my assumptions, I'm just thinking loudly because I had the same issue - just another architecture :-)

Reply by email

Maik Wojcieszak
to Alexander Grau

Alexander,

Show quoted text 6 lines
Try the following to limit the problem: in your callback do not call *ANY* FPC runtime related code - Just a "Win32 MessagBox(0, "hi", "text", MB_OK)". Add the process id (GetProcessId) to that dialog output and also in your main program (to see if you are really in an external thread).

I am in an external thread.

Show quoted text 8 lines
If that works, call a simple global FPC function (no object) and again just that callback there. After that works, just a simple FPC object call and again that message box. I still assume in the compiler version you are using, you simply can't call FPC objects inside external threads.

this is the full code of the callback

procedure ExecuteCommandClb(tmlhandle : TML_COMMAND_HANDLE; pCBData :
Pointer); cdecl;
var
  hsidex : SIDEX_HANDLE;
  val    : SIDEX_VARIANT;
begin
  tml_Cmd_Get_Sidex_Handle(tmlhandle, at hsidex);
  if assigned(hsidex) then
  begin
    val := sidex_Variant_New_Boolean(true) ;
    sidex_Variant_Write(hsidex,'Answer', 'Accept', val);
    sidex_Variant_DecRef(val);
  end;
end;

no FPC objects are called. I've thought that this might be the reason and changed the code to try, but it didn't work either. All calls are API calls of my dll.

Also have a look at the mantis - e.g. see if 0012987 is related to your problem? (http://bugs.freepascal.org/view.php?id=12987)

This sounds pretty much like my problem.
I'm new to lazarus and I've downloaded the installer package as it is. Can I change the FPC Compiler without changing my current IDE and how can I do this ?
I think it might be a good idea to use the current version to double check if the problem is still there.
The latest release is 2.4.0. Why does the lazarus installer for windows install 2.2.4, it seems a little bit out of date ?

I might be fully wrong with my assumptions, I'm just thinking loudly because I had the same issue - just another architecture :-)

I am happy about any hint to solve this one. The dll is already ported to Linux and Mac OS-X and I've decided to use lazarus for some demo applications.

Maik

Reply by email

Michael Schnell
to Maik Wojcieszak: The interface is as flat as possible. I am using __declspec(dllexport) on windows with…
Maik Wojcieszak wrote
I am using __declspec(dllexport) on windows with Visual Studio 2005.

Exactly what my colleagues do - just using Delphi instead of FPC/Lazarus.

I know that I can use stdcall instead but currently I saw now reason to do so.

Agreed.

That's the problem with most language bindings. The object I've mentioned is implented in Pascal. It's only the pointer which is stored and passesd to the callback to access it.

OK passing a pointer to the C DLL just to store it, providing it in a callback to Pascal and use it there should be OK.

Show quoted text 7 lines
> Of course a multithreaded DLL offers some even more advanced > problems. I would do a test with a singlethreaded DLL just to test > all aspects of the interface to your FPC project and activate the > threads later. Great, let me know if there is some more information I can provide.

Ooops. "I would..." did not mean an offer to actually do it here :(. Take it for "you should" :).

I know about the LCL limitations regarding multithreading. It is the same with VCL. There should be now problem to use objects dreived from TComponent as long as they don't do any visual stuff.

Unfortunately this nowhere is decently documented - neither with Lazarus nor with Delphi. But we handle this in the same way....

-Michael

Reply by email

Michael Schnell
to Alexander Grau: Try the following to limit the problem: in your callback do not call *ANY* FPC runtime…

It's a known problem that a DLL done in FPC has it's own memory manager unless you do certain additional things to unify the memory managers of the DLL and the main program. This is known to create at least heap memory holes just wen passing strings between the main program and the DLL. Maybe if using TThread this hits even "automatically".

But as a C DLL does not use Delphi strings nor TThread and needs to use it's own heap, anyway, I suppose the problem is not related.

-Michael

Reply by email

Alexander Grau
to Maik Wojcieszak: I am in an external thread. this is the full code of the callback procedure…

Maik,

I have written the counterpart of the "textthr.pp" test (I mentioned before) for Win32 below - it works for my compiler (release 2.4.0 ). Maybe you want to try it on your version (2.2.4) too.

Now I think your problem is more related to something inside the DLL (just another assumption). Try to do just a simple call inside your callback to that DLL, and do not create or pass FPC strings ('someString') and do not call anything from the FPC runtime (assigned(x)). If that it still does work, you have some call parameter issues. If it works, try to use more and more from the runtime (e.g. try to use PChar strings instead of strings)....

Regards,
Alexander

program project1;
{$APPTYPE CONSOLE}
{$mode objfpc}{$H+}

uses

  {$IFDEF UNIX}{$IFDEF UseCThreads}
  cthreads,
  {$ENDIF}{$ENDIF}
  Classes,
  { you can add units after this }
  windows;
{$R project1.res}
type
  TSomeClass = class
  public
    procedure someMethod();
  end;
procedure TSomeClass.someMethod();
begin
  writeln('TSomeClass.someMethod called');
end;
var
  ThreadID: DWORD;       //Thread-ID
  ThreadHandle: THandle; //R?ckgabewert von CreateThread
  someObj: TSomeClass;
procedure someFunc(param: pchar); cdecl;
begin
  WriteLn('External is called - ThreadID=', GetCurrentThreadId);
  if assigned(param) then writeln('blub');
  writeln('param=', param);
  someObj.someMethod();
end;
function ExternalThread(zahl: Pointer): LongInt; stdcall;
begin
  //Sleep(2000);
  someFunc('test');
  Result:=0;
end;
begin
  WriteLn('Main ThreadID=', GetCurrentThreadId);
  someObj:=TSomeClass.create;
  WriteLn('Creating external thread');
  ThreadHandle:=CreateThread(nil, 0, TFNThreadStartRoutine(@ExternalThread),
    nil, 0, ThreadID);
  if ThreadHandle = 0 then writeln('ERROR creating external thread');
  readln;
  WriteLn('Freeing external thread');
  if ThreadHandle0 then CloseHandle(ThreadHandle);
  someObj.free;
end.
Show quoted text 64 lines
Alexander, > Try the following to limit the problem: in your callback do not call > *ANY* FPC runtime related code - Just a "Win32 MessagBox(0, "hi", > "text", MB_OK)". Add the process id (GetProcessId) to that dialog > output and also in your main program (to see if you are really in an > external thread). > I am in an external thread. > If that works, call a simple global FPC function (no object) and > again just that callback there. > > After that works, just a simple FPC object call and again that > message box. > > I still assume in the compiler version you are using, you simply > can't call FPC objects inside external threads. this is the full code of the callback procedure ExecuteCommandClb(tmlhandle : TML_COMMAND_HANDLE; pCBData : Pointer); cdecl; var hsidex : SIDEX_HANDLE; val : SIDEX_VARIANT; begin tml_Cmd_Get_Sidex_Handle(tmlhandle, at hsidex); if assigned(hsidex) then begin val := sidex_Variant_New_Boolean(true) ; sidex_Variant_Write(hsidex,'Answer', 'Accept', val); sidex_Variant_DecRef(val); end; end; no FPC objects are called. I've thought that this might be the reason and changed the code to try, but it didn't work either. All calls are API calls of my dll. > > Also have a look at the mantis - e.g. see if 0012987 is related to > your problem? (http://bugs.freepascal.org/view.php?id=12987) This sounds pretty much like my problem. I'm new to lazarus and I've downloaded the installer package as it is. Can I change the FPC Compiler without changing my current IDE and how can I do this ? I think it might be a good idea to use the current version to double check if the problem is still there. The latest release is 2.4.0. Why does the lazarus installer for windows install 2.2.4, it seems a little bit out of date ? > > > I might be fully wrong with my assumptions, I'm just thinking loudly > because I had the same issue - just another architecture :-) I am happy about any hint to solve this one. The dll is already ported to Linux and Mac OS-X and I've decided to use lazarus for some demo applications. Maik -- _______________________________________________ Lazarus mailing list http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Reply by email

Maik Wojcieszak
to Michael Schnell: Exactly what my colleagues do - just using Delphi instead of FPC/Lazarus. Agreed. OK…

Michael,

Show quoted text 9 lines
> >> Of course a multithreaded DLL offers some even more advanced >> problems. I would do a test with a singlethreaded DLL just to test >> all aspects of the interface to your FPC project and activate the >> threads later. > > Great, let me know if there is some more information I can provide. Ooops. "I would..." did not mean an offer to actually do it here :(. Take it for "you should" :).

Ok, wishful thinking ;-)
I will write a test project right now.

Maik

Reply by email

Vincent Snijders
to Maik Wojcieszak: I am in an external thread. this is the full code of the callback procedure…

2010/8/3 Maik Wojcieszak :

The latest release is 2.4.0. Why does the lazarus installer for windows install 2.2.4, it seems ?a little bit out of date ?

FPC 2.2.4 was the latest release, when the latest Lazarus release was made. FPC 2.4.0 contains some bugs with respect to winlike resources, which disqualifies it for use with a Lazarus release. So no new Lazarus release will be made until fpc 2.4.2 has been released.

Vincent

Reply by email

Maik Wojcieszak
to Alexander Grau: I have written the counterpart of the "textthr.pp" test (I mentioned before) for Win32…

Alexander,

This is the result of the test:

G:\Workdir HEAD\cb Test\thr_test>project1.exe
Main ThreadID=864
Creating external thread
External is called - ThreadID=6000
blub
param=test
TSomeClass.someMethod called

Freeing external thread

G:\Workdir HEAD\cb Test\thr_test>

it's ok, isn't it ?

Maik

Show quoted text 164 lines
Maik, I have written the counterpart of the "textthr.pp" test (I mentioned before) for Win32 below - it works for my compiler (release 2.4.0 ). Maybe you want to try it on your version (2.2.4) too. Now I think your problem is more related to something inside the DLL (just another assumption). Try to do just a simple call inside your callback to that DLL, and do not create or pass FPC strings ('someString') and do not call anything from the FPC runtime (assigned(x)). If that it still does work, you have some call parameter issues. If it works, try to use more and more from the runtime (e.g. try to use PChar strings instead of strings).... Regards, Alexander program project1; {$APPTYPE CONSOLE} {$mode objfpc}{$H+} uses {$IFDEF UNIX}{$IFDEF UseCThreads} cthreads, {$ENDIF}{$ENDIF} Classes, { you can add units after this } windows; {$R project1.res} type TSomeClass = class public procedure someMethod(); end; procedure TSomeClass.someMethod(); begin writeln('TSomeClass.someMethod called'); end; var ThreadID: DWORD; //Thread-ID ThreadHandle: THandle; //R?ckgabewert von CreateThread someObj: TSomeClass; procedure someFunc(param: pchar); cdecl; begin WriteLn('External is called - ThreadID=', GetCurrentThreadId); if assigned(param) then writeln('blub'); writeln('param=', param); someObj.someMethod(); end; function ExternalThread(zahl: Pointer): LongInt; stdcall; begin //Sleep(2000); someFunc('test'); Result:=0; end; begin WriteLn('Main ThreadID=', GetCurrentThreadId); someObj:=TSomeClass.create; WriteLn('Creating external thread'); ThreadHandle:=CreateThread(nil, 0, TFNThreadStartRoutine(@ExternalThread), nil, 0, ThreadID); if ThreadHandle = 0 then writeln('ERROR creating external thread'); readln; WriteLn('Freeing external thread'); if ThreadHandle0 then CloseHandle(ThreadHandle); someObj.free; end. Maik Wojcieszak schrieb: > Alexander, > >> Try the following to limit the problem: in your callback do not >> call *ANY* FPC runtime related code - Just a "Win32 MessagBox(0, >> "hi", "text", MB_OK)". Add the process id (GetProcessId) to that >> dialog output and also in your main program (to see if you are >> really in an external thread). >> > I am in an external thread. >> If that works, call a simple global FPC function (no object) and >> again just that callback there. >> >> After that works, just a simple FPC object call and again that >> message box. >> >> I still assume in the compiler version you are using, you simply >> can't call FPC objects inside external threads. > this is the full code of the callback > > procedure ExecuteCommandClb(tmlhandle : TML_COMMAND_HANDLE; pCBData : > Pointer); cdecl; > var > hsidex : SIDEX_HANDLE; > val : SIDEX_VARIANT; > begin > tml_Cmd_Get_Sidex_Handle(tmlhandle, at hsidex); > if assigned(hsidex) then > begin > val := sidex_Variant_New_Boolean(true) ; > sidex_Variant_Write(hsidex,'Answer', 'Accept', val); > sidex_Variant_DecRef(val); > end; > end; > > no FPC objects are called. I've thought that this might be the reason > and changed the code to try, but it didn't work either. > All calls are API calls of my dll. >> >> Also have a look at the mantis - e.g. see if 0012987 is related to >> your problem? (http://bugs.freepascal.org/view.php?id=12987) > This sounds pretty much like my problem. > I'm new to lazarus and I've downloaded the installer package as it > is. Can I change the FPC Compiler without changing my current IDE and > how can I do this ? > I think it might be a good idea to use the current version to double > check if the problem is still there. > The latest release is 2.4.0. Why does the lazarus installer for > windows install 2.2.4, it seems a little bit out of date ? >> >> >> I might be fully wrong with my assumptions, I'm just thinking loudly >> because I had the same issue - just another architecture :-) > I am happy about any hint to solve this one. The dll is already > ported to Linux and Mac OS-X and I've decided to use lazarus for > some demo applications. > > Maik > > -- > _______________________________________________ > Lazarus mailing list > > http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus > -- _______________________________________________ Lazarus mailing list http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Reply by email

Alexander Grau
to Maik Wojcieszak
Show quoted text 17 lines
Alexander, This is the result of the test: G:\Workdir HEAD\cb Test\thr_test>project1.exe Main ThreadID=864 Creating external thread External is called - ThreadID=6000 blub param=test TSomeClass.someMethod called Freeing external thread G:\Workdir HEAD\cb Test\thr_test> it's ok, isn't it ?

No crash => it is ok - As I wrote before, on the Mac counterpart of this test failed for me - In this test you can see an external thread (like C does) that is calling the FPC runtime.

Because it succeeds, I think there are must be another issues in your case as the principal test works on your side!

After you managed to isolate your problem in another test program, I would be interested to see it :-)

Regards,
Alexander

Reply by email

Maik Wojcieszak
to Alexander Grau

Hi,

After my holiday I've done done more research on this problem and hope I can get an answer while continuing this thread.

My Versions (again)

Lazarus IDE v0.9.28.2 Beta
MS Windows XP Prof DE sp3
FPC Version 2.2.4

What I basically do is passing a pointer to a calback funtion to a dll writte in c++ (Visual Studio 2005) and call it from there. ** The follwing code is from the example attached to this mail **

procedure OnExecuteClb(pCBData : Pointer); cdecl;
begin
...
end;

The function is passed to the dll like this

...
     FHandle    := uVCMTDLL.vcmtdll_NewThread();
     uVCMTDLL.vcmtdll_RegisterCallback(FHandle, at OnExecuteClb,self);
...

The test includes a dll which has the following interface

...
#define VCMTDLL_API __declspec(dllexport)

namespace VCMTDLL {

typedef void* MT_THREAD_HANDLE;

extern "C" {

VCMTDLL_API MT_THREAD_HANDLE vcmtdll_NewThread(); // create a new thread object
VCMTDLL_API void vcmtdll_DeleteThread(MT_THREAD_HANDLE ahandle); // delete an existing thread object
VCMTDLL_API void vcmtdll_RegisterCallback(MT_THREAD_HANDLE ahandle, void* pCBFunc, void* pCBData); // pass a callback to execute inside the thread context
VCMTDLL_API void vcmtdll_StartThread(MT_THREAD_HANDLE ahandle, unsigned int iDelay); // start the execution with a delay of <iDelay> milliseconds

}// extern "C"

...

with a loop in the callback function everything is fine

...
    x := 0;
    while x < 100 do
    begin
      x := x + 1;
    end;
...

I can execute it without problems.

BUT if I create and destroy an object inside the procedure the app crahes or hangs after several runs of the test (between 1 an 10 starts).

the object:

...
TVCMTDummyObject = class
private
public
       Tag : Integer;
end;
...

the code in the callback

...
    a := TVCMTDummyObject.Create;
    a.Free;
...

Another problem is that a PostMessage from the callback's thread context immediatly crashes the app. But this may be resolved later.

If there is anybody who uses a similar mechanism in his lazarus project and knows how to solve this problem it'd be great. The demo project together with dll is attached too.

Best Regards
Maik

-------------- next part --------------
A non-text attachment was scrubbed...
Name: cb Test.zip
Type: application/x-zip-compressed
Size: 107105 bytes
Desc: not available
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20100824/8b587cb6/attachment-0001.bin>

Reply by email

Michael Schnell
to Maik Wojcieszak
Maik Wojcieszak wrote
BUT if I create and destroy an object inside the procedure the app crahes or hangs after several runs of the test (between 1 an 10 starts).

I suppose you need a unified memory manager for the dll and the program. I do know that several such products are available for Delphi, but I don't know if/how they or whatever can be used for Lazarus projects.

-Michael

Reply by email

José Mejuto
to Maik Wojcieszak: After my holiday I've done done more research on this problem and hope I can get an…

Hello Lazarus-List,

Tuesday, August 24, 2010, 1:41:51 PM, you wrote:

MW> After my holiday I've done done more research on this problem and hope I MW> can get an answer while continuing this thread. MW> My Versions (again)
MW> Lazarus IDE v0.9.28.2 Beta
MW> MS Windows XP Prof DE sp3
MW> FPC Version 2.2.4
MW> What I basically do is passing a pointer to a calback funtion to a dll MW> writte in c++ (Visual Studio 2005) and call it from there. MW> ** The follwing code is from the example attached to this mail **

I was trying to test your code but I was totally unable to load the DLL (I have VC++ 2005 redist. installed).

Best regards, Jos?

Reply by email

Marco van de Voort
to Maik Wojcieszak: After my holiday I've done done more research on this problem and hope I can get an…
Maik Wojcieszak wrote 12 lines
After my holiday I've done done more research on this problem and hope I can get an answer while continuing this thread. My Versions (again) Lazarus IDE v0.9.28.2 Beta MS Windows XP Prof DE sp3 FPC Version 2.2.4 What I basically do is passing a pointer to a calback funtion to a dll writte in c++ (Visual Studio 2005) and call it from there. ** The follwing code is from the example attached to this mail **

Is something exported, but not explicitely declared __cdecl really cdecl?

Try stdcall?

Reply by email

Maik Wojcieszak
to José Mejuto: MW> After my holiday I've done done more research on this problem and hope I MW> can get…

Jos?,

I've packed the runtime env dlls and put them on my ftp server. They are too big to attach to the mail on the list.

Server : support-ftp.wobe-team.com
User : lazarus
Pwd : test

The file u need : Microsoft.VC80.CRT.zip

Put the files into the directory of the dll and it works.

Best Regards
Maik

Show quoted text 18 lines
Hello Lazarus-List, Tuesday, August 24, 2010, 1:41:51 PM, you wrote: MW> After my holiday I've done done more research on this problem and hope I MW> can get an answer while continuing this thread. MW> My Versions (again) MW> Lazarus IDE v0.9.28.2 Beta MW> MS Windows XP Prof DE sp3 MW> FPC Version 2.2.4 MW> What I basically do is passing a pointer to a calback funtion to a dll MW> writte in c++ (Visual Studio 2005) and call it from there. MW> ** The follwing code is from the example attached to this mail ** I was trying to test your code but I was totally unable to load the DLL (I have VC++ 2005 redist. installed).

Reply by email

Maik Wojcieszak
to Marco van de Voort: Is something exported, but not explicitely declared __cdecl really cdecl? Try stdcall?

Marco,

Thanks for your reply.

Show quoted text 18 lines
On Tue, Aug 24, 2010 at 01:41:51PM +0200, Maik Wojcieszak wrote: > After my holiday I've done done more research on this problem and hope I > can get an answer while continuing this thread. > > My Versions (again) > > Lazarus IDE v0.9.28.2 Beta > MS Windows XP Prof DE sp3 > FPC Version 2.2.4 > > What I basically do is passing a pointer to a calback funtion to a dll > writte in c++ (Visual Studio 2005) and call it from there. > ** The follwing code is from the example attached to this mail ** > Is something exported, but not explicitely declared __cdecl really cdecl?

I am using cdecl in Delphi and it works very well. So, I guess it is ok. I've tested stdcall and it show's the same problem.

Maik

Show quoted text 10 lines
Try stdcall? -- _______________________________________________ Lazarus mailing list http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Reply by email

Marco van de Voort
to Maik Wojcieszak
Maik Wojcieszak wrote 5 lines
> > Is something exported, but not explicitely declared __cdecl really cdecl? > I am using cdecl in Delphi and it works very well. So, I guess it is ok. I've tested stdcall and it show's the same problem.

Then I don't know. The only thing I can recommend is to be very careful with "works in Delphi" like statements, since that can be coincidence due to memory layout.

Reply by email

Maik Wojcieszak
to Marco van de Voort

Marco,

Show quoted text 13 lines
On Wed, Aug 25, 2010 at 09:28:14AM +0200, Maik Wojcieszak wrote: >> Is something exported, but not explicitely declared __cdecl really cdecl? >> >> > I am using cdecl in Delphi and it works very well. So, I guess it is ok. > I've tested stdcall and it show's the same problem. > Then I don't know. The only thing I can recommend is to be very careful with "works in Delphi" like statements, since that can be coincidence due to memory layout.

Yes, it can be coincidence and that is another reason, why I really like to know what I am doing wrong here.
There is at least nothing special in using a flat dll interface in another language. Also using callbacks
is nothing special. The only thing is the multithreaded callback which is a more special, but should be
standard when using a high level language on a multithreaded OS.

I am porting the components to linux now and try if I have the same problem there. I am using
gcc on linux to build the shared libs. Let's see if this leads us to a solution.

Maik

Reply by email

Alexander Grau
to Maik Wojcieszak: After my holiday I've done done more research on this problem and hope I can get an…

Hello Maik!

I have tried out your test program, and I get the crash too. However your program is already doing 10 things at the same time ;-) - This is too difficult to find out the real problem - you need to scale down everything ... So, I have written a minimal test program for you here to start with - it works for me (no crash), how does it work at your side?

Regards,
Alexander

program project1;
{$mode objfpc}{$H+}

uses

  {$IFDEF UNIX}{$IFDEF UseCThreads}
  cthreads,
  {$ENDIF}{$ENDIF}
  Classes,
  sysutils
  { you can add units after this };
{$R project1.res}
type
  MT_THREAD_HANDLE   = Pointer;
  tdummy = class
    name: string;
    constructor create(aname: string);
    procedure test;
  end;
function vcmtdll_NewThread: MT_THREAD_HANDLE; stdcall; external
'vcmtdll.dll';
procedure vcmtdll_DeleteThread(ahandle : MT_THREAD_HANDLE); stdcall;
external 'vcmtdll.dll';
procedure vcmtdll_RegisterCallback(ahandle : MT_THREAD_HANDLE; pCBFunc :
Pointer; pCBData : Pointer); stdcall; external 'vcmtdll.dll';
procedure vcmtdll_StartThread(ahandle : MT_THREAD_HANDLE; iDelay :
Cardinal); stdcall; external 'vcmtdll.dll';
var
  handle: MT_THREAD_HANDLE;
constructor tdummy.create(aname: string);
begin
  inherited create;
  name:=aname;
end;
procedure tdummy.test;
begin
  writeln('tdummy.test: ', name);
end;
procedure OnExecuteClb(pCBData : Pointer); cdecl;
var
  dummy: tdummy;
  i: integer;
begin
  writeln('onExecuteClb');
  dummy:=tdummy.create('callbackdummy');
  tdummy(pCBData).test;
  for i:=1 to 30 do
    dummy.test();
  dummy.free;
end;
var
  globaldummy: tdummy;
begin
  globaldummy:=tdummy.create('globaldummy');
  writeln('start');
  Handle  := vcmtdll_NewThread();
  writeln('handle=', hexStr(handle));
  vcmtdll_RegisterCallback(Handle, @OnExecuteClb, globaldummy);
  vcmtdll_StartThread(handle, 0);
  sleep(1000);
  vcmtdll_DeleteThread(handle);
  writeln('bye');
  globaldummy.free;
end.
Show quoted text 111 lines
Hi, After my holiday I've done done more research on this problem and hope I can get an answer while continuing this thread. My Versions (again) Lazarus IDE v0.9.28.2 Beta MS Windows XP Prof DE sp3 FPC Version 2.2.4 What I basically do is passing a pointer to a calback funtion to a dll writte in c++ (Visual Studio 2005) and call it from there. ** The follwing code is from the example attached to this mail ** procedure OnExecuteClb(pCBData : Pointer); cdecl; begin ... end; The function is passed to the dll like this ... FHandle := uVCMTDLL.vcmtdll_NewThread(); uVCMTDLL.vcmtdll_RegisterCallback(FHandle, at OnExecuteClb,self); ... The test includes a dll which has the following interface ... #define VCMTDLL_API __declspec(dllexport) namespace VCMTDLL { typedef void* MT_THREAD_HANDLE; extern "C" { VCMTDLL_API MT_THREAD_HANDLE vcmtdll_NewThread(); // create a new thread object VCMTDLL_API void vcmtdll_DeleteThread(MT_THREAD_HANDLE ahandle); // delete an existing thread object VCMTDLL_API void vcmtdll_RegisterCallback(MT_THREAD_HANDLE ahandle, void* pCBFunc, void* pCBData); // pass a callback to execute inside the thread context VCMTDLL_API void vcmtdll_StartThread(MT_THREAD_HANDLE ahandle, unsigned int iDelay); // start the execution with a delay of <iDelay> milliseconds }// extern "C" ... with a loop in the callback function everything is fine ... x := 0; while x < 100 do begin x := x + 1; end; ... I can execute it without problems. BUT if I create and destroy an object inside the procedure the app crahes or hangs after several runs of the test (between 1 an 10 starts). the object: ... TVCMTDummyObject = class private public Tag : Integer; end; ... the code in the callback ... a := TVCMTDummyObject.Create; a.Free; ... Another problem is that a PostMessage from the callback's thread context immediatly crashes the app. But this may be resolved later. If there is anybody who uses a similar mechanism in his lazarus project and knows how to solve this problem it'd be great. The demo project together with dll is attached too. Best Regards Maik ------------------------------------------------------------------------ -- _______________________________________________ Lazarus mailing list http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Reply by email

Maik Wojcieszak
to Alexander Grau

Alexander,

I found something in the lazarus package for vortex :

{ FPC specific thread create function to replace Vortex's thread create. This is required because FPC doesn't work when C libraries create their own threads}

This may lead us to a solution I guess.
Some more background about that would be nice. Is this documented anywhere ?

Maik

Show quoted text 221 lines
Hello Maik! I have tried out your test program, and I get the crash too. However your program is already doing 10 things at the same time ;-) - This is too difficult to find out the real problem - you need to scale down everything ... So, I have written a minimal test program for you here to start with - it works for me (no crash), how does it work at your side? Regards, Alexander program project1; {$mode objfpc}{$H+} uses {$IFDEF UNIX}{$IFDEF UseCThreads} cthreads, {$ENDIF}{$ENDIF} Classes, sysutils { you can add units after this }; {$R project1.res} type MT_THREAD_HANDLE = Pointer; tdummy = class name: string; constructor create(aname: string); procedure test; end; function vcmtdll_NewThread: MT_THREAD_HANDLE; stdcall; external 'vcmtdll.dll'; procedure vcmtdll_DeleteThread(ahandle : MT_THREAD_HANDLE); stdcall; external 'vcmtdll.dll'; procedure vcmtdll_RegisterCallback(ahandle : MT_THREAD_HANDLE; pCBFunc : Pointer; pCBData : Pointer); stdcall; external 'vcmtdll.dll'; procedure vcmtdll_StartThread(ahandle : MT_THREAD_HANDLE; iDelay : Cardinal); stdcall; external 'vcmtdll.dll'; var handle: MT_THREAD_HANDLE; constructor tdummy.create(aname: string); begin inherited create; name:=aname; end; procedure tdummy.test; begin writeln('tdummy.test: ', name); end; procedure OnExecuteClb(pCBData : Pointer); cdecl; var dummy: tdummy; i: integer; begin writeln('onExecuteClb'); dummy:=tdummy.create('callbackdummy'); tdummy(pCBData).test; for i:=1 to 30 do dummy.test(); dummy.free; end; var globaldummy: tdummy; begin globaldummy:=tdummy.create('globaldummy'); writeln('start'); Handle := vcmtdll_NewThread(); writeln('handle=', hexStr(handle)); vcmtdll_RegisterCallback(Handle, @OnExecuteClb, globaldummy); vcmtdll_StartThread(handle, 0); sleep(1000); vcmtdll_DeleteThread(handle); writeln('bye'); globaldummy.free; end. Maik Wojcieszak schrieb: > Hi, > > After my holiday I've done done more research on this problem and hope I > can get an answer while continuing this thread. > > My Versions (again) > > Lazarus IDE v0.9.28.2 Beta > MS Windows XP Prof DE sp3 > FPC Version 2.2.4 > > What I basically do is passing a pointer to a calback funtion to a dll > writte in c++ (Visual Studio 2005) and call it from there. > ** The follwing code is from the example attached to this mail ** > > procedure OnExecuteClb(pCBData : Pointer); cdecl; > begin > ... > end; > > The function is passed to the dll like this > > ... > FHandle := uVCMTDLL.vcmtdll_NewThread(); > uVCMTDLL.vcmtdll_RegisterCallback(FHandle, at OnExecuteClb,self); > ... > > > The test includes a dll which has the following interface > > ... > #define VCMTDLL_API __declspec(dllexport) > > namespace VCMTDLL { > > typedef void* MT_THREAD_HANDLE; > > extern "C" { > > VCMTDLL_API MT_THREAD_HANDLE vcmtdll_NewThread(); // create a new > thread object > VCMTDLL_API void vcmtdll_DeleteThread(MT_THREAD_HANDLE ahandle); // > delete an existing thread object > VCMTDLL_API void vcmtdll_RegisterCallback(MT_THREAD_HANDLE ahandle, > void* pCBFunc, void* pCBData); // pass a callback to execute inside the > thread context > VCMTDLL_API void vcmtdll_StartThread(MT_THREAD_HANDLE ahandle, unsigned > int iDelay); // start the execution with a delay of <iDelay> > milliseconds > > }// extern "C" > > ... > > with a loop in the callback function everything is fine > > ... > x := 0; > while x < 100 do > begin > x := x + 1; > end; > ... > > I can execute it without problems. > > BUT if I create and destroy an object inside the procedure the app > crahes or hangs after several runs of the test (between 1 an 10 starts). > > the object: > > ... > TVCMTDummyObject = class > private > public > Tag : Integer; > end; > ... > > the code in the callback > > ... > a := TVCMTDummyObject.Create; > a.Free; > ... > > Another problem is that a PostMessage from the callback's thread context > immediatly crashes the app. But this may be resolved later. > > If there is anybody who uses a similar mechanism in his lazarus project > and knows how to solve this problem it'd be great. > The demo project together with dll is attached too. > > Best Regards > Maik > > > > > > > > > > ------------------------------------------------------------------------ > > -- > _______________________________________________ > Lazarus mailing list > > http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus > -- _______________________________________________ Lazarus mailing list http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Reply by email

Sven Barth
to Maik Wojcieszak
Maik Wojcieszak wrote 13 lines
Alexander, I found something in the lazarus package for vortex : { FPC specific thread create function to replace Vortex's thread create. This is required because FPC doesn't work when C libraries create their own threads} This may lead us to a solution I guess. Some more background about that would be nice. Is this documented anywhere ? Maik

You can read this thread on the fpc-pascal list:
http://www.hu.freepascal.org/lists/fpc-pascal/2010-July/025939.html

Jonas has implemented a possibilty for external C threads to work, but you need to use a SVN FPC and you need to create at least one thread in FPC (it can terminate immediatly, it's just needed to initialise the threading subsystem). (And this fix is *nix only)

Regards,
Sven

Reply by email

Alexander Grau
to Maik Wojcieszak: I found something in the lazarus package for vortex : { FPC specific thread create…

Maik,

Did you try out the console test program I posted? It uses your DLL that runs a C thread and that C thread calls back the FPC function. Even more, this FPC callback function then calls a FPC object method.

My program shows that there is nothing wrong with FPC (on Win32) and external C threads - it just works.

My assumption : there is something else wrong in your code - use my code and add one piece after the other of your code. For example, first just try to make my test app a simple LCL app (just a simple form). If that works too, add the dynamic function mapping you used. If that works too, make the wrapper (TComponent) you wrote etc.

Again : You can call FPC functions/objects in external C threads - there must be something else wrong in your program ;-) ...

Alexander

Show quoted text 249 lines
Alexander, I found something in the lazarus package for vortex : { FPC specific thread create function to replace Vortex's thread create. This is required because FPC doesn't work when C libraries create their own threads} This may lead us to a solution I guess. Some more background about that would be nice. Is this documented anywhere ? Maik Alexander Grau schrieb: > Hello Maik! > > I have tried out your test program, and I get the crash too. However > your program is already doing 10 things at the same time ;-) - This > is too difficult to find out the real problem - you need to scale > down everything ... So, I have written a minimal test program for you > here to start with - it works for me (no crash), how does it work at > your side? > > Regards, > Alexander > > > program project1; > > {$mode objfpc}{$H+} > > uses > {$IFDEF UNIX}{$IFDEF UseCThreads} > cthreads, > {$ENDIF}{$ENDIF} > Classes, > sysutils > { you can add units after this }; > > {$R project1.res} > > type > MT_THREAD_HANDLE = Pointer; > > tdummy = class > name: string; > constructor create(aname: string); > procedure test; > end; > > > function vcmtdll_NewThread: MT_THREAD_HANDLE; stdcall; external > 'vcmtdll.dll'; > procedure vcmtdll_DeleteThread(ahandle : MT_THREAD_HANDLE); stdcall; > external 'vcmtdll.dll'; > procedure vcmtdll_RegisterCallback(ahandle : MT_THREAD_HANDLE; > pCBFunc : Pointer; pCBData : Pointer); stdcall; external 'vcmtdll.dll'; > procedure vcmtdll_StartThread(ahandle : MT_THREAD_HANDLE; iDelay : > Cardinal); stdcall; external 'vcmtdll.dll'; > > var > handle: MT_THREAD_HANDLE; > > > constructor tdummy.create(aname: string); > begin > inherited create; > name:=aname; > end; > > procedure tdummy.test; > begin > writeln('tdummy.test: ', name); > end; > > procedure OnExecuteClb(pCBData : Pointer); cdecl; > var > dummy: tdummy; > i: integer; > begin > writeln('onExecuteClb'); > dummy:=tdummy.create('callbackdummy'); > tdummy(pCBData).test; > for i:=1 to 30 do > dummy.test(); > dummy.free; > end; > > var > globaldummy: tdummy; > > begin > globaldummy:=tdummy.create('globaldummy'); > > writeln('start'); > Handle := vcmtdll_NewThread(); > writeln('handle=', hexStr(handle)); > > vcmtdll_RegisterCallback(Handle, @OnExecuteClb, globaldummy); > vcmtdll_StartThread(handle, 0); > > sleep(1000); > > vcmtdll_DeleteThread(handle); > writeln('bye'); > globaldummy.free; > end. > > > > > > > > Maik Wojcieszak schrieb: >> Hi, >> >> After my holiday I've done done more research on this problem and >> hope I >> can get an answer while continuing this thread. >> >> My Versions (again) >> >> Lazarus IDE v0.9.28.2 Beta >> MS Windows XP Prof DE sp3 >> FPC Version 2.2.4 >> >> What I basically do is passing a pointer to a calback funtion to a dll >> writte in c++ (Visual Studio 2005) and call it from there. >> ** The follwing code is from the example attached to this mail ** >> >> procedure OnExecuteClb(pCBData : Pointer); cdecl; >> begin >> ... >> end; >> >> The function is passed to the dll like this >> >> ... >> FHandle := uVCMTDLL.vcmtdll_NewThread(); >> uVCMTDLL.vcmtdll_RegisterCallback(FHandle, at OnExecuteClb,self); >> ... >> >> >> The test includes a dll which has the following interface >> >> ... >> #define VCMTDLL_API __declspec(dllexport) >> >> namespace VCMTDLL { >> >> typedef void* MT_THREAD_HANDLE; >> >> extern "C" { >> >> VCMTDLL_API MT_THREAD_HANDLE vcmtdll_NewThread(); // create a new >> thread object >> VCMTDLL_API void vcmtdll_DeleteThread(MT_THREAD_HANDLE ahandle); // >> delete an existing thread object >> VCMTDLL_API void vcmtdll_RegisterCallback(MT_THREAD_HANDLE ahandle, >> void* pCBFunc, void* pCBData); // pass a callback to execute inside the >> thread context >> VCMTDLL_API void vcmtdll_StartThread(MT_THREAD_HANDLE ahandle, unsigned >> int iDelay); // start the execution with a delay of <iDelay> >> milliseconds >> >> }// extern "C" >> >> ... >> >> with a loop in the callback function everything is fine >> >> ... >> x := 0; >> while x < 100 do >> begin >> x := x + 1; >> end; >> ... >> >> I can execute it without problems. >> >> BUT if I create and destroy an object inside the procedure the app >> crahes or hangs after several runs of the test (between 1 an 10 >> starts). >> >> the object: >> >> ... >> TVCMTDummyObject = class >> private >> public >> Tag : Integer; >> end; >> ... >> >> the code in the callback >> >> ... >> a := TVCMTDummyObject.Create; >> a.Free; >> ... >> >> Another problem is that a PostMessage from the callback's thread >> context >> immediatly crashes the app. But this may be resolved later. >> >> If there is anybody who uses a similar mechanism in his lazarus project >> and knows how to solve this problem it'd be great. >> The demo project together with dll is attached too. >> >> Best Regards >> Maik >> >> >> >> >> >> >> >> >> >> ------------------------------------------------------------------------ >> >> >> -- >> _______________________________________________ >> Lazarus mailing list >> >> http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus >> > > > -- > _______________________________________________ > Lazarus mailing list > > http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus > -- _______________________________________________ Lazarus mailing list http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Reply by email

Alexander Grau
to Sven Barth: You can read this thread on the fpc-pascal list:…
Show quoted text 23 lines
Am 25.08.2010 13:53, schrieb Maik Wojcieszak: > Alexander, > > I found something in the lazarus package for vortex : > > { FPC specific thread create function to replace Vortex's thread create. > This is required because FPC doesn't work when C libraries create > their own > threads} > > This may lead us to a solution I guess. > Some more background about that would be nice. Is this documented > anywhere ? > > Maik You can read this thread on the fpc-pascal list: http://www.hu.freepascal.org/lists/fpc-pascal/2010-July/025939.html Jonas has implemented a possibilty for external C threads to work, but you need to use a SVN FPC and you need to create at least one thread in FPC (it can terminate immediatly, it's just needed to initialise the threading subsystem). (And this fix is *nix only)

If you read this thread carefully, you can notice that I already posted this here in my first answer to Maik ;-)

I also showed in a test program in a later post that this problem doesn't seem to exist in the Win32 target, it was just the Darwin target. Here's again the test program I wrote . I creates a non-FPC thread and inside that thread it calls back a FPC funtion and object. This fully works without crashes.

program project1;
{$APPTYPE CONSOLE}
{$mode objfpc}{$H+}

uses
{$IFDEF UNIX}{$IFDEF UseCThreads}
cthreads,
{$ENDIF}{$ENDIF}
Classes,
{ you can add units after this }
windows;

{$R project1.res}
type
 TSomeClass = class
 public
   procedure someMethod();
 end;
procedure TSomeClass.someMethod();
begin
 writeln('TSomeClass.someMethod called');
end;
var
 ThreadID: DWORD;       //Thread-ID
 ThreadHandle: THandle; //R?ckgabewert von CreateThread
 someObj: TSomeClass;
procedure someFunc(param: pchar); cdecl;
begin
 WriteLn('External is called - ThreadID=', GetCurrentThreadId);
 if assigned(param) then writeln('blub');
 writeln('param=', param);
 someObj.someMethod();
end;
function ExternalThread(zahl: Pointer): LongInt; stdcall;
begin
 //Sleep(2000);
 someFunc('test');
 Result:=0;
end;
begin
 WriteLn('Main ThreadID=', GetCurrentThreadId);
 someObj:=TSomeClass.create;
 WriteLn('Creating external thread');
 ThreadHandle:=CreateThread(nil, 0, TFNThreadStartRoutine(@ExternalThread),
   nil, 0, ThreadID);
 if ThreadHandle = 0 then writeln('ERROR creating external thread');
 readln;
 WriteLn('Freeing external thread');
 if ThreadHandle0 then CloseHandle(ThreadHandle);
 someObj.free;
end.

Reply by email

Sven Barth
to Alexander Grau
Alexander Grau wrote 11 lines
Sven Barth schrieb: > You can read this thread on the fpc-pascal list: > http://www.hu.freepascal.org/lists/fpc-pascal/2010-July/025939.html > > Jonas has implemented a possibilty for external C threads to work, but > you need to use a SVN FPC and you need to create at least one thread > in FPC (it can terminate immediatly, it's just needed to initialise > the threading subsystem). (And this fix is *nix only) If you read this thread carefully, you can notice that I already posted this here in my first answer to Maik ;-)

That was on 3rd August... How am I supposed to remember this? :P

Show quoted text 6 lines
I also showed in a test program in a later post that this problem doesn't seem to exist in the Win32 target, it was just the Darwin target. Here's again the test program I wrote . I creates a non-FPC thread and inside that thread it calls back a FPC funtion and object. This fully works without crashes.

I looked at your example program and I also looked into FPC's Thread implementation for Windows. You need to play around with the Heap a bit more to make things interesting:

Modify someFunc like in the following and someObj.someMethod won't be executed any more, because an exception is raised somewhere after the second Writeln.

Regards,
Sven

procedure someFunc(param: pchar); cdecl;
var
   t: TObject;
begin
   WriteLn('External is called - ThreadID=', GetCurrentThreadId);
   if assigned(param) then writeln('blub');
   writeln('param=', param);
   t := TObject.Create;
   try
     Writeln(t.ClassName);
   finally
     t.Free;
   end;
   // this point will not be reached
   someObj.someMethod();
end;

Reply by email

Maik Wojcieszak
to Alexander Grau: Did you try out the console test program I posted? It uses your DLL that runs a C thread…

Alexander,

I tried you're test app already and it worked as you said. So does my test app if I do only one changing you're test app to process
multiple threads at the same time it crashed too.

Show quoted text 10 lines
Maik, Did you try out the console test program I posted? It uses your DLL that runs a C thread and that C thread calls back the FPC function. Even more, this FPC callback function then calls a FPC object method. My program shows that there is nothing wrong with FPC (on Win32) and external C threads - it just works.

Well, it doesn't if you stress this stuff a bit.

Show quoted text 6 lines
My assumption : there is something else wrong in your code - use my code and add one piece after the other of your code. For example, first just try to make my test app a simple LCL app (just a simple form). If that works too, add the dynamic function mapping you used. If that works too, make the wrapper (TComponent) you wrote etc.

The question is: What is wrong ?

Again : You can call FPC functions/objects in external C threads - there must be something else wrong in your program ;-) ...

...

I've choosen the vortex (beep )binding as an example because my dll does exactly the same.
In fact it uses the vortex library to implement user protocols. There must be a reason for those guys (and the
vortex programmers) to add a mechanism to use a different threading when using FPC.
At the moment I will follow this idea to check if it solves my problem and post my result here.

Maik

Reply by email

Alexander Grau
to Sven Barth: That was on 3rd August... How am I supposed to remember this? :P I looked at your example…
Show quoted text 28 lines
Am 25.08.2010 15:46, schrieb Alexander Grau: > Sven Barth schrieb: >> You can read this thread on the fpc-pascal list: >> http://www.hu.freepascal.org/lists/fpc-pascal/2010-July/025939.html >> >> Jonas has implemented a possibilty for external C threads to work, but >> you need to use a SVN FPC and you need to create at least one thread >> in FPC (it can terminate immediatly, it's just needed to initialise >> the threading subsystem). (And this fix is *nix only) > If you read this thread carefully, you can notice that I already posted > this here in my first answer to Maik ;-) > That was on 3rd August... How am I supposed to remember this? :P > I also showed in a test program in a later post that this problem > doesn't seem to exist in the Win32 target, it was just the Darwin > target. Here's again the test program I wrote . I creates a non-FPC > thread and inside that thread it calls back a FPC funtion and object. > This fully works without crashes. > I looked at your example program and I also looked into FPC's Thread implementation for Windows. You need to play around with the Heap a bit more to make things interesting: Modify someFunc like in the following and someObj.someMethod won't be executed any more,

Using exactly your modifications, all code ist executed correctly - the line you indicated is correctly reached and executed. No 'm using FPC 2.4.0 for all my tests - I'm assuming you are using 2.2.4 ? (If that is the case there must have been a fix for this to make this bug gone away).

Show quoted text 30 lines
because an exception is raised somewhere after the second Writeln. Regards, Sven procedure someFunc(param: pchar); cdecl; var t: TObject; begin WriteLn('External is called - ThreadID=', GetCurrentThreadId); if assigned(param) then writeln('blub'); writeln('param=', param); t := TObject.Create; try Writeln(t.ClassName); finally t.Free; end; // this point will not be reached someObj.someMethod(); end; -- _______________________________________________ Lazarus mailing list http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Reply by email

Sven Barth
to Alexander Grau
Alexander Grau wrote 14 lines
Sven Barth schrieb: > I looked at your example program and I also looked into FPC's Thread > implementation for Windows. You need to play around with the Heap a > bit more to make things interesting: > > Modify someFunc like in the following and someObj.someMethod won't be > executed any more, Using exactly your modifications, all code ist executed correctly - the line you indicated is correctly reached and executed. No exceptions at all. I'm using FPC 2.4.0 for all my tests - I'm assuming you are using 2.2.4 ? (If that is the case there must have been a fix for this to make this bug gone away).

Yes, I'm using 2.2.4... shame on me for not testing with a current FPC -.-

Does it even work if you use threadvars? It should as the Heap uses threadvars, but I want to be sure. (and now I'm also curious to know why it works with 2.4.0 ^^)

Regards,
Sven

Reply by email

Alexander Grau
to Maik Wojcieszak: I tried you're test app already and it worked as you said. So does my test app if I do…

Maik,

As you did, I did write a new test program - this one uses 30 threads - it works fine (no crash) until the moment I enable the a simple 'writeln' inside a callack. You can see that it works by the increased counters for each thread that are displayed after the program successfully finished.

So: external multiple threads calling fine until you use FPC runtime code :/ ...

Regards,
Alexander

program project1;
{$mode objfpc}{$H+}

uses

  {$IFDEF UNIX}{$IFDEF UseCThreads}
  cthreads,
  {$ENDIF}{$ENDIF}
  Classes,
  sysutils
  { you can add units after this };
{$R project1.res}
type
  MT_THREAD_HANDLE   = Pointer;
  tdummy = class
    idx: integer;
    constructor create();
    procedure test;
  end;
function vcmtdll_NewThread: MT_THREAD_HANDLE; cdecl; external 'vcmtdll.dll';
procedure vcmtdll_DeleteThread(ahandle : MT_THREAD_HANDLE); cdecl;
external 'vcmtdll.dll';
procedure vcmtdll_RegisterCallback(ahandle : MT_THREAD_HANDLE; pCBFunc :
Pointer; pCBData : Pointer); cdecl; external 'vcmtdll.dll';
procedure vcmtdll_StartThread(ahandle : MT_THREAD_HANDLE; iDelay :
Cardinal); cdecl; external 'vcmtdll.dll';
const
  COUNT = 30;
var
  handles: array[1..COUNT] of MT_THREAD_HANDLE;
  objects: array[1..COUNT] of tdummy;
constructor tdummy.create();
begin
  inherited create;
  idx:=0;
end;
procedure tdummy.test;
begin
  inc(idx);
end;
procedure OnExecuteClb(pCBData : Pointer); cdecl;
var
  i: integer;
begin
  //  writeln('onExecuteClb');  <-- this line will crash
  for i:=1 to 30 do
    tdummy(pCBData).test();
end;
var
  i: integer;
begin
  writeln('start');
  for i:=1 to COUNT do objects[i]:=TDummy.create;
  for i:=1 to COUNT do
    Handles[i]  := vcmtdll_NewThread();
  for i:=1 to COUNT do
    vcmtdll_RegisterCallback(handles[i], @OnExecuteClb, objects[i]);
  for i:=1 to COUNT do
    vcmtdll_StartThread(handles[i], 1000);
  writeln('go');
  sleep(2000);
  writeln('freeing');
  for i:=1 to COUNT do
    vcmtdll_DeleteThread(handles[i]);
  for i:=1 to COUNT do writeln(objects[i].idx);
  writeln('bye');
end.
Show quoted text 49 lines
Alexander, I tried you're test app already and it worked as you said. So does my test app if I do only one changing you're test app to process multiple threads at the same time it crashed too. Alexander Grau schrieb: > > > Maik, > > Did you try out the console test program I posted? It uses your DLL > that runs a C thread and that C thread calls back the FPC function. > Even more, this FPC callback function then calls a FPC object method. > > My program shows that there is nothing wrong with FPC (on Win32) and > external C threads - it just works. Well, it doesn't if you stress this stuff a bit. > > My assumption : there is something else wrong in your code - use my > code and add one piece after the other of your code. For example, > first just try to make my test app a simple LCL app (just a simple > form). If that works too, add the dynamic function mapping you used. > If that works too, make the wrapper (TComponent) you wrote etc. The question is: What is wrong ? > > Again : You can call FPC functions/objects in external C threads - > there must be something else wrong in your program ;-) ... ... I've choosen the vortex (beep )binding as an example because my dll does exactly the same. In fact it uses the vortex library to implement user protocols. There must be a reason for those guys (and the vortex programmers) to add a mechanism to use a different threading when using FPC. At the moment I will follow this idea to check if it solves my problem and post my result here. Maik -- _______________________________________________ Lazarus mailing list http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Reply by email

Alexander Grau
to Sven Barth: Yes, I'm using 2.2.4... shame on me for not testing with a current FPC -.- Does it even…
Show quoted text 23 lines
Am 25.08.2010 16:12, schrieb Alexander Grau: > Sven Barth schrieb: >> I looked at your example program and I also looked into FPC's Thread >> implementation for Windows. You need to play around with the Heap a >> bit more to make things interesting: >> >> Modify someFunc like in the following and someObj.someMethod won't be >> executed any more, > Using exactly your modifications, all code ist executed correctly - the > line you indicated is correctly reached and executed. No exceptions > at all. > > I'm using FPC 2.4.0 for all my tests - I'm assuming you are using 2.2.4 > ? (If that is the case there must have been a fix for this to make this > bug gone away). > Yes, I'm using 2.2.4... shame on me for not testing with a current FPC -.- Does it even work if you use threadvars? It should as the Heap uses threadvars, but I want to be sure. (and now I'm also curious to know why it works with 2.4.0 ^^)

I have modified the test - now it's using multiple external threads and that works fine too until the moment I enable any 'writeln'. Using threadvars or not doesn't make any difference. Using any writeln will make it crash. If not using writeln, the external threads can call FPC objects to increase some counters that are correctly displayed at the end of the program for each thread.

program project1;
{$APPTYPE CONSOLE}
{$mode objfpc}{$H+}

uses

  {$IFDEF UNIX}{$IFDEF UseCThreads}
  cthreads,
  {$ENDIF}{$ENDIF}
  Classes,
  { you can add units after this }
  windows;
{$R project1.res}
type
  TSomeClass = class
  public
    procedure someMethod(idx: integer);
  end;
const
  COUNT = 30;
var
  i: integer;
  threadid: DWORD;
  ThreadHandles: array[1..COUNT] of THandle; //R?ckgabewert von CreateThread
  counters: array[1..COUNT] of integer;
  someObj: TSomeClass;
procedure TSomeClass.someMethod(idx: integer);
begin
  //writeln('TSomeClass.someMethod called');
  inc(counters[idx]);
end;
procedure someFunc(param: pointer); cdecl;
var
  t: tobject;
begin
  //WriteLn('External is called - ThreadID=', GetCurrentThreadId);
  //if assigned(param) then writeln('blub');
  //writeln('param=', param);
  t := TObject.Create;
  try
    //Writeln(t.ClassName); // <--- here we get a crash
  finally
    t.Free;
  end;
  someObj.someMethod(integer(param));
end;
function ExternalThread(param: Pointer): LongInt; stdcall;
begin
  someFunc(param);
  Result:=0;
end;
begin
  WriteLn('Main ThreadID=', GetCurrentThreadId);
  someObj:=TSomeClass.create;
  for i:=1 to COUNT do counters[i]:=0;
  WriteLn('Creating external threads');
  for i:=1 to COUNT do
  begin
    ThreadHandles[i]:=CreateThread(nil, 0,
TFNThreadStartRoutine(@ExternalThread),
      pointer(i), 0, ThreadID);
    if ThreadHandles[i] = 0 then writeln('ERROR creating external thread');
  end;
  readln;
  WriteLn('Freeing external thread');
  for i:=1 to COUNT do
  begin
    if ThreadHandles[i]0 then CloseHandle(ThreadHandles[i]);
  end;
  someObj.free;
  for i:=1 to COUNT do
    writeln (counters[i]);
end.

Reply by email

Sven Barth
to Alexander Grau
Alexander Grau wrote 9 lines
> Does it even work if you use threadvars? It should as the Heap uses > threadvars, but I want to be sure. (and now I'm also curious to know > why it works with 2.4.0 ^^) I have modified the test - now it's using multiple external threads and that works fine too until the moment I enable any 'writeln'. Using threadvars or not doesn't make any difference. Using any writeln will make it crash. If not using writeln, the external threads can call FPC objects to increase some counters that are correctly displayed at the end of the program for each thread.

So the runtime must be tested more when using external threads... Does the writeln work when you use FPC threads?

Regards,
Sven

Reply by email

Maik Wojcieszak
to Alexander Grau: As you did, I did write a new test program - this one uses 30 threads - it works fine (no…

Alexander,

Not only writeln crashes the app. It is not really predictibable what exactly crashes the app.
normally it should be ok to use PostMessage inside a thread to send a message to the main thread. BUT
doing this crashes the app immediatly.

Maik

Show quoted text 166 lines
Maik, As you did, I did write a new test program - this one uses 30 threads - it works fine (no crash) until the moment I enable the a simple 'writeln' inside a callack. You can see that it works by the increased counters for each thread that are displayed after the program successfully finished. So: external multiple threads calling fine until you use FPC runtime code :/ ... Regards, Alexander program project1; {$mode objfpc}{$H+} uses {$IFDEF UNIX}{$IFDEF UseCThreads} cthreads, {$ENDIF}{$ENDIF} Classes, sysutils { you can add units after this }; {$R project1.res} type MT_THREAD_HANDLE = Pointer; tdummy = class idx: integer; constructor create(); procedure test; end; function vcmtdll_NewThread: MT_THREAD_HANDLE; cdecl; external 'vcmtdll.dll'; procedure vcmtdll_DeleteThread(ahandle : MT_THREAD_HANDLE); cdecl; external 'vcmtdll.dll'; procedure vcmtdll_RegisterCallback(ahandle : MT_THREAD_HANDLE; pCBFunc : Pointer; pCBData : Pointer); cdecl; external 'vcmtdll.dll'; procedure vcmtdll_StartThread(ahandle : MT_THREAD_HANDLE; iDelay : Cardinal); cdecl; external 'vcmtdll.dll'; const COUNT = 30; var handles: array[1..COUNT] of MT_THREAD_HANDLE; objects: array[1..COUNT] of tdummy; constructor tdummy.create(); begin inherited create; idx:=0; end; procedure tdummy.test; begin inc(idx); end; procedure OnExecuteClb(pCBData : Pointer); cdecl; var i: integer; begin // writeln('onExecuteClb'); <-- this line will crash for i:=1 to 30 do tdummy(pCBData).test(); end; var i: integer; begin writeln('start'); for i:=1 to COUNT do objects[i]:=TDummy.create; for i:=1 to COUNT do Handles[i] := vcmtdll_NewThread(); for i:=1 to COUNT do vcmtdll_RegisterCallback(handles[i], @OnExecuteClb, objects[i]); for i:=1 to COUNT do vcmtdll_StartThread(handles[i], 1000); writeln('go'); sleep(2000); writeln('freeing'); for i:=1 to COUNT do vcmtdll_DeleteThread(handles[i]); for i:=1 to COUNT do writeln(objects[i].idx); writeln('bye'); end. Maik Wojcieszak schrieb: > Alexander, > > I tried you're test app already and it worked as you said. So does my > test app if I do only one changing you're test > app to process > multiple threads at the same time it crashed too. > > Alexander Grau schrieb: >> >> >> Maik, >> >> Did you try out the console test program I posted? It uses your DLL >> that runs a C thread and that C thread calls back the FPC function. >> Even more, this FPC callback function then calls a FPC object method. >> >> My program shows that there is nothing wrong with FPC (on Win32) and >> external C threads - it just works. > Well, it doesn't if you stress this stuff a bit. >> >> My assumption : there is something else wrong in your code - use my >> code and add one piece after the other of your code. For example, >> first just try to make my test app a simple LCL app (just a simple >> form). If that works too, add the dynamic function mapping you used. >> If that works too, make the wrapper (TComponent) you wrote etc. > The question is: What is wrong ? >> >> Again : You can call FPC functions/objects in external C threads - >> there must be something else wrong in your program ;-) ... > ... > > I've choosen the vortex (beep )binding as an example because my dll > does exactly the same. > In fact it uses the vortex library to implement user protocols. There > must be a reason for those guys (and the > vortex programmers) to add a mechanism to use a different threading > when using FPC. > At the moment I will follow this idea to check if it solves my > problem and post my result here. > > Maik > > > -- > _______________________________________________ > Lazarus mailing list > > http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus > -- _______________________________________________ Lazarus mailing list http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Reply by email

Alexander Grau
to Sven Barth: So the runtime must be tested more when using external threads... Does the writeln work…
Show quoted text 13 lines
Am 25.08.2010 17:02, schrieb Alexander Grau: >> Does it even work if you use threadvars? It should as the Heap uses >> threadvars, but I want to be sure. (and now I'm also curious to know >> why it works with 2.4.0 ^^) > I have modified the test - now it's using multiple external threads and > that works fine too until the moment I enable any 'writeln'. Using > threadvars or not doesn't make any difference. Using any writeln will > make it crash. If not using writeln, the external threads can call FPC > objects to increase some counters that are correctly displayed at the > end of the program for each thread. So the runtime must be tested more when using external threads... Does the writeln work when you use FPC threads?

Yes, using FPC threads it works - it crashes only when using external threads.

Here's my latest and finest test code for both cases:

program project1;
{$APPTYPE CONSOLE}
{$mode objfpc}{$H+}

uses

  {$IFDEF UNIX}{$IFDEF UseCThreads}
  cthreads,
  {$ENDIF}{$ENDIF}
  Classes,
  { you can add units after this }
  windows;
{$R project1.res}
type
  TSomeClass = class
  public
    procedure someMethod(idx: integer);
  end;
const
  USE_EXT_THREADS = true;
  COUNT = 30;
var
  i: integer;
  threadid: DWORD;
  ThreadHandles: array[1..COUNT] of THandle; //R?ckgabewert von CreateThread
  counters: array[1..COUNT] of integer;
  someObj: TSomeClass;
  FPCthreads: array[1..COUNT] of tthread;
type
  TFPCThread = class(TThread)
  protected
    param: pointer;
    ffinished: boolean;
    procedure Execute; override;
  public
    constructor Create(aparam: pointer; CreateSuspended: boolean);
    property finished: boolean read ffinished;
  end;
  constructor TFPCThread.Create(aparam: pointer; CreateSuspended: boolean);
  begin
    param:=aparam;
    ffinished:=false;
    FreeOnTerminate := false;
    inherited create(CreateSuspended);
  end;
  procedure TFPCThread.execute;
  var
    t: tobject;
  begin
    t := TObject.Create;
    try
      Writeln(t.ClassName); // <--- here we get *NO* crash
    finally
      t.Free;
    end;
    someObj.someMethod(integer(param));
    ffinished:=true;
  end;
// ----------------------------------------------
procedure TSomeClass.someMethod(idx: integer);
begin
  //writeln('TSomeClass.someMethod called');
  inc(counters[idx]);
end;
procedure someFunc(param: pointer); cdecl;
var
  t: tobject;
begin
  //WriteLn('External is called - ThreadID=', GetCurrentThreadId);
  //if assigned(param) then writeln('blub');
  //writeln('param=', param);
  t := TObject.Create;
  try
    Writeln(t.ClassName); // <--- here we get a crash
  finally
    t.Free;
  end;
  someObj.someMethod(integer(param));
end;
function ExternalThread(param: Pointer): LongInt; stdcall;
begin
  someFunc(param);
  Result:=0;
end;
begin
  WriteLn('Main ThreadID=', GetCurrentThreadId);
  someObj:=TSomeClass.create;
  for i:=1 to COUNT do counters[i]:=0;
  WriteLn('Creating external threads');
  for i:=1 to COUNT do
  begin
    if USE_EXT_THREADS then
    begin
      ThreadHandles[i]:=CreateThread(nil, 0,
TFNThreadStartRoutine(@ExternalThread),
        pointer(i), 0, ThreadID);
      if ThreadHandles[i] = 0 then writeln('ERROR creating external
thread');
    end else FPCThreads[i]:=TFPCThread.Create(pointer(i), false);
  end;
  readln;
  WriteLn('Freeing external thread');
  for i:=1 to COUNT do
  begin
    if USE_EXT_THREADS then
    begin
      if ThreadHandles[i]0 then CloseHandle(ThreadHandles[i]);
    end else FPCThreads[i].free;
  end;
  someObj.free;
  for i:=1 to COUNT do
    writeln (counters[i]);
end.

Reply by email

Alexander Grau
to Maik Wojcieszak: Not only writeln crashes the app. It is not really predictibable what exactly crashes the…
Show quoted text 7 lines
Alexander, Not only writeln crashes the app. It is not really predictibable what exactly crashes the app. normally it should be ok to use PostMessage inside a thread to send a message to the main thread. BUT doing this crashes the app immediatly.

I can confirm now that the FPC runtime seem to have a problem when external threads calling it. It works fine when using FPC threads. See my latest test program in my previous post to Sven.

Show quoted text 179 lines
Maik Alexander Grau schrieb: > Maik, > > As you did, I did write a new test program - this one uses 30 threads > - it works fine (no crash) until the moment I enable the a simple > 'writeln' inside a callack. You can see that it works by the > increased counters for each thread that are displayed after the > program successfully finished. > > So: external multiple threads calling fine until you use FPC runtime > code :/ ... > > > Regards, > Alexander > > > program project1; > > {$mode objfpc}{$H+} > > uses > {$IFDEF UNIX}{$IFDEF UseCThreads} > cthreads, > {$ENDIF}{$ENDIF} > Classes, > sysutils > { you can add units after this }; > > {$R project1.res} > > type > MT_THREAD_HANDLE = Pointer; > > tdummy = class > idx: integer; > constructor create(); > procedure test; > end; > > > function vcmtdll_NewThread: MT_THREAD_HANDLE; cdecl; external > 'vcmtdll.dll'; > procedure vcmtdll_DeleteThread(ahandle : MT_THREAD_HANDLE); cdecl; > external 'vcmtdll.dll'; > procedure vcmtdll_RegisterCallback(ahandle : MT_THREAD_HANDLE; > pCBFunc : Pointer; pCBData : Pointer); cdecl; external 'vcmtdll.dll'; > procedure vcmtdll_StartThread(ahandle : MT_THREAD_HANDLE; iDelay : > Cardinal); cdecl; external 'vcmtdll.dll'; > > const > COUNT = 30; > > var > handles: array[1..COUNT] of MT_THREAD_HANDLE; > objects: array[1..COUNT] of tdummy; > > > constructor tdummy.create(); > begin > inherited create; > idx:=0; > end; > > procedure tdummy.test; > begin > inc(idx); > end; > > procedure OnExecuteClb(pCBData : Pointer); cdecl; > var > i: integer; > begin > // writeln('onExecuteClb'); <-- this line will crash > for i:=1 to 30 do > tdummy(pCBData).test(); > end; > > var > i: integer; > > begin > writeln('start'); > for i:=1 to COUNT do objects[i]:=TDummy.create; > > for i:=1 to COUNT do > Handles[i] := vcmtdll_NewThread(); > > for i:=1 to COUNT do > vcmtdll_RegisterCallback(handles[i], @OnExecuteClb, objects[i]); > > for i:=1 to COUNT do > vcmtdll_StartThread(handles[i], 1000); > > writeln('go'); > sleep(2000); > > writeln('freeing'); > for i:=1 to COUNT do > vcmtdll_DeleteThread(handles[i]); > > for i:=1 to COUNT do writeln(objects[i].idx); > > writeln('bye'); > end. > > > > > Maik Wojcieszak schrieb: >> Alexander, >> >> I tried you're test app already and it worked as you said. So does >> my test app if I do only one changing you're >> test app to process >> multiple threads at the same time it crashed too. >> >> Alexander Grau schrieb: >>> >>> >>> Maik, >>> >>> Did you try out the console test program I posted? It uses your DLL >>> that runs a C thread and that C thread calls back the FPC function. >>> Even more, this FPC callback function then calls a FPC object method. >>> >>> My program shows that there is nothing wrong with FPC (on Win32) >>> and external C threads - it just works. >> Well, it doesn't if you stress this stuff a bit. >>> >>> My assumption : there is something else wrong in your code - use >>> my code and add one piece after the other of your code. For >>> example, first just try to make my test app a simple LCL app (just >>> a simple form). If that works too, add the dynamic function mapping >>> you used. If that works too, make the wrapper (TComponent) you >>> wrote etc. >> The question is: What is wrong ? >>> >>> Again : You can call FPC functions/objects in external C threads - >>> there must be something else wrong in your program ;-) ... >> ... >> >> I've choosen the vortex (beep )binding as an example because my dll >> does exactly the same. >> In fact it uses the vortex library to implement user protocols. >> There must be a reason for those guys (and the >> vortex programmers) to add a mechanism to use a different threading >> when using FPC. >> At the moment I will follow this idea to check if it solves my >> problem and post my result here. >> >> Maik >> >> >> -- >> _______________________________________________ >> Lazarus mailing list >> >> http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus >> > > > -- > _______________________________________________ > Lazarus mailing list > > http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus > -- _______________________________________________ Lazarus mailing list http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Reply by email

Sven Barth
to Alexander Grau: Yes, using FPC threads it works - it crashes only when using external threads. Here's my…
Alexander Grau wrote 17 lines
Sven Barth schrieb: > Am 25.08.2010 17:02, schrieb Alexander Grau: >>> Does it even work if you use threadvars? It should as the Heap uses >>> threadvars, but I want to be sure. (and now I'm also curious to know >>> why it works with 2.4.0 ^^) >> I have modified the test - now it's using multiple external threads and >> that works fine too until the moment I enable any 'writeln'. Using >> threadvars or not doesn't make any difference. Using any writeln will >> make it crash. If not using writeln, the external threads can call FPC >> objects to increase some counters that are correctly displayed at the >> end of the program for each thread. > > So the runtime must be tested more when using external threads... Does > the writeln work when you use FPC threads? Yes, using FPC threads it works - it crashes only when using external threads.

Might be the time to report this to the FPC devs. Maybe I'll take a look into the RTL by myself as well, but I can't promise anything.

Regards,
Sven

Reply by email

Alexander Grau
to Sven Barth
Show quoted text 22 lines
Am 25.08.2010 17:48, schrieb Alexander Grau: > Sven Barth schrieb: >> Am 25.08.2010 17:02, schrieb Alexander Grau: >>>> Does it even work if you use threadvars? It should as the Heap uses >>>> threadvars, but I want to be sure. (and now I'm also curious to know >>>> why it works with 2.4.0 ^^) >>> I have modified the test - now it's using multiple external threads >>> and >>> that works fine too until the moment I enable any 'writeln'. Using >>> threadvars or not doesn't make any difference. Using any writeln will >>> make it crash. If not using writeln, the external threads can call FPC >>> objects to increase some counters that are correctly displayed at the >>> end of the program for each thread. >> >> So the runtime must be tested more when using external threads... Does >> the writeln work when you use FPC threads? > Yes, using FPC threads it works - it crashes only when using external > threads. > Might be the time to report this to the FPC devs. Maybe I'll take a look into the RTL by myself as well, but I can't promise anything.

I think so - can you report it if you cannot find something after digging into the RTL?

Thanks,
Alexander

Reply by email

José Mejuto
to Maik Wojcieszak: I've packed the runtime env dlls and put them on my ftp server. They are too big to…

Hello Lazarus-List,

Wednesday, August 25, 2010, 8:44:53 AM, you wrote:

MW> I've packed the runtime env dlls and put them on my ftp server. MW> They are too big to attach to the mail on the list. MW> Server : support-ftp.wobe-team.com
MW> The file u need : Microsoft.VC80.CRT.zip
MW> Put the files into the directory of the dll and it works.

Thank you. I was able to debug the code and seems that the problem is in the DLL side, because setting the callback to "nil" the code should never callback due:

if (NULL != data && NULL != data->pCBFunc){

And even when no callback occurs the crash happends. I'll try to refine the problem search.

Best regards, Jos?

Reply by email

José Mejuto
to Maik Wojcieszak: I've packed the runtime env dlls and put them on my ftp server. They are too big to…

Hello Lazarus-List,

Wednesday, August 25, 2010, 8:44:53 AM, you wrote:

MW> I've packed the runtime env dlls and put them on my ftp server. MW> They are too big to attach to the mail on the list.

Do you have compiled the "vcmtdll.dll" with Multithread runtime library (/MT) ?

To avoid crashes just simply:

// start threads
for index := 0 to FThreads.Count-1 do
begin
   TVCMTThread(FThreads.Items[index]).Start(10);
   Sleep(50);
end;

Adding the Sleep(50) avoids any crash, and the AsyncWrite works as expected. You should also add a Sleep(50) after each FThreads[n].free

If a multithread problem is solved with sleeps usually it is a race condition in the RTL, in this case you create a new thread while the provious one is being created (or destroyed), so to me looks like a VC++ non multithread RTL problem.

It is also desirable to add the line "IsMultithread:=true" in the first line of .lpr (after the begin) to activate the Multithread RTL in the FPC, otherwise it could not be active.

I had tested it with 200 threads and using the sleeps no problem at all.

Best regards, Jos?

Reply by email

Alexander Grau
to José Mejuto
Show quoted text 34 lines
Hello Lazarus-List, Wednesday, August 25, 2010, 8:44:53 AM, you wrote: MW> I've packed the runtime env dlls and put them on my ftp server. MW> They are too big to attach to the mail on the list. Do you have compiled the "vcmtdll.dll" with Multithread runtime library (/MT) ? To avoid crashes just simply: // start threads for index := 0 to FThreads.Count-1 do begin TVCMTThread(FThreads.Items[index]).Start(10); Sleep(50); end; Adding the Sleep(50) avoids any crash, and the AsyncWrite works as expected. You should also add a Sleep(50) after each FThreads[n].free If a multithread problem is solved with sleeps usually it is a race condition in the RTL, in this case you create a new thread while the provious one is being created (or destroyed), so to me looks like a VC++ non multithread RTL problem. It is also desirable to add the line "IsMultithread:=true" in the first line of .lpr (after the begin) to activate the Multithread RTL in the FPC, otherwise it could not be active. I had tested it with 200 threads and using the sleeps no problem at all.

Hello Jos?,

I have tried to reduce the DLL problem into a test that doesn't need any external DLL. It crashes too.

Here's the test program:

http://grauonline.de/alexwww/tmp/extthread.zip

Can you see any analogy to the problem? I have tried to add your suggested "IsMultithread:=true" and it doesn't make any difference. I think there's something wrong in the FPC runtime.

Regards,
Alexander

Reply by email

Vincent Snijders
to Alexander Grau

2010/8/25 Alexander Grau :

Show quoted text 8 lines
Here's the test program: http://grauonline.de/alexwww/tmp/extthread.zip Can you see any analogy to the problem? I have tried to add your suggested "IsMultithread:=true" and it doesn't make any difference. I think there's something wrong in the FPC runtime.

I took a quick look at it, but missed the code where the first FPC thread is created, before any external threads that use the FPC RTL is created.

That is required, as is mentioned earlier in this thread by Sven too: http://www.hu.freepascal.org/lists/fpc-pascal/2010-July/025939.html

Vincent

Reply by email

Sven Barth
to Alexander Grau: I have tried to reduce the DLL problem into a test that doesn't need any external DLL. It…
Alexander Grau wrote 19 lines
Jos? Mejuto schrieb: > It is also desirable to add the line "IsMultithread:=true" in the > first line of .lpr (after the begin) to activate the Multithread RTL > in the FPC, otherwise it could not be active. > > I had tested it with 200 threads and using the sleeps no problem at > all. Hello Jos?, I have tried to reduce the DLL problem into a test that doesn't need any external DLL. It crashes too. Here's the test program: http://grauonline.de/alexwww/tmp/extthread.zip Can you see any analogy to the problem? I have tried to add your suggested "IsMultithread:=true" and it doesn't make any difference. I think there's something wrong in the FPC runtime.

IsMultiThread is only a variable. Setting it is only useful recognized once FPC threading code is called. So instead of setting IsMultiThread to true you must create at least one thread in FPC (it can be a dummy thread that immediately terminates).

That might not solve the Writeln problems that Alexander discovered, though.

Regards,
Sven

Reply by email

Alexander Grau
to Vincent Snijders: 2010/8/25 Alexander Grau : I took a quick look at it, but missed the code where the first…
Show quoted text 16 lines
2010/8/25 Alexander Grau : > Here's the test program: > > http://grauonline.de/alexwww/tmp/extthread.zip > > Can you see any analogy to the problem? I have tried to add your suggested > "IsMultithread:=true" and it doesn't make any difference. I think there's > something wrong in the FPC runtime. > > I took a quick look at it, but missed the code where the first FPC thread is created, before any external threads that use the FPC RTL is created.

I have added this and it does not make any difference -

I have updated the test program here:

http://grauonline.de/alexwww/tmp/extthread.zip

Alexander

Reply by email

José Mejuto
to Alexander Grau: I have tried to reduce the DLL problem into a test that doesn't need any external DLL. It…

Hello Lazarus-List,

Wednesday, August 25, 2010, 8:24:38 PM, you wrote:

AG> I have tried to reduce the DLL problem into a test that doesn't need any AG> external DLL. It crashes too.
AG> Here's the test program:

Change "somefunc" by:

procedure someFunc(param: pointer); cdecl;
var
  t: tobject;
  a: string;
begin
//  WriteLn('External is called - ThreadID=', GetCurrentThreadId);
//  if assigned(param) then writeln('blub');
//  writeln('param=', integer(param));
  t := TObject.Create;
  try
    a:=TObject.ClassName;
    FileWrite(StdOutputHandle,a[1],Length(a));
  finally
    t.Free;
  end;
  someObj.someMethod(integer(param));
end;

And it works, so the problem is not in the RTL itself, but in the writeln in a multithread environment.

Best regards, Jos?

Reply by email

José Mejuto
to Sven Barth: IsMultiThread is only a variable. Setting it is only useful recognized once FPC threading…

Hello Lazarus-List,

Wednesday, August 25, 2010, 8:47:23 PM, you wrote:

SB> IsMultiThread is only a variable. Setting it is only useful recognized SB> once FPC threading code is called. So instead of setting IsMultiThread SB> to true you must create at least one thread in FPC (it can be a dummy SB> thread that immediately terminates).

In Delphi setting IsMultithread to true is a must to use non TThread threads. Taken from Embarcadero QA:

Question:
Why do I get "Access violation" or "invalid pointer operation" when doing multi-threading in Delphi when not using the TThread class or the BeginThread function (i.e. using the CreateThread API function, or writing an Entera server)?

Answer:
If you use non-Delphi threads (not using TThread or BeginThread) in a Delphi application, then you must manually set the IsMultiThread variable from the System unit to true. See "IsMultiThread variable" in the online help files for more information.

Also the one thread creation is only for Unix systems, win32 does not have this problem since a lot of time.

On the other hand, if Unix needs one thread creation then it should be automagically done in the initialization section of cthreads, do not ?

Best regards, Jos?

Reply by email

Alexander Grau
to José Mejuto: AG> I have tried to reduce the DLL problem into a test that doesn't need any AG> external…
Show quoted text 16 lines
Hello Lazarus-List, Wednesday, August 25, 2010, 8:24:38 PM, you wrote: AG> I have tried to reduce the DLL problem into a test that doesn't need any AG> external DLL. It crashes too. AG> Here's the test program: Change "somefunc" by: ... And it works, so the problem is not in the RTL itself, but in the writeln in a multithread environment.

Correct - my test just shows there's a problem with write in multithread env - the other runtime calls work. Also, my test doesn't replicate the original DLL problem of this mail thread, that must be another reason (missing IsMultiThread, missing /MT etc.)

Anyway, I really learned much more about multi-threading issues here and will make all this information available on a Wiki page ('FPC multithreading pitfalls') soon.

Alexander

Reply by email

Sven Barth
to José Mejuto: SB> IsMultiThread is only a variable. Setting it is only useful recognized SB> once FPC…
Jos? Mejuto wrote 12 lines
Hello Lazarus-List, Wednesday, August 25, 2010, 8:47:23 PM, you wrote: SB> IsMultiThread is only a variable. Setting it is only useful recognized SB> once FPC threading code is called. So instead of setting IsMultiThread SB> to true you must create at least one thread in FPC (it can be a dummy SB> thread that immediately terminates). In Delphi setting IsMultithread to true is a must to use non TThread threads. Taken from Embarcadero QA:

FPC is not Delphi and in this regard those two differ. The method SysInitMultiThreading in rtl/win/systhrd.pas which initializes the thread local storage is only called on two locations:
- when a new thread is created using FPC routines (BeginThread, TThread)
- when the unit system is initalized (using InitSystemThreads) AND if the application is a library

If you don't create a thread the thread local storage won't be initialized and thus threadvars won't work and thus the heap will not work correctly.

Also the one thread creation is only for Unix systems, win32 does not have this problem since a lot of time.

Windows DOES have this problem.

On the other hand, if Unix needs one thread creation then it should be automagically done in the initialization section of cthreads, do not ?

This might indeed be a solution.

Regards,
Sven

Reply by email

Sven Barth
to Alexander Grau: I have added this and it does not make any difference - I have updated the test program…
Alexander Grau wrote 5 lines
I have added this and it does not make any difference - I have updated the test program here: http://grauonline.de/alexwww/tmp/extthread.zip

If you extend ExternalThread as in the following it works at least on Wine:

function ExternalThread(param: Pointer): LongInt; stdcall;
var
   tm: TThreadManager;
begin
   GetThreadManager(tm);
   tm.AllocateThreadVars;
   InitThread(100000000);
   someFunc(param);
   Result:=0;
end;

With this extension the thread behaves similar to a FPC created one.

Regards,
Sven

Reply by email

José Mejuto
to Sven Barth: FPC is not Delphi and in this regard those two differ. The method SysInitMultiThreading…

Hello Lazarus-List,

you wrote
> In Delphi setting IsMultithread to true is a must to use non TThread > threads. Taken from Embarcadero QA:

SB> FPC is not Delphi and in this regard those two differ.

Ok, I quoted Delphi just as a side note, and the reason I included it in the message. In fact there is some routines here and there that checks isMultiThread in order to perform or not locks. Of course, if heap is not properly initialized... ;)

> Also the one thread creation is only for Unix systems, win32 does not > have this problem since a lot of time.

SB> Windows DOES have this problem.

In that case I misunderstood a quite recent message in fpc mailing list, my apologizes.

> On the other hand, if Unix needs one thread creation then it should be > automagically done in the initialization section of cthreads, do not ?

SB> This might indeed be a solution.

If windows have the same problem, threading system should be initialized in... somewhere automagically as windows multithread is always present.

Best regards, Jos?

Reply by email

Maik Wojcieszak
to José Mejuto

Hi guys,

thank you for the tons of information about the multithreading system in FPC.
In one of my mails I've mentioned a project (beepfp) which uses external threads
from the vortex dll.

There is a hint to the instability problem we are discussing here:

{ FPC specific thread create function to replace Vortex's thread create. This is required because FPC doesn't work when C libraries create their own threads}

The solution they found here was to implement a callback in the dll that passes
a FPC function to create threads to use inside the dll. The code for unix to create the thread is:

...
    //Start thread
    BeginThread(@C2P_Translator, ThreadData, TThreadID(thread_def^) );
    if TThreadID(thread_def) > 0 then
      //Don't free memory here, it is done in the thread function
      Result := axl_true
    else
    begin
      //Free memory
      dispose(ThreadData);
      Result := axl_false;
    end;
...
initialization
  // Set pascal specific thread creators
  vortex_thread_set_create(@fpc_vortex_thread_create);
  vortex_thread_set_destroy(@fpc_vortex_thread_destroy);
...

this seems to solve the problem and that is what I am testing now too. If the dll calls the fpc callbacks
now it will be in the context of it's own thread.

The question is now: Why does this work ?

If BeginThread initializes the stack which needs to be done to make external threads work it will work with this example too. Did the creator of this code find a regular solution for this problem or did he just work around the problem by making external threads internal ?

Maik

Reply by email

Sven Barth
to Maik Wojcieszak
Maik Wojcieszak wrote 14 lines
initialization // Set pascal specific thread creators vortex_thread_set_create(@fpc_vortex_thread_create); vortex_thread_set_destroy(@fpc_vortex_thread_destroy); ... The question is now: Why does this work ? If BeginThread initializes the stack which needs to be done to make external threads work it will work with this example too. Did the creator of this code find a regular solution for this problem or did he just work around the problem by making external threads internal ?

I took a quick look at the vortex library.

The "magic" happens in those two functions that are passed in the initialization section. Those tell the vortex library to not use it's own threading creation and destruction methods, but to let the calling application (in this case the FPC one) create the threads. That was a rather intelligent approach by the developers of the vortex library as it allows threading systems that need special initialisation (like ours) to be used.

Effectively those calls start a "FPC internal" thread and is the best way to circumvent external thread problems.

Regards,
Sven

Reply by email

Sven Barth
to José Mejuto: SB> FPC is not Delphi and in this regard those two differ. Ok, I quoted Delphi just as a…
Jos? Mejuto wrote 8 lines
>> On the other hand, if Unix needs one thread creation then it should be >> automagically done in the initialization section of cthreads, do not ? SB> This might indeed be a solution. If windows have the same problem, threading system should be initialized in... somewhere automagically as windows multithread is always present.

This only solves one point (but is needed nevertheless).

As I've showed in the last message to Alexander yesterday evening you need to initialize some things PER THREAD and I don't know whether this can be automised. We might need to ask the FPC devs for help regarding this. Perhaps the first time a threadvar in this new thread is accessed might be a good possibilty, but I don't know (yet) how we could track whether this is the first call, because threadvars are not yet allocated (hen-egg-problem). I'll have to take a deeper look into the threading system on Win32 :)

Regards,
Sven

Reply by email

Alexander Grau
to Sven Barth
Show quoted text 23 lines
Am 25.08.2010 23:07, schrieb Jos? Mejuto: >>> On the other hand, if Unix needs one thread creation then it should be >>> automagically done in the initialization section of cthreads, do not ? > SB> This might indeed be a solution. > > If windows have the same problem, threading system should be > initialized in... somewhere automagically as windows multithread is > always present. > This only solves one point (but is needed nevertheless). As I've showed in the last message to Alexander yesterday evening you need to initialize some things PER THREAD and I don't know whether this can be automised. We might need to ask the FPC devs for help regarding this. Perhaps the first time a threadvar in this new thread is accessed might be a good possibilty, but I don't know (yet) how we could track whether this is the first call, because threadvars are not yet allocated (hen-egg-problem). I'll have to take a deeper look into the threading system on Win32 :) Regards, Sven

I'm currently trying to summarize all aspects we discussed to add them to the FreePascal Multithreading wiki page (see below) - did I forget something that should be mentioned?

Thanks,
Alexander

External threads

To make Free Pascal's threading system to work properly, each newly created FPC thread needs to be initialized (more exactly, the thread local storage per thread needs to be initialized so threadvars and heap are working). That is fully automatically done for you if you use BeginThread (or indirectly by using the TThread class). However, if you use threads that were created without BeginThread (i.e. external threads), additional work (currently) might be required. External threads also include those that were created in external C libraries (.DLL/.so).

Things to consider when using external threads (might not be needed in all or future compiler versions):

A) Initialize the FPC's threading system by creating a dummy thread:

  { initialise threading system }
   with tc.create(false) do
   begin
     waitfor;
     free;
   end;

B) Make the runtime aware that your application uses multiple threads in the first line of your application (.dpr):

IsMultithread:=true;

(This will make FPC routines to perform locks here and there.)

C) Do not use external threads at all - use FPC threads.

D) If for some reason this doesn't work for you, try this code in your external thread function:

function ExternalThread(param: Pointer): LongInt; stdcall;
var
  tm: TThreadManager;
begin
  GetThreadManager(tm);
  tm.AllocateThreadVars;
  InitThread(100000000);
  someFunc(param);
  Result:=0;
end;

How to identify if your code is executed in an external thread context

1. Ask the OS for the ID of the current thread at your application's start

Win32: WriteLn('Thread ID=', GetCurrentThreadID);
Darwin: WriteLn('Thread ID=', GetThreadID);
Linux: Writeln('Thread ID=', TThreadID(pthread_self) );

2. Ask again for the ID of the current thread inside the thread function and compare this by the result of step 1.

Reply by email

Sven Barth
to Alexander Grau
Alexander Grau wrote 22 lines
I'm currently trying to summarize all aspects we discussed to add them to the FreePascal Multithreading wiki page (see below) - did I forget something that should be mentioned? Thanks, Alexander External threads To make Free Pascal's threading system to work properly, each newly created FPC thread needs to be initialized (more exactly, the thread local storage per thread needs to be initialized so threadvars and heap are working). That is fully automatically done for you if you use BeginThread (or indirectly by using the TThread class). However, if you use threads that were created without BeginThread (i.e. external threads), additional work (currently) might be required. External threads also include those that were created in external C libraries (.DLL/.so).

It's not only initializing the thread local storage (which is Windows only by the way, the threadvar system on e.g. Linux might have a different name), but also the exception and the I/O system

Show quoted text 13 lines
Things to consider when using external threads (might not be needed in all or future compiler versions): A) Initialize the FPC's threading system by creating a dummy thread: { initialise threading system } with tc.create(false) do begin waitfor; free; end;

You should not forget to write how a class of type "tc" looks like :)

B) Make the runtime aware that your application uses multiple threads in the first line of your application (.dpr): IsMultithread:=true;

This is not needed. That is set to true automatically if a thread is created inside FPC.

(This will make FPC routines to perform locks here and there.) C) Do not use external threads at all - use FPC threads.

Should be put at the top of the list ^^

Show quoted text 14 lines
D) If for some reason this doesn't work for you, try this code in your external thread function: function ExternalThread(param: Pointer): LongInt; stdcall; var tm: TThreadManager; begin GetThreadManager(tm); tm.AllocateThreadVars; InitThread(100000000); someFunc(param); Result:=0; end;

The number in InitThread is the initial stack length and as I was a bit lazy I simply set it to a huge number. So don't nail me on that one :D

You might also replace the "someFunc(param)" with "{ do something threaded }" or something like that.

Show quoted text 11 lines
How to identify if your code is executed in an external thread context 1. Ask the OS for the ID of the current thread at your application's start Win32: WriteLn('Thread ID=', GetCurrentThreadID); Darwin: WriteLn('Thread ID=', GetThreadID); Linux: Writeln('Thread ID=', TThreadID(pthread_self) ); 2. Ask again for the ID of the current thread inside the thread function and compare this by the result of step 1.

I don't know whether this works (reliably) when the threadvars haven't been allocated yet. I don't even know what happens if you access unallocated threadvars O.o

Regards,
Sven

Reply by email

Maik Wojcieszak
to Alexander Grau: I'm currently trying to summarize all aspects we discussed to add them to the FreePascal…

Hi,

What about the callback from the dll. I think this should at least be mentioned as a possible solution.
Here is the code:

type
  PVCMTThread = Pointer;
  TVCMTThreadFunc = function (user_data:Pointer):Pointer;cdecl;
  PCdeclThreadFunc = ^TCdeclThreadFunc;
  TCdeclThreadFunc = record
    Func: TVCMTThreadFunc;  //cdecl function
    Data: Pointer;            //orig data
  end;
{ C to pascal wrapper function.
  BeginThread expects a 'register' function. Vortex supplies a 'cdecl'
function.}
function C2P_Translator(FuncData: pointer) : ptrint;
var
  ThreadData: TCdeclThreadFunc;
begin
  //Copy the supplied info
  ThreadData := PCdeclThreadFunc(FuncData)^;
  //Release memory allocated in fpc_vortex_thread_create()
  dispose(PCdeclThreadFunc(FuncData));
  //Finally we run the thread function
  Result := ptrint(ThreadData.Func(ThreadData.Data));
end;
{ FPC specific thread create function to replace Vortex's thread create.
 This is required because FPC doesn't work when C libraries create their own
 threads}
//{$WARNINGS OFF} //For "cdecl'd functions have no high parameter"
function fpc_vcmt_thread_create(thread_def : PVCMTThread;
                                  func       : TVCMTThreadFunc;
                                  user_data  : Pointer):longint; cdecl;
var
  ThreadData: PCdeclThreadFunc;
begin
  //Pass thread information to C to Pascal wrapping function
  New(ThreadData);
  ThreadData^.Func := func;
  ThreadData^.Data := user_data;
  {$IFDEF WINDOWS}
    //Start thread
    TThreadID(thread_def^) := BeginThread(@C2P_Translator, ThreadData );
    if TThreadID(thread_def) > 0 then
      //Don't free memory here, it is done in the thread function
      Result := 1
    else
    begin
      //Free memory
      dispose(ThreadData);
      Result := 0;
    end;
  {$ENDIF}
  //{$IFDEF UNIX}
  //  //Start thread
  //  BeginThread(@C2P_Translator, ThreadData, TThreadID(thread_def^) );
  //
  //  if TThreadID(thread_def) > 0 then
  //    //Don't free memory here, it is done in the thread function
  //    Result := True
  //  else
  //  begin
  //    //Free memory
  //    dispose(ThreadData);
  //    Result := False;
  //  end;
  //{$ENDIF}
end;
{$WARNINGS ON}
{ FPC specific thread destroy function to replace Vortex's thread destroy.
 This is required because FPC doesn't work when C libraries create their own
 threads}
function fpc_vcmt_thread_destroy(thread_def    : PVCMTThread;
                                 free_data  : boolean):longint; cdecl;
var
  Err: integer;
begin
  {$IFDEF WINDOWS}
    //Wait for thread
    Err := WaitForThreadTerminate(TThreadID(thread_def), 0);
    //Free resources
    //if free_data then
    //  vcmtdll_Sys_Free(thread_def);
  {$ENDIF}
  //{$IFDEF UNIX}
  //  //Wait for thread
  //  Err := WaitForThreadTerminate(TThreadID(thread_def^), 0);
  //
  //  //Free resources
  //  if 1 = free_data then
  //    tml_Sys_Free(thread_def);
  //{$ENDIF}
  if 0 = Err then
    Result := 1
  else
    Result := 0;
end;
initialization
{$IFDEF FPC}
  // Set pascal specific thread creators
  vcmtdll__Thread_Set_OnCreate(@fpc_vcmt_thread_create);
  vcmtdll__Thread_Set_OnDestroy(@fpc_vcmt_thread_destroy);
{$ENDIF}

The example with my dll can be downloaded for testing:

Server : support-ftp.wobe-team.com
User : lazarus
Pwd : test

It works with communication to the GUI baed on events (PostMessage). This example crashed before. Is there any hint that the FPC team will provide a fix for this in the near future ?

Thanks
Maik

Show quoted text 102 lines
Sven Barth schrieb: > Am 25.08.2010 23:07, schrieb Jos? Mejuto: >>>> On the other hand, if Unix needs one thread creation then it >>>> should be >>>> automagically done in the initialization section of cthreads, do >>>> not ? >> SB> This might indeed be a solution. >> >> If windows have the same problem, threading system should be >> initialized in... somewhere automagically as windows multithread is >> always present. >> > > This only solves one point (but is needed nevertheless). > > As I've showed in the last message to Alexander yesterday evening you > need to initialize some things PER THREAD and I don't know whether > this can be automised. We might need to ask the FPC devs for help > regarding this. Perhaps the first time a threadvar in this new thread > is accessed might be a good possibilty, but I don't know (yet) how we > could track whether this is the first call, because threadvars are > not yet allocated (hen-egg-problem). I'll have to take a deeper look > into the threading system on Win32 :) > > Regards, > Sven I'm currently trying to summarize all aspects we discussed to add them to the FreePascal Multithreading wiki page (see below) - did I forget something that should be mentioned? Thanks, Alexander External threads To make Free Pascal's threading system to work properly, each newly created FPC thread needs to be initialized (more exactly, the thread local storage per thread needs to be initialized so threadvars and heap are working). That is fully automatically done for you if you use BeginThread (or indirectly by using the TThread class). However, if you use threads that were created without BeginThread (i.e. external threads), additional work (currently) might be required. External threads also include those that were created in external C libraries (.DLL/.so). Things to consider when using external threads (might not be needed in all or future compiler versions): A) Initialize the FPC's threading system by creating a dummy thread: { initialise threading system } with tc.create(false) do begin waitfor; free; end; B) Make the runtime aware that your application uses multiple threads in the first line of your application (.dpr): IsMultithread:=true; (This will make FPC routines to perform locks here and there.) C) Do not use external threads at all - use FPC threads. D) If for some reason this doesn't work for you, try this code in your external thread function: function ExternalThread(param: Pointer): LongInt; stdcall; var tm: TThreadManager; begin GetThreadManager(tm); tm.AllocateThreadVars; InitThread(100000000); someFunc(param); Result:=0; end; How to identify if your code is executed in an external thread context 1. Ask the OS for the ID of the current thread at your application's start Win32: WriteLn('Thread ID=', GetCurrentThreadID); Darwin: WriteLn('Thread ID=', GetThreadID); Linux: Writeln('Thread ID=', TThreadID(pthread_self) ); 2. Ask again for the ID of the current thread inside the thread function and compare this by the result of step 1. -- _______________________________________________ Lazarus mailing list http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

Reply by email

Sven Barth
to Maik Wojcieszak
Maik Wojcieszak wrote 6 lines
Hi, What about the callback from the dll. I think this should at least be mentioned as a possible solution. Here is the code:

The callback solution can only be used if the library in question provides such an API. And sadly not every library developer sees a need for this. But it might be mentioned of course for the case a library provides this.

Show quoted text 6 lines
It works with communication to the GUI baed on events (PostMessage). This example crashed before. Is there any hint that the FPC team will provide a fix for this in the near future ?

I hope to report this to the FPC devs this weekend. Then only time will tell. But as a similiar fix was already done for POSIX systems and we've also found a workaround for the problem, the solution might(!) be rather simple/near.

Regards,
Sven

Reply by email

Michael Schnell
to Alexander Grau: I'm currently trying to summarize all aspects we discussed to add them to the FreePascal…
Alexander Grau wrote 9 lines
To make Free Pascal's threading system to work properly, each newly created FPC thread needs to be initialized (more exactly, the thread local storage per thread needs to be initialized so threadvars and heap are working). That is fully automatically done for you if you use BeginThread (or indirectly by using the TThread class). However, if you use threads that were created without BeginThread (i.e. external threads), additional work (currently) might be required. External threads also include those that were created in external C libraries (.DLL/.so).

FPC uses it's own way to create threadvars that is OS-specific and not identical with what other compilers (i.e. Microsoft C and GNU C) do.

While I do think that it would be a good idea to have the FPC RTL follow theses paradigms (i.e. look at the GNU docs and reverse engineer what M$ C does: we already investigate that; see the appropriate thread(s) in the FPC developers forum), I in fact don't see why this would get in the way with external procedures that independently use their own threadvars.

Of course when using heap managed variables (such as Pascal Strings) in both the main and external functions, a common heap management is needed, which maybe uses threadvars.

-Michael

Reply by email

Sven Barth
to Michael Schnell
Michael Schnell wrote 23 lines
On 08/26/2010 08:52 PM, Alexander Grau wrote: > To make Free Pascal's threading system to work properly, each newly > created FPC thread needs to be initialized (more exactly, the thread > local storage per thread needs to be initialized so threadvars and > heap are working). That is fully automatically done for you if you use > BeginThread (or indirectly by using the TThread class). However, if > you use threads that were created without BeginThread (i.e. external > threads), additional work (currently) might be required. External > threads also include those that were created in external C libraries > (.DLL/.so). FPC uses it's own way to create threadvars that is OS-specific and not identical with what other compilers (i.e. Microsoft C and GNU C) do. While I do think that it would be a good idea to have the FPC RTL follow theses paradigms (i.e. look at the GNU docs and reverse engineer what M$ C does: we already investigate that; see the appropriate thread(s) in the FPC developers forum), I in fact don't see why this would get in the way with external procedures that independently use their own threadvars. Of course when using heap managed variables (such as Pascal Strings) in both the main and external functions, a common heap management is needed, which maybe uses threadvars.

It's not about using the same threadvar system or the same heap management in FPC binary and non-FPC library. It's about initialising the FPC-internal systems PER THREAD so that you can USE theadvars, I/O, exceptions and such in callbacks that are coming back from the library.

Regards,
Sven

Reply by email

Michael Schnell
to Alexander Grau: I'm currently trying to summarize all aspects we discussed to add them to the FreePascal…
Alexander Grau wrote
C) Do not use external threads at all - use FPC threads.

This is a very hard requirement :(.

A colleague of mine uses the "PJSIP" open source library in his project done with Delphi. Here he created a DLL in C with Microsoft Visual Studio and calls the "flattened" functions in this DLL with his Pascal project. The C-DLL is statically linked to the PJSIP library functions provided.

Now the PJSIP library creates lots of threads and preforms callbacks to the DLL from same. The DLL forwards theses callbacks towards the Pascal code.

All this seems to work fine.

It would be a major shame if using Lazarus would prevent this kind of projects.

.-Michael

Reply by email

Alexander Grau
to Michael Schnell
Show quoted text 18 lines
On 08/26/2010 08:52 PM, Alexander Grau wrote: > C) Do not use external threads at all - use FPC threads. This is a very hard requirement :(. A colleague of mine uses the "PJSIP" open source library in his project done with Delphi. Here he created a DLL in C with Microsoft Visual Studio and calls the "flattened" functions in this DLL with his Pascal project. The C-DLL is statically linked to the PJSIP library functions provided. Now the PJSIP library creates lots of threads and preforms callbacks to the DLL from same. The DLL forwards theses callbacks towards the Pascal code. All this seems to work fine. It would be a major shame if using Lazarus would prevent this kind of projects.

It's not meant as a requirement - it's meant as list of considerations where you pick one or more of them to solve your problem. The "problem" might be too OS specific that you can say it always works or it doesn't work. It depends on the case. I'll make this more clearly. Also, I'll add all missing aspects Maik and Sven mentioned.

Regards,
Alexander

Reply by email

Maik Wojcieszak
to José Mejuto: SB> FPC is not Delphi and in this regard those two differ. Ok, I quoted Delphi just as a…

Hi,

What exactly does this

System.IsMultiThread := true;

do ?

If I use external threads with my thread creation callback it works fine with
System.IsMultiThread := false;
If I set it to true the app crashes when creating the first threads. If I use the suggested initialisation with the dummy thread it works too, but with
System.IsMultiThread := true;

Thanks in advance,
Maik

Show quoted text 35 lines
Hello Lazarus-List, Wednesday, August 25, 2010, 10:05:32 PM, you wrote: >> In Delphi setting IsMultithread to true is a must to use non TThread >> threads. Taken from Embarcadero QA: >> SB> FPC is not Delphi and in this regard those two differ. Ok, I quoted Delphi just as a side note, and the reason I included it in the message. In fact there is some routines here and there that checks isMultiThread in order to perform or not locks. Of course, if heap is not properly initialized... ;) >> Also the one thread creation is only for Unix systems, win32 does not >> have this problem since a lot of time. >> SB> Windows DOES have this problem. In that case I misunderstood a quite recent message in fpc mailing list, my apologizes. >> On the other hand, if Unix needs one thread creation then it should be >> automagically done in the initialization section of cthreads, do not ? >> SB> This might indeed be a solution. If windows have the same problem, threading system should be initialized in... somewhere automagically as windows multithread is always present.

Reply by email

Sven Barth
to Maik Wojcieszak
Maik Wojcieszak wrote 19 lines
Hi, What exactly does this System.IsMultiThread := true; do ? If I use external threads with my thread creation callback it works fine with System.IsMultiThread := false; If I set it to true the app crashes when creating the first threads. If I use the suggested initialisation with the dummy thread it works too, but with System.IsMultiThread := true; Thanks in advance, Maik

I did a quick search in the source of the RTL. IsMultiThread is mostly tested to check whether multi threading was already initialised. Especially on POSIX systems you MUST NOT set IsMultiThread to True or the threadvar system won't be initialized (win32 does a more "intelligent" test).

The interesting thing is though that IsMultiThread is never set to True by the RTL in POSIX systems...

The creation of a dummy thread should be enough. You can set IsMultiThread to true after the thread has run though (to circumvent the POSIX problem I mentioned above).

Regards,
Sven

Reply by email

Alexander Grau
to Sven Barth
Show quoted text 33 lines
Am 27.08.2010 12:27, schrieb Maik Wojcieszak: > Hi, > > What exactly does this > > System.IsMultiThread := true; > > do ? > > If I use external threads with my thread creation callback it works fine > with > System.IsMultiThread := false; > If I set it to true the app crashes when creating the first threads. > If I use the suggested initialisation with the dummy thread it works > too, but with > System.IsMultiThread := true; > > Thanks in advance, > Maik > I did a quick search in the source of the RTL. IsMultiThread is mostly tested to check whether multi threading was already initialised. Especially on POSIX systems you MUST NOT set IsMultiThread to True or the threadvar system won't be initialized (win32 does a more "intelligent" test). The interesting thing is though that IsMultiThread is never set to True by the RTL in POSIX systems... The creation of a dummy thread should be enough. You can set IsMultiThread to true after the thread has run though (to circumvent the POSIX problem I mentioned above).

My attempt to summarize all (practical) aspects is online now (at the end of the page):

http://wiki.freepascal.org/Multithreaded_Application_Tutorial

If I should make corrections, just let me know.

Regards,
Alexander

Reply by email