Hello,

I am making a Pascal wrapper for a library, which states in its API description:

"
Note that the Callback function will be called from an independent child thread, not from the Application’s Window thread. It should not therefore be used to draw data directly to a Windows screen. The safest use of the callback is to use a threadsafe interface (such as Windows messaging) to pass the message to Windows for handling by the Windows thread. "

and c(++) example

void Callback( int SerialNumber, int Event, int Data )
{
// Send the message to Windows thread for processing
PostMessage( NULL, WM_USER_MULTIPORTEVENT, Event,SerialNumber );
// Note: Data ignored. We can poll for an up-to-date
// value when this message is handled.
}

This would be fine, IFF i was adding this to the main form of my application... but I'm not. I'm trying to wrap this .dll in a separate unit, only communicating to the form through a wrapper object, defined in a separate unit. The object itself, would be created /by/ the form window, but won't be a GUI object, so won't be in the message queue;

Doing a chain like:

(Setup)
(form passes its handle to the object) -> (object installs callback function) and then
(normal operation)
(form receives callback via Windows Message) -> (form calls e.g OnCallback function of the object)

seems a bit backwards.

I can imagine, that I need to have the Callback procedure as

procedure Callback(Port, Event, Value : Integer); stdcall;

but how do I call functions/procedures of MyObject in a threadsafe manner from there?

(Lazarus 1.0.10 2013-06-09, FPC 2.6.2 rev. 41613 i386-win32-win32/win64)

Sorry if it's lame,
Kind Regards,
Lukasz

AFAIU: The callback is just a callback. So you can handle it in the normal way (either in an application, in a command line tool without a main form or in a dll.) You just can't do any GUI stuff in that callback.

I understand that you in fact don't want to do that, anyway.

-Michael

Lazarus has QueueAsyncCall, which is a thread safe, cross platform way of queueing method calls asynchronously.

E.g. for USB hotplug notification, I register this function with my device library:

	procedure NotifyPlug(status: cint); cdecl;
	begin
	  Application.QueueAsyncCall(@MainForm.NotifyAsync, status);
	end;

Henry

Oh, I missed the whole point ... sorry.

in fact I DO want to be able to call gui-interfacing functions of the form. The library I am loading (dynamically) will be giving me callbacks when something changes in an external USB device... When that happens, I want to be able to do e.g.

MyObject.OnStateChange := TForm1.ShowStateChange;

and e.g. safely call ShowMessage or display a custom dialog box,

(as well as I will be calling procedures/functions of another non-vis object(s) doing something else with the data provided by callback)

So it seems, if I call my objects' InternalCallback function from the installed callback, then it's my object that has to care for thread safety to the GUI then ?

(Henry Vermaak mentions QueueAsyncCall, going there...

Kind Regards,
Lukasz

Hello Henry,

but I would then need to make the wrapper object aware of the Application or its forms then?

I wanted the wrapper unit to be as self-contained as much as possible, (so the callback function is defined in a unit that only has Classes, SysUtils and dynlibs as uses, i.e. no Forms. Controls or Graphics)
but still be thread-safe w/r/t to where the wrapper object is created (within a GUI form or /not/)

Unless I can do:

procedure Callback (...); stdcall;
begin
  {Application}.QueueAsyncCall(MyObject.CallbackNotify, ...);
end;

no?

Thanks,
Lukasz

Lukasz Sokol wrote

in fact I DO want to be able to call gui-interfacing functions of the form.

As Henry states, doing GUI calls from a thread is easy by Application.QueueAsyncCall().

But, AFAIK, the dll does not know the main Form instance and does not know the Application instance of the application that loads the dll.

Moreover the memory management of the dll is independent of that of the Application it is called by. That is why you can't easily pass objects and strings between an application and the dll.

AFAIK, there are means to unify the memory management and to make the Form and/or the Application known to the dll.

But these issues need to be soled independently from the fact that the dll calls another dll.

-Michael

Lukasz Sokol wrote

but I would then need to make the wrapper object aware of the Application or its forms then?

This is provided by "Runtime Packages" with Delphi.

Lazarus does not support Runtime packages (yet) so you are quite on your own to implement something similar.

Hopefully the Wiki might provide some hints.

-Michael

Hello Michael,

i see then, having a chain

(callback procedure) -> MyObject.CallbackNotify -> (processing callback data) -> call a function within a form unit that does Application.QueueAsyncCall()

is really my best bet then?

Lukasz

Create an event in your wrapper class and make sure that it's documented that this event will be called from a thread. Then if you use this wrapper class from a gui app, you can use Application.QueueAsyncCall or PostMessage in the event handler. If you use the wrapper from a command line app, you'll have to use a thread-safe queue that you query from time to time in the main thread of the app, or build it into your event loop, if you have one.

Henry

Thanks - I see it's the only way forward then :)

Lukasz

I don't see another way.

And I suppose all the steps can and should be debugged separately.

-Michael

Very well then, thanks :)

Lukasz

Lukasz Sokol wrote

Very well then, thanks :)

I suggest that - to exclude problems that might be imposed by some external non-Lazarus and software not by yourself - you do a test creating a combination of a Lazarus Application and a Lazarus-made dll (and/or so) that features a TThread in the dll and uses Application.QueuAsyncCall(Form1.DisplayCallFromDLL, NIL) to display some running counter or similar stuff.

I understand for this you need to
- unify the Heaps of Application and dll
- do a call to the dll to provide the Application and the Form1 main form instances of the calling program

If you succeed in doing a really reliable Test implementation of this, I would be happy to see the source code.

This could be the basis of a "standard" implementation of something similar to Delphi runtime packages.

Thanks,
-Michael

You seem to think that Lukasz is making a library, but I can't see an email where he says that. He does say that he's making a wrapper for a library, though.

Henry

Michael Schnell wrote

On 08/08/2013 01:15 PM, Lukasz Sokol wrote: > but I would then need to make the wrapper object aware of the > Application or its forms then? This is provided by "Runtime Packages" with Delphi.

Michael, it seems that you understood Lukasz wrong, this has nothing to do with runtime packages. He has a Pascal/Lazarus application and has a C(++) library where he wants to provide a unit for. Now this library has a callback (for the application) which is called in a different thread. In this context it would be perfectly legal to use e.g. Application.QueueAsyncCall, but he wants to keep his interface unit independant of Lazarus. Thus in this situation he should - as Henry wrote - provide an event that is called by the callback and which can then be set by the calling application.

Regards,
Sven

Henry Vermaak wrote

You seem to think that Lukasz is making a library,

This is what I understood from "self contained wrapper unit". But maybe I am wrong.

-Michael

Sven Barth wrote

Michael, it seems that you understood Lukasz wrong,

Yep. I thought he intents to do a dll himself, but maybe I am wrong.

-Michael

I, erm, second what Sven and Henry have said - I am not creating my own library, but merely trying to build a wrapper...

(OT) (On a side note to the runtime packages, some years ago I had to look for and indeed I came on someone's 'plugin system' for delphi (7) that ran with .dll (as opposed to .bpk where it usually is possible) and could add forms to your main application at runtime... needless to say, it was a paid-for (and IIRC with NDA attached) system; But I was /under impression/ that it would be as easy as having same memory/heap manager instance for main program and for dll (of which some were open source), and that was the main hurdle too, and passing the Application instance into it by using some Interfaces, and the dll capabilities back in the same manner;
but before I had time to get /less/ inept on Interfaces, the demand for this feature has ended...
so I can't really remember anything else out of that area, sorry.

[Also it may all be rubbish what I wrote, t'is only what I remember of my impressions, only... I don't have any hard data of this saved anywhere (my then-employer was really insisting on not taking this work home)...] )

-L

Lukasz Sokol wrote

Also it may all be rubbish what I wrote...

Sorry If I misled you...

-Michael

It's not only the memory that needs to be shared. All DLLs need to use the same RTL functions and the VMT pointers need to be equal as well. If you have two DLLs and both use TObject than for a TObject variable of DLL A "is" is not true for a TObject from DLL B. This is what runtime packages solve correctly (in addition to dependency loading, etc).

Regards,
Sven

In our previous episode, Sven Barth said:

I assume I don't have to tell you, but for the benefits of others:

In general, there is only one copy of anything in a system with packages, so the unit's code and const parts (rtti/vmts) and its state (variables) only exist once. Both from RTL and user parts.

And there already is much more crucial global state (locale, exception/synchronize system, unicode codepage stuff in trunk etc) than just memory manager.