Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2011-01

Even more radical suggestion regarding Event Queues

23 messages · 6 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Michael Schnell

After some more thinking about the issue:

Obviously doing Event Queues with the current Linux based Widget Types is not easy to do (in the way it is done right now). In some revisions PostMessage does not work at all, in some revisions TApplication.QueueAsyncCall does not work as expected, in some revisions TThread.Synchronize does not work as expected (While TThread.Queue is not implemented at all).

As far as I understand, (e.g.) the gtk (2) Widget Set is used by a "loop" (repeated call or callback) that alternately runs the Widget Set code and the application program code, while the widget set code (additionally) performs callbacks into the application program code when a GUI event is fired by the user's interaction.

Now, at some point the code (GUI Thread) needs to go to sleep if nothing is to be done. It seems like the Widget Set is done in a way that it automatically wakes up the GUI Thread when user interaction is detected and thus the appropriate Callback is done and seemingly as a consequence, the "loop"ed code in the application program is executed, too, resulting in execution of all events scheduled up till now.

This results in the problem, that waking up the program whenever an event is to be handled that is not generated by the Widget Set, is not that easy to do. (Especially regarding that doing too much polling will degrade the overall performance.)

With Windows, this problem does not appear, as non-GUI events are scheduled by a PostMessage() or PostThreadMessage() API call, that automatically wake up the target-thread.

So a more radical way of Event handling with Linux could be worth considering.

The Widget Set could be assigned to a dedicated thread, while it's callbacks are transferred to the main thread via the application's message queue (that is implemented anyway) by Application.QueueAsyncCall().

The Pro being:
- The Main thread is completely independent of the GUI, the sleeping and waking of the main thread can be done with pure Linux means, automatically scheduling the GUI- and the non-GUI- events appropriately, while the GUI thread does it's work undisturbed by that.
- no necessity to permanently run (poll) the Widget Set loop, reducing the overall CPU time
- the Widget Set code (running in another thread) can run on another CPU in an SMP system (very common nowadays), thus providing more performance both for the GUI and for the application itself.

The obvious Cons being:
- much work for developing the code and testing it
- adding another level of complexity to the code for the LCL's GUI controls due to inter-thread data transfer when calling GUI functions and when taking GUI callbacks.

-Michael

Reply by email

Unknown author
to Michael Schnell
Michael Schnell wrote 44 lines
After some more thinking about the issue: Obviously doing Event Queues with the current Linux based Widget Types is not easy to do (in the way it is done right now). In some revisions PostMessage does not work at all, in some revisions TApplication.QueueAsyncCall does not work as expected, in some revisions TThread.Synchronize does not work as expected (While TThread.Queue is not implemented at all). As far as I understand, (e.g.) the gtk (2) Widget Set is used by a "loop" (repeated call or callback) that alternately runs the Widget Set code and the application program code, while the widget set code (additionally) performs callbacks into the application program code when a GUI event is fired by the user's interaction. Now, at some point the code (GUI Thread) needs to go to sleep if nothing is to be done. It seems like the Widget Set is done in a way that it automatically wakes up the GUI Thread when user interaction is detected and thus the appropriate Callback is done and seemingly as a consequence, the "loop"ed code in the application program is executed, too, resulting in execution of all events scheduled up till now. This results in the problem, that waking up the program whenever an event is to be handled that is not generated by the Widget Set, is not that easy to do. (Especially regarding that doing too much polling will degrade the overall performance.) With Windows, this problem does not appear, as non-GUI events are scheduled by a PostMessage() or PostThreadMessage() API call, that automatically wake up the target-thread. So a more radical way of Event handling with Linux could be worth considering. The Widget Set could be assigned to a dedicated thread, while it's callbacks are transferred to the main thread via the application's message queue (that is implemented anyway) by Application.QueueAsyncCall(). The Pro being: - The Main thread is completely independent of the GUI, the sleeping and waking of the main thread can be done with pure Linux means, automatically scheduling the GUI- and the non-GUI- events appropriately, while the GUI thread does it's work undisturbed by that. - no necessity to permanently run (poll) the Widget Set loop, reducing the overall CPU time

That's wrong. The thread would then run the widget set loop, since that must be run in every scenario. And Poll() does not consume any CPU.

I would never use a widget set with the structure you describe. Threading problems are extremely difficult to debug, doubly so in an event based system. What's more, X11 is not .

Reply by email

Henry Vermaak
to Michael Schnell: After some more thinking about the issue: Obviously doing Event Queues with the current…
Michael Schnell wrote 26 lines
After some more thinking about the issue: Obviously doing Event Queues with the current Linux based Widget Types is not easy to do (in the way it is done right now). In some revisions PostMessage does not work at all, in some revisions TApplication.QueueAsyncCall does not work as expected, in some revisions TThread.Synchronize does not work as expected (While TThread.Queue is not implemented at all). As far as I understand, (e.g.) the gtk (2) Widget Set is used by a "loop" (repeated call or callback) that alternately runs the Widget Set code and the application program code, while the widget set code (additionally) performs callbacks into the application program code when a GUI event is fired by the user's interaction. Now, at some point the code (GUI Thread) needs to go to sleep if nothing is to be done. It seems like the Widget Set is done in a way that it automatically wakes up the GUI Thread when user interaction is detected and thus the appropriate Callback is done and seemingly as a consequence, the "loop"ed code in the application program is executed, too, resulting in execution of all events scheduled up till now. This results in the problem, that waking up the program whenever an event is to be handled that is not generated by the Widget Set, is not that easy to do. (Especially regarding that doing too much polling will degrade the overall performance.)

No, it's very easy to do. At least gtk and qt has dedicated functions to do just this (g_main_context_wakeup() and QEventLoop::wakeUp()). Lazarus just has to do this after QueueAsyncCall (I do this manually myself for the moment).

Henry

Reply by email

Michael Schnell
to Henry Vermaak: No, it's very easy to do. At least gtk and qt has dedicated functions to do just this…
Henry Vermaak wrote 5 lines
No, it's very easy to do. At least gtk and qt has dedicated functions to do just this (g_main_context_wakeup() and QEventLoop::wakeUp()). Lazarus just has to do this after QueueAsyncCall (I do this manually myself for the moment).

I suppose this is (or should be) done in WakeMainThread that called by Synchronize in classes.inc.

But nonetheless my example shows that Synchronize does not work correctly (only firing the Event when additionally a GUI Event is handled).

Moreover I feel that it would be appropriate to implement TThread.Synchronize and PostMessage, (plus the yet not existing PostThreadMessage and TThread.Queue) on top of
TApplication.QueuAsyncCall, so that all of them should work, once QueuAsyncCall is done.

-Michael

Reply by email

Henry Vermaak
to Michael Schnell: I suppose this is (or should be) done in WakeMainThread that called by Synchronize in…
Michael Schnell wrote 11 lines
On 01/20/2011 01:20 PM, Henry Vermaak wrote: > > No, it's very easy to do. At least gtk and qt has dedicated functions > to do just this (g_main_context_wakeup() and QEventLoop::wakeUp()). > Lazarus just has to do this after QueueAsyncCall (I do this manually > myself for the moment). I suppose this is (or should be) done in WakeMainThread that called by Synchronize in classes.inc. But nonetheless my example shows that Synchronize does not work correctly (only firing the Event when additionally a GUI Event is handled).

Report a bug with a simple example program, please.

Show quoted text 5 lines
Moreover I feel that it would be appropriate to implement TThread.Synchronize and PostMessage, (plus the yet not existing PostThreadMessage and TThread.Queue) on top of TApplication.QueuAsyncCall, so that all of them should work, once QueuAsyncCall is done.

Why do you think QueueAsyncCall isn't done? As far as I can see, it just needs to wake up the main thread when a message is queued (something I do manually for now). This bug is reported in http://bugs.freepascal.org/view.php?id=13120.

No one that knows lcl message loop internals have really commented on how this must be solved.

Henry

Reply by email

Michael Schnell
to Henry Vermaak
Henry Vermaak wrote
Report a bug with a simple example program, please.

AFAIK, there already is a bug report on that in the bug tracker (I once read it but some days ago I failed to find it, though). Moreover it would be good to do a test with qt as well, before filing another report.

Why do you think QueueAsyncCall isn't done?

In the thread "Event handling example project: results for gtk2 and fpGUI" I already stated that it suffers from the same problem as TThread.Synchronize.

So in fact it might be a good idea to file another bug report. I think it would be best to put the "Event handling example project" in the "examples" directory in the svn, so it can easily be used for testing and I can mention same in the bug report.

As far as I can see, it just needs to wake up the main thread when a message is queued (something I do manually for now).

Hmm. If you do the same that synchronize does (i. e. call "WakeUpMainThread" ) I don't think it will work (with gkt).

This bug is reported in http://bugs.freepascal.org/view.php?id=13120.

Supposedly this in fact is the bug report I had been reading (and failed to find as I searched for "synchronize). As (with gtk) the same issue can be seen with Synchronize, I'll just add a comment to this report.

No one that knows lcl message loop internals have really commented on how this must be solved.

But AFAIK, in the latest _binary_release_ of the LCL synchronize did work on gtk, so someone _has_ been working on that and did not notice that he broke it. :(

-Michael

Reply by email

Henry Vermaak
to Michael Schnell
Michael Schnell wrote
In the thread "Event handling example project: results for gtk2 and fpGUI" I already stated that it suffers from the same problem as TThread.Synchronize.

That doesn't mean it isn't "done", that means it has a bug.

So in fact it might be a good idea to file another bug report. I think it would be best to put the "Event handling example project" in the "examples" directory in the svn, so it can easily be used for testing and I can mention same in the bug report.

Yes, report another bug for synchronise and attach the test project there. I don't see a reason why it should be in the repo.

> As far as I can see, it just needs to wake up the main thread when a > message is queued (something I do manually for now). Hmm. If you do the same that synchronize does (i. e. call "WakeUpMainThread" ) I don't think it will work (with gkt).

As I've said in the bug report, I call
g_main_context_wakeup(g_main_context_default) myself to wake up the loop. This works perfectly, but is obviously suboptimal, since it need to be inside an ifdef.

Henry

Reply by email

Michael Schnell
to Henry Vermaak
Henry Vermaak wrote 5 lines
As I've said in the bug report, I call g_main_context_wakeup(g_main_context_default) myself to wake up the loop. This works perfectly, but is obviously suboptimal, since it need to be inside an ifdef.

Am I wrong assuming that WakeUpMainThread (which is a defined as

var
   { method proc that is called to trigger gui thread to execute a method }
   WakeMainThread : TNotifyEvent = nil;

) in fpc's classes.inc, thus something provided independent of the underlying Widget Type should be used for that purpose ?

I understand that in the initialization phase this variable somehow is filled with the address of a procedure that (depending on the underlying Widget Type), is supposed to do what is necessary.

-Michael
(... hating that this is called the "GUI thread even in FPC, while it is obvious that this is necessary in non-GUI applications that use timers etc, as well ;). )

Reply by email

Henry Vermaak
to Michael Schnell
Michael Schnell wrote 18 lines
On 01/21/2011 01:05 PM, Henry Vermaak wrote: > > As I've said in the bug report, I call > g_main_context_wakeup(g_main_context_default) myself to wake up the > loop. This works perfectly, but is obviously suboptimal, since it need > to be inside an ifdef. Am I wrong assuming that WakeUpMainThread (which is a defined as var { method proc that is called to trigger gui thread to execute a method } WakeMainThread : TNotifyEvent = nil; ) in fpc's classes.inc, thus something provided independent of the underlying Widget Type should be used for that purpose ? I understand that in the initialization phase this variable somehow is filled with the address of a procedure that (depending on the underlying Widget Type), is supposed to do what is necessary.

Yes. in
gtk2widgetset.inc. It creates a pipe, which gets watched by the main loop. PrepareSynchronize then just writes something to the pipe, which wakes up the main func via the watched pipe. Or so the theory goes. I haven't really looked into it more closely.

Henry

Reply by email

Michael Schnell
to Henry Vermaak
Henry Vermaak wrote 6 lines
Yes. in gtk2widgetset.inc. It creates a pipe, which gets watched by the main loop. PrepareSynchronize then just writes something to the pipe, which wakes up the main func via the watched pipe. Or so the theory goes. I haven't really looked into it more closely.

OK, once again, IMHO, crafted a little bit short sighted. The init for Synchronize should not create it's own pope (AKA "Event Queue"), but all similar functionalities (i.e. TApplication.QueueAsyncCall, TThread.Synchronize, SendMessage, SendThreadMessage, TThread.Queue) should use a common queue (and in fact be based on the most flexible function (being QueuAsyncCall, providing a pointer as a parameter to the callee). This would result in more manageable code, Especially, if same is moved out of the Widget Type code and provided for all Widget Types to use, if they want to.

-Michael

Reply by email

Michael Schnell
to Michael Schnell
Michael Schnell wrote
Very funny: I did a slightly less complex test program and here TThread.synchronize does work as expected in gtk2.

I found that the difference is this:

If the memo that displays the messages resulting from the threads activity (via TThread.Synchronize or via Application.QueueAsyncCall) has the focus, the event does work. If the memo does not have the focus (as another control element on the Form or the Window of another application has the GUI focus) the queued Events do get handled in the main (GUI) thread.

Supposed this queer behavior is the reason why the problem has not yet been detected by the testers.

Hope this helps,
-Michael

Reply by email

Henry Vermaak
to Michael Schnell
Michael Schnell wrote 14 lines
On 01/21/2011 05:57 PM, Michael Schnell wrote: > Very funny: > > I did a slightly less complex test program and here > TThread.synchronize does work as expected in gtk2. I found that the difference is this: If the memo that displays the messages resulting from the threads activity (via TThread.Synchronize or via Application.QueueAsyncCall) has the focus, the event does work. If the memo does not have the focus (as another control element on the Form or the Window of another application has the GUI focus) the queued Events do get handled in the main (GUI) thread.

The QueueAsyncCall bug is known, as I've said before. Please create a bug report for Synchronize with an example project. They can be marked as related if it turns out that way.

Henry

Reply by email

Alexsander Rosa
to Joshua Lim

Internet Conection Sharing?

2011/1/24 Joshua Lim

Show quoted text 9 lines
Hi, i've a major project that uses ICS, and I intend to port it to Lazarus. Has anyone used ICS on Lazarus successfully? Thanks. Rgds, Joshua
Atenciosamente, Alexsander da Rosa Linux User #113925 "Extremismo na defesa da liberdade n?o ? defeito. Modera??o na busca por justi?a n?o ? virtude." -- Barry Goldwater -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20110124/acc63f70/attachment.html>

Reply by email

Joshua Lim
to Alexsander Rosa

Hi Alexsander, thanks, i meant the ICS TCP component. But I found my answer, it can't work in Linux, so no go. :-(

Alexsander Rosa wrote 23 lines
Internet Conection Sharing? 2011/1/24 Joshua Lim < <mailto:>> Hi, i've a major project that uses ICS, and I intend to port it to Lazarus. Has anyone used ICS on Lazarus successfully? Thanks. Rgds, Joshua -- Atenciosamente, Alexsander da Rosa Linux User #113925 "Extremismo na defesa da liberdade n?o ? defeito. Modera??o na busca por justi?a n?o ? virtude." -- Barry Goldwater

Reply by email

Howard Page-Clark
to Joshua Lim: Hi, i've a major project that uses ICS, and I intend to port it to Lazarus. Has anyone…
Joshua Lim wrote
Hi, i've a major project that uses ICS, and I intend to port it to Lazarus. Has anyone used ICS on Lazarus successfully?

Note - you seem to have hijacked a thread with a different subject, I presume by mistake.
The webservices toolkit (WST) available at:

http://wiki.freepascal.org/Web_Service_Toolkit

uses ICS (and other libraries as well) very successfully.

H

Reply by email