Hi All.

Maybe not a Lazarus/FPC specific question but development whill be done using Lazarus/FPC.
I wonder if I can build an application (service) having multiple threads running (Indy TCP/IP server, multiple connections) and every thread calls some functions in a library (dll on Windows, .so on Linux) I can imagine that if every thread loads dynamically the library it should work as I assume that a library shares its code among all callers (processes ?) but each instance has its own data segment tied tp the calling process (thread ?).

Thank you for comments.

Antonio.

---
Ce courrier ?lectronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

The data segment is tied to the process, not to the thread.

So global variables in the DLL are shared by all threads of the calling process. If you define threadvars in the DLL, they will normally be specific to threads of the calling program (just as they would be if the library created the thread).

Michael.

Le 17/01/2014 12:53, Michael Van Canneyt a ?crit :

Ok, got it. The library has to be made thread safe as well. This was already in my mind.
Tnaks Michael.

Antonio.

---
Ce courrier ?lectronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

There will be no problems if the DLL is thread-safe, ie, is better not use global variables in the DLL.

Marcos Douglas

Marcos Douglas wrote

is better not use global variables in the DLL.

Of course I agree.

A reason for using global variables in a DLL might be to access that value as well in the DLL as in the main program.

Is this even possible ?

(To allow for this loading the DLL would need to modify the variable's address anywhere within the DLL code. Up til now I only know about the the "jump table interface" that allows the main code to call DLL functions. )

-Michael

Yes, this is possible.

On *nix systems this is easy, as they don't have these indirect references. On Windows it needs support on the side which uses the variable and at least in theory FPC should support this (though it might be buggy).

Regards,
Sven

Yes, see the Sven Barth answer.

(To allow for this loading the DLL would need to modify the variable's address anywhere within the DLL code. Up til now I only know about the the "jump table interface" that allows the main code to call DLL functions. )

But why do you need a global var in the DLL? IMHO (and for many programmers around the world), global var is not good, even in the application code.

Regards,
Marcos Douglas

Vrey interesting indeed.

I agree with Marcos on his last remark. I'm concerned with multi-threading from a long time now and I'm used to avoid global vars in any case.
Reading your answers I'm more strongly convinced that this position should be the master rule when dealing with threads. But one should also know who does what.
This is the origin of may quenstion

Thank you all again for your pertinent comments.

Antonio.

---
Ce courrier ?lectronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

Marcos Douglas wrote

But why do you need a global var in the DLL? IMHO (and for many programmers around the world), global var is not good, even in the application code.

I don't suppose I'll ever want to do this. :-) - I just was being nosy.

-Michael

Marcos Douglas wrote

But why do you need a global var in the DLL? IMHO (and for many programmers around the world), global var is not good, even in the application code.

Although I think that this is in part "received wisdom", dating back to some of the earliest ALGOL computers where globals were a comparatively scarce resource which couldn't be swapped with the same ease as variables on the stack or storage on the heap.

If they really were so unutterably bad, I'd like to think that the designers of Delphi would have gone to more trouble to define forms as something non-global.

So going back to the specific question, as I understand it you get a global in a DLL if you define a form in it. In practical terms it might not be immediately usable, but as an example I've used a form in a DLL as a template and copied menu entries from it to an appropriate form in the main app.

I said: IMHO, not only "programmers around the world"... so, for me is not a "received wisdom", but a lesson learned. ;-)

If they really were so unutterably bad, I'd like to think that the designers of Delphi would have gone to more trouble to define forms as something non-global.

I think so. The design using global variables, components as published not private... I'd like to think they chosen a simple and fast way to code.

I think this is not his problem... but I could be wrong.

Best regards,
Marcos Douglas

Marcos Douglas wrote

I said: IMHO, not only "programmers around the world"... so, for me is not a "received wisdom", but a lesson learned. ;-)

Unfortunately Lazarus uses global variables in the LCL. Otherwise it would be possible to create a project with a TApplication instance per thread and have it run with multiple simultaneous GUIs.

-Michael

"Michael Schnell" wrote

On 01/17/2014 05:47 PM, Marcos Douglas wrote: > > I said: IMHO, not only "programmers around the world"... so, for me is

not a "received wisdom", but a lesson learned. ;-)

Unfortunately Lazarus uses global variables in the LCL. Otherwise it

would be possible to create a project with a TApplication instance per thread and have it run with multiple simultaneous GUIs.

Wouldn't help much, because e.g. Qt does not allow usage of GUI widgets in multiple threads.

Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20140117/8dfcc17f/attachment.html>

I was being somewhat speculative. However, working back through the thread I'm not sure that anybody's mentioned the problems that can be caused if the app and DLL/so have distinct memory managers, e.g. the library tries to allocate an ansistring from one pool as a return result and the main program tries to free it in the context of a different heap.

http://qt-project.org/doc/qt-4.8/threads.html

I did not thoroughly read this but it seems that QT is thread-aware

-Michael

If you do read thoroughly, you'll notice:

http://qt-project.org/doc/qt-4.8/threads-starting.html

"Note that QCoreApplication::exec() must always be called from the main thread (the thread that executes main()), not from a QThread. In GUI applications, the main thread is also called the GUI thread because it's the only thread that is allowed to perform GUI-related operations."

Confirming what Sven already said.

Michael.

in multiple threads.

Trust me, I'm currently working with Qt's internals as I've just finished the port of Qt to my company's OS, so I know what I'm talking about... (and I've already hit an assertion regarding this)

Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20140120/3cec8d45/attachment.html>

I see.

In fact I already came across applications where starting multiple independent GUI's would make sense and I did a "proof of concept project" (using Delphi in Windows) that this started to work (but certainly would need a lot of tweaking) using a thread and a DLL per GUI instance.

I had _some_ hope that Lazarus might one day decently support this feature but of course this very seldom asked for.

Thanks,
-Michael

I didn't think that in Linux with NPTL there not even is a thing like a "main thread" of a program, but (before binding to a GUI provider) all threads of a project are equivalent. As a consequence, multiple of them should be able to be bound to a GUI provider instance - as long as same does not impose global variable.

http://www.gentoo.org/doc/en/articles/l-posix1.xml says:

Things are a little more interesting with POSIX threads. You may have noticed that I have intentionally avoided using the terms "parent thread" and "child thread" so far. That's because with POSIX threads this hierarchical relationship doesn't exist. While a main thread may create a new thread, and that new thread may create an additional new thread, the POSIX threads standard views all your threads as a single pool of equals. So the concept of waiting for a child thread to exit doesn't make sense. The POSIX threads standard does not record any "family" information. This lack of genealogy has one major implication: if you want to wait for a thread to terminate, you need to specify which one you are waiting for by passing the proper tid to pthread_join(). The threads library can't figure it out for you.

-Michael

thread

> (the thread that executes main()), not from a QThread. In GUI

applications,

> the main thread is also called the GUI thread because it's the only

thread

> that is allowed to perform GUI-related operations." > I didn't think that in Linux with NPTL there not even is a thing like a

"main thread" of a program, but (before binding to a GUI provider) all threads of a project are equivalent. As a consequence, multiple of them should be able to be bound to a GUI provider instance - as long as same does not impose global variable.

For Qt the main thread is the thread that first called QCoreApplication::exec() (or that first constructed a QCoreApplication instance? I'd need to check that again...). This is ensured using the thread's ID.

Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20140120/e997eea5/attachment.html>

I stumbled over this:

http://tronche.com/gui/x/xlib/display/XInitThreads.html

So Xlib is thread aware and they seem to suggest that it dopes make sense to create multiple X sessions (i.e. one per thread) in a single program.

(And I learned that in fact XInitThreads() can be called in Lazarus projects. ->
http://lists.lazarus.freepascal.org/pipermail/lazarus/2013-January/078799.html )

So in fact X does support multiple "!GUIs" (e.g. one per thread) in a multithreadded project.

Of course it might be that QT (or KDE) impose additional restrictions (though I still suppose that this can be handles somehow).

-Michael

Thread aware does not necessarily mean thread-safe.

I think you should read

http://tronche.com/gui/x/xlib/display/XLockDisplay.html

It kind of defeats the purpose, as it suggests that only 1 thread at a time can access the display.

Michael.

Michael Van Canneyt wrote

It kind of defeats the purpose, as it suggests that only 1 thread at a time can access the display.

Of course concentrating the GUI stuff to a single thread makes things a lot easier to handle.

But I already did explain the purpose I (once) had in mind with this:

If you have multiple very actively automatically updated GUI elements it does make sense to allow "rendering" same using multiple processors of a modern multicore system. An easy way (for the user, provided the LCL- / RTL- / WidgetSet- infrastructure would support this) is assigning a thread to each of them. Here the rendering could be done in true parallel manner up to the point that X needs to file the updates to the hardware.

-Michael

BTW.:

What we did to take advantage of the multicore architecture in fact was creating multiple applications, each of which handles such a GUI element in a dedicated Window that is automatically placed at the appropriate location "above" the main window. But of course the overhead for managing and placing the multiple windows and transferring the process data between the multiple applications is enormous, not even calculating the huge additional memory needed.

-Michael

At that point why not bite the bullet and exploit GPU parallelism?

KDE and Qt are minor issues, far more significant is arbitrary restrictions that GTK imposes because they envisage some usage case that will break Gnome.

This may make sense with remote displays.

So in fact X does support multiple "!GUIs" (e.g. one per thread) in a multithreadded project.

This is not a matter of X (Windows: GDI), but of the application GUI code and libraries.

Of course it might be that QT (or KDE) impose additional restrictions (though I still suppose that this can be handles somehow).

Right, and the restriction to one widgetset resides in the Lazarus widgetset handling.

DoDi

Mark Morgan Lloyd wrote

At that point why not bite the bullet and exploit GPU parallelism?

Yep ! :-)
(And this hint is going out to whom ? ;-) )

KDE and Qt are minor issues, far more significant is arbitrary restrictions that GTK imposes because they envisage some usage case that will break Gnome.

Gnome easily handles multiple applications calling it's API, and as with NPTL in Linux, the difference between a process and a thread is close to Zero (in fact the API that is called, would need to actively check whether or not two calls come in with the same memory mapping). So for me it's astonishing that GTK / Gnome should suffer from this issue.

-Michael

Hans-Peter Diettrich wrote

Right, and the restriction to one widgetset resides in the Lazarus widgetset handling.

I do know. But it does not make much sense to be nagging towards the powers here about LCL not being reentrant, if the library (Widget Set, ...) it binds to does not decently support multiple GUIs in multiple threads of an application.

-Michael

Michael Schnell wrote

Of course it might be that QT (or KDE) impose additional restrictions (though I still suppose that this can be handles somehow).

No, this can not be handled somehow (and it's not KDE, but Qt itself): Upon creation of the QApplication class it remembers the thread ID and every GUI widget is checked that it is owned by this thread ID as well. Certain events like rendering are also checked for the thread ID.

Regards,
Sven

Sven Barth wrote

Upon creation of the QApplication class it remembers the thread ID and every GUI widget is checked that it is owned by this thread ID as well. Certain events like rendering are also checked for the thread ID.

While I don't exactly see the point (as each thread of the application will have it's own thread id and thus should be handled as "unrelated"), I am sure that you do know what you say, denying the possibility in question.

Anyway this is really bad news regarding the performance of KDE, QT etc (and consequence the LCL) on multiprocessor systems:

Doing multiple automatically updated widgets will perform a lot faster when done in multiple applications than in a single one. (We did check this with Delphi/Windows: same here.)

(Maybe in Windows this might be better: I once tried fooling the system by doing a DLL per thread and run each of them as a thread with it's own VCL instance. It did work but crashed after some time and I did not have the guts to go on hacking and track down that problem.)

-Michael

Sorry, but I think you are totally misdesigning your system.

The GUI is for use by humans. That means that there is no point whatsoever in updating the GUI more than 10 times per second: the human eye cannot process information faster than that, let alone that the brain can grasp the *meaning* of what the eye has seen in such a short time.

Designating 1 GUI thread to update the entire screen 10 times per second should therefor
a) be entirely possible with any GUI widgetset.
b) be complete overkill for any human operator.

if I play a video at full HD, 1 thread handles the screen display. It works just fine, and probable needs to transfer a lot more data to the screen than any normal program would have to transfer. It would be very surprising if you would not be able to show some statistical data (or whatever) on screen with the same hardware.

Michael.

Michael Van Canneyt wrote

I play a video at full HD, 1 thread handles the screen display. It works just fine,

Playing two videos at the same time does make sense in some applications (we "create" 20 small "videos" in our application to have the user see what the system behind his work-place is doing).

-Michael

Exactly:
you can perfectly collect 20 stills and display a composed view of all 20 stills, 10 times a second. Should be perfectly in reach of a single thread.

Michael.

I'm afraid I disagree. As a trivial example, the position of the mouse pointer is updated much more often than every 100mSec, not to mention things like animated pointers and icons. Just because a GUI invites user interaction doesn't mean that it can't run asynchronously.

I'm afraid I disagree with that even more, since when you play a video you're relying on hardware decompression which you can't reliably characterise as either single- or multi-threaded. Believe me, I've got Linux etc. on so many odd and ageing systems around here that I regularly come up against the suppurating lacerations that were "bleeding edge" until some developer lost interest and didn't stitch it up, and in my experience the thing that's most likely to work is DVD playback because the data stream is (as I understand it) sent via a backdoor to the graphics chips bypassing most of the kernel and X.

Having said which, I'm not unreservedly siding with MichaelS's position, since unless it completely bypassed any existing widget set (and preferably drew directly to the framebuffer) it could be screwed at any time by off-project developers (i.e. the Gnome, Qt or Windows teams).

You didn't prove anything with this statement. It is obviously possible. But is it necessary ? I don't think so. For 'simple' things like mouse cursor, you can do this because it requires no resources.

But complicated information like statistics or information of 20 cameras, needs time to be processed by the brain in order to react on it. There is therefor no need to have it refreshed in intervals that the brain cannot discern anyway. What could possibly be achieved with displaying information of 20 cameras at 60 FPS ? Nothing, unless you want to brag about your hardware.

I'm not saying that you should wait more than a second, but 1/10th, or even 1/24th should be ample for the stated purpose, and well within reach of a single thread.

Based on that I just don't think that what Michael Schnell aims to accomplish calls for a full-blown multi-threaded GUI architecture. The overhead that would introduce just isn't worth it. It's killing a mosquito with a cannon.

Michael.

When you call a method of a widget that results in some GUI related stuff Qt checks the current thread ID against the thread ID the QApplication was created in (or more precisely: the thread ID the event loop is running in) using an Assert. While there can be multiple event loops in Qt and there can be signal passing across threads it is not possible to have more than one active GUI event loop.

Anyway this is really bad news regarding the performance of KDE, QT etc (and consequence the LCL) on multiprocessor systems:

I agree with Michael van Canneyt that I don't even remotely see the need for multi threaded GUIs.

Regards,
Sven

The GUI is for use by humans. That means that there is no point whatsoever in updating the GUI more than 10 times per second: the human eye cannot process information faster than that, let alone that the brain can grasp the *meaning* of what the eye has seen in such a short time.

<snip>

if I play a video at full HD, 1 thread handles the screen display. It works just fine,

Combining these two statements sounds like you are running your video at 10 frames per second, which you hopefully don't do. And I am very interested where you got the limitations for the human eye from. Knowing a thing or two about animation I have heard quite different values. Examples:

Nearly everyone I know at least feels a difference between an animation running in 60 FPS and one in 30 FPS (both frame rates need the eye and brain to process faster than your claimed 10 updates a second, else there would be no difference).

The USAF tested their pilots visual response time by flashing a picture of an aircraft on a screen in a dark room for 1/220th of a second. The pilots were consistently able to identitfy the shown craft.

There are more examples but I am not interested in starting a discussion about this.

Closing I must admit that for a GUI 10 FPS is mostly enough. It is just that 1/10th of a second as the human eyes limit sounds way to low to me.

R.

No, obviously not.

And I am very interested where you got the limitations for the human eye from. Knowing a thing or two about animation I have heard quite different values.

Traditional Film uses 24 FPS, this was considered twice the speed of what was/is needed to experience motion 'fluently'.

The human eye can *discern* changes at faster rates, meaning it will recognize the difference between a film played at 30 and 60, the latter being perceived as more 'smooth'.

But this does not enable you to process the *information* displayed at such a speed.

using a practical example:
Let me display random numbers at speed > 10 FPS on screen. Will you be able to repeat the numbers ? I doubt it.

You may want to watch "Fight Club", where this is demonstrated in a cinema ;)

Michael.

I'm afraid that's... highly questionable. The more video streams (etc.) that are visible the smoother you want the overall presentation to be: the human brain is /very/ sensitive to small timing differences. In addition, there are standards for e.g. flashing indicators in safety-critical applications which are specified with much finer resolution than 100 mSec.

So a 10Hz update is totally out of the question these days: 100Hz would be more realistic.

But I still don't entirely agree with MichaelS: just because something is highly desirable doesn't mean it's attainable, and some sort of double-buffering would probably be a good compromise.

and in my experience the thing that's most likely to work is DVD playback because the data stream is (as I understand it) sent via a backdoor to the graphics chips bypassing most of the kernel and X.

On Linux and BSD at least it's the standard to bypass X when talking to the GPU. You can talk to the GPU from different threads (with a little overhead) but you actually gain nothing, because of how the drivers work. There may be some way to get better performance if you are talking to multiple GPUs and you can do a better job than the vendors driver to utilize them (highly unlikely). There are reasons to talk to the GPU from different threads but that's only usefull for loading resources while not interupting the main rendering thread, there is no speed gain.

R.

Yes. I don't argue with that.

I would be very interested to see experiments proving that this will make an actual difference in human operator efficiency.

If something is that time critical, I would not leave it to a human operator to begin with :)

Michael.

Traditional Film uses 24 FPS, this was considered twice the speed of what was/is needed to experience motion 'fluently'.

It is not unusual to show the same frame three times to bring the framerate up to 72fps to reduce the "black screen time" which will be percieved as flickering. Motion blur helps a lot to make the experience 'smooth'. Why do you think it is recreated in 3D graphics?

The human eye can *discern* changes at faster rates, meaning it will recognize the difference between a film played at 30 and 60, the latter being perceived as more 'smooth'.

And the general human can see changes up to 150 fps so the eyes must be able to "see at that rate".

But this does not enable you to process the *information* displayed at such a speed.

As stated in my last mail the USAF and their pilots do not share your opinion.

using a practical example: Let me display random numbers at speed > 10 FPS on screen. Will you be able to repeat the numbers ? I doubt it.

That's no argument, that's speculation.

You may want to watch "Fight Club", where this is demonstrated in a cinema ;)

Thanks, but I do not watch Hollywood blockbusters. No matter the frame rate your brain is not able to process anything when it is exposed to such trash.

R.

Michael Van Canneyt wrote

[...]

I would be very interested to see experiments proving that this will make an actual difference in human operator efficiency.

I know this is completely different ballpark, but, why do you think, 3d games are all in pursuit of FPS ? And some (older ones but not just - case Starcraft (II)) being capable to /rendrer/ (not display!) 100-s of frames per second on todays' hardware, even though 60FPS would be enough for anybody's eyes? And why gamers won't buy a mouse that at least doesn't try to communicate to computer at 1000Hz ?

Answer is what MML said: it's about precision - if you are able to calculate objects' coordinates more precisely than your brain can perceive them, you'll get absolute smoothness of any movement.

So far, I think, not many can actually achieve that though. Even your plain old 2d interface however sexed up it may be...

If something is that time critical, I would not leave it to a human operator to begin with :)

:) tell this to Starcraft players ;)

Michael.

-L ;)

Not arguing with that.

Not arguing with that either.

> > But this does not enable you to process the *information* displayed at such a speed. As stated in my last mail the USAF and their pilots do not share your opinion.

Please... These people are primed for this kind of thing.

Not really. Tests show that:

Average reaction time on an emergency situation in e.g. a car is 1 second, more often 2 seconds.

Here is another one, grabbing a ruler:
http://virtuallabs.stanford.edu/tech/images/ReactionTime.SU-Tech.pdf It shows reaction times of 0.1 seconds as "fast"

Another one, stating 215 milliseconds (0.2 sec) as median click reaction time: http://www.humanbenchmark.com/tests/reactiontime/

I'm assuming that the purpose of displaying 20 videos simultaneously is so people can react on it, not simply to induce epilepsy attacks. Hence my conclusion that 10FPS is ample for a GUI - take even 24 FPS if you prefer.

The point was simply that this relatively low speed does not warrant thread-aware GUIs.

Michael.

>> But this does not enable you to process the *information* displayed at such a speed. > As stated in my last mail the USAF and their pilots do not share your opinion. Please... These people are primed for this kind of thing.

Irrelevant. If people can be trained to do such a thing the eyes and brains of humans are capable of doing it.

Even if you backpedal now (and in other mails, too), you started by stating that "there is no point whatsoever in updating the GUI more than 10 times per second: the human eye cannot process information faster than that". You are right that it makes not much sense to update a GUI faster than that (I said that in my first mail already) but you are wrong that the eyes (and in some cases the brain) cannot process visual information faster. Regarding GUI update times and threaded GUIs we were on the same side from the beginning. Looks like you did not read my first mail thoroughly enough.

All I wanted to make clear is that the human eye can process information faster than that and the brain can process the information faster than that, too. There are real world examples that it is beneficial for special software to update faster than 10 FPS (movies, games, etc.).

Pattern recognition (like in the aircraft example) is what the human eyes and brains are especially good at (and therefor fast). More complex tasks like reading (which is needed for your random number example and most GUIs) are quite a different beast.

Your reaction time examples don't add to the discussion as they involve much more than the eyes and brains visual processing speed.

The point was simply that this relatively low speed does not warrant thread-aware GUIs.

I never argued with that, quite the opposite. Quoting my first mail: "I must admit that for a GUI 10 FPS is mostly enough. It is just that 1/10th of a second as the human eyes limit sounds way to low to me."

So eyes and brains can process faster than at 10Hz (simple tasks), but nonetheless multi threaded GUIs are nonsense. A single rendering thread is good enough for complex real time graphics at 60 FPS and beyond, so it is good enough for GUIs.

R.

Right, nobody tries to support threads in GUI related libraries, because this doesn't make sense. Every programming language that supports threads also supports inter-thread communication, which can be used to send update requests to an dedicated GUI thread.

DoDi

I'm afraid that further experimentation isn't relevant: if you want to sell into e.g. drilling platform safety systems then you /have/ to adhere to the specifications that they mandate, and checking earlier with a colleague confirmed that the timings required weren't close to multiples of 100 mSec (please excuse me if I don't go back asking him for more detail, I don't want to tie him of for three days' research :-)

Obviously, this was one of the strong points of the cell-oriented PC screen: once CPUs were running at 10s of MHz you could get them to flash etc. in a predictable way.

With the major caveat that the interfaces used for playing video and for program-generated graphics are distinct (hardware MPEG decompression in one case, OpenGL in the other) and having one work implies nothing about the other.

But the general discussion didn't say that the user of [whatever it was we were talking about] wasn't similarly primed.

I agree with that conclusion, but primarily on account of the known implementation problems.

With the major caveat that the interfaces used for playing video and for program-generated graphics are distinct (hardware MPEG decompression in one case, OpenGL in the other) and having one work implies nothing about the other.

"Implies nothing" is a little too strong for my liking. I give you that this is no proof of OpenGL and hardware video decompression using the same mechanics of bypassing X. I don't know much about older implementations of hardware video decompression (my impressions at the time was that they were Windows only) but VDPAU is immplemented in the device driver, so I have a hard time believing that they do not transfer data the same way as they do for OpenGL. I know that at least for some codecs (and vendors) the decompression is done in the shader processors and I see no reason to use a radically different system to transfer video streams to these processors than they use to transfer OpenGL data. IMHO being implemented in the same driver is a hint that they use similar mechanics.

R.

At the risk of getting off-topic, my intent was to indicate that just because a machine can play a DVD doesn't mean it can play the same content from a file, and just because it can play content from a DVD or file doesn't mean that it has sufficient OpenGL support to support something like Google Earth. And the ultimate test of a system is being able to handle something like BBC audiovisual content or YouTube properly, both of which (at least until recently) made a lot of assumptions about a Flash player which itself made a lot of assumptions about the robustness of OpenGL's hardware acceleration.

Now in general, things like video content aren't a problem with recent PC hardware and a recent version of Windows or Linux. And they aren't too much of a problem with something like ARM-based tablets provided that the manufacturer has either stuck to a reference design or has invested lots of money getting a custom player from Adobe. But for older kit (e.g. laptops with limited video RAM), or non-x86 systems (e.g. SPARC with any version of Linux) or minority operating systems (e.g. Solaris, unless you can afford to pay protection money to Oracle) all bets are off: the fact that one type of video works implies nothing about whether other types work.

IMHO being implemented in the same driver is a hint that they use similar mechanics.

But they aren't implemented in the same driver: video (i.e. mpeg playback etc.) uses the standard X drivers while OpenGL uses kernel drivers accessed via the DRI interface.

Fortunately, this sort of thing doesn't usually have a direct bearing on the sort of apps that Lazarus is usually used for. Let's try to keep it that way.

Hans-Peter Diettrich wrote

Right, nobody tries to support threads in GUI related libraries, because this doesn't make sense.

IMHO it does make sense in certain applications.

The one my colleagues did is a "user interfaces application" for a huge (including many PCs plus hundreds of small embedded controllers) embedded system, that of course features many complex internal states that rapidly change.

Here the user can select a big number of "state display windows" to be shown on his - say - four HD-screen wide MDI-like GUI.

Originally the "state display windows" were just normal Windows done with normal means provided by the VCL (this application at the moment is done in Delphi).

But as the customer requests got more demanding, the count of the "state display windows" increased and their content got more complex. As a result the GUI got too sluggish to be usable. A really hot eight core engine did not help a bit.

Multithreadding the GUI would had helped, but as this is not available they finally did a "multi-application upgrade" and which does work great but needs a huge overhead managing the overplayed Windows and the data transfer (via a database) between the applications.

-Michael

Mark Morgan Lloyd wrote

I'm not unreservedly siding with MichaelS's position, since unless it completely bypassed any existing widget set (and preferably drew directly to the framebuffer) it could be screwed at any time by off-project developers (i.e. the Gnome, Qt or Windows teams)

Yep.

See my mail to DoDi for details of the application in question.

Thanks,
-Michael

While this is not what out application does, this is perfectly normal for our customers. In their control-rooms they have a huge number of screens, each permanently showing the video streams coming in from many surveillance cameras. The person sitting their can request things like full screen display of some cameras instead of "thumbnails", playing back recordings, ..

-Michael

Mark Morgan Lloyd wrote

I'm afraid that's... highly questionable. The more video streams (etc.) that are visible the smoother you want the overall presentation to be: the human brain is /very/ sensitive to small timing differences.

Yep.

We don't do such systems, but our customer do run them. They never would use a system that does not always grant all video displays to be smooth. And they apply the same standard to our "state display windows".

-Michael

Reimar Grabowski wrote

You can talk to the GPU from different threads (with a little overhead) but you actually gain nothing, because of how the drivers work.

Very wrong, IMHO.

There is a lot to do for the CPU between the user programmer's code calls an Language RTL- (LCL-) or Operating System (Widget Set-, Windowing-System-, X-) API and the point where the driver is called. Multithreading this code will speed up things a lot on a multicore engine. (And the user programmer can't do anything to that but just define the threads and do the calls within them ... or - as my colleagues did - split up the application in multiple programs.)

-Michael

Reimar Grabowski wrote

It is not unusual to show the same frame three times to bring the framerate up to 72fps to reduce the "black screen time" which will be percieved as flickering. Motion blur helps a lot to make the experience 'smooth'. Why do you think it is recreated in 3D graphics?

In fact we once did build LED based video displays and thus did a lot of experiments on this issue.

It really is true that - if you avoid any "black screen time"by hardware-double buffering in the display drivers, a frame rate of 10 fps is considered as "smooth" most of the time. But it dies not suffice if you are decently trained and really concentrate on details, which professional surveillance watches permanently do.

Moreover a decent system for an appropriate purpose features at least four HD-Screens and might be used to display some 20 videos at the same time.
Of course only one of them is really watched at a given second, but the software does not know which one.

-Michael

:-)

I agree with Michael van Canneyt that I don't even remotely see the need for multi threaded GUIs.

:-(

-Michael

Sounds correct on the first sight, but my colleagues found that this was not good enough for their needs (see my mail to DoDi) (In fact I did not doubt their analysis and hence did not do a research on my own.)

-Michael

I think a realistic question is how close can multiple threads get to a form without fouling up the LCL or widget set.

If it were possible- and I mean provably possible here, not "works most of the time"- for a background thread to prepare the graphical rendition of an object (or possibly something like a TFrame) with the main thread marshalling the GUI then there might be a useful performance gain.

But the LCL isn't in a position to try to hack around underlying limitations of the widget set and X.

Well, if you say so.
I was under the impression that VDPAU (which I was talking about) is implemented in the video card driver (see http://us.download.nvidia.com/XFree86/Linux-x86_64/331.38/README/vdpausupport.html). Btw it's in Mesa and the open source ATI/AMD driver, too.

The interoperability between VDPAU and OpenGL is implemented as OpenGL extension (http://www.opengl.org/registry/specs/NV/vdpau_interop.txt). I wonder how they achieve this if different drivers are in use.

Fortunately, this sort of thing doesn't usually have a direct bearing on the sort of apps that Lazarus is usually used for. Let's try to keep it that way.

Too late. The pascal code to decode video via libav is already written (years ago and with Lazarus) and it uses OpenGL, too. Most likely it will use VDPAU in the future as well, so that decoding on the CPU is just a fallback if your video card and/or driver do not support VDPAU. Sorry for my unusual apps not fitting in your "Lazarus apps category".

R.

Michael Van Canneyt wrote

Judging by your explanation to Dido, it seems you must explain your users the limits of their own brains. Adding monitors and indicators does not guarantee that they will process more info :)

Please see the discussion on surveillance camera systems.

You can't sell a (huge) system that does not fulfill the customer's expectations regarding their workflow. :-( .

-Michael
(Sorry for (again) replying to your personal address instead of the list )

On 02/04/2014 04:02 PM, Reimar Grabowski wrote: > You can talk to the GPU from different threads (with a little > overhead) but you actually gain nothing, because of how the drivers work. Very wrong, IMHO.

This is not about your opinion. Either you know how it works or you don't. Talking to the GPU means Direct3D or OpenGL (not any legacy VESA compatible way from the 90s). Both utilize sequential command queues. Feeding the driver from multiple threads does not gain you anything.

There is a lot to do for the CPU between the user programmer's code calls an Language RTL- (LCL-) or Operating System (Widget Set-, Windowing-System-, X-) API and the point where the driver is called.

Talk to the driver directly, that's why there are APIs to do so (namely OpenGL and Direct3D). If you need speed utilize your hardware correctly and bypass the LCL, widgetset and X.

R.

Mark Morgan Lloyd wrote

I think a realistic question is how close can multiple threads get to a form without fouling up the LCL or widget set.

Yep !
Seemingly not doable at the moment. The LCL _could_ be updated to be reentrant, but if the Widget Set it binds to does not provide decent support, there is no need to try :-(

But the LCL isn't in a position to try to hack around underlying limitations of the widget set and X.

Been there (with Windows/Delphi/VCL) and gave up.

-Michael

Reimar Grabowski wrote

Talk to the driver directly ...

OK, I could drop considering using KDE, QT, Gnome, the LCL and Lazarus and fpc and do everything in ASM.

But in fact Lazarus is more fun :-) .

-Michael

Then you have to include appropriate hardware, where multiple GPUs are preferable to multiple CPUs when it comes to processing multiple video streams concurrently.

E.g. my over 10 year old TV card talks directly to the graphics card, so that TV display frequently continues even if the system is unresponsive due to some crash. When nowadays hardware doesn't allow for more video sources, fed into one video card, then still multiple (small) monitors with dedicated graphics cards could be used to display one video stream each. Such a configuration should outperform every threaded software solution, where the video data must go through a single CPU bottleneck.

Please note that even a CPU with multiple cores doesn't have multiple data busses for the cores. That's the narrowest bottleneck, which limits the throughput on every standard machine. Next comes the DMA over one common data bus, which again limits the overall throughput. When instead you spend each camera its own small machine and monitor, the overall throughput can be increased (scaled) without limitations.

DoDi

Hans-Peter Diettrich wrote

Please note that even a CPU with multiple cores doesn't have multiple data busses for the cores. That's the narrowest bottleneck, which limits

That's a good point, but strictly it's an implementation detail which could be very different on even different chips of the same architecture.

the throughput on every standard machine. Next comes the DMA over one common data bus, which again limits the overall throughput. When instead you spend each camera its own small machine and monitor, the overall throughput can be increased (scaled) without limitations.

But some CPU types have multiple "hoses" each driving a group of PCI busses with a processor in the middle. And plenty of larger systems are NUMA to a greater or lesser extent, with provision for some peripherals to be nearer one CPU (and bank of memory etc.) than others.

Particularly on systems with some sort of "interconnect" between processor units, it's entirely possible to find the threads associated with a single program spread over a comparatively large physical volume, with each thread able to talk efficiently to "nearby" peripherals. The result of that is that data (e.g. video) collected from the peripherals can be got into memory without contention.

Being able to use multiple threads to marshal that data into e.g. a video collage would be desirable, making the assumption that the user would choose appropriate hardware. And if X or OpenGL imposes crippling overhead I don't see that as our problem, provided that we've done our bit to work as well as possible within the limitations.

Hans-Peter Diettrich wrote

Then you have to include appropriate hardware, where multiple GPUs are preferable to multiple CPUs when it comes to processing multiple video streams concurrently.

In fact the system runs just fine for multiple customers right now with many dynamical Status Windows on multiple HD-Screens attached to a singe standard fast multicore PC. This has been accomplished by using multiple separate Programs for the "dynamical Status Windows" and the main Program that provides the complex (but standard) GUI.

As already said, this is not "video" but dynamic graphics showing states in the system, that are calculated in realtime.

-Michael

Hans-Peter Diettrich wrote

Please note that even a CPU with multiple cores doesn't have multiple data busses for the cores. That's the narrowest bottleneck, which limits the throughput on every standard machine.

Right, But this is why 1st Level cache has been invented :-) , (and why expensive multicore chips have more cache than cheaper ones).

-Michael

And loose your job for missing deadline after deadline. ;)

But in fact Lazarus is more fun :-) .

It's perfectly doable with FPC and Lazarus. They come with all you need to start developing with OpenGL (a more direct access to the driver is not possible no matter which language you use). Why else should I post this to the Lazarus list?

R.

What I meant by the sarcasm is: Even if there are other ways to reach a goal, it does make sense to work with the "powers" to help improving a tool that shows shortcomings in certain aspects, while being perfectly useful in many others.

-Michael

On an 7 year old athlon X2 I can stream 5 Gigabit cameras at about 50MB/s each. CPU will show 0% utilization.

The offloading of the intel pro cards delivers it directly into memory, without measurable contention.

Of current systems (stock desktops, nothing NUMA) memory <-> cpu bandwidths are measured in tens of GByte/s. Even for that Athlon X2 it was in the single GByte/s.

The question is more how you are going to display that stuff swiftly. GDI? Forget it. OpenGL gives up at about 40-60 MByte/s on a discrete model, and half on integrated video. (HD4000)

One can try to create a GUI for this already remote scenario (independent processes display to independent sections of the screen), but if the systems below can't handle it, what's the point?

I agree, with the caveat that an improved X or improved OpenGL on a suitable architecture could possibly boost performance significantly without losing API compatibility.

But having application-level components trying to guess the limitations of lower-level system elements is far too risky.

What I meant by the sarcasm is: Even if there are other ways to reach a goal, it does make sense to work with the "powers" to help improving a tool that shows shortcomings in certain aspects, while being perfectly useful in many others.

You see that as shortcomings. Most others (even widgetset developers) see thread-safe GUIs as nonsense without real benefit, else they would be available. I think we can agreee to disagree.

R.

Reimar Grabowski wrote

You see that as shortcomings. Most others (even widgetset developers) see thread-safe GUIs as nonsense without real benefit, else they would be available. I think we can agreee to disagree.

This is a funny opinion indeed.

- we did a project in a standard (single-threaded GUI) way
- with enhanced demand of the customers this did not work decently (GUI getting sluggish)
- we found that using a multi-threaded GUI would solve the problem
- we found that this is not possible due to shortcomings of the underlying system
- we did a redesign with multiple applications and found that this does work fine on exactly the same hardware
- it was a huge overhead regarding the software design, the effort to transfer data between the multiple applications, and RAM demand
- hence it is obviously a shortcoming that might be solved by the designers of the infrastructure system
- I agree that this is only relevant in a rather small range of applications. But this does not deny that it is a shortcoming nonetheless, as solving it would not impose any disadvantages for anyone.

-Michael

And this is a funny way to solve performance problems. Have you just experimented with multi-threaded GUIs (which one did you use as none supports this)? Or is your way of choosing multiple apps to do the work your proof that multi is faster? What is the real bottleneck of your application?
How did you identify this bottleneck?
How did you arrive at the conclusion that multi would solve your problem?

- I agree that this is only relevant in a rather small range of applications.

So a gain for a small amount of apps but a lot of work to support the solution. Currently there are other options but you do not want to use them. Your choice.

But this does not deny that it is a shortcoming nonetheless, as solving it would not impose any disadvantages for anyone.

Multi-threaded is always more work than single (more complex, different errors that appear only in multi-threaded apps, harder to debug). It is a disadvatage for at least the widgetset developers (more work, more complex code). The Lazarus developers (same reason). Maybe even for the users of the widgetset. How about you take fpGUI (or GTK or QT if your skills allow that) and make it thread-safe? The code is there for you. You sound like for example removing the check in QT if a widget is updated from the main thread is enough to make it thread safe, it most likely is not. Who does the work to make it thread safe?
If it is so beneficial to your work and so easy to achieve without disadvantages, why don't you do the work?

R.

Reimar Grabowski wrote

So a gain for a small amount of apps but a lot of work to support the solution.

Correct, but this is not the definition of not being a shortcoming.

Currently there are other options but you do not want to use them. Your choice.

Can't you read ? just a few sentences ago I explained how we got the system decently running, avoiding the sa(i)d problem - just with considerable additional effort.

-Michael

It's based on the fact that there is only one window/desktop manager, which handles screen updates in strictly sequential fashion.

When an application relies on high speed updates, it should serialize these requests itself, as required, instead of leaving this to the desktop manager and its arbitrary request scheduling.

DoDi

Hans-Peter Diettrich wrote

When an application relies on high speed updates, it should serialize these requests itself, as required, instead of leaving this to the desktop manager and its arbitrary request scheduling.

Once again, this is not about serializing "requests" but about allowing as much code as possible to be distributed on multiple CPUs as possible.

The "Widget Provider" (including X or whatever) does a fine job serializing the requests to the graphics driver, as you can see when doing the same stuff in multiple applications instead of multiple threads.

( On top of that you are right that serializing the GUI inputs (i.e. the event queue handling) could benefit from multiple threads as well, as with that e.g. events from a certain Windows could be handled in a higher priority queue than others, but this is a different issue ).

-Michael

Can't you read ?

Nope. The batteries of my cyberpig which does the reading for me are running low. Can you send a voice recording of your last mail?
Or a video of you dancing it (perhaps multi-threaded)?

Have a nice day.

R.

Note that the numbers I cited are already for a framework that is geared toward this application. In fact, displaying the image eats the bulk of the cpu time of the machine.

Even if the backend improves, all things must line up to make it possible to exploit that, which is hard with a general framework. Specially since the backends also have limitations wrt threads (like e.g. Opengl).

To present a general threadsafe model to the user, that would require general locking solution in the LCL, which would slow down normal use for fringe scenarios (if possible at all)

Yes, agreed. And I think that your observation about attainable speed (i.e. the limits of X and OpenGL) is pretty much the last word in the debate- at least as far as current PCs and OSes go.

To present a general threadsafe model to the user, that would require general locking solution in the LCL, which would slow down normal use for fringe scenarios (if possible at all)

I think that the only thing remotely worth pursuing, bearing in mind the current state of widget sets etc., would be whether an LCL in a dll/so animated by its own thread (which MichaelS has already said was doable) could output to a buffer, which could then be copied in a thread-safe fashion to the main form. I speculate that even if that was doable, getting UI events into the background LCL would be difficult, which in practice would severely curtail the components which could be used.

Mark Morgan Lloyd wrote

I think that the only thing remotely worth pursuing, bearing in mind the current state of widget sets etc....

I don't suppose this makes much sense unless the infrastructure called by the LCL supports it in a proven way.

OTOH:
In the other Thread we discussed that an LCL Widget Type allowing to do a remote Browser based GUI (using some Java script Toolset) would be a "good thing".

Here of course it would be advantageous to allow for two GUIs at the same time (local and remote) using threads.

But besides another instance of the "Application" object (and thus same being thread-safe) - including a second event queue - this would need a second set of resources to define the two GUIs, plus appropriate GUI editing means.

Does not sound easy at all. ;-)

-Michael

OK, so assuming an OS similar to X, can a single app support two forms on different displays, and/or move forms between displays? Because if it could, one of those displays could be a local X session and the other could be something like a VNC server.

That's exactly how it should be done. Threads fill their video buffer, then send a message to the GUI thread, to copy the frame into the output buffer and request an screen refresh. That's a total of 3 messages per stream and frame.

Eventually the video threads can write directly into the output buffer, eliminating one copy action. Then the GUI thread can switch between multiple buffers, one to be filled and the other one ready for display.

DoDi

OK, so assuming an OS similar to X, can a single app support two forms on different displays, and/or move forms between displays?

That's the job of the desktop/window manager, on every OS. When a form shall be updated on screen, the manager determines the monitor from the form's screen coordinates, then asks the form to paint itself. Moving forms means nothing but changing their screen coordinates, then request an screen update.

DoDi

The desktop manager does not care about threads, it only manages windows as areas on a monitor.

The "Widget Provider" (including X or whatever) does a fine job serializing the requests to the graphics driver, as you can see when doing the same stuff in multiple applications instead of multiple threads.

Right, but the *timing* of the screen updates is undefined. On Windows a screen update (WM_PAINT) is performed at lowest priority, when the application (GUI thread) has no other outstanding messages. Then it depends on when the application has time to fetch and serve the message. This should not be very different in other managers (X11...).

The worst thing an application can do, WRT screen update timing, is having the GUI thread busy with other things, like copying large data structures (frame bitmaps...).

DoDi

My understanding is that moving forms *between* *displays* requires rather more than that, in fact I'm not sure it's doable on X (if it were then we wouldn't need Xinerama, Xdmx etc.).

I don't know too many details (from just reading LWN...) but it seems very similar to the way Wayland is said/described to be working... (At least on Linux) Everything it handles, is only pixbuffers, clients/applications fill them in. Or so I read. YUMV. Not sure about thread awareness/safety though.

But with GTK or QT running on top of it, these libraries become the actual 'buffer fillers' ... and yet we have LCL running on top of that still.

-L.

Try to move a form with the mouse to an different monitor. This works on Linux without additional libraries. Linux also can show the same form on all desktops, without additional code or libraries.

If you mean "desktops" for "displays", or different X sessions, that's a bit different. But even then it's only a process property, on which desktop or X session a form is shown.

DoDi

Oh no it doesn't. You need something like Xinerama (RandR, Xdmx) installed, or hardware/drivers that support multiheaded operation. Otherwise the two monitors are separate *screens* (specific, technocal term- go look at your xorg.conf) identified as :0, :1 and so on.

If you mean "desktops" for "displays", or different X sessions, that's a bit different. But even then it's only a process property, on which desktop or X session a form is shown.

You're jumbling terms up badly there. "Desktop" has no meaning in classic X organisation, these days it's roughly synonymous with "window manager" where a window manager operates on a single screen (i.e. identified as display :0, display :1 and so on). Where a desktop has some sort of pager then forms can be moved around trivially: my understanding is that on unix this is handled by minimisation/hiding (on Windows there is the windowstation layer, which is somewhat different).

So I stick to my position that it's at best unclear whether forms can be moved around between displays (I think you'd like to call those sessions, in any event corresponding to a screen) and whether an app can distribute forms over multiple displays (e.g. one form on display :0, another on :1 and so on).

For completeness, displays are usually identified as :0 etc. but there's actually another layer hence :0.0 :0.1 etc. I've not come across this being used even on e.g. Xinerama, and I don't know what it's supposed to represent.

In fact allowing for a remote GUI by having the application be a VNC server (or attach to a remote X server or be a "NoMachine reverse X" server) is a nice additional idea, leading to similar threading issues.

-Michael

Hans-Peter Diettrich wrote

fill their video buffer,

This only sounds appropriate for pixel graphics, not for the complex stuff "widgets" create and that result in "object descriptions" that are rendered by the processor in the video adapter.

-Michael

Okay with multiheaded graphics cards, dunno how to work with multiple cards.

Otherwise the two monitors are separate *screens* (specific, technocal term- go look at your xorg.conf) identified as :0, :1 and so on.

I cannot test that right now, having Linux only in virtual machines. But from my Drag&Drop and IDE experiments I know that nothing special is required to spread the IDE windows across multiple monitors, once the monitors are arranged to form an desktop.

AFAIR the term "desktop" became popular with the multiple-desktop feature of X11, never implemented by Windows. Multi-monitor desktops are different, in Delphi/Lazarus they are described by the Screen object.

You can test that yourself, when you have a system with multiple monitors. Write a test form that displays its screen coordinates, then move it around. AFAIR moving a form to an different desktop can be achieved by the window context menu.

For completeness, displays are usually identified as :0 etc. but there's actually another layer hence :0.0 :0.1 etc. I've not come across this being used even on e.g. Xinerama, and I don't know what it's supposed to represent.

The :0.0 notation may denote a desktop and monitor number. Deciphering such an ID requires access to the "native" window implementation, which is hidden from LCL users. Of course the system and window manager has to know about all that, see the Monitors object in Delphi/Lazarus and related methods/functions.

I assume that low-level access to distinct monitors requires such addressing, but ordinary applications don't have to bother with the availability, arrangement and resolution of multiple monitors on their desktop. The widgetset experts may contribute more precise information.

DoDi

What kind of "objects" should that be?

Window managers do not care about widgets, these reside in a widgetset layer between the application and window manager. A window manager only splits up the available desktop extent, into application specific windows, and identifies the application that has to receive events (mouse, touchpad...) and paint the window interior (client area).

DoDi

Hans-Peter Diettrich wrote

Window managers do not care about widgets,,,

So maybe I am confused about the term "Window Manager" (I think I did not introduce it here)

-Michael

MS Windows does not distinguish between the window manager and widgetset, it provides both. The Windows-provided widgets again are windows (have a HWND...), so that the window manager must handle *nested* windows. Also both send messages to a window's message queue (WM_...), so that an ordinary Windows user doesn't notice a difference between both.

X11 instead does not provide widgets itself, these must be added by dedicated libraries (widgetsets). The communication between an application and the window manager and an widgetset works differently, events from both sources must be merged into the LCL message queue, for VCL/Windows compatibility.

DoDi

and in my experience the thing that's most likely to work is DVD playback because the data stream is (as I understand it) sent via a backdoor to the graphics chips bypassing most of the kernel and X.

On Linux and BSD at least it's the standard to bypass X when talking to the GPU. You can talk to the GPU from different threads (with a little overhead) but you actually gain nothing, because of how the drivers work. There may be some way to get better performance if you are talking to multiple GPUs and you can do a better job than the vendors driver to utilize them (highly unlikely). There are reasons to talk to the GPU from different threads but that's only usefull for loading resources while not interupting the main rendering thread, there is no speed gain.

R.

My last message was just a failure in using my mail client.

Sorry for the noise.

R:

Reimar Grabowski wrote

but you actually gain nothing, because of how the drivers work.

Are we (still) talking multi-core systems gaining performance by really parallel work ?

You will gain nothing in the work of the GUI framework called by the Lazarus program *if* the Framework software decides to use a semaphore to block multiple thread concurrent working directly when entering any framework function. We can't (easily) know this, but in fact I *suppose* the widget set framework libraries will only block when entering the driver.

But you will gain performance during the LCL's widget code if same would support concurrency (which it currently does not).

-Michael

Are we (still) talking multi-core systems gaining performance by really parallel work ?

No, we aren't talking anything (see second mail).
If you did not get it there may be a problem with your mail client(1), if you got it (must have been about 2,5h before you replied) there may be a problem with your reading or comprehension skills(2). Choose 1 or 2.
I am not thread-necrophile and this one is as dead as it gets.

R.