Hi FPC experts.

I read this thread in the list archive and as I'm very interested in this topic (since years), I'm very happy do see this evolving and would like to provide some comment (and maybe contribute some code once I'm up to speed with FPC) and, of course, ask questions.

1) How is the state on this issue? Does the latest released version of the FPC RTL already provide a decent implementation of the threading primitives known from Delphi ? (e.g. TThread.Suspend, TThread.Synchronize, TEvent, TCriticalSection, TThreadList, etc) as well for Linux as for Windows etc.? What about WinCE on ARM and Linux on ARM and 68K ?

2) IMHO FPC is a lot about platform independency. In Delphi/Kylix there never was a decent platform independent way of inter thread message passing. Are there any plans on this ? IMHO a perfect way would need a language enhancement (a way to fire events that are to run in another thread context, which would also involve enhancing/creating the message scheduler in the RTL to work not only for the main thread but for any worker thread, too), but supposedly a thing easier to agree about could be something like a "TThreadMessage" class in the RTL that fires an event in the main thread when a message from a worker thread comes in and for worker threads provides a means to check if a message is available and optionally wait for one.

3) Does TTimer work in Linux, now, ? Last time I checked I did not get it running.

-Michael

It does.

2) IMHO FPC is a lot about platform independency. In Delphi/Kylix there never was a decent platform independent way of inter thread message passing. Are there any plans on this ?

None currently. Only 'Synchronize' is implemented.

Feel free to implement this, but not as a language enhancement, a simple unit should be enough.

3) Does TTimer work in Linux, now, ? Last time I checked I did not get it running.

It works.

Michael.

Really ? Does TThread.Suspend work on linux ?

IMHO, the need for this function means your design is broken, but maybe it's just me ...

Micha

It should, AFAIK ?

Michael.

It does.

Great !

None currently. Only 'Synchronize' is implemented.

Like with Delphi :-) .... :-( .

Feel free to implement this, but not as a language enhancement, a simple unit should be enough.

..., at least for the beginning !
I'm sure I'll do this (supposedly in January/February), unless somebody is faster than that

How is TTimer (and TThread.synchronize) done ? I supposed this needs a TApplication object to do the event scheduling. Is this provided in the RTL, even though TApplication is not hooked to the GUI (as Lazarus would do) ?

-Michael

Really ? Does TThread.Suspend work on linux ? IMHO, the need for this function means your design is broken, but maybe it's just me ...

As Delphi does provide TThread.Suspend, I suppose FPC should do so on any platform (unless that is really impossible, which I don't think).

Of course you are right that using messages and semaphores is a _much_ more "professional", so I would avoid it. But anyway...

-Michael

Why ? You don't need language enhancements for this IMHO. Scheduling and messaging is OS territory, not programming language territory.

I'm sure I'll do this (supposedly in January/February), unless somebody is faster than that

I seriously doubt someone will be faster, but one never knows. But I'm looking forward to your contribution in this area :-)

TTimer is only available in Lazarus, not in FPC. It's hooked in the event loop, I suppose.

Michael.

e.g. Ada provides multithreading means as language elements and AFAIK, other languages also provide "joins" for that.

I do agree that right now this should be done without any language enhancement, but in a platform independent way.

Having this we will face two shortcomings:

1) following the Delphi ways we have a message scheduler for the main thread (GUI thread in Delphi) but not for the worker threads created by TThread. This "asymmetrical" approach makes using threads a bit clumsy. Once we provide a standard means to send messages to a thread we will want to allow a worker thread react on a message like the main thread: by firing an event and by using "Application.ProcessMessages" (so here we would have a TThread.Application property). The TThread.Execute thus would (optionally) be enabled to finish without ending the thread. This enhancement can be done without enhancing the ) With that solution of (1) we would have a new thread model that now follows the event driven programming paradigm like the main thread does. With that we would want to integrate the thread programming more tightly and more easy to use in the language. Here I would like to see a means to create "thread events". With that I mean a way to create "callback" properties for classes that not just provide a function call but can be defined (either statically with the class implementation or dynamically when fired) to run on a certain thread context. This would of course mean sending a message with the parameters to the thread, queuing it, and have the thread execute it once it's free to do so. The "calling" thread should optionally be made waiting for the called thread to do it's thing (like with "Synchronize", but more versatile as parameters can be passed"). In that case a function result and "out" parameters can be used.

This is just an idea I had some time ago and not yet a fully grown concept.

I already tried something like that several years ago, (see below).

TTimer is only available in Lazarus, not in FPC. It's hooked in the event loop, I suppose.

Ah ! At that time I did get it running in Lazarus and was struck, as it did compile but not work without.

That's really bad ! IMHO it should be possible to use TTimer and TThread in a simple non-GUI way. (I maybe will need to do an embedded project on an ARM/Linux system that does not have X or any widget library.) What is the problem with that ? can we work together on making this happen (e.g. extracting a part of the "Application" code from Lazarus, but doing it's own event loop instead of using the OS for that.) I already tried (and contributed) some thing like that several years ago (creating my own Application object and trying to provide most of the Delphi thread primitives for Linux), but at that time all this was bound to fail, as the memory management was not thread save :-( . (Lazarus was not started at that time AFAIK.)

-Michael

Michael Schnell wrote

That's really bad ! IMHO it should be possible to use TTimer and TThread in a simple non-GUI way. (I maybe will need to do an embedded project on an ARM/Linux system that does not have X or any widget library.)

I would also love to see a TTimer in FPC rather than Lazarus. TTimer without any GUI depenencies.

Regards,
- Graeme -

Graeme Geldenhuys wrote

I would also love to see a TTimer in FPC rather than Lazarus. TTimer without any GUI depenencies.

If the Eventer class in lNet can be moved into general FCL this would be possible because I was planning to create a TLTimer, for use with lNet eventer.

Micha

If the Eventer class in lNet can be moved into general FCL this would be possible because I was planning to create a TLTimer, for use with lNet eventer.

Could you elaborate on that (I don't know what lNet is at the moment).

But IMHO there should not be a separate TLTimer but the RTL (that already does contain TTimer, IMHO) should somehow be enhanced to make TTimer work. I suppose the use of TThread.Synchronize also need the event loop and thus making same work needs the same means as TTimer does.

-Michael

lNet is a networking library, and has an internal event loop already implemented, cross-platform.

But IMHO there should not be a separate TLTimer but the RTL (that already does contain TTimer, IMHO) should somehow be enhanced to make TTimer work. I suppose the use of TThread.Synchronize also need the event loop and thus making same work needs the same means as TTimer does.

If lNet's eventer is moved into the RTL/FCL, then 'fixing' TTimer may be possible. As my timer was meant for lNet, the name was going to be TLTimer. Can't find at the moment where TTimer is declared and implemented.

Micha

lNet is a networking library, and has an internal event loop already implemented, cross-platform.

Wouldn't it b e better to first implement an independent cross-platform event loop ("Application!" thingy) in the RTL that can be used by TTimer, TThread.Synchronize _and_ lNet ?!?!?

If lNet's eventer is moved into the RTL/FCL, then 'fixing' TTimer may be possible.

As said the eventer (sh)could be separated from lNet in that moment to create a behavior like Lazarus provides. Supposedly care must be taken not top break Lazarus when doing this.

Can't find at the moment where TTimer is declared and implemented.

I seem to remember that I once found it after a long search, but I don't remember right now.

Please keep us posted.

BTW.: will/does lNet provide access to sockets etc in a platform independent way ?

-Michael

I'm not sure I see the value behind these ideas.

If you need an event-based loop and callbacks you can make TApplication. If you don't want LCL ok, then make your own, it's generaly needed to be very problem-specific anyhow, eg: the TLEventers I use are ment for handling events on handles (files and sockets atm, sockets only in windows).

If you want a "Timer" a "polling" mechanism has to take place which looks if something happened on some timer for example, OnTimer is "ready to be fired" (you can't call callbacks in threads, and if you go single-threaded you have to provide the main loop hooks).

So basicly, let's say TTimer is in RTL and there's some event mechanism already too, you'd have to do something like this:

procedure init;
begin
  Timer.Timeout:=somenumber;
  ;
  EventHandler.RegisterTimer(Timer); // could be the other way around
end;
procedure MainLoop;
begin
  EventHandler.HandleEvents; // if eventhandler after polling all timers in
it's watchlist find out some has it's interval done, it fires it's callback
end;
begin
  Init;
  MainLoop;
end.

Apart from asynchronous stuff like watching handles etc. I don't see much use for this relativly complicated program model. Remember you need to call the "HandleEvents" as fast as possible to stay effective (otherwise you lose precision)

Ales Katona wrote

Apart from asynchronous stuff like watching handles etc. I don't see much use for this relativly complicated program model. Remember you need to call the "HandleEvents" as fast as possible to stay effective (otherwise you lose precision)

This is how lNet already works, no ? Where's the difference ?

Micha

Ales Katona wrote

If you want a "Timer" a "polling" mechanism has to take place which looks if something happened on some timer for example,

Is that so?

OnTimer is "ready to be fired" (you can't call callbacks in threads,

I can't call callbacks in threads? Why's that? Or am I understanding something wrong?

and if you go single-threaded you have to provide the main loop hooks).

Do I? I always thought a timer is more comparable to an interrupt or signal than a polling loop.

Timer.OnTimer  := @Some_Callback;
Timer.Timeout  := Some_Number;
Timer.Periodic := True;

And somewhere a "Timer.Start;", that should be it. There's no need to register the timer in a global event loop or even explicitely calling it. If that's needed on a particalur target, the timer's implementation should take care of this.

Apart from asynchronous stuff like watching handles etc. I don't see much use for this relativly complicated program model.

Don't know what's complicated here. It's asynchronous by definition, of course, but apart from that?

Well, for me it would have saved me some time which I wasted in creating threads that implement "timing events" by simply sleeping for a particular time. And there's really nothing complicated in queuing log messages and saving the queued messages every five seconds.

Vinzent.

TApplication supposedly is platform specific, as e.g. Delphi or LCL use OS mechanism for GUI events and there is no platform independent message implementation in FPC. I don't see why it should be "problem-specific". A mechanism that waits for a message and creates an event from same should be quite universal.

I don't think so. To do a TTimer implementation that does not use GUI events, I would e.g. use a thread that sleeps periodically wakes up and sends a message to the main thread. Here the TApplication event scheduler would fire an Event in the main thread when appropriate (at once if idle, when coming back from the previous event or when "ProcessMessages is called).

Using an OS-Timer would save a thread for each timer. Here The Event Scheduler would need to be aware not only of messages but of timers as well. Maybe this is a better way to do it.

It's necessary to be compatible with Delphi paradigms. Here I will need to port "embedded" programs done in Delphi by my colleagues to GUI-less ARM-Linux. Of course this is just _my_ motivation, but IMHO prototyping embedded applications in Delphi (or Lazarus) and porting them to run on a smaller/cheaper platform than a Windows PC is a great way to do things like communication servers.

-Michael

Remember you need to call the "HandleEvents" as fast as possible to stay effective (otherwise you lose precision)

I want to avoid this by any means !

That is why I want to have a message driven system (as described in the other post):

TAppllication creates an event from a message generated by another thread (or maybe from an OS timer if it can do this). In Delphi/Windows all GUI events are messages and so are Timer and TThread.synchronize events. In Linux we could use a System 5 message queue for this (I once did a working draft of that). A Timer might need a thread to create a message (Threads are cheap in Linux). Signals might be an alternative, but here we would need to implement a queue on top of them.

-Michael

I can't call callbacks in threads? Why's that? Or am I understanding something wrong?

Of course you can use "events" (aka class properties that are functions) in threads. That is just the Object Pascal syntax. But in the main thread in Delphi or Lazarus events of GUI objects (e.g. TButtons) are called "automatically" by user actions (thus in effect by hardware events, which cause messages in Windows). This is not possible with threads in Delphi, as system messages are sent to the main (GUI) thread of a project. A thread can _programmatically_ wait for a message, but the Delphi (and Lazarus) event scheduler that calls events of GUI based objects only works for the main thread.

In Delphi this is done by system messages, so no permanent polling (spinning) but using a wait for message Windows API. I don't know how this is done in Kylix or Lazarus/Linux.

Well, for me it would have saved me some time which I wasted in creating threads that implement "timing events" by simply sleeping for a particular time.

IMHO this should be done following the Delphi paradigms to be nice to Delphi users coming over to FCP (only enhancement allowed). Of course Lazarus does this, but IMHO it would be great to have this in a system without a GUI, too.

-Michael

I always thought a timer is more comparable to an interrupt or signal than a polling loop.

That is a very common misconception. A TTimer event is not an interrupt in that it does not interrupt any code in the Application. such an "automatic event" is always done in the main thread and only started when (1) the application is idle, (2) the application comes back from some other event, or (3) Application.ProcessMessages is called. Only another thread can be interrupted ("preempted") by such an event.

The big advantage is that you do not need to protect any resources from multi-threaded access if you do not explicitly create a thread by means of TThread.

-Michael

At first: It doesn't matter how it's done to me as long as it works. :)

My system is a system without a GUI, or to be more precise: with a remote (G)UI. That's why I don't know about LCL or whatever and I don't even care, because I simply don't understand what "time" as in "timer" has to do with "graphics" as in "graphical". For me that's two completely different subjects unless we're talking about animations where those two subjects meet.

All I needed was a simple timer, so that some user supplied routine was being called after some specified time has been elapsed. There's nothing complicated in that view of the programming model, because it's the user's view. I implemented that by using an additional thread in a relatively simple way, although its execute method was merely a

|while not self.Terminated do
|begin
|   SysUtils.Sleep (self.Interval);
|   self.Logger.Flush;
|end {while};

to take care of my requirement that the Flush-routine of the logger object would be called once in a while.

If "following the Delphi paradigm" leads to such complicated solutions as layed out here by Ales (and I'm still talking about the user's view here, not any particular implementation) I surely can live without such a solution.

Vinzent.

This is how lNet already works, no ? Where's the difference ?

Indeed but it's only there because there's no other way, if I could just make it "magicly" work without "CallAction" I'd love to.

You can't call callbacks in threads (you can but only in one thread, not between threads), and you can't magicly "jump into execution" if something happens, with the exception of OS signals (which use a hack in hardware afaik).

So eg: if you want to do something every 1000ms, you could put a TThread based timer in, and make it Sleep(1000) and then call the code, but you must be sure that your main thread cannot have conflicts if this code is called at any time (eg: variable writes etc.).

My point was that making a non-blocking or blocking event based main-loop-hooked mechanism for polling events is IMHO overkill in most cases (there are obvious exceptions but they need obvious specifics anyhow). It's easier for the user to simply check some times and see if the interval is hit and then call the function in his main loop mostly.

IMHO ofcourse.

Ales

Michael Schnell wrote

> I always thought a timer is more comparable to an interrupt or > signal than a polling loop. That is a very common misconception.

Indeed. Even a paper in "Embedded Computing Design" about Ada 2005 follows that "very common" misconception:

-- quote -- 8< --
Timing events allow for a handler to be executed at a future point in time in an efficient way, as it is a standalone timer that is executed directly in the context of the interrupt handler (it does not need a server task).
The use of timing events may reduce the number of tasks in a program and the overhead of context switching. It provides an effective solution for programming short time-triggered procedures and implementing some specific scheduling algorithms, such as those used for imprecise computation. Imprecise computation increases the utilization and effectiveness of real-time applications by means of structuring tasks into two phases (one mandatory and one optional). Scheduling algorithms that try to maximize the likelihood that optional parts are completed typically require changing asynchronously the priority of a task, which can be implemented elegantly and efficiently with timing events. -- /quote -- 8< --

But of course, the Delphi "concept" is right.

Those are details of _one_ particular implementation.

The big advantage is that you do not need to protect any resources from multi-threaded access if you do not explicitly create a thread by means of TThread.

And the big disadvantage seems to be a overly complicated programming model. Guess, what I'd prefer.

Vinzent.

Of course, because the common concept of a timer is as asynchronous as in "multi-threaded" or even "interrupt".

You're not seriously trying to work around that simple fact, do you?

Vinzent.

MSEgui has teventthread (lib/common/kernel/msethread.pas), there is also a timer implementation (lib/common/kernel/msetimer.pas).

http://mypage.bluewin.ch/msegui/

Martin

Actualy I am, because lNet is single-threaded non-blocking.

As for general use, you can't do a Timer this way. You can't just put a TTimer in which works in it's own thread and then calls some callback in it's own thread, unless the users are informed, and-or the thing is locked and protected but you can't do that from within a TThread (because you don't know what the callback code is).

Basicly a solution might be a thread which CriticalSections the callback part, but that still doesn't protect from wrong stuff: eg the user is in a loop and the code inside the OnTimer callback modifies something which breaks or loopies the loop.

In case of gui TTimer this cannot happen because it's not threaded, so the userloop would first finish, then the user function returns and the main gui loop does it's stuff (this is my oh-so-complicated part, done by the gui).

Of course, because the common concept of a timer is as asynchronous as in "multi-threaded" or even "interrupt".

That is not true ! (See my other post.)

Timer events are queued in a line (message-queue) with all "hardware"/"GUI" events. Main Thread event callbacks are never interrupted (preempted) by other Main Thread event callbacks unless Application.ProcessMessages is called. And this _is_ important and makes using Object pascal a lot easier !

-Michael

Viewing from top you are right, and I do need a platform independent implementation along theses lines.

But (at least in Delphi) TTimer and GUI events are implemented using system messages and a Windows-internal message scheduling API. FPC does not know about messages, here Lazarus gets involved, as the LCL supplies the TApplication class that is able to fire events when a message arrives.

So what I suggest is implementing a platform independent TApplication class that can handle messages and implement TTimer and TThread.synchronize and other Delphi stuff on top of this, without using GUI related system APIs, but using a System 5 Message queue in Linux.

-Michael

As said, we need to implement TTimer to work as it does in Delphi (and Lazarus), as otherwise we would brake the code of everybody who ports his application.

And using this way (something like that is sometimes called "cooperative multitasking") is by far less complicated that it would be with real threads (aka "preemptive multitasking"), as no protection of the objects against multithreaded access needs to be used (hence e.g. you can use a TList in a TTimer event while you need to use a TThreadList in a TThread, if the list can be accessed by the main thread, too).

No Delphi program that uses TTimer would work reliably, if TTimer would be implemented similar to a thread. As most application developers don't ever use threads, they even are not aware that they would need to take lots of precautions if a TTimer would be able to interrupt the main line code at any place.

-Michael

Thanks for the pointer. I'll take a look at this tomorrow.

What is the purpose of msetimer ?

Do you think this can show a way to do a GUI-less, portable, Delphi work alike implementation of TTimer ?

Thanks again,
-Michael

There is TCustomApplication in fcl/inc/custapp. The Lazarus TApplication descends from it. All you need to do is create a descendent of TCustomApplication which can implement all the messaging you wish for...

Michael.

It is the timer facility of the MSEgui framework.

Do you think this can show a way to do a GUI-less, portable, Delphi work alike implementation of TTimer ?

Yes, with a simplified tapplication and some changes in the tick generation. Essential are iobjectlink and the tevent class
(lib/common/kernel/mseevents.pas) and tobjectlinker (lib/common/kernel/mseclasses.pas).

Martin

Michael Schnell wrote

> Of course, because the common concept of a timer is as asynchronous > as in "multi-threaded" or even "interrupt". That is not true ! (See my other post.)

Well, a timer is, even though the Delphi implementation of a so-called TTimer object might not.

Basically a timer starts, later then it expires an that expiration creates an event (in the old days we called that interrupt). This event is no way synchronous to the code currently running.

Timer events are queued in a line (message-queue) with all "hardware"/"GUI" events.

Ok. So this would do nothing more than synchronize the asynchronous event of an expired timer to the main thread. Of course, the "time" of execution of the code associated with the expiration is now determined by the time the message queue is handled the next time.

So why don't use CheckSynchronized for this?

Vinzent.

Ales Katona wrote

As for general use, you can't do a Timer this way.

Believe me, I can. :)

You can't just put a TTimer in which works in it's own thread and then calls some callback in it's own thread,

I even call the callback of another thread. :P

unless the users are informed, and-or the thing is locked and protected but you can't do that from within a TThread (because you don't know what the callback code is).

Well, in Turbo Pascal I could write interrupt handlers the wrong way, without locking code/data in question and nobody cared about that simple fact.

But if that's so complex: Why don't you use the usual synchronization stuff then to put the asynchronous timer event in a message queue using Synchronized?

In case of gui TTimer this cannot happen because it's not threaded, so the userloop would first finish, then the user function returns and the main gui loop does it's stuff (this is my oh-so-complicated part, done by the gui).

The problem in a portable general solution is that the may be no "main gui loop" and you can't just write one and force anyone to use it, right?

So why don't you use Synchronized/CheckSynchronize (or whatever it's called) then?

If you'd do that, the timer event callbacks would be queued and then executed in the context of the main thread.
The user is only required to call CheckSynchronized from time to time, but because the accuracy of the time of execution of any associated handler is in the hands of the user anyway as long as you stick to a synchronous solution to avoid putting the burden of locking data to the programmer.
As long as you don't want to implement real-time behaviour, I don't even see a problem in that. If the user decides to not call/execute any provided "message event loop", the execution of the timer handler code will be deferred until the end of the universe anyway.

Vinzent.

As for general use, you can't do a Timer this way.

Right ! The beauty of TTimer (and GUI events as well) is that it does not work in a thread. Otherwise it e.g. would not be possible to do GUI stuff in a timer event, as VCL and LCL are not thread save (and probably can't be made thread save).

But that does not mean that you can't implement a decent (firing main-thread-events) TTimer in a non-GUI way, not using the GUI API of the system. e.g. You can use System 5 message queues to do the scheduling and use a thread that sends messages to the main thread as a trigger.

-Michael

It can't because X-Windows or MS-Windows are not thread-safe.

Michael.

Michael Schnell wrote

Right ! The beauty of TTimer (and GUI events as well) is that it does not work in a thread.

Well, I don't consider it beautiful, I consider it a hack. :)

Vinzent.

No you can't.

This is exactly the "oh-so-complex" solution I use in lNet, and is basicly the only solution.

Yeah, right. And threading in DOS is not possible either, because DOS is non-reentrant, is it?

Vinzent.

A TTimer by definition is what a Delphi TTimer is (and of course the Lazarus implementation is done according these lines).

Of course you can do a TVinzentTimer that does a callback in a thread. In Linux this even is a very useful thing in some projects, as (other than in Windows) creating threads is very cheap here. The user needs to be aware of what he is doing and that hes required to protect resources from mutual access and that it's not possible to uses LCL functions (many of which are not thread save) in a TVinzentTimer event.

Basically a timer starts, later then it expires an that expiration creates an event (in the old days we called that interrupt). This event is no way synchronous to the code currently running.

As said, his is useful in some projects, but by far more difficult to use than the Delphi-like TTimer.

Exactly

Of course, the "time" of execution of the code associated with the expiration is now determined by the time the message queue is handled the next time.

You are absolutely right. (Delphi like) TTimer can't be used for "realtime" stuff. Here you need threads and taking the pain to care about mutual access problems is required and sensible.

So why don't use CheckSynchronized for this?

What is CheckSynchronized ? Can this help queuing multiple (e.g. Timer) events so that they get executed in order of their occurrence in a non preempted way ?

To enable TTimer to work in the Delphi/Lazarus-like way, of course the main program needs to call an RTL function that never returns and _all_ actions need to be events that are queued and handled by callbacks in the sequence of their occurrence. Application.ProcessMessages needs to be in place to allow a less important event granting precedence to more important ones.

-Michael

There is TCustomApplication in fcl/inc/custapp. The Lazarus TApplication descends from it. All you need to do is create a descendent of TCustomApplication which can implement all the messaging you wish for...

That sounds very good. :-) I'll take a look at this. (Supposedly in January :-( . )
Of course I'll publish all I can get working.

Many thanks,
-Michael

I don't know. I didn't make any statements about DOS, AFAIK.

Michael.

You still refuse to tell me why, so please stop telling me what I can't do, especially when I *am* doing it. Of course the procedures in question are protected against concurrent execution from different threads at the same time. This was needed anyway, because the object was designed to be called from different threads anyway. So adding a "timed" call from another thread wasn't too hard, as you probably can imagine.

Of course I am not using this GUI based TTimer object, I was talking about the concept "timer" in a more general way. Call it signal, if you like that better.

Let me quote yourself: "Apart from asynchronous stuff like watching handles etc. I don't see much use for this relativly complicated program model."

So you yourself seem to look at it as "complex" somehow.

and is basicly the only solution.

Well, using a (priority) message queue is a common solution, yes. Yet it's not necessarily the only one. :P

It all depends on what you're trying to accomplish. I still like to have a "asynchronous timer event" model. It would still simplify some code.

Vinzent.

No you didn't. But I heard the same before and yet there *is* a thread-safe implementation on top of GTK (which isn't thread-safe, either) and there *are* threading implementations under DOS.

So I'd be careful with such "it can't"s. ;)

Vinzent.

Of course you can call any callback that exists in the application from anywhere else in the application independently from in which thread context you actually are running. But doing a function call (or hence using a callback) does not change the thread context you are running in and thus you are still the same thread. IOW: code lines are not _dedicated_ to a thread context.

-Michael

It all depends on what you're trying to accomplish. I still like to have a "asynchronous timer event" model. It would still simplify some code.

You are right that this would be desirable.

Normal Delphi/Lazarus like TTimers are non-preemptive (you may call it synchronous) for a good reason (as already discussed at some length).

Preemptive timers ("asynchronous" timers, using threads) can be very useful too, and Delphi/Lazarus don't offer any portable means to do them. So there is a good reason for enhancement.

If you take a look the start of this thread, I mentioned that after implementing Delphi/Lazarus like behavior for the main thread without the need for a GUI, I'd like to enhance the concept by optionally having an TApplication object for any thread (e.g. TThread.Application) and thus having an event scheduler for same. By this it would be possible to implement TTimer in a way that it's callbacks can be fired in the context of a thread and thus "asynchronous timer event" are possible in a completely compatible way, allowing the user to define, regarding what context the event is asynchronous. IMHO this would be a great plus for any embedded "real-time" use of FPC.

-Michael

Michael Schnell wrote

But doing a function call (or hence using a callback) does not change the thread context you are running in and thus you are still the same thread. IOW: code lines are not _dedicated_ to a thread context.

Oh that, of course. Seems I got confused by all the talking that my code can't work at all, and started to think in TThread objects instead of execution context. Well, right. I wasn't going to write a task scheduler. ;)

But now ... rereading the post, the original text was "You can't just put a TTimer in which works in it's own thread and then calls some callback in it's own thread," and I still wonder why I shouldn't be able to do that (well, if TTimer would actually do what its name suggests and no-one else would change the subject to one particular Delphi-way-implementation called TTimer, when I am still talking about a timer object in general).

Vinzent.

You _can_ use threads in an X-Windows, MS-Windows or DOS environment, you just can't do X-Windows, MS-Windows or DOS API calls from a thread.

-Michael

You can call a callback in same thread, but since you can't ensure what the callback does from your "lib" you can't make it threadsafe in any way. Even if you put the callback itself into a "criticalsection" it might eg: change some variable which was "just in use" by the main thread, and once the callback finished the main thread booms.

This is an old problem, and not fixable by wishing it. If you want a good async. timer, sure make one, but don't expect it to work safely by magic. Make sure to inform users that certain things are simply not a good idea to be done from their callbacks, or tell them to ensure thread-safety in main thread too.

That's what I ment, it won't simply "just work without implications" like TThread SEEMS to.

Ales

Michael Van Canneyt wrote

It can't because X-Windows or MS-Windows are not thread-safe.

FTR: AFAIK MS WinAPI is thread safe, just don't do stupid things yourself of course (like passing buffers that are going to be messed with by another thread and such).

Micha

Sorry, if it caused confusion, but I don't know about TApplication or its particular implementation. I don't do GUI. ;) I was just reading about timer objects ... sorry.

Now we're talking. :)

Vinzent.

Well, I don't consider it beautiful, I consider it a hack. :)

It's beautiful, as the user does not need to care about mutually exclude access of the objects that are used both in the main line code and the timer callbacks. Most "application programmers" would not be able to deal with this at all and produce nothing but buggy programs. Additionally as X-Windows and MS-Windows are not thread save in such an environment it would not be possible to do GUI stuff in a timer callback.

Of course there _are_ reasons to have preemptive timer callbacks and providing decent means for that would be a great plus of FPCs.

-Michael

If you take a look the start of this thread, I mentioned that after implementing Delphi/Lazarus like behavior for the main thread without the need for a GUI, I'd like to enhance the concept by optionally having an TApplication object for any thread

Not every thread or program is event driven. To be honest I have missed that term entirely till now.

Also I think a base thread should remain free from this. You can of course create a descendant of tthread that is event driven (e.g. messagequeue, pump and list of known events/methodvars in overriden tthread.execute)

Ales Katona wrote

This is an old problem, and not fixable by wishing it. If you want a good async. timer, sure make one, but don't expect it to work safely by magic.

I'm sure I won't, I'm paranoid. :)

That's what I ment, it won't simply "just work without implications" like TThread SEEMS to.

Alright, yes. But as soon as you start overriding Thread.Execute the user (as in programmer) should start thinking about those issues anyway, so I didn't consider that a show-stopper. Maintaining thread-safety ain't so complex once you start following some simple rules.

Vinzent.

Vinzent Hoefler wrote

Maintaining thread-safety ain't so complex once you start following some simple rules.

Either you're underestimating thread-safety or you're a genius. Can you document those simple rules on the wiki ?

Micha

Of course it would be great if a TTimer would fire it's events in a thread context that can be selected by the user (e.g. the thread context the TTimer.Create was called in, thus in a compatible way, the TTimers that are placed visually and therefor created by the application start code do main thread events). But (unfortunately :-) ) this is done neither in Delphi nor in Lazarus, and it's not easy to do, as I suppose it would need something like TThread.Application and with that a completely event-driven programming paradigm for the thread in question. This is exactly what I would like to see someday for FPC.

-Michael

Sorry, if it caused confusion, ...

No problem, taking more ideas as an input is always useful !

Thanks,
-Michael

Michael Schnell wrote

the need for a GUI, I'd like to enhance the concept by optionally having an TApplication object for any thread (e.g. TThread.Application) and

It shouldn't be named Application, because there is only one application per process.

thus having an event scheduler for same. By this it would be possible to

Separate event scheduler and application.

No, the timer will still be synchronous, only to the current thread's event handler, not to the main (GUI) one.

I don't see how async timers can be useful for software (maybe to control hardware perhaps, but only the trivial kind as well, no complex state allowed), as you cannot take any locks, and must be re-entrant. This implies you can almost call no

Also I think a base thread should remain free from this. You can of course create a descendant of tthread that is event driven (e.g. messagequeue, pump and list of known events/methodvars in overriden tthread.execute)

Right. That is a decent way to do it "optionally" (as I of course would do it, not to break any running code). So a TApplicationThread class might be defined in the RTL as a descendant of TThread. It might have a TApplicationThread property the type of which might be a descendant of TCustomApplication and provides the non preemptive scheduling stuff based in messages.

But before doing this I'd like to do/have a TApplication that might be a descendant of TCustomApplication for the main thread for the same purpose here. (similar as Lazarus, but not using any GUI stuff).

TTimer, TThread (.Synchronize ! ) etc. might be affected by this, too.

-Michael

Separate event scheduler and application.

Hint registered and saved in the to-do list :-)
-Michael

No, the timer will still be synchronous, only to the current thread's event handler, not to the main (GUI) one.

These timers are assigned to a thread.
They are synchronous (non-preemptive) regarding their own thread and asynchronous (preemptive) regarding all other threads. So, by accessing objects from the correct thread, the user can select between complexity and latency as needed.

I don't see how async timers can be useful for software

It decreases the latency. This is always the reason to do _preemptive_ multitasking.

(maybe to control hardware perhaps, but only the trivial kind as well, no complex state allowed), as you cannot take any locks, and must be re-entrant. This implies you can almost call no

Michael Schnell wrote

> I don't see how async timers can be useful for software It decreases the latency. This is always the reason to do _preemptive_ multitasking.

This is not an async timer in the way Vincent meant it, assuming I read his mails correctly. Also, pre-emptive multitasking is different from the async timer as you're explaining here in exactly this detail.

Micha

This is not an async timer in the way Vincent meant it, assuming I read his mails correctly. Also, pre-emptive multitasking is different from the async timer as you're explaining here in exactly this detail.

An asynchronous timer needs to do a preemptive callback. Preemptive execution needs it's own thread. This is a way to provide the timer with a thread. The advantage of this way (explicitly use a TThread instead of having the timer create it's own thread) is (1) compatibility the timer works like commonly known only extending the paradigm to using the timer in a thread and (2) when using the associated thread for this objects can manipulated without caring about mutual access with the timer callback. If you simply want an asynchronous timer, just don't do anything else in the associated TThread. Of course you can do a TAsnycTimer class by using this scheme.

-Michael

Michael Schnell wrote

An asynchronous timer needs to do a preemptive callback.

No.

Preemptive execution needs it's own thread.

No.

This is a way to provide the timer with a thread.

Preemptive means same thread, as it 'preempts' it. That's what a (unix) signal is. That's what Vincent meant (I think).

Micha

No, not reentrant, unless you use the same event in the queues of multiple execution points. (which is unlikely I think)

Micha Nelissen wrote

Michael Schnell wrote: > An asynchronous timer needs to do a preemptive callback. No.

Ooops ? What is an asynchronous timer according to your definition ?

> Preemptive execution needs it's own thread. No.

Ooops ? What do you think is preemptive ? By definition preemption means that according to a hardware event the execution of a line of code can be "preempted" by other code an any position. As I understand it this is only possible by threads or processes or by multiple CPUs (that are by the OS are handled as Threads, as well)

Preemptive means same thread, as it 'preempts' it. That's what a (unix) signal is. That's what Vincent meant (I think).

OK, a signal is a special kind of (preemptive) thread that the OS can schedule.

IMHO it's not a good idea to have pascal code run in the context of a signal. I feel that restrictions that might hold here ask for lots of problems.

Moreover FPC is a lot about platform independency and signals are not an option in all supported platforms.

-Michael

Micha Nelissen wrote

Preemptive means same thread, as it 'preempts' it. That's what a (unix) signal is. That's what Vincent meant (I think).

Sorry, "Vinzent", it is.

Micha

On Wednesday 08 November 2006 10:53, Micha Nelissen wrote:

Sorry for all the confusion, I'm trying to clarify what I actually mean (or what I would *expect* from such a timer object):

I don't see how async timers can be useful for software (maybe to control hardware perhaps, but only the trivial kind as well, no complex state allowed), as you cannot take any locks,

First: Of course you can take locks. Such a "timed execution" is - just like any other thread - still scheduled by the OS and thus follows the same rules. A higher priority to make sure an expired timer actually preempts other running threads doesn't change this fact.

and must be re-entrant. This implies you can almost call no function at all.

Second: You could call any function you can call safely from within any other plain thread.

That's the difference to a "real" signal (where you can merely set some variable to be polled later and bail out), and makes it so much nicer to use than OS' signals.

So, the semantics would be about the same as in a signal (apart from the "softer" timing), but you would have much less restrictions in the handler's implementation compared to a real OS signal, which acts more like an interrupt.

Vinzent.

Vinzent Hoefler wrote

So, the semantics would be about the same as in a signal (apart from the "softer" timing), but you would have much less restrictions in the handler's implementation compared to a real OS signal, which acts more like an interrupt.

Impossible. You'll end with TThread.Synchronize, but without its advantage of knowing in which thread it runs, because the entire *point* of a GUI thread is that it's not doing anything for 99% of the time (waiting for user input only).

Micha

Yeah, right. So this:

--- 8< ---
{/= tFlusher.Execute =================================================\} { } {\====================================================================/} procedure tFlusher.Execute;
begin

   self.Started.SetEvent;

   repeat
      SysUtils.Sleep (self.Interval);
      self.Logger.Flush;
   until self.Terminated;
   // Last Action Hero.
   self.Logger.Flush;
end {tFlusher.Execute};
--- 8< ---

is totally impossible to work? To implement something less crude you could either implement a "delay until" by checking the point in time, the threads gets woken up or on UNIX you could even sigwait() for SIGALARM and (re)set it accordingly.

You'll end with TThread.Synchronize, but without its advantage of knowing in which thread it runs, because the entire *point* of a GUI thread is that it's not doing anything for 99% of the time (waiting for user input only).

I don't a GUI thread. Currently I have up to 24 threads processing image data and writing log information (which I don't want to be written line by line, that's why I collect them in a queue for a couple of seconds before actually writing it to a file) and some other threads processing RPC requests. And only one of them provides an RPC interface to something you'd call GUI.

But of course, automatically calling methods by a "timer" with such semantics is totally impossible. Guess I have to rethink, why it actually works.

Vinzent.

It works, but it's not realtime by any definition.

Micha

There is TCustomApplication in fcl/inc/custapp. The Lazarus TApplication descends from it. All you need to do is create a descendent of TCustomApplication which can implement all the messaging you wish for...

I took a does not provide anything that
is related to scheduling (e.g.: ProcessMessages), but everything else that TApplication has. Supposedly the scheduling stuff is added by Lazarus.

So the way to go is
(1a) do the scheduling stuff based on messages in it's own object. (1b) do TApplication as a descendant of TCustomApplication and have it use the scheduling object.
(1c) do TTimer in a way that it sends messages to the main thread. (1d )do TThread.Synchronize in a way that it sends messages to the main thread.

and later
(2a) do TSchedThread as a descendant of TThread in a way that it uses the scheduling object. (it will have things like TSchedThread.Run and TSchedThread.ProcessMessages).
(2b) do TTimer in a way that when used with TSchedThread it sends it's messages to the appropriate thread

-Michael

Oh wonder. A couple of lines ago these semantics were still "impossible".

but it's not realtime by any definition.

First, I never said "real-time" in that context and second: it sure can be, if the thread's priority demands an immediate wake-up after the sleep.

Then and if implemented a litte bit less crude - as I suggested - it would be no less real-time than

|declare
|   Next_Cycle : Ada.Real_Time.Time := Ada.Real_Time.Clock;
|   CYCLE_TIME : constant Ada.Real_Time.Time_Span :=
|                  Ada.Real_Time.To_Time_Span (1.0);
|   use type Ada.Real_Time.Time;
|begin
|   loop
|      Next_Cycle := Next_Cycle + CYCLE_TIME;
|      Do_Some_Work;
|      delay until Next_Cycle;
|   end loop;
|end;

in Ada would be. And the jitter in the above Ada code was about 30 microseconds running on an RTOS, of course.

Vinzent.

TThread.Synchronize does not need to be changed. You can set a series of callbacks (from classes unit) which you can then use to configure the whole synchronize issue...

Seems correct. And all this cross-platform, of course :-)

Michael.

Sounds great !

Seems correct. And all this cross-platform, of course :-)

Of course !!! :-)

So I gather that I have your support for that project (once I might be able to start (as I supposedly will have a lot of questions on how to integrate it with the existing FPC RTL code) ...

Thanks,

-Michael

What integration would you like to see ?

Your code can perfectly be implemented as an add-on unit, which we can distribute.

If you implement native threading, that will certainly be 'integrated'.

Michael.

Vinzent Hoefler wrote

> but it's not realtime by any definition. First, I never said "real-time" in that context and second: it sure can

I concluded that from your quote:

<meta-quote>
Timing events allow for a handler to be executed at a future point in time in an efficient way, as it is a standalone timer that is executed directly in the context of the interrupt handler (it does not need a server task).
</meta-quote>

I read server task as thread, and a timer as interrupting any thread at the time it was fired and calling some callback then.

Maybe Ada creates a hidden thread itself in which it calls these callbacks (so cheating on its own definition IMHO), but otherwise the only value for such interrupts would be realtime requirements.

microseconds running on an RTOS, of course.

That's not the common OS case.

Micha

I mainly quoted that to clarify my intention of the term "timer", because there seemed to be some confusion about that. The answer to my "I always thought a timer is more comparable to an interrupt or signal than a polling loop." was "That is a very common misconception." which I still refuse as being a correct answer.

[It seems that a lot of people seem to put all their efforts in looking at a single particular implementation before trying to see the big picture. ;)]

I didn't say that it *is* necessarily an interrupt or signal, I just said it is comparable to one, by which I meant its preemption of ongoing operations.

To implement the semantics of "timing events allow for a handler to be executed at a future point" a polling-loop could prove to be a very problematic solution.
Especially outside of the usual GUI-application, we seemed to be talking about, because the user/programmer somehow has to make sure that the polling is actually done some time in the future.
Considering that a simply "ReadLn" can block indefinitely already, this is quite hard to accomplish then, isn't it?

I read server task as thread, and a timer as interrupting any thread at the time it was fired and calling some callback then.

Yes. And if a timer implemented as thread has the highest priority it would even do exactly that, and I also said that in one of my last mails. Still no word of "real-time", although now we're catching up. ;)

Maybe Ada creates a hidden thread itself in which it calls these callbacks (so cheating on its own definition IMHO),

Maybe. :) I didn't read the new standard yet, but I guess, the actual implementation is left to the implementor as long as the required semantics are kept. Although it seems rather unliky it actually might be an additional thread on top of the OS' scheduling system, or it might be a real hardware interrupt, or it might be the scheduler of the runtime without any OS running at all ...

but otherwise the only value for such interrupts would be realtime requirements.

Well, Ada is - of course - a language designed with real-time systems in mind. Yet, this doesn't mean, that the general concept is useless outside of a hard real-time scope.

Considering that in the GUI-application such things seem to be used very commonly (due to the polling-loop implementation without the implications of being executed concurrently), the value seems to be a little bit greater than to "useful in real time environments only".

As far as I understood, the only problem here we discussed were applications without such dedicated event loop.

> microseconds running on an RTOS, of course. That's not the common OS case.

Precisely, and that's why I even tried to avoid the term "real time" thus far. Yet, such feature can be useful.

Vinzent.

The answer to my "I always thought a timer is more comparable to an interrupt or signal than a polling loop." was "That is a very common misconception." which I still refuse as being a correct answer.

I misunderstood that you meant a Delphi TTimer. And I wanted to point out that many Delphi users think that a TTimer works like an interrupt and that of course is untrue.

-Michael

Vinzent Hoefler wrote

The answer to my "I always thought a timer is more comparable to an interrupt or signal than a polling loop." was "That is a very common misconception." which I still refuse as being a correct answer.

The irony is of course that you clamp to your interrupt definition, but still implement your log timed event with a loop :-).

Well, readln is used in the trivial (T)UIs; TUIs/GUIs are usually event-driven, like networked apps also can be. A Timer fits in a event-driven app just fine, and then a timer doesn't interrupt anything anymore; but your intuition may say that it should, that's why it's a common misconception.

Micha

Your code can perfectly be implemented as an add-on unit, which we can distribute.

The first goal I see is that FPC comes with portable support for TTimer and Delphi/Lazarus compatible thread support, that can be used in a GUI-less Application - thus without Lazarus. If that's possible with an add-on unit: fine, but if that needs some support in the RTL (as we need Application.Run, Application.ProcessMessages and maybe a ways to separate the "Application Source" from the user code) , it should go there.

I suppose we'll think about if and how to implement something like TSchedThread once the first step is done and verified.

-Michael

Well, yes and no, just like I would do in Esterel, for example:

|loop
|   await NEXT_TIME;
|   emit EXPIRED;
|end loop;

Because I do not know of any other construct to express indefinite repeatition, I used a loop. Any other ideas? ;)

Yes, of course, this still *is* nothing more than an event loop, and its asynchronous character would only come from the fact that it is done in parallel with some main thread, so it neither does interfere with the execution of the main program nor does it depend on it.

Whatchamacallit, it was an example where calling this "event loop" might be close to impossible. Yet, admitted, the expiring timer still wouldn't stop the ReadLn from reading its file, so you're screwed either way, but you could for example emit a "beep" to the user from time to time. ;)

like networked apps also can be. A Timer fits in a event-driven app just fine,

I didn't say, it doesn't, did I? If so, I'm sorry. But I even remember mentioning priority queues in that context (which could somehow ensure a timer event would be handled in time, so even "real time" is possible if enough care is taken by the programmer).

and then a timer doesn't interrupt anything anymore; but your intuition may say that it should, that's why it's a common misconception.

Ok, I now understand where "my misconception" comes from. :P

We're back to the usual GUI application where everything depends on that event loop. :) Of course, that's fine for most "event-driven" applications, especially when being GUI. IOW, it can be considered the difference between being preemptive and being cooperative. ;)

Problem here is that I would call my event loop only every couple of hours, because responding to on the event "now process image data" takes so long. :) And as far as I understood, the problem was exactly that: To implement something like TTimer in an application which is *not* based on a regularly called event loop.

Vinzent.

Yes, of course, this still *is* nothing more than an event loop, ...

A decent event loop should use an OS API to free the processor until the next event is scheduled:

Meta code:

  repeat
    if (event in event queue) begin // this is either an OS API call or

the event queue is done with user land means
read event from event queue // "

      end else begin
       (sleep until a new event is detected in the event queue) // this

needs to be an OS API call

    end;
  until false;

So there is no polling and no CPU time is wasted that might be valuable for other processes or threads.

Decent OS means usable include message queues and semaphores.

AFAIK, in Delphi and Lazarus this event loop is not done in the program, but an event scheduler provided by the GUI system is used.

-Michael

That's why I wrote the example in Esterel.

Give me that API and I sure will use it. Modula had signals at least (although I don't remember it too good), Ada has task entries (and you still write your tasking code in a loop, then). Alas, Object Pascal lacks a decent portable OS tasking interface.

Vinzent.

Alas, Object Pascal lacks a decent portable OS tasking interface.

With Delphi we successfully used Windows Messages for inter Thread communication.

With FPC on Linux a friend of mine used System 5 Messages queues for the same purpose.

So IMHO inter Thread messages is a good way to go.

As neither Delphi nor FPC have a portable way to use inter Thread messages, in both simply the system API encapsulations provided were used.

This is why I vote for doing a portable implementation of inter Thread messaging, including firing an event (at first only in the main thread) when a message arrives. Of course a Delphi/Lazarus compatible GUI-less TTimer implementation can easily be made on top of this.

In Delphi there is a way to register Windows message callbacks for the GUI thread in a very Windowish way. I don't know if the Lazarus people implemented something like this and if it's portable.

-Michael

No. The message loop is very low level.

Why don't you simply go ahead, implement something (cross-platform) and then we can see how this can be integrated in FPC, if this integration is needed at all...

We can discuss for days in the air....
Better show some working code, then we can discuss about specifics.

Michael.

Why don't you simply go ahead, implement something (cross-platform) and then we can see how this can be integrated in FPC, if this integration is needed at all...

I did do first steps of this already some years ago. I hope I will be able to start over in January.

-Michael

BTW.:

TPC comes with Free Vision.

The FPC IDE based on Free Vision seems to work fine.

The Free Vision "Non-GUI" seems to be event driven and thus needs to implement a kind of event scheduler.

Maybe same might be worth looking at.

-Michael

It's an event handling loop using queues for input events. Very primitive :)

Maybe same might be worth looking at.

Don't think so, technology of the 80ths ;)

Op Thu, 9 Nov 2006, schreef Florian Klaempfl:

Hmmm... Actually I think event handling mechanism in FV is quite advanced.

I.e. compare TV with VCL. In the VCL you define an event property (i.e. key pressed) which gets magically called, and can modify the methods in the object.

A first difference is that the event doesn't magically arrive at a control, but is distributed by the event handler logic, which is through overides completely under control of the programmer.

But I'm most impressed what happens when the event arrives at the right view. In general, you don't start to modify fields directly, but a new event is generated: a command. In effect, unlike the VCL, the program logic is steered by commands, and not by direct events like mouse clicks.

Add to this the enable/disable system of commands. I.e. disable cmCut and all menu bars, status bars, buttons, context menu's and whatever thing that can generate a cmCut turns grey.

IMO it is still even today one of the best designed event systems and I have to see a system that does it better...

Daniël

Daniël Mantione wrote

Add to this the enable/disable system of commands. I.e. disable cmCut and all menu bars, status bars, buttons, context menu's and whatever thing that can generate a cmCut turns grey.

In Delphi/VCL this is called a TAction, which can be assigned to speedbutton, toolbutton, and menu item, IIRC.

Micha

At least the same exists in Delphi/lazarus. Just use actions.

Michael.

I should have read all posts first, before posting a duplicate :-)

Mea Culpa...

Michael.