Lets say you have a unit, with code that can be called from threads.
- The code  does not own the thread object, so it can *not* use TThread.OnTerminate (assuming it could get hold of the thread object).
- Yet the code wants to use "threadvar" and create objects (or otherwise allocate mem). The code should return the memory, when the thread terminates.
- The user of the code should not need to make any calls, to initiate the clean up.

So that would need something like
   GetCurrentThreadObject.AddThreadTermationHandler(MyHandler);
or
   GlobalThreadTerminationHandler.Add(MyHandler);

MyHandler does not need the thread-id, or -object. But it needs to run in the correct context for the ThreadVar. That is it needs to see the value the threadvar has for the thread that is terminating. So the Handler could do something like "myThreadVar.Free".

Any ideas?

I know the memory manager gets called "MemoryManager.DoneThread". It is possible to intercept this, by getting the current mem-mgr, replacing the method with a wrapper method (calling the original method), and setting the mem-mgr.... Not particular nice.

Are threadvar variables being freed like regular managed vars? I guess that is all you need.

02.02.2019 14:00, пишет:

Are threadvar variables being freed like regular managed vars

Threadvars are not are like global vars, only that each running thread automatically has its own copy.

-Michael

Hello,

If you are targeting Windows, there is a special area in the PE structure for a DLL that contains addresses that will be jumped to whenever a thread gets created/destroyed. This has been used by libtcmalloc from gperftools:
https://github.com/gperftools/gperftools/blob/master/src/windows/port.cc

I don't know how to populate the appropriate locations in the PE header with freepascal though.
The comments in that same file hint that there is a function called pthread_key_create for systems that implement the pthread library, but I have not looked into that.

If you want an entirely FPC based solution, I would patch the running exe so that the system thread routine that calls OnTerminate jumps to my override which calls the original one before doing any finalization work that it needs to do.
But installing such a trampoline is very platform specific, I only know how to do it on Intel based architectures.

Michael Schnell wrote

> Are threadvar variables being freed like regular managed vars Threadvars are not are like global vars, only that each running thread automatically has its own copy.

Yes, I know this. (did my google).
Even managed vars are not released and leak. And according to what I googled, same happens in Delphi.

My question was, how I can write my own code to free them. There is a unit with functions (not methods), but they create an object (singleton). Without threads, that is dealt with by a unit finalization handler.
With out threads it just so happens that the finalization runs when the main thread ends.

But if you have such a singleton per thread?
The finalization runs to late, and it can not see the threadvar of other threads.
The TThread object is not accessible from that code, and if it was it has only one callback, that may already be used. (For this one would need "AddOnTerminate(AHandler)" instead of "property OnTerminate"

So my question was, if there is any other place, any hook, that is called at the time of a thread terminating? I haven't found one.

FPC seems to know when the thread ends. It calls
MemoryManager.THreadDone. And it seems that is for all platforms?

How forward compatible would it be, to pretend to install your own memory manager? Simply creating a wrapper around the current one? Then you would get ThreadDone calls (which appear to runs inside the thread, or at least they see the correct threadvar)

Anything else?

Anything else?

Maybe the other thread could set a callback event property provided by the TThread sibling with a function it defines, and the thread that is associated with the TThread sibling calls this property in it's OnTerminate handler.

-Michael

Michael Schnell wrote

> Anything else? Maybe the other thread could set a callback event property provided by the TThread sibling with a function it defines, and the thread that is associated with the TThread sibling calls this property in it's OnTerminate handler.

There are no "siblings".

The use case here would be LazLogger. User code can call a function "Debugln" this calls a method on a global object.
The object provides storage for log-level (by name), and indentation, and others.

Indentation would be best per thread. So if user code in a thread calls "Debugln" a thread-instance of the logger (or a wrapper to it) should be created. But that must be destroyed at some time.
User code should not be expected to deal with this destruction. The logger has no knowledge of the TThread object. User code decides, if and how it is subclassed, and if and how OnTerminate is used.

As for the question, why create an object, why not just "threadvar indent". User code can create (within the same thread) more than one instance of the logger.
This I something that can be used in testcase, where the testcase wants to have a logger, but does not want to mix with the log of the tested code (maybe capture that log).

So the logger software could check a Thradvar (I *suppose* they are initiated with Zero) and create or use an internal object the self of which is stored in the Threadvar and with that having an instance for any thread that calls "Debugln"

-Michael

George Bakhtadze wrote

As an option, you can store all thread-specific data within a thread-safe data structure (Map) instead of threadvars and free that structure in unit's finalization.

Yes, but you can end up with a lot of data for dead threads, if an app create many short lived threads.

Yes, but how do I get notified, when the thread is gone, and I can free the memory (of the object that was created)?

Yes, but how do I get notified, when the thread is gone, and I can free the memory (of the object that was created)?

Ah, now I finally see the problem :)

The only idea that comes in my mind is creating yet another thread or a TTimer (by means of "QueueAsyncCall") to poll if the original Threads still living. This supposedly would involve OS calls for getting the thread ID and for checking same for validity. Ugly ! :( .

-Michael

If the LazLogger user has to put debugln() statements in his code, me thinks it would be an acceptable price to tell him:

"If you are using debugln() inside threads, let each thread register with the logger before first usage."

So the user could do sth. like

  LazLoggerObj.RegisterThread(<thread handle or ID>);

in the construction code of his threads. So you know the threads using the logger. Same could be done in the destruction code, using the ".DeregisterThread()" method.

So you now the threads and their lifetime inside the logger.

HTH anyhow,
Marc

Then I may as well "hack" the memory manager.
Just wanted to see, if there was anything better. Given that MemManager.DoneThread exists, it should be possible to add a feature. Is that a plausible request?

I believe writing and installing your own memory manager is a documented feature, so the following steps should be future save. (unless something else install yet another mem manager after this. But since at that stage, code is already running and allocating memory, that should have happened before)

var
   CurMemMgr: TMemoryManager;
   CurDoneThreat: procedure;
threadvar
   logger: TMyThreadLogger; // might be created by each thread
procedure MyDoneThread;
begin
   Logger.Free;
   CurDoneThreat();
end;
   GetMemoryManager(CurMemMgr);
   CurDoneThreat := CurMemMgr.DoneThread;
   CurMemMgr.DoneThread := MyDoneThread;
   SetMemoryManager(CurMemMgr);

It is not the "nicest" solution but should work.

What I still need to find out is:
- Is DoneThread called under every OS/platform?
  but if not, it is not fatal, just a leak. Would be good to be able to have some IFDEF in this case (eg not create logger)
- Is DoneThread called inside the thread? or otherwise with access to the correct threadvar "logger" instance?
  and is that so on all OS/platform. (I can add tests, to protect against freeing the main threads object)

Am Di., 12. Feb. 2019, 16:56 hat Marc Santhoff
geschrieben:

Not every LazLogger user knows that their code is run in a thread. And not every thread user knows that the code they are using is using LazLogger in turn.

Regards,
Sven

Marc Santhoff wrote

".DeregisterThread()" method. So you now the threads and their lifetime inside the logger.

Yes, I wanted to avoid that. Put as little on the user as possible.

The user may use some THttpRequest.OnProgress which may be in a thread, but the user does not have the thread either.

Sven Barth wrote

Not every LazLogger user knows that their code is run in a thread. And not every thread user knows that the code they are using is using LazLogger in turn.

Ok.

But if used that way, would LazLogger need to notice at all?

As I understand it the use case will be to discriminate and not mix up the log entries from different threads. If the user does not want this at all, why bother inside the logger code?

I can imagine race conditions or misplaced log lines. If used in debugging code this is mostly irrelevant, imho...

Yes it is not critical, and may be made optional (global switch in main thread / or different unit in uses (once in main unit))

The current idea I am pursuing is this.

LazLogger already has
DebuglnEnter
DebuglnExit
which add/remove indent.

So your log can have lines that are indented.
procedure DoFoo;
begin
   DebuglnEnter('start foo');
   CallBar;
   CallSome;
   DebuglnExit('end foo');
end;

The called code can contain debugln, and all of them will be nicely indented.

Of course that is limited in threading. Should the code in other threads also log indented? Worse, if the other thread goes through a DebuglnExit, when the last DebuglnEnter was not in that other thread, then the indent becomes entirely messed up.

Maybe I will also combine this with prefixing the thread ID. But for that I do not need extra objects.

And yes, at first it would like that the indent needs no extra object either. just have a global
  threadvar LazLoggerIndent: integer;
But that fails, if there are (in the main thread, or in any one thread) more than one logger. (Yes I do have that, eg in testcase where the test logs independent of the tested code)
Once there is more than one instance of logging (in the main thread), then which of them gets to use the global threadvar?

Well on 2nd thought it could be done. (Discussing an issue has a tendency to bring up new ideas)
the functional coded "debugln" uses the main instance. So that could have a flag, to use the global var.

2ndary loggers must anyway be called via there object, and therefore do not have the benefit of automatic wrappers.

So yes maybe I will go that way. It is a bit against OO paradigm. But it should yield the same result.

Thanks for all the feedback.

Out of interest I still keep the mem-manager question....