Hi,

I am running FPC 2.3.1 under Linux 2.6.26 on a twin CPU AMD Athlon MP system.

Whenever I instantiate multiple descendants of the TThread class, they all seem to run on the same CPU, which is the same CPU as the parent thread is running on. With 21 threads in the program, the system monitor shows one CPU at 100% busy and the other CPU idle.

Does this class, by default, impose CPU affinity, in the manner of the Windows NT/2K/XP CreateThread() API? If so, how can I disable this? [I.e., I want all my threads to be runnable by any CPU inside the box.]

No CPU affinity is imposed as far as I know;
Under Linux, this is not possible, at least I've never heard of such a call.

Michael.

-------- Original-Nachricht --------

sched_set_affinity()
sched_get_affinity()

(Now you've heard. ;)

The default for the CPU masks should be all 1s though, so the scheduler should distribute the threads evenly to all available CPUs. But... AFAIR the affinity mask is inherited by child processes, so maybe the main thread already has some affinity mask set (well, I don't see no reason why it should, but you never know).

So for the original poster I'd suggest inserting a sched_get_affinity() call and check the resulting bit mask. If that's not all 1s, one should investigate the reason... if it is, then maybe the scheduler detects too many dependencies between the threads, so it doesn't distribute the thread on both CPUs, or maybe there's one totally different reason for the behaviour...

Vinzent.

In our previous episode, "Vinzent H�fler" said:

Maybe the dynloader tries to detect somehow if the binary is modern, by searching for some gcc generated symbol that is not there in the FPC bins, and reverts to something old. (e.g. to keep older threading code running).

Note that there is also a trend towards more asymetric multiprocessing, as already a dual cpu system of multicore processors is not symmetric anymore (moving threads between CPUs too often decreases cache utilization). It might be a byproduct/bug of such modifications in that particular kernel, specially if he is on a distro that likes to add own patches to kernels. (since it seems the OP is using a dual singlecore cpu system)

Vinzent Höfler wrote

-------- Original-Nachricht -------- > Datum: Tue, 20 Jan 2009 09:07:54 +0100 (CET) > Von: Michael Van Canneyt

[snip]

Correct spelling:

sched_setaffinity()
sched_getaffinity()

(Now you've heard. ;)

There are also pthread_setaffinity_np() and pthread_getaffinity_np(), where the "_np" suffix is a mnemonic for "non-portable".

I have checked the CPU masks for all threads and the first byte is always 0x03, which indicates that both CPUs can be considered for dispatching the thread.

I have noticed one thing that might shed a little light on the topic: whenever I use a pipe for the output and I allow the pipe to stall with a full buffer, after execution is resumed the program sometimes restarts on the other CPU, but again all threads run on the same CPU. This would indicate to me that the child threads are tightly associated with the parent thread, as far as CPU selection goes.

I will reimplement my code in C++ using wxWidgets and its wxThread class, and see what behaviour that produces.

I might also reimplement the code in C, using the pthread_??? family of API calls in their raw state. This could be a bit ugly, as the code is decidedly object oriented.

As it appears that this issue is at a lower level than the TThread class, we might need to examine the BeginThread() function of the RTL. However, it could be attributable to the threading library I use with Linux (NPTL, in my case).

I shall report back with whatever I find.

On Tue, 2009-01-20 at 21:20 +0000, David W Noon wrote:

[snip]

I have noticed one thing that might shed a little light on the topic: whenever I use a pipe for the output and I allow the pipe to stall with a full buffer, after execution is resumed the program sometimes restarts on the other CPU, but again all threads run on the same CPU.

I have finally tracked down what is going on here, and the above was a significant hint.

In years past, the SMP Linux kernel would dispatch threads to CPUs using a more-or-less round-robin basis: whenever a thread was preempted or yielded its timeslice, the kernel would try to use the next available CPU to dispatch the next true-ready thread. This had the nice effect of spreading the thermal stress across all CPUs fairly evenly. However, if the next true-ready thread happened to be in the same process as the preempted/yielding thread, the caches, TLBs, etc., would have to be flushed from the "current" CPU and then reloaded on the "next" CPU.

With current (2.6) kernels, this approach has changed. The "current" CPU is reused, so that when the next true-ready thread happens to be in the same process, the caches, TLBs, etc., are already valid.

Now, the thread routines in my test program all pivot on the one mutex semaphore and do very little work before blocking again, so only one of them can execute at any point in time. This has the effect of reducing the execution model to that of a uniprocessor system. I added some bogus CPU-intensive workload that ran outside of the mutex, so that this could be executed by more than one thread concurrently. The upshot was that I could get both CPUs at 100% busy.

This also led me to the discovery of a similar concept tp CPU affinity, called thread scope. There are 2 APIs, called pthread_attr_setscope() and pthread_attr_getscope(), the first of which allows a thread to be created with a scope of PTHREAD_SCOPE_PROCESS or PTHREAD_SCOPE_SYSTEM. The behaviour I was seeing was akin to PTHREAD_SCOPE_PROCESS, which ties all threads of that scope within a process to dispatch on the same CPU. However, Linux does not implement this thread scope; it only implements PTHREAD_SCOPE_SYSTEM, which allows a thread to be dispatched on any CPU to which it has an affinity.