Hi All,

If my application needs SPEED, i.e. take full advantage of CPU capabilities, and the application has no I/O operation at all (neither disk nor network), it seems no need to create threads more than the number of CPU cores （hyper-threads)?

Am I right?

Xiangrong

In our previous episode, Xiangrong Fang said:

My rule of thumb is physical cores + a percentage. (like 10-20%).

The idea is to saturate the CPU, but only just so, since saturating by way too many threads increasing switching time and thus decreases performance.

The percentage is a rule of thumb approach to account for some minor I/O (if only OS background), scheduler/locks switch time, and, possibly hyperthreading.

But in my apps I always make it configurable, and use this sum only to seed the default of the configuration setting.

This because you don't always want to fully utilize the computer (e.g. to give some air to background services, or because you are working on it), and because automatic detection regularly goes wrong with new CPUs etc.

2014-09-15 11:10 GMT+02:00 Xiangrong Fang :

（hyper-threads)?

If the CPU has hyper threading then these should show up as individual cores (CPUs). They call a "core" with 2 processors on it a "core" but each of these "cores" is actually two complete processors (cores in the conventional meaning of the word) that just share some common circuitry to speed up some things (and slow down other things).

If you have 2 hyper cores then your operating system should actually report 4 CPUs (cores) because that is what it actually is if you remove all the intel marketing terminology, no need to double any numbers. If it reports 4 CPUs then you have exactly 4 CPUs.

In our previous episode, Bernd said:

Hyperthreading doubles the execution stream logic, so that one core can take two incoming streams of instructions. The idea is that when one thread stalls, or executes unoptimal code, the other thread can run, increasing utilization. But that is more something like 10%, not doubling it.

Since the rest of the core (the execution units) can't actually handle processing two streams at full speed.

You are probably confused with AMD's module system, which does duplicate cores, and only shares some logic (like AVX)

Marco van de Voort wrote

Hyperthreading doubles the execution stream logic, so that one core can take two incoming streams of instructions. The idea is that when one thread stalls, or executes unoptimal code, the other thread can run, increasing utilization. But that is more something like 10%, not doubling it.

I'd expect somewhat better than 10%, but another complicating factor is that it will depend on the cache architecture. Also in the general case some architectures can manage a lot of integer threads but have a massive bottleneck if multiple threads try to do floating point operations.

In our previous episode, Mark Morgan Lloyd said:

It's a bit low. It is more like 15%. Based on kernel compile benchmarks on phoronix.com done with and without HT.

Marco van de Voort wrote

too many threads increasing switching time and thus decreases performance

Not only this.

Additionally Linux tries to keep a thread/Task at the same CPU when re-scheduling the CPUs. By this the count of misses on the (primary) caches is decreased.

AFAIK, cache misses can result in a huge degradation.

Hence - depending on the task to be done - using more threads than CPUs might not be a good idea.

-Michael

Bernd wrote

If it reports 4 CPUs then you have exactly 4 CPUs.

Correct from a algorithm POV, but not from a performance POV.

-Michael

Marco van de Voort wrote

My rule of thumb is physical cores + a percentage. (like 10-20%).

Tell that to most developers out there, they clearly don't know that "rule of thumb". :-) Mozilla Thunderbird under Win7 shows 47 threads. EditPad Pro 7 shows 18 threads. etc etc... I only have a mere dual-core system.

Regards,
- Graeme -

And look at that, I'm sure that you have more than one process running.

Only because there are 47 or 18 threads does not mean that every thread is doing something right now. Most threads in those applications are just waiting (blocking) for the right event to trigger them to do something. And in that case the "rule of thumb" doesn't help much. But when we're talking about really active threads that do calculate something or so then the "rule of thumb" is quite good, because otherwise there will be more thread switching between the threads your program has (plus the other threads of other processes) than when you only have a bit more threads active than cores.

Regards,
Sven

In our previous episode, Graeme Geldenhuys said:

Not all thraeds are about increasing performance. Usually threads are just to do (or monitor) things asynchroniously.

IOW, this is the about guestimating optimal threadcount if you divide one task over multiple threads. The point is that this is often not equal to the number of hardware threads.

Increasing Performance is only one of the purposes you might have in mind when using threads. "Traditionally" threads are just that: unrelated threads of execution in a project, i.e. multitasking-tasks that share some memory.

-Michael