Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2008-09

parallel compilation support

10 messages · 4 people ·

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Marco van de Voort
to Graeme Geldenhuys

In our previous episode, Graeme Geldenhuys said:

Does FPC support parallel compilation? For compiling itself, or for applications?

What do you mean with parallel compilation? Starting up multiple compilers like make -j? Or generating paralel code?

The tree supports make -j x due to Jonas' efforts.

The yields are fairly low though (+/- 10% iirc, but it is a while back that I timed).

Reply by email

Graeme Geldenhuys
to Marco van de Voort
Marco van de Voort wrote 5 lines
What do you mean with parallel compilation? Starting up multiple compilers like make -j? Or generating paralel code? The tree supports make -j x due to Jonas' efforts.

I meant multiple compilers running in parallel compiling various units to ultimately build a single application executable or FPC itself. You answered my question for the latter.

From your response, I assume it's not an automated process (FPC can

figure it out by itself)? Does it need some manual tweaking from the developer (via Makefiles maybe)?

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Reply by email

Marco van de Voort
to Graeme Geldenhuys

In our previous episode, Graeme Geldenhuys said:

Show quoted text 12 lines
> What do you mean with parallel compilation? Starting up multiple compilers > like make -j? Or generating paralel code? > > The tree supports make -j x due to Jonas' efforts. I meant multiple compilers running in parallel compiling various units to ultimately build a single application executable or FPC itself. You answered my question for the latter. >From your response, I assume it's not an automated process (FPC can figure it out by itself)? Does it need some manual tweaking from the developer (via Makefiles maybe)?

Via makefiles yes, and only on directory level. But this requires to specify the dependancies real precise. Currently afaik the packages/ dir uses parallelization.

To take this a step further, it would take a multithreaded compiler I guess, and a way to offer a lot of source at once to the compiler, with a hint to use paralellism. But that is far, far off.

Reply by email

Jonas Maebe
to Marco van de Voort
Marco van de Voort wrote 7 lines
In our previous episode, Graeme Geldenhuys said: > > From your response, I assume it's not an automated process (FPC can > figure it out by itself)? Does it need some manual tweaking from the > developer (via Makefiles maybe)? Via makefiles yes, and only on directory level.

It also works at the unit level, e.g. the Darwin rtl. It goes from 8.6 to 5.3 seconds when adding -j 2 on my MacBook.

Jonas

Reply by email

Marco van de Voort
to Jonas Maebe

In our previous episode, Jonas Maebe said:

Show quoted text 10 lines
> In our previous episode, Graeme Geldenhuys said: >> >> From your response, I assume it's not an automated process (FPC can >> figure it out by itself)? Does it need some manual tweaking from the >> developer (via Makefiles maybe)? > > Via makefiles yes, and only on directory level. It also works at the unit level, e.g. the Darwin rtl. It goes from 8.6 to 5.3 seconds when adding -j 2 on my MacBook.

Yes of course, since the RTL is special in that it has non build unit default.

Reply by email

Jonas Maebe
to Marco van de Voort
Marco van de Voort wrote 12 lines
In our previous episode, Jonas Maebe said: >> In our previous episode, Graeme Geldenhuys said: > >> >> Via makefiles yes, and only on directory level. > > It also works at the unit level, e.g. the Darwin rtl. It goes from > 8.6 > to 5.3 seconds when adding -j 2 on my MacBook. Yes of course, since the RTL is special in that it has non build unit default.

Build units are the exception rather than the rule under packages as well. Most directories under packages are not parallellised themselves because no one went through the trouble of adding dependencies for individual units.

Jonas

Reply by email

Jonas Maebe
to Jonas Maebe
Jonas Maebe wrote
Build units are the exception rather than the rule under packages as well. Most directories under packages are not parallellised themselves because no one went through the trouble of adding dependencies for individual units.

Which in itself doesn't matter that much, since there is probably enough parallellism at the directory level.

Jonas

Reply by email

Marco van de Voort
to Jonas Maebe: Build units are the exception rather than the rule under packages as well. Most…

In our previous episode, Jonas Maebe said:

Show quoted text 9 lines
>> to 5.3 seconds when adding -j 2 on my MacBook. > > Yes of course, since the RTL is special in that it has non build unit > default. Build units are the exception rather than the rule under packages as well. Most directories under packages are not parallellised themselves because no one went through the trouble of adding dependencies for individual units.

The larger ones have afaik. But that is not so much what I meant.

I meant more that due to the autobuild capabilities parallelization might not be as big as a leap as for gcc.

And if you need to rearrange anyway, rearranging to a buildunit is an option too.

It would be interesting what would be faster (and maybe then even as a function of -j instances), a buildunit or parallelization.

At least I always though that that was the idea behind the packages/ paralellization: Fine granularity done by autobuilding (a package), coarse granularity by make.

It's not that I want to really start optimizing everything, just curious.

Reply by email

Peter Vreman
Show quoted text 21 lines
On 15 Sep 2008, at 09:52, Marco van de Voort wrote: > In our previous episode, Jonas Maebe said: >>> In our previous episode, Graeme Geldenhuys said: >> >>> >>> Via makefiles yes, and only on directory level. >> >> It also works at the unit level, e.g. the Darwin rtl. It goes from >> 8.6 >> to 5.3 seconds when adding -j 2 on my MacBook. > > Yes of course, since the RTL is special in that it has non build unit > default. Build units are the exception rather than the rule under packages as well. Most directories under packages are not parallellised themselves because no one went through the trouble of adding dependencies for individual units.

For the a lot of packages the fpmake.pp already contains correct include file and unit dependencies. If somebody is interested in parallelism he can extend fpmake to use this in calling the compiler multiple times.

In general a build unit is quicker because it can keep already compiled units in memory.

Peter

Peter

Reply by email