Sign in Join
Wiki copytemporary

Parallel procedures: what changed

Revision 0 to 1. Removed lines have a minus sign; added lines have a plus sign.

Title: Parallel procedures

OldNewLine
1+# Overview
2+
3+This page describes how to run single procedures in parallel using the MTProcs unit, which simplifies running procedures in parallel and simplifies implementing parallel algorithms.
4+
5+Parallel procedures and methods are often found in parallel algorithms and some programming languages provide built-in support for them (e.g. OpenMP in gcc). See [[OpenMP support]] for the plans adding such language features to FPC. Embedding these things into the language can save some typing and allows the compiler to generate code with less overhead. On the other hand there are many ways to convert a single threaded piece of code into parallel code. In fact simple approaches often slows down the code. To get good results one must specify some parameters, which a compiler can not guess. For examples see the vast amount of settings and discussions of [OpenMP](http://openmp.org/wp/) and [OpenCL](http://www.khronos.org/opencl/). You need parallel algorithms. MTProcs helps to implement parallel algorithms.
6+
7+# Getting MTProcs
8+
9+The unit **mtprocs.pas** is part of the **multithreadprocslaz.lpk** package, which needs no other packages, only FPC \>= 2.6.0.
10+
11+You can find its sources either on sourceforge
12+
13+ svn co https://lazarus-ccr.svn.sourceforge.net/svnroot/lazarus-ccr/components/multithreadprocs multithreadprocs
14+
15+Or in Lazarus *components/multithreadprocs*.
16+
17+As always: Open the package multithreadprocslaz.lpk in the IDE once, so that it learns the path. To use the package in your project: Either use *IDE Menu / Package / Open recent package / .../multithreadprocslaz.lpk / More / add to project* to use it in your current project. Or add it via the project inspector.
18+
19+# Simple example
20+
21+Here is a short example, that does not do anything useful, but to demonstrate how a parallel procedure looks like and how it is called:
22+
23+``` pascal
24+program Test;
25+
26+{$mode objfpc}{$H+}
27+
28+uses
29+ {$IFDEF UNIX}
30+ cthreads, cmem,
31+ {$ENDIF}
32+ MTProcs;
33+
34+// a simple parallel procedure
35+procedure DoSomethingParallel(Index: PtrInt; Data: Pointer; Item: TMultiThreadProcItem);
36+var
37+ i: Integer;
38+begin
39+ writeln(Index);
40+ for i:=1 to Index*1000000 do ; // do some work
41+end;
42+
43+begin
44+ ProcThreadPool.DoParallel(@DoSomethingParallel,1,5,nil); // address, startindex, endindex, optional data
45+end.
46+```
47+
48+The output will be something like this:
49+
50+ 2
51+ 3
52+ 1
53+ 4
54+ 5
55+
56+Here are some short notes. The details follow later.
57+
58+- Multithreading needs under unix the unit **cthreads** as explained in [[Multithreaded Application Tutorial]].
59+- For speed reasons the **cmem** heap manager is recommended, although it does not make any difference in this example.
60+- The parallel procedure **DoSomethingParallel** gets some fixed and predefined parameters.
61+- **Index** defines what chunk of work should be done by this call.
62+- **Data** is the pointer, that was given to *ProcThreadPool.DoParallel* as fourth parameter. It is optional and you are free to use it for anything.
63+- **Item** can be used to access some of the more sophisticated features of the thread pool.
64+- **ProcThreadPool.DoParallel** works like a normal procedure call. It returns when it has run completely - that means all threads have finished their work.
65+- The output shows the typical multi threading behavior: the order of the calls is not determined. Several runs can result in different orders.
66+
67+# Features
68+
69+Running a procedure in parallel means:
70+
71+- a procedure or method is executed with an Index running from an arbitrary StartIndex to an arbitrary EndIndex.
72+- One or more threads execute these index' in parallel. For example if the Index runs from 1 to 10 and there are 3 threads available in the pool, then 3 threads will run three different index at the same time. Every time a thread finishes one call (one index) it allocates the next index and runs it. The result may be: Thread 1 executes index 3,5,7, thread 2 executes 1,6,8,9 and thread 3 runs 2,4,10.
73+- The number of threads may vary during run and there is no guarantee for a minimum of threads. In the worst case all index will be executed by one thread - the thread itself.
74+- The maximum number of threads is initialized with a good guess for the current system. It can be manually set at any time via
75+
76+``` pascal
77+ProcThreadPool.MaxThreadCount := 8;
78+```
79+
80+- You can set the maximum threads for each procedure.
81+- A parallel procedure (or method) can call recursively parallel procedures (or methods).
82+- Threads are reused, that means they are not destroyed and created for each index, but there is a global pool of threads. On a dual core processor there will be two threads doing all the work - the main thread and one extra thread in the pool.
83+
84+# Overhead, slow down
85+
86+The overhead heavily depends on the system (number and types of cores, type of shared memory, speed of critical sections, cache size). Here are some general hints:
87+
88+- Each chunk of work (index) should take at least some milliseconds.
89+- The overhead is independent of recursive levels of parallel procedures.
90+
91+Multi threading overhead, which is independent of the MTProcs units, but simply results from todays computer architectures:
92+
93+- As soon as one thread is created your program becomes multi threaded and the memory managers must use critical sections, which slows down. So even if you do nothing with the thread your program might become slower.
94+- The cmem heap manager is on some systems much faster for multi threading. In my benchmarks especially on intel systems and especially under OS X the speed difference can be more than 10 times.
95+- Strings and interfaces are globally reference counted. Each access needs a critical section. Processing strings in multiple threads will therefore hardly give any speed up. Use PChars instead.
96+- Each chunk of work (index) should work on a disjunctive part of memory to avoid cross cache updates.
97+- Do not work on vast amounts of memory. On some systems one thread alone is fast enough to fill the memory bus speed. When the memory bus maximum speed is reached, any further thread will slow down instead of making it faster.
98+
99+# Setting the maximum number of threads for a procedure
100+
101+You can specify the maximum number of threads for a procedure as the fifth parameter.
102+
103+``` pascal
104+begin
105+ ProcThreadPool.DoParallel(@DoSomethingParallel,1,10,nil,2); // address, startindex, endindex,
106+ // optional: data (here: nil), optional: maximum number of threads (here: 2)
107+end.
108+```
109+
110+This can be useful, when the threads work on the same data, and too many threads will create so many cache conflicts, that they slow down each other. Or when the algorithm uses a lot of WaitForIndex, so that only a few threads can actually work. Then the threads can be used for other tasks.
111+
112+# Wait for index / executing in order
113+
114+Sometimes an Index depends on the result of a former Index. For example a common task is to first compute chunk 5 and then combine the result with the result of chunk 3. Use the WaitForIndex method for that:
115+
116+``` pascal
117+procedure DoSomethingParallel(Index: PtrInt; Data: Pointer; Item: TMultiThreadProcItem);
118+begin
119+ ... compute chunk number 'Index' ...
120+ if Index=5 then begin
121+ if not Item.WaitForIndex(3) then exit;
122+ ... compute ...
123+ end;
124+end;
125+```
126+
127+WaitForIndex takes as an argument an Index that is lower than the current Index or a range. If it returns true, everything worked as expected. If it returns false, then an exception happened in one of the other threads.
128+
129+There is an extended function WaitForIndexRange that waits for whole range of Index:
130+
131+``` pascal
132+if not Item.WaitForIndexRange(3,5) then exit; // wait for 3,4 and 5
133+```
134+
135+# Exceptions
136+
137+If an exception occur in one of the threads, the other threads will finish normally, but will not start a new Index. The pool waits for all threads to finish and will then raise the exception. That's why you can use *try..except* like always:
138+
139+``` pascal
140+try
141+ ...
142+ ProcThreadPool.DoParallel(...);
143+ ...
144+except
145+ On E: Exception do ...
146+end;
147+```
148+
149+If there are multiple exceptions, only the first exception will be raised. To handle all exceptions, add a try..except inside your parallel method.
150+
151+# Synchronize
152+
153+When you want to call a function in the main thread, for example to update some gui element, you can use the class method *TThread.Synchronize*. It takes as arguments the current TThread and the address of a method. Since 1.2 mtprocs provides a threadvar **CurrentThread**, which makes synchronizing simple:
154+
155+``` pascal
156+TThread.Synchronize(CurrentThread,@YourMethod);
157+```
158+
159+This will post an event on the main event queue and wait until the main thread has executed your method. Keep in mind that a bare fpc program does not have an event queue. A LCL or fpgui program has it.
160+
161+If you create your own TThread descendants, you should set the variable in your Execute method. For example:
162+
163+``` pascal
164+procedure TYourThread.Execute;
165+begin
166+ CurrentThread:=Self;
167+ ...work...
168+end;
169+```
170+
171+# Example: Parallel loop
172+
173+This example explains step by step how to convert a loop into a parallel procedure. The example computes the maximum number of an integer array *BigArray*.
174+
175+## The original loop
176+
177+``` pascal
178+type
179+ TArrayOfInteger = array of integer;
180+
181+function FindMaximum(BigArray: TArrayOfInteger): integer;
182+var
183+ i: PtrInt;
184+begin
185+ Result:=BigArray[0];
186+ for i:=1 to length(BigArray)-1 do begin
187+ if Result<BigArray[i] then Result:=BigArray[i];
188+ end;
189+end;
190+```
191+
192+## Splitting the work
193+
194+The work should be equally distributed over n threads. For this the BigArray is split into equally sized blocks and an outer loop runs over every block. Typically n is the number of cpus/cores in the system. MTProcs has some utility functions to compute the block size and count:
195+
196+``` pascal
197+function FindMaximum(BigArray: TArrayOfInteger): integer;
198+var
199+ BlockCount, BlockSize: PtrInt;
200+ i: PtrInt;
201+ Index: PtrInt;
202+ BlockStart, BlockEnd: PtrInt;
203+begin
204+ Result:=BigArray[0];
205+ ProcThreadPool.CalcBlockSize(length(BigArray),BlockCount,BlockSize);
206+ for Index:=0 to BlockCount-1 do begin
207+ Item.CalcBlock(Index,BlockSize,length(BigArray),BlockStart,BlockEnd);
208+ for i:=BlockStart to BlockEnd do begin
209+ if Result<BigArray[i] then Result:=BigArray[i];
210+ end;
211+ end;
212+end;
213+```
214+
215+The added lines can be used for any loop. Eventually a tool can be written to automate this.
216+
217+The work is now split into smaller pieces. Now the pieces must become more independent.
218+
219+## Local and shared variables
220+
221+For each used variable in the loop you must decide whether it is shared variable used by all threads or if each thread uses its own local variable. The shared variables BlockCount and BlockSize are only read and do not change, so no work is needed for them. But a shared variable like Result will be changed by all threads. This can either be achieved with synchronization (e.g. critical section), which is slow, or each thread use a local copy and these local variable are later combined.
222+
223+Here is a solution replacing the *Result* variable with an array, which is combined in the end:
224+
225+``` pascal
226+function FindMaximum(BigArray: TArrayOfInteger): integer;
227+var
228+ // shared variables
229+ BlockCount, BlockSize: PtrInt;
230+ BlockMax: PPtrInt;
231+ // local variables
232+ i: PtrInt;
233+ Index: PtrInt;
234+ BlockStart, BlockEnd: PtrInt;
235+begin
236+ ProcThreadPool.CalcBlockSize(length(BigArray),BlockCount,BlockSize);
237+ BlockMax:=AllocMem(BlockCount*SizeOf(PtrInt)); // allocate space for local variables
238+ // compute maximum for each block
239+ for Index:=0 to BlockCount-1 do begin
240+ // compute maximum of block
241+ Item.CalcBlock(Index,BlockSize,length(BigArray),BlockStart,BlockEnd);
242+ BlockMax[Index]:=BigArray[BlockStart];
243+ for i:=BlockStart to BlockEnd do begin
244+ if BlockMax[Index]<BigArray[i] then BlockMax[Index]:=BigArray[i];
245+ end;
246+ end;
247+ // compute maximum of all blocks
248+ // (if you have hundreds of threads there are better solutions)
249+ Result:=BlockMax[0];
250+ for Index:=1 to BlockCount-1 do
251+ Result:=Max(Result,BlockMax[Index]);
252+
253+ FreeMem(BlockMax);
254+end;
255+```
256+
257+This approach is straightforward and could be automated. The process will need some hints from the programmer though.
258+
259+## DoParallel
260+
261+The final step is to move the inner loop into a sub procedure and replace the loop with a call to DoParallelNested.
262+
263+``` pascal
264+...
265+{$ModeSwitch nestedprocvars}
266+uses mtprocs;
267+...
268+function TMainForm.FindMaximum(BigArray: TArrayOfInteger): integer;
269+var
270+ BlockCount, BlockSize: PtrInt;
271+ BlockMax: PPtrInt;
272+
273+ procedure FindMaximumParallel(Index: PtrInt; Data: Pointer;
274+ Item: TMultiThreadProcItem);
275+ var
276+ i: integer;
277+ BlockStart, BlockEnd: PtrInt;
278+ begin
279+ // compute maximum of block
280+ Item.CalcBlock(Index,BlockSize,length(BigArray),BlockStart,BlockEnd);
281+ BlockMax[Index]:=BigArray[BlockStart];
282+ for i:=BlockStart to BlockEnd do
283+ if BlockMax[Index]<BigArray[i] then BlockMax[Index]:=BigArray[i];
284+ end;
285+var
286+ Index: PtrInt;
287+begin
288+ // split work into equally sized blocks
289+ ProcThreadPool.CalcBlockSize(length(BigArray),BlockCount,BlockSize);
290+ // allocate local/thread variables
291+ BlockMax:=AllocMem(BlockCount*SizeOf(PtrInt));
292+ // compute maximum for each block
293+ ProcThreadPool.DoParallelNested(@FindMaximumParallel,0,BlockCount-1);
294+ // compute maximum of all blocks
295+ Result:=BlockMax[0];
296+ for Index:=1 to BlockCount-1 do
297+ Result:=Max(Result,BlockMax[Index]);
298+ FreeMem(BlockMax);
299+end;
300+```
301+
302+This was mostly copy and paste, so again this could be automated.
303+
304+# Example: parallel sort
305+
306+The unit mtputils contains the function **ParallelSortFPList** which uses mtprocs to sort a TFPList in parallel. A compare function must be given.
307+
308+``` pascal
309+procedure ParallelSortFPList(List: TFPList; const Compare: TListSortCompare; MaxThreadCount: integer = 0; const OnSortPart: TSortPartEvent = nil);
310+```
311+
312+This function uses the parallel MergeSort algorithm. The parameter MaxThreadCount is passed to DoParallel. A 0 means to use the system default.
313+
314+Optionally you can provide your own sort function (OnSortPart) to sort the part of each single thread. For example you can sort the blocks via QuickSort, which are then merged. Then you have a **parallel QuickSort**. See TFPList.Sort for an example implementation of QuickSort.
315+
316+# Using nested procedure
317+
318+``` pascal
319+procedure DoSomething(Value: PtrInt);
320+var
321+ p: array[1..2] of Pointer;
322+
323+ procedure SubProc(Index: PtrInt; Data: Pointer; Item: TMultiThreadProcItem);
324+ begin
325+ p[Index]:=Pointer(Value); // accessing local variables and parameters is possible!
326+ end;
327+
328+var
329+ i: Integer;
330+begin
331+ ProcThreadPool.DoParallelNested(@SubProc,1,2);
332+end;
333+```
334+
335+This can save a lot of refactoring and makes the parallel procedure much more readable.
336+
337+# See also
338+
339+- [[Multithreaded Application Tutorial]]
340+- [[OpenCL]]
341+
342+---
343+*From the Free Pascal wiki, page "Parallel procedures", as it stood on 2020-02-10 after a change by Andry, out of the wiki's dump of 2024-05-26. Copyright stays with the author who wrote it, according to the old wiki's [[Licensing]] page. If you have concerns about this material, [message the admin](/wiki-contact).*

Back to history