OpenMP support: what changed

Revision 2 to 3. Removed lines have a minus sign; added lines have a plus sign.

Old	New		Line
1	1		## What is OpenMP?
2	2
3	3		OpenMP is an API accessed by language directives to do multi threaded programming, see also <http://www.openmp.org>. Currently, there is only OpenMP syntax defined for C and Fortran. This page tries to collect some stuff to settle down pascal syntax for it.
4	4
5	5		## Pascal syntax for OpenMP
6	6
7	7		### Proposal 1
8	8
9	9		#### Foreword
10	10
11	11		At first, I must admit that some parts of the OpenMP specification I still don't understand. They did a terrible good job throwing away all common terms ever used in multi threading context, and invented their own ones.
12	12
13	13		#### Syntax vs. Compiler directives
14	14
15	15		OpenMP for C and C++ is implemented by using compiler directives mainly due to the reasons of source code compatibility (or: standards compliance). So a conforming program is intended to behave the same regardless if the actual compiler compiling the program supports those special pragmas or not.
16	16
17	17		For FreePascal I don't think this is the way to go, because first it changes comments into code and second, it makes the program far less readable. For C programs this doesn't seem to be an issue, if you get my meaning. But in my opinion, readability is a far more important issue than compatibility to older/different compilers. If all else fails, a preprocessor could be provided to strip out the parallel specific stuff, as has been suggested by Marco. Note that you would need this preprocessor for the directives too because older FPCs and Delphi don't skip unknown directives.
18	18
19	19		Well, enough talk, I start with the easier directives which are luckily the more fundamental ones.
20	20
21	21		Ok, I got more input than I'd expected and less time than I wished. :) Anyway, against my own objection, the idea of enclosing the parallel code into (local) functions looks very appealing, so I've changed the example accordingly.
22	22
23	23		#### parallel
24	24
25	25		The `parallel` construct can only be used for a structured block. That means in Pascal it should be enclosed in some sort of `begin`/`end` pair anyway - so, as it has been suggested, we could use a (in this particular example non-local) function instead. Though, I don't know yet, if this may bite with other parts of the spec as this is evolving. Let's try:
26	26
27	27		(Original example A.4.1.c of the OpenMP V2.5 specification):
28	28
29		-	~~` ``procedure`` SubDomain (``var`` x : ``array of`` Float;`\~~
30		-	~~` istart : Integer;`\~~
31		-	~~` ipoints : Integer)`\~~
32		-	~~` ``var`\~~
33		-	~~` i : Integer;`\~~
34		-	~~` ``begin`\~~
35		-	~~` ``for`` i := 0 ``to`` ipoints - 1 ``do`\~~
36		-	~~` x[istart + i] := 123.456;`\~~
37		-	~~` ``end`` ``{SubDomain}``;`\~~
38		-	~~` `\~~
39		-	~~` ``parallel`` ``procedure`` Sub (``var`` x : ``array of`` Float);`\~~
40		-	~~` ``// Variables declared here have ``private`` context.`\~~
41		-	~~` ``// So each instance of the parallel function has its own set, as usual.`\~~
42		-	~~` ``var`\~~
43		-	~~` iam : Integer;`\~~
44		-	~~` nt : Integer;`\~~
45		-	~~` ipoints : Integer;`\~~
46		-	~~` ``// Any variable access outside of the function's scope accesses the variable in`\~~
47		-	~~` ``// a ``shared`` context.`\~~
48		-	~~` ``// This might prove problematic, especially because it causes special semantics`\~~
49		-	~~` ``// on the function's parameters, probably depending on the parameter mode or worse:`\~~
50		-	~~` ``// On the calling convention actually used (call-by-value vs. call-by-reference).`\~~
51		-	~~` ``begin`` ``// of (possibly) parallel section`\~~
52		-	~~` iam := OMP.Get_Thread_Num; ``// OMP library calls.`\~~
53		-	~~` nt := OMP.Get_Num_Threads;`\~~
54		-	~~` `\~~
55		-	~~` ipoints := Length (x) ``div`` nt; ``// size of partition`\~~
56		-	~~` istart := iam * ipoints; ``// starting array index`\~~
57		-	~~` `\~~
58		-	~~` ``if`` iam = Pred (nt) ``then`\~~
59		-	~~` ipoints := Length (x) - istart; ``// last thread may do more`\~~
60		-	~~` `\~~
61		-	~~` SubDomain (x, istart, ipoints);`\~~
62		-	~~` ``end`` ``{Sub}``;`\~~
63		-	~~` `\~~
64		-	~~` ``var`\~~
65		-	~~` arr : ``array``[0 .. 9999] ``of`` Float;`\~~
66		-	~~` ``begin`` `` // Main program`\~~
67		-	~~` Sub (arr);`\~~
68		-	~~` ``end``.`~~
	29	+	``` pascal
	30	+	<b>procedure</b> SubDomain (<b>var</b> x : <b>array of</b> Float;
	31	+	istart : Integer;
	32	+	ipoints : Integer)
	33	+	<b>var</b>
	34	+	i : Integer;
	35	+	<b>begin</b>
	36	+	<b>for</b> i := 0 <b>to</b> ipoints - 1 <b>do</b>
	37	+	x[istart + i] := 123.456;
	38	+	<b>end</b> <i>{SubDomain}</i>;
	39	+
	40	+	<b>parallel</b> <b>procedure</b> Sub (<b>var</b> x : <b>array of</b> Float);
	41	+	<i>// Variables declared here have </i><b>private</b><i> context.</i>
	42	+	<i>// So each instance of the parallel function has its own set, as usual.</i>
	43	+	<b>var</b>
	44	+	iam : Integer;
	45	+	nt : Integer;
	46	+	ipoints : Integer;
	47	+	<i>// Any variable access outside of the function's scope accesses the variable in</i>
	48	+	<i>// a </i><b>shared</b><i> context.</i>
	49	+	<i>// This might prove problematic, especially because it causes special semantics</i>
	50	+	<i>// on the function's parameters, probably depending on the parameter mode or worse:</i>
	51	+	<i>// On the calling convention actually used (call-by-value vs. call-by-reference).</i>
	52	+	<b>begin</b> <i>// of (possibly) parallel section</i>
	53	+	iam := OMP.Get_Thread_Num; <i>// OMP library calls.</i>
	54	+	nt := OMP.Get_Num_Threads;
	55	+
	56	+	ipoints := Length (x) <b>div</b> nt; <i>// size of partition</i>
	57	+	istart := iam * ipoints; <i>// starting array index</i>
	58	+
	59	+	<b>if</b> iam = Pred (nt) <b>then</b>
	60	+	ipoints := Length (x) - istart; <i>// last thread may do more</i>
	61	+
	62	+	SubDomain (x, istart, ipoints);
	63	+	<b>end</b> <i>{Sub}</i>;
	64	+
	65	+	<b>var</b>
	66	+	arr : <b>array</b>[0 .. 9999] <b>of</b> Float;
	67	+	<b>begin</b> <i> // Main program</i>
	68	+	Sub (arr);
	69	+	<b>end</b>.
	70	+	```
69	71
70	72		I don't like the idea of declaring variables inside the actual statments, this looks very unpascalish. Maybe we can find a way around it. --FPK 10:22, 26 July 2006 (CEST)
71	73
72	74		I agree with Florian that this is not the way to go. Why not require all parallelizable code to be in local functions ? After all, that's almost what you are doing: declaring a local function. That would be a simple extension of the current syntax. You have access to all local variables; all you'd need is to add a parallel keyword to the local function declaration.
73	75
74	76		Ok, so what do you think about the changed example above? OpenMP really is about coarse grain parallelism, so I see indeed no strong reason, why parallel blocks shouldn't be enclosed in procedures. Parallel functions obviously do not make sense, as every thread could return its own return value, but the block calling the parallel function can only evaluate one. I would have liked the notion of a local block, though (I'm quite used to it), but as I seem to be the only one... --V.hoefler 21:03, 27 July 2006 (CEST)
75	77
76	78		How would a try/finally type blocking approach work? (No begin needed) Like,
77	79
78		-	~~` ``Parellel`\~~
79		-	~~` SomeCodeHere;`\~~
80		-	~~` SomeCodeHereToo;`\~~
81		-	~~` ``End``; ``{ Parellel Block }`~~
	80	+	``` pascal
	81	+	<b>Parellel</b>
	82	+	SomeCodeHere;
	83	+	SomeCodeHereToo;
	84	+	<b>End</b>; <i>{ Parellel Block }</i>
	85	+	```
82	86
83	87		Very simple, just as easy when we first discovered Exception handling. --Raid 19:11, 1 September 2009 (CEST)
84	88
85	89		#### parallel for
86	90
87	91		This is simply a parallel for-loop. There's nothing special to it. Although OMP2.5 states a for-loop iteration variable is `private` in that construct, which I consider rather redundant, I hardly can imagine correctly behaving code with a `shared` loop iteration variable. It also places some restrictions onto the allowed loop-statements (no change of iteration variable inside the loop, simple iteration constructs, ...), but these are already implemented in the language, so there's no need to elaborate on that much further.
88	92
89	93		(Example A.1.1c):
90	94
91		-	~~` ``procedure`` a1 ( n : Integer;`\~~
92		-	~~` ``const`` a : ``array`` ``of`` Float;`\~~
93		-	~~` ``var`` b : ``array`` ``of`` Float);`\~~
94		-	~~` ``var`\~~
95		-	~~` i : Integer;`\~~
96		-	~~` ``begin`\~~
97		-	~~` ``parallel`` ``for`` i := Succ (Low (a)) ``to`` High (a) ``do`\~~
98		-	~~` b[i] := (a[i] + a[i - 1]) / 2.0;`\~~
99		-	~~` ``end`` ``{a1}``;`~~
	95	+	``` pascal
	96	+	<b>procedure</b> a1 ( n : Integer;
	97	+	<b>const</b> a : <b>array</b> <b>of</b> Float;
	98	+	<b>var</b> b : <b>array</b> <b>of</b> Float);
	99	+	<b>var</b>
	100	+	i : Integer;
	101	+	<b>begin</b>
	102	+	<b>parallel</b> <b>for</b> i := Succ (Low (a)) <b>to</b> High (a) <b>do</b>
	103	+	b[i] := (a[i] + a[i - 1]) / 2.0;
	104	+	<b>end</b> <i>{a1}</i>;
	105	+	```
100	106
101	107		That's it. Now probably someone sees the reason why I wouldn't use the `parallel` keyword as function modifier like `inline` or `cdecl` are used, but rather prepend it to the function header itself. I think, it's a more consistent usage of a new keyword. -- V.hoefler 21:17, 27 July 2006 (CEST)
102	108
103	109		#### data sharing attributes
104	110
105	111		To me these seem quite complex constructs considering that most of the time you probably won't need it at all, because the default is fine and follows normal programming logic. So if anyone has an idea, if and why we need to support them explicitely, here's the place.
106	112
107	113		##### threadprivate
108	114
109	115		This attribute closely resembles, what FreePascal already knows as `threadvar`, so I even think, we can reuse this keyword here. I see some semantic issue though:
110	116
111	117		The OMP2.5 specification states:
112	118
113	119		> The values of the data in the threadprivate objects of threads other than the initial thread are guaranteed to persist between two consecutive parallel regions only if all the conditions hold:
114	120
115	121		and then follows a list of condition, which basically state, that the number of threads in both sections must be the same.
116	122
117	123		So to write some simple pseudo code to demonstrate:
118	124
119		-	~~`procedure`` Thread_Vars;`\~~
120		-	~~`threadvar`\~~
121		-	~~` Count : Integer;`\~~
122		-	~~`var`\~~
123		-	~~` i : Integer;`\~~
124		-	~~`begin`\~~
125		-	~~` Count := 0; ``// initial state`\~~
	125	+	``` pascal
	126	+	<b>procedure</b> Thread_Vars;
	127	+	<b>threadvar</b>
	128	+	Count : Integer;
	129	+	<b>var</b>
	130	+	i : Integer;
	131	+	<b>begin</b>
	132	+	Count := 0; <i>// initial state</i>
126	133
127		-	~~` ``parallel`` ``for`` i := 1 ``to`` SOME_VALUE ``do`\~~
128		-	~~` Count := Count + 1;`\~~
	134	+	<b>parallel</b> <b>for</b> i := 1 <b>to</b> SOME_VALUE <b>do</b>
	135	+	Count := Count + 1;
129	136
130		-	~~` ``// Point A`\~~
	137	+	<i>// Point A</i>
131	138
132		-	~~` ``parallel`` ``for`` i := 1 ``to`` SOME_OTHER_VALUE ``do`\~~
133		-	~~` Count := Count + 2;`\~~
	139	+	<b>parallel</b> <b>for</b> i := 1 <b>to</b> SOME_OTHER_VALUE <b>do</b>
	140	+	Count := Count + 2;
134	141
135		-	~~` ``// Point B`\~~
	142	+	<i>// Point B</i>
136	143
137		-	~~`end`` ``{Thread_Vars}``;`~~
	144	+	<b>end</b> <i>{Thread_Vars}</i>;
	145	+	```
138	146
139	147		Now, each iteration of the loop is executed in parallel, so each copy also gets its own copy of `Count`. At Point A after the loop, `Count` would equal `1`, because if each loop iteration was executed by a single thread, the incrementing operation would have happened only once (per thread). Let aside the question which copy is seen after the loop, the thing gets more interesting. What value is seen at Point B? Well, if I understood the specification correctly, the value would be `3` if and only if the actual values of the place-holding constants `SOME_VALUE` and `SOME_OTHER_VALUE` are equal. In any other case, the value of `Count` at Point B would be undefined.
140	148
141	149		### Proposal 2: Using local functions
142	150
143	151		Instead of using new block types (like parallel), it uses a nested procedure, with the parallel modifier.
144	152
145	153		I think we could use a sequential keyword for the sub() procedure. More on talk page. -- MarkMLl 12:46, 9 December 2007 (CET)
146	154
147	155		#### parallel
148	156
149		-	~~` ``procedure`` SubDomain (``var`` x : ``array of`` Float;`\~~
150		-	~~` istart : Integer;`\~~
151		-	~~` ipoints : Integer); `\~~
152		-	~~` ``var`\~~
153		-	~~` i : Integer;`\~~
154		-	~~` ``begin`\~~
155		-	~~` ``for`` i := 0 ``to`` ipoints - 1 ``do`\~~
156		-	~~` x[istart + i] := 123.456;`\~~
157		-	~~` ``end`` ``{SubDomain}``;`\~~
158		-	~~` `\~~
159		-	~~` ``procedure`` Sub (``var`` x : ``array of`` Float);`\~~
160		-	~~` `\~~
161		-	~~` ``procedure`` ParallelBlock; ``parallel``;`\~~
162		-	~~` ``var`` `\~~
163		-	~~` iam : Integer;`\~~
164		-	~~` nt : Integer;`\~~
165		-	~~` ipoints : Integer;`\~~
166		-	~~` ``begin`\~~
167		-	~~` iam := OMP.Get_Thread_Num; ``// OMP library calls.`\~~
168		-	~~` nt := OMP.Get_Num_Threads;`\~~
169		-	~~` `\~~
170		-	~~` ipoints := Length (x) ``div`` nt; ``// size of partition`\~~
171		-	~~` istart := iam * ipoints; ``// starting array index`\~~
172		-	~~` `\~~
173		-	~~` ``if`` iam = Pred (nt) ``then`\~~
174		-	~~` ipoints := Length (x) - istart; ``// last thread may do more`\~~
175		-	~~` `\~~
176		-	~~` SubDomain (x, istart, ipoints);`\~~
177		-	~~` ``end``;`\~~
	157	+	``` pascal
	158	+	<b>procedure</b> SubDomain (<b>var</b> x : <b>array of</b> Float;
	159	+	istart : Integer;
	160	+	ipoints : Integer);
	161	+	<b>var</b>
	162	+	i : Integer;
	163	+	<b>begin</b>
	164	+	<b>for</b> i := 0 <b>to</b> ipoints - 1 <b>do</b>
	165	+	x[istart + i] := 123.456;
	166	+	<b>end</b> <i>{SubDomain}</i>;
	167	+
	168	+	<b>procedure</b> Sub (<b>var</b> x : <b>array of</b> Float);
	169	+
	170	+	<b>procedure</b> ParallelBlock; <b>parallel</b>;
	171	+	<b>var</b>
	172	+	iam : Integer;
	173	+	nt : Integer;
	174	+	ipoints : Integer;
	175	+	<b>begin</b>
	176	+	iam := OMP.Get_Thread_Num; <i>// OMP library calls.</i>
	177	+	nt := OMP.Get_Num_Threads;
	178	+
	179	+	ipoints := Length (x) <b>div</b> nt; <i>// size of partition</i>
	180	+	istart := iam * ipoints; <i>// starting array index</i>
	181	+
	182	+	<b>if</b> iam = Pred (nt) <b>then</b>
	183	+	ipoints := Length (x) - istart; <i>// last thread may do more</i>
	184	+
	185	+	SubDomain (x, istart, ipoints);
	186	+	<b>end</b>;
178	187
179		-	~~` ``begin`\~~
180		-	~~` ParallelBlock;`\~~
181		-	~~` ``end`` ``{Sub}``;`\~~
182		-	~~` `\~~
183		-	~~` ``var`\~~
184		-	~~` arr : ``array``[0 .. 9999] ``of`` Float;`\~~
185		-	~~` ``begin`` `` // Main program`\~~
186		-	~~` Sub (arr);`\~~
187		-	~~` ``end``.`~~
	188	+	<b>begin</b>
	189	+	ParallelBlock;
	190	+	<b>end</b> <i>{Sub}</i>;
	191	+
	192	+	<b>var</b>
	193	+	arr : <b>array</b>[0 .. 9999] <b>of</b> Float;
	194	+	<b>begin</b> <i> // Main program</i>
	195	+	Sub (arr);
	196	+	<b>end</b>.
	197	+	```
188	198
189	199		### Proposal 3
190	200
191	201		`parallel`, `future`, and `async` keywords like implemented in the "Oxygen" Pascal dialect. Oxygen uses features of the CIL (aka ".NET") framework to implement this. (IMHO this does not qualify the paradigm of these keywords as bad.) With FP, this can be implemented in native code in the RTL.
192	202
193	203		see
194	204
195		-	~~`- `[`http://wiki.oxygenelanguage.com/en/Parallel_Loops`](http://wiki.oxygenelanguage.com/en/Parallel_Loops)\~~
196		-	~~`- `[`http://wiki.oxygenelanguage.com/en/Futures`](http://wiki.oxygenelanguage.com/en/Futures)\~~
197		-	~~`- `[`http://wiki.oxygenelanguage.com/en/Asynchronous_Statements`](http://wiki.oxygenelanguage.com/en/Asynchronous_Statements)~~
	205	+	```text
	206	+	- http://wiki.oxygenelanguage.com/en/Parallel_Loops
	207	+	- http://wiki.oxygenelanguage.com/en/Futures
	208	+	- http://wiki.oxygenelanguage.com/en/Asynchronous_Statements
	209	+	```
198	210
199	211		### Proposal 4
200	212
201	213		Benefit from the efforts of Modula-2+ and Modula-2* and maybe use (or build upon) their ideas.
202	214
203	215		### See also
204	216
205	217		- [[ThreadEvent]]
206	218		- [[Proposal of a Widget Set independent Event Queue implementation]]
207	219
208	220		---
209	221		From the Free Pascal wiki, page "OpenMP support", as it stood on 2019-04-06 after a change by Amalawi, out of the wiki's dump of 2024-05-26. Copyright stays with the author who wrote it, according to the old wiki's [[Licensing]] page. If you have concerns about this material, [message the admin](/wiki-contact).
210	222		Category: Parallel programming
211	223		Category: Proposals

Back to history