I'am currently thinking about implementing OpenMP support in FPC. However, there is currently (to my knowledge) no pascal syntax defined for OpenMp support. Do you think we can find a common syntax to simplify things for users? I've some ideas how it be done, but I want to hear other ideas first so they are maybe better if they aren't influenced by my ideas :)

I started also a wiki page about it
http://www.freepascal.org/wiki/index.php/OpenMP_support where ideas can be written down and shared.

I'am currently thinking about implementing OpenMP support in FPC.

There seems to be something in the air.
I had the same idea two weeks ago. Well, to be honest, I had the idea as I heard of OpenMP 3 years ago, but now I want to finally do it. I want to first write a tutorial for MPICH for FPC and then take a closer look on OpenMP, to get used to it and find out it advantages and disadvantages. We don't need to make the same errors as the C and Fortran compilers.

However, there is currently (to my knowledge) no pascal syntax defined for OpenMp support.

Right. AFAIK OpenMP is only a standard for C/C++ and Fortran. It's not an implementation.

As soon as I done some more OpenMP experiments and made up my mind, I will join. :)

Mattias

2006/7/17, Mattias Gaertner :

If ever it needed an implementation (which I'm glad to hear it is not), maybe a section should be used like with asm (assembler) code. At first glance (2 minutes) I was not able to grasp the big thing about that; are we not yet able to do multi-threaded programming in FPC?

Yes, we are.
But creating and synchronizing threads needs often a lot of overhead, measured in lines of code and in performance.
OpenMP is a specification, how to parallize code blocks easily and give the compiler enough hints to optimize the code.
It is up to compiler developers to support it and provide an implementation. Probably we can not name ours "OpenMP".

Mattias

Florian Klaempfl wrote

I'am currently thinking about implementing OpenMP support in FPC.

Is this similar as polyphinic C# ? (you gave me a link a while ago)

What I'm puzzeled with, usually an API specifies the interface to an external library, where here it seems a spec how a language should "behave" when implementing Multi Processing (and in this case for C++ and Fortran).

Is the idea to implement a similar, based on the ideas of openMP, version for FPC?

Marc

The only suggestion that I have may be not very useful and may be very obvious - but here goes anyway. Please make the interface as simple (read "high level") as possible, providing high level wrappers for low level routines if necessary. In many cases, writing Pascal interfaces to C libraries often produces something that is no easier to use than the original C interface, and for which one might as well use gcc.

Best regards, The Chief
--------
Prof. Abimbola A. Olowofoyeku (The African Chief)
web: http://www.greatchief.plus.com/

I would suggest something along the lines of the C/C++ implementation, but using the Pascal form of compiler directives.

Using some of the documented examples in the v2.5 spec:

Example A.1.1:
procedure a1(n: Integer; a: PSingleArray; b: PSingleArray);
var
  i: Integer;
begin
  {$omp parallel for}
  for i := 1 to n - 1 do
    b^[i] := (a^[i] + a^[i - 1]) / 2.0;
end;
Example A.5.1:
uses omp;
begin
  omp_set_dynamic(1);
  {$omp parallel num_threads(10)}
  begin
    (* Do work here *)
  end;
end;
Example A.13.1:
interface
function dequeue(var a: Single): Integer;
procedure work(i: Integer; var a: Single);
implementation
procedure a13(var x: Single; var y: Single);
var
  ix_next, iy_next: Integer;
begin
  {$omp parallel shared(x, y) private(ix_next, iy_next)}
  begin
    {$omp critical (xaxis)}
      ix_next := dequeue(x);
    work(ix_next, x);
    {$omp critical (yaxis)}
      iy_next := dequeue(y);
    work(iy_next, y);
  end;
end;

2006/7/20, Steve Williams :

I fear this just looks like a terrible stranger-outsider pach to a superbe language. :) If fpc peoples want to implement new language syntax... may it be pascal-like and good looking. At least in the previous example, a closing block would be required... as for myself:

{$omp parallel for}
...
[$omp end}

The same for the other examples. So if the directive is not understood by the compiler, it becomes standard code without any problems. For me, the best approach would be an unit to be used; not language implementation. (In that unit you could have ugly compiler magic if you want, but at least, when using this technology, it would be much more like using a standard class or set of procedures.)

Any-way; Reagards.

Isn't there a copascal that already has established concurent pascal syntax?

if not, the other wirthian languages look like logical providers?

Marco van de Voort wrote

Isn't there a copascal that already has established concurent pascal syntax?

Yes, there is, but its syntax is very limited AFAICS.

if not, the other wirthian languages look like logical providers?

Ada tasking? Well, too much overkill, I think. ;)

My problem with the OpenMP stuff is that it's more C-centric than concurrency-centric: Basically they give a compiler "hint"s that some sequential statement could be executed concurrently.

A concurrent language goes a different way, there you would simply instantiate the assignment multiple times whether its like "cobegin" in ConcurrentPascal, "PAR" in Occam2, or "task" types in Ada.

So at first the question should be answered if OpenMP in FreePascal should be implemented on language (means: special syntax) level or with compiler directives more similar to the C-version.

Vinzent.

I'am currently thinking about implementing OpenMP support in FPC.

Florian,

Have you looked at Pascal-FC (a language developped based Pascal/0, I believe, by Alan Burns)? I've used it to teach multiprogramming, and it incorporates many of the items which are available in several other MP languages (such as ADA). It's probably too basic, but there may be some useful ideas.

Is _is_ very clean though. No {$xxx's etc...

John

Alan Burns? That's a name which rings a bell. You could have send the URL, though. ;)

http://www-users.cs.york.ac.uk/~burns/pf.html

Hmm, and taking a peek look at the examples, it doesn't really surprise me, that this just looks like the Pascal version of Ada's tasking.

Regards,

Vinzent.

Brrr.... using local defines look not native to the language. Why not something like as (refered in another thread) pascal-fc which uses cobegin..coend or known blocks like asm..end; try..end;

for instance:
  omp..end;
  parralel..end;

so:

procedure a1(n: Integer; a: PSingleArray; b: PSingleArray);
var
  i: Integer;
begin
  for i := 1 to n - 1 do
  parallel
    b^[i] := (a^[i] + a^[i - 1]) / 2.0;
  end;
end;

  uses omp;
  begin
   omp_set_dynamic(1);
//  {$omp parallel num_threads(10)}
// why not:
   omp_set_num_threads(10);
   parallel
     (* Do work here *)
   end;
  end;

  procedure a13(var ax: Single; var ay: Single);
  begin
   parallel
   shared
     x: Single; absolute ax;
     y: Single; absolute ay;
   private
     ix_next, iy_next: Integer;
   begin
     // where does xaxis come from ?
     // {$omp critical (xaxis)}
     ix_next := dequeue(x);
     work(ix_next, x);
     // where does yaxis come from ?
     // {$omp critical (yaxis)}
     iy_next := dequeue(y);
     work(iy_next, y);
   end;
  end;

or:

  procedure a13(var ax: Single; shared var y: Single);
  shared var
    x: Single; absolute ax;
// y: Single; absolute ay;
  private var
    ix_next, iy_next: Integer;
  begin
   parallel
     // where does xaxis come from ?
     // {$omp critical (xaxis)}
     ix_next := dequeue(x);
     work(ix_next, x);
     // where does yaxis come from ?
     // {$omp critical (yaxis)}
     iy_next := dequeue(y);
     work(iy_next, y);
   end;
  end;

Marc

"Not native". Well, I would have prefered the term "darn ugly". :)

Yes, this idea looks *much* nicer. And much more Pascalish.

Vinzent.

One of the pretexts behind OpenMP is that the code will still compile if OpenMP is not available or disabled on a particular compiler. That's the reason behind using compiler directives instead of new keywords. Using compiler directives similar to the C/C++ directives also means that the wealth of information already out there is instantly applicable to the Pascal version.

Hi Vinzent.

Yes - quite famous.

But apart from the ADA style resources, it also implement a number of other tools, such as channels/remote invocation, etc. The cobegin/coend pair to specify concurrency is elegant, though maybe somewhat 'flat'.

John

Mwah... in that case you can still use the same keywords, only it won't be much parallel, since it's executed in one thread. And beeing executed in one or more threads should not matter in parallel blocks.

Marc

That's the reason behind using compiler directives instead of new keywords. Using compiler directives similar to the C/C++ directives also means

eehh... to repeat what someone else already mentioned in this thread, if I want to use C, I don't use pascal.

Marc.

that the wealth of information already out there is instantly applicable to the Pascal version.

But then your code is locked to that version of FPC that supports those new keywords.

Marc Weustink wrote: >> One of the pretexts behind OpenMP is that the code will still compile

if

Other compilers will give you warnings about illegal compiler directives. Therefore if your code should work with multiple compilers, you will probably enclose the omp additions with IFDEFs anyway.

{$IFDEF HasOMP}cobegin{$ENDIF}

OpenMP uses pragmas, because
- They didn't want to break ANSI C. FPC has its own mode and compatibility modes for others.
- It takes time before all the C compilers support the OMP additions. For example default gcc does not yet.
Do we plan a syntax for FPC or for pascal in general?
- C programmers are more used to macros and pragmas, than pascal programmers to compiler directives.
IMO Directives are ugly.
- C programmers are more used to hints and notes, than pascal programmers. Although recent fpc additions changed that (hints about unused/uninitialised variables).
IMO Noisy notes make it harder to find the relevant compiler messages.

Mattias

> new keywords. Other compilers will give you warnings about illegal compiler directives.

Bad assumption, the only one that matters, Delphi errors on unknown compiler directives. So you will have to ifdef anyway. (tested D6)

And then I prefer the clean syntax.

Yes, he borrowed from quite some concurrent languages (the semaphore could be Modula, and the channels maybe occam, I'm not sure). It still looks very much like Ada to me, especially the remote invocation just looks like Ada's rendezvous. Well, he calls it process, not task - just like in VHDL - but apart from that it looks very similar.

The cobegin/coend pair to specify concurrency is elegant, though maybe somewhat 'flat'.

What do you mean by 'flat'? That it can't be nested? If I read the OpenMP specs correctly, they say there, that they don't expect vendors to implement nested stuff real soon anyway. :)

Yet, I don't see a compelling reason - apart from the question how to implement it - why a nice new syntax couldn't nest "cobegin"s just like we already do with "begin"s:

cobegin
    Do_This;
    Do_That;
    cobegin
      Do_More;
      Do_Much_More;
    coend;
coend;

Not sure of that would work. ;)

Well, I guess, I have to read the specs more thoroughly, perhaps I come up with an idea, how all this stuff can be mapped nicely to this ugly C hack. They even state, that they are so much more special an better compared with all the existing concurrent languages, that I'd like to prove them otherwise. ;)

Vinzent.

Does Delphi support FPC style macros? In that case, perhaps as sort of
include file to map "cobegin" or whatever to normal ones ...

And then I prefer the clean syntax.

Yes, absolutely.

Vinzent.

No. There is not much that can be done about this. Except postprocessing FPC source and stripping out paralel syntax. Be it directives or keywords.

In that case directives don't make any sense like they'd do in a C-version.

So it'll have to be syntax. :)

Vinzent.

Ok. Beaten. :)

But at least it provoked discussion on it.

I think at least in this case GPC also matters, since Florian is also soliciting feedback on the syntax on the GPC list (and code written for parallel calculations can probably be easily made compatible with multiple Pascal dialects). I don't know what they do with unknown compiler directives though.

Jonas

I meant more that GPC also still can implement the new syntax. This will be a bit harder with Delphi :-)

Florian Klaempfl wrote

I started also a wiki page about it http://www.freepascal.org/wiki/index.php/OpenMP_support where ideas can be written down and shared.

Well, I just added some stuff there, yesterday. It's far from being complete yet (it just covers a basic "parallel" construct), nor is it really thought through yet, but well, it might be a start; something to begin with.

Any suggestions are welcome, of course.

Due to the fact, that the OpenMP specification is neither the easiest one to read or understand, and it's not particularly tiny, neither, I don't want to put hours and hours of then wasted effort into this, if everyone here thinks that this idea is total crap.

And yes, I admit, I like code being a little bit more verbose. :)

Vinzent.

Does parallel mean all the statements in the block can be executed in parallel, or that multiple copies of the block of statements can be started in parallel ?

Micha

The latter.

Vinzent.

Which is why I think that it's better to have them as local functions, instead of having to introduce a lot of new functions.

Local functions are very pascal-ish. C doesn't have it, which is why they can't use it. Let's use the language features to their full extent.

Michael.

Strange. How many copies ?

That somehow applies to the whole original specification. :)

How many copies ?

Omp.Get_Num_Threads(), AFAICS.

Vinzent.

Looking at the example, I would say as many copies as there are threads: OMP.Get_Thread_Num

Vincent

Vinzent Hoefler wrote

On Wednesday 26 July 2006 09:07, Micha Nelissen wrote: > How many copies ? Omp.Get_Num_Threads(), AFAICS.

Ah the number of threads is determined by the RTL, and any parallel block must be written flexible, so that it can work for any given number of threads ?

Micha

Hey, it's only *one* new keyword yet. :)

Admitted, local blocks are more Adaish, but with this approach, I was trying to avoid introducing a lot more keywords. There still probably will be some, because there are more constructs like "barrier", "workshare" (which I didn't fully understand yet) and "section".

And well, such blocks are already known in Object Pascal. Just look at them as in object declarations the public/protected/private modifiers.

Local functions are very pascal-ish. C doesn't have it, which is why they can't use it. Let's use the language features to their full extent.

Yes. I agree with that. The problem is that there are more "variable" types than just shared and private (like "firstprivate"), so it might prove to be quite difficult to do that if you only have local functions. I just didn't come to that part yet.

I'll probably update the WiKi this evening/night (can't do that from work, where freepascal.org still times out).

Vinzent.

  *procedure* SubDomain (*var* x       : *array of* Float;
                           istart  : Integer;
                           ipoints : Integer)
  *var*
     i : Integer;
  *begin*
     *for* i := 0 *to* ipoints - 1 *do*
        x[istart + i] := 123.456;
  *end* /{SubDomain}/;
  *procedure* Sub (*var* x : *array of* Float);
  /// Variables declared here should have /|*shared*|/ context./
  /// This would include the function's parameters then.../
    *procedure* SubTask(*var* x : *array of* Float);
    *var*
      /// Variables declared here have /|*private*|/ context./
      iam     : Integer;
      nt      : Integer;
      ipoints : Integer;
    *parallel*
      iam := OMP.Get_Thread_Num;  /// OMP library calls./
      nt  := OMP.Get_Num_Threads;
      ipoints := Length (x) *div* nt; /// size of partition/
      istart  := iam * ipoints;     /// starting array index/
      *if* iam = Pred (nt) *then*
        ipoints := Length (x) - istart; /// last thread may do more/
      SubDomain (x, istart, ipoints);
    *end*;
  *begin*
    SubTask(x);
  *end* /{Sub}/;
  *var*
     arr = *array*[0 .. 9999] *of* Float;
  *begin* / // Main program/
     Sub (arr);
  *end*.

Damn Thunderbird.

Yes and no. I think, there are constructs to set a specific number of threads. And as I tried to say, the specification is quite large. If not to say even convoluted. I've seen easier to read stuff recently.

Vinzent.

I added an example with nested procedures.

Vincent

Michael Van Canneyt wrote

> The latter. Which is why I think that it's better to have them as local functions, instead of having to introduce a lot of new functions.

There is no real reason to restrict 'parallel' to local functions, is there ?

Micha

Well, it tried to mimic my syntax highlighting. :) Apart from that, your change is quite nice, yes. That's what Michael meant, I guess.

Vinzent.

No, but I used that because in the example you make use of variables defined in the local function, but also in the parent function.

It seems obvious to me that a global function can be called in parallel at any time. The compiler can perfectly detect whether a global function writes to variables outside it's own scope, in which case it's probably a no-no to paralellize the function. The programmer could help there by adding the parallel keyword indicating that the programmer knows it is safe to parallelize it.

For local functions it is less obvious, there the compiler could use some help in most cases...

Michael.

Michael Van Canneyt wrote

It seems obvious to me that a global function can be called in parallel at any time. The compiler can perfectly detect whether a global function writes to variables outside it's own scope, in which case it's probably a no-no to paralellize the function.

Hey, you're trying to put more burden on the compiler here as the spec even allows, I'd say.

Don't go on or you'll end up with the requirement for compile time dead-lock detection. :)

Vinzent.

Well, my statement is moderate in the sense that the compiler does this anyway already (see the hints/warnings about unused params/vars); let's use this information as much as we can. To avoid more difficult detections such as possible dead-locks, I would let the programmer add some keywords...

Michael.

Wouldn't it be better to write it like this:
    procedure SubTask(var x: array of Float); *parallel*;
    var
       ...
    begin
       ...
    end;

True. Using information that's already there, can't be wrong. I second that.

We're not C, where everything is standardized to be the programmer's problem. So yes, if concurrency will ever be implemented in the language, no matter how, some decent warnings/hints/notes about several questionable constructs would be nice.

Vinzent.

Actually no. I thought about it, but I didn't get through all that stuff yesterday evening or else I would have updated the WiKi already. The problem I see is that the parallel directive has more meanings (basically it is the main directive). For that reason, I wouldn't want to put in on the callee, but rather on the caller.

I think it would also be easier for the compiler then to detect alls those parallel regions.

Let's see. Anyway, there was already more response I would have expected after the days of silence. :)

Vinzent.

2006/7/27, Vinzent Hoefler :

What about the same approach as the class procedures?

Actually we have in a class the possibility to:
class procedure MyProc;
class function MyFunc: Integer;
Then we could very simply have:
parallel procedure ParallelBlock;
parallel function ParallelFunction; //if this can happen...

Also, I read that in example 1: "Variables declared here should have shared context." In fact by default they have private context and to me it looks like more normal. Maybe we should have a way to do the contrary: specify that a variable has shared context... What others think?

Alexandre Leclerc wrote

Then we could very simply have: parallel procedure ParallelBlock; parallel function ParallelFunction; //if this can happen...

Yes. I thought of something like that, because it could quite easily match with a "parallel for" construct. That's why I don't like the idea of a function modifier.

But pleeeaaase, people. This was only one single and little example and it's not nearly close to what the spec says. It just scratched the surface of it. So it's a bit early to hang on this only parallel keyword in that particular example. In the end, *all* constructs should nicely match together.

What about those "parallel section" and "parallel workshare" constructs? The latter bothers me a lot (section might prove to be quite easy) plus all the /parameters/ those stuff can have:

You don't want to write

|parallel (Num_Threads := 3, ...)
|function

or something similar, do you?

Or think of this "reduction" keyword...

Also, I read that in example 1: "Variables declared here should have shared context."

Yes, if they're declared outside of the parallel block, they can be seen by every single one, so "shared" would IMO be a more natural visibility rule here. This could remove the need for another keyword.

In fact by default they have private context and to me it looks like more normal.

What do you mean "by default"? The OpenMP spec? Well, I wasn't trying to copy the idiocies mainly caused by the chosen base language(s). :->

Thread private variables don't make sense outside of the parallel block, especially *after* it, so why should the default be "private" here?

Maybe we should have a way to do the contrary: specify that a variable has shared context... What others think?

The approach allowed both in a (I think, logical way, because of the implied scoping rules). The problems begin with the "firstprivate" and such directives.

Vinzent.