Hello,

I'd like to help with the experimental LLVM backend if anyone is interested. It's been years since I've used Pascal though, so perhaps my familiarity with C and C++ on the LLVM end will be beneficial.

While we're on the subject, LLVM has a very powerful optimizer that, when used in conjunction with Mono, speeded up execution time of calculations quite substantially. I thought I read somewhere on the roadmap of using Static Single Assignment-related optimizations being needed on FPC. LLVM does all of those optimizations quite naturally since its instruction set uses SSA throughout.

My reason for wanting to help is that I'm already involved in a project for bringing an old version of BASIC up to date from the days of the Amiga computer. The member of our team responsible for writing the runtime library is using Delphi but the parser writer and I want to run it on any LLVM-supported platform. Thus enters FPC into the scene.

If there's anything I can do to help, please reply on the list or drop me a private email.

--Sam Crow

Samuel Crow wrote

I'd like to help with the experimental LLVM backend if anyone is interested. It's been years since I've used Pascal though, so perhaps my familiarity with C and C++ on the LLVM end will be beneficial.

Thanks for the offer!

Yes, that's the main reason I started the LLVM backend.

The main things that basically halted my work on the LLVM backend are: a) FPC does not propagate type info down into the register level. There is no notion even of pointers, everything is just an integer of a particular size once it's inside a virtual register. The low level code generator routines are completely oblivious to the symbol table and high level type system (which in general is nice and helps modular design, but it's not nice in this case). It would be possible to simply typecast these integers to i8*/i16*/i32*/i64*/... when loading/ storing a value and forgetting about it for the rest, but I'm not sure to what extent that affect LLVM's ability to generate good code (and since getting better code is the whole point...). It would also be possible to implement the LLVM code generation at a higher level (bypassing the low level code generator, and directly translating the parse tree nodes into LLVM byte code), but that would both be a lot of work, and mean that any code generation fix or addition would need to be implemented twice.
b) the code generator currently does not support CPUs where the size of the integer registers is > sizeof(pointer). I've worked a bit on trying to separate the sizeof(pointer) from sizeof(integer registers), but it's not complete (the {$ifdef cpu64bitaddr} vs {$ifdef cpu64bitalu} defines in the compiler).

Finally, there are also the i386 problems
- for Delphi compatibility, FPC's default calling convention on all i386 targets is "Borland fastcall", which is different from all other i386calling conventions (see http://en.wikipedia.org/wiki/X86_calling_conventions#Borland_fastcall ). And you'd be surprised how much code is out there that depends on this (even our RTL won't work without this, a.o. the signal handling depends on this). So at least for i386, LLVM would first need support for this calling convention.
- I'm also not sure whether or not LLVM has support for the x87 extended floating type in the mean time (80 bits floating point), another requirement for our i386 port. Other cpu targets are more standard though, and should not pose such problems.

Jonas

Hello Jonas,

(Replies inline.)

Samuel Crow wrote

The current way to implement pointer arithmetic in LLVM is to use an i64 for all instances of pointers

We can't do that at the high level, since then all pointer fields in records etc would become 64 bit. And the low level currently assumes that register sizes are the same as the pointer size. I have started in the past at trying to break these assumptions, but it's not straightforward.

and then bitcast them to and from the correct pointer type when they are needed as actual pointers and then use the GetElementPointer operation to do the pointer-specific addressing modes.

Somehow, we have to integrate the high level type information in the assembler instructions though. For example when doing a function call, you have no choice but to use the real high level function types, otherwise LLVM can't know how to pass the parameters (e.g. record/ struct by value). This requires changes in all code generators for all platforms if we want to keep using the same code generator abstraction for everything (which I would very much prefer -- we currently maintain 6 different code generators with maybe 3 or 4 people, and that's only possible because so much is shared and generic).

Yes, I saw that (although I'm 8000+ mails behind on the llvm-dev list :)

That would indeed be very nice!

That's fine, as the same goes for FPC.

We have a C++ name mangler in the compiler, but it's a) barely used/tested
b) only usable for calling regular functions/procedures, not for calling methods (someone recently started some preliminary work on supporting that though)

In general, the most common way to interface with external C++ libraries is indeed via C wrappers.

Directly linking against LLVM (be it via C or C++) would also only seem advisable if that API is considered stable. One of the main pluses of FPC is that both the compiler and the generated programs are generally very much forward and backward compatible. E.g., you can run FPC and FPC-compiled binaries on any 2.4+ kernel Linux system, with no dependencies on glibc versions or anything else. Same goes for Mac OS X: the PPC compiler run on 10.3+, and the compiled binaries on 10.2.8+. For Windows it's also similar.

With an LLVM backend, there will obviously be a dependency on LLVM (either as a library or as an installed tool chain) and hence on its dependencies, but it would be nice if we could be compatible with as many different LLVM releases as possible at the same time (so that people having to stick to an older LLVM version for whatever reason can upgrade FPC independently of that). In that respect, I don't know to what extent the C++ interface of the LLVM code generator is more or less stable than the assembler format (I know that the LLVM developers reserve the right to break backwards compatibility in the assembler format only when releasing new major versions).

I don't see any problem at all with announcing that people are working on an LLVM code generation backend for FPC. I even already did so myself a long time ago on llvm-dev (http://lists.cs.uiuc.edu/pipermail/llvmdev/2007-June/009282.html -- it's quite annoying that the mailing list archive is blocked from all search engines via a robots.txt file, not sure why that is). Obviously, that work stalled at some point.

Jonas

In our previous episode, Jonas Maebe said:

Since LLVM is a specific codegen, there would be simply a bunch of llvm-<arch> targets with each their own PPC binary? So only the LLVM ones would be libc dependant.

(This sounds a bit like FPC is going to use it exclusively? Maybe make clear that that is not the case or plan?)

Yes of course, but I was saying that those compiler binaries should be compatible with an as wide selection of different LLVM versions as possible.

Jonas

Hello again Jonas,

----- Original Message ----

-snip-

The IRBuilder class http://llvm.org/doxygen/classllvm_1_1IRBuilder.html is as stable as the instruction set it builds and is maintained as such since it is used by Clang, LLVM-GCC and other frontends. As noted in other messages on the list, the LLVM-FPC compiler would be heavily dependent on LLVM and, in turn, would require newer operating system support for the compiler to run on MacOSX 10.4, for example, (primarily because of the linker and debugger situation on the earlier models' XCode versions) since that's the earliest that LLVM supports. I have a 10.3.9-based PowerMac G4 that we could test the resultant code on, but I'm not holding my breath to think that it will work.

I think the runtime library and frontend parser would be needed more than the rest of the FPC compiler itself. Most of the backend and intermediate code in the FPC optimizer would be redundant with LLVM. The reason I'm not afraid to redo most of the internals of FPC is that I am involved in a compiler project already and would need to reuse the code for that project also. Before I noticed that there was an LLVM backend already started for FPC I was going to take the parser description from the documentation and start from scratch anyway.

I'm looking forward to the challenges of this project. I hope I haven't scared you all with my brashness (and yes I am a little crazy also). I'll talk to you more later.

--Sam Crow

Samuel Crow wrote

I'm looking forward to the challenges of this project.

I am planing for another project: doing an FPC compiler for the NIOS processor (which is similar to MIPS32) on a Linux target. Do you think it's viable to consider LLVM instead of trying to modify the "native" ARM FPC code generator ?

-Michael

Samuel Crow wrote

The IRBuilder class http://llvm.org/doxygen/classllvm_1_1IRBuilder.html is as stable as the instruction set it builds and is maintained as such since it is used by Clang, LLVM-GCC and other frontends.

Ok, that would be fine then. The only problem is actually interfacing with the C++ classes then, because as mentioned that's not supported currently.

In a sense it's no problem if the LLVM backend doesn't support all targets that the other backends support, after all it'll be just another choice that people can use. I just want to avoid the LLVM backend being completely disconnected from the other backends or require an inordinate amount of maintenance to keep it in sync with the other backends, because in that case it would probably be quite short-lived (unless some dedicated maintainer would step up and constantly keep it in sync with the rest).

I think the runtime library and frontend parser would be needed more than the rest of the FPC compiler itself. Most of the backend and intermediate code in the FPC optimizer would be redundant with LLVM.

As I see it, there are basically two approaches to integrate a compiler with LLVM:
a. the LLVM-GCC approach, where you bolt an existing compiler's frontend onto llvm
b. the dragonegg approach (http://dragonegg.llvm.org/), where you make use of the support in an existing compiler (again GCC in this case) for abstract code generator support to emit LLVM assembler/bitcode rather than machine code

I am very much in favour of approach b, because I think it's the only one that can stand the test of time (in case a, you you constantly have to keep your "bolts" up-to-date as the frontend evolves).

FPC already has a fairly abstract code generator interface, because we don't use intermediate code: parse tree nodes are translated into machine code by calling the appropriate code generator class methods. The existing abstract code generator's interface is lower level than LLVM's as far as the type info is concerned though, so we need a higher level (either sitting above the current code generator, or replacing existing code generator).

As mentioned before, I do not think the end result of such an approach would be sustainable in the long term (unless all code generators would be removed in favour of the LLVM backend, but I don't see that happening for a variety of reasons).

I'm looking forward to the challenges of this project. I hope I haven't scared you all with my brashness (and yes I am a little crazy also). I'll talk to you more later.

Thanks for your interest!

Jonas

Jonas Maebe wrote

b. the dragonegg approach (http://dragonegg.llvm.org/), where you make use of the support in an existing compiler (again GCC in this case) for abstract code generator support to emit LLVM assembler/bitcode rather than machine code

For my upcoming project: I (of course) do have a gcc4 compiler for that processor. Would dragonegg now allow generating code for the processor from FPC sources ?

IMHO it would be a great advantage for FPC to be able to generate code for any processor supported by gcc. Maybe this even forks for 16 Bit CPUs (lately here I saw a Request to generate code for PIC).

-Michael

I have really no idea why you think that would be the case. Please read the above paragraph again, and also the DragonEgg website, to see what it is.

Jonas

Hello Michael,

The MIPS32 code for LLVM's code generator is out-of-date but salvageable. If you're familiar with C++ I'd recommend anybody use LLVM for its massive optimizations, documentation and support classes. If you want C/C++/Objective C compiler as well as other compilers working on your processor, LLVM would certainly be worth it.

If, on the other hand, you want only a Pascal compiler and don't care about the other languages, it is too soon to tell. LLVM-FPC is still in its infancy. I wish I could give you a more certain answer. The best I can tell you is this: If and when the become available commercially, I will probably be making an LLVM backend for the 68050 rather than using the existing GCC backend for the 680x0 even though I program in C++. Does that answer your question?

--Sam Crow

----- Original Message ----

What I originally had in mind was to use the LLVM support classes based on C bindings for libstdc++ and LLVM's template libraries. See http://llvm.org/docs/ProgrammersManual.html for LLVM's best practices. So, as such, it wouldn't be entirely disconnected from the rest of the project. It would just be another LLVM frontend. As for the code maintenance, the Mattathias BASIC team would be using the code for our compiler in the future as well so we'd have to protect it from bitrot. Unfortunately something so drastic would be like a fork of the project and I like the fact that FPC will work on my old PowerMac G4 as well as the newer systems. I was also hoping to avoid having to write our compilers as GCC frontends as well since that framework is poorly documented.

We may want to replace the existing code generator, unfortunately. The reason that LLVM requires such high-level code for the calling routines is that it supports not only different calling conventions but it promotes from the stack-based C calling convention to a register-loaded fastcall convention whenever possible. This may not have been necessary when using the Borland Fastcall convention but sometimes being able to rearrange the registers when calling different functions can avoid some spillage to the stack.

I think the biggest obstacles to using LLVM at this point is that Borland Fastcall isn't supported by the x86 backend for LLVM and that the type information isn't high-level enough in the FPC compiler. That's two problems, both fairly significant (although the latter is definitely heavier than the former). Do you think it's too soon to divide out the work? Is there more that needs discussion?

Let me know what you think.

--Sam Crow

I think we're talking about two different things here: I meant that it could get disconnected from the rest of the FPC project, not from the LLVM project.

Not only that: a fork without any modifications in the "regular FPC" would probably be quite hard to keep in sync, because new frontend features still get added to FPC. Sometimes these also require (abstract, non-cpu-specific) code generator support, and if the regular FPC only keeps its current code generator, then a lot of those changes will require porting to be integrated the LLVM backend.

I agree that the current function call infrastructure in FPC's code generator is way over-engineered for what LLVM requires, and that part would probably not be used. An LLVM call instruction handles all of the parameter passing and return value handling by itself, so that is no problem. There are however other things where you do may want to keep the current code generator, because you'd have to implement support just as low level in LLVM as it currently exists in FPC.

One example is bitpacked arrays. Last I checked, LLVM did not support this yet (only bitpacked structs were functional), so you'd also need the explicit loading, masking, inserting and storing (unless you translate that into an array of structs with 1-byte elements consisting of 8 bits each, but then the debug info will suffer).

Another example is the Objective-Pascal support we've added: unless you'd build the FPC frontend on top of Clang rather than on top of LLVM, you'll have to re-implement all of the metadata generation for Objective-C classes in an LLVM way if you'd throw away all of FPC's low level code generation stuff.

I think the biggest obstacles to using LLVM at this point is that Borland Fastcall isn't supported by the x86 backend for LLVM and that the type information isn't high-level enough in the FPC compiler.

The code generation in FPC is implemented in basically two steps. The parse tree nodes are all class instances with a "generate_code" method. The generic implementations of these methods use the various (generic) code generator methods to actually emit the (cpu-specific) assembler instructions. If you just create descendent classes for all of these parse tree nodes whose generate_code methods call into the LLVM run time rather than in FPC's abstract code generator, then you have access to all high level information that you could want.

This approach is what would correspond to the "replace the existing code generator" you mentioned. The big downside to this approach, as mentioned earlier, is that it would result in a lot of maintenance work to keep the LLVM version up-to-date, because newly added tree node types and changes to the regular FPC-versions of the generate_code methods of the FPC would have to be manually ported to the LLVM version (the language/dialects that FPC supports, and hence the parse tree classes and/or their implementation, are still evolving -- we're not just fixing implementation bugs or optimising).

Another approach would be to introduce a higher level code generator in FPC that can be implemented via either
a) the current low level code generator
b) an LLVM backend

In that case, new parse tree nodes etc would be implemented using this high level code generator, and no extra porting effort would be required for supporting both the LLVM backend and the regular code generators (once the initial implementation is finished, obviously). Since designing such a beast is hard and a lot of work, one possibility could be to model it on the interface that LLVM offers. Then we immediately know that it's complete and appropriate for compiler use (since LLVM was designed as a compiler backend), and that by definition it will fulfil the needs of the LLVM backend (and presumably also that of other high level backends, should anyone ever want to implement that).

Of course, that's probably quite a bit of work (and it would slow down the compiler due to the extra abstraction level), but I think it offers the best chances of avoiding a fork and lots of additional porting efforts afterwards.

That's two problems, both fairly significant (although the latter is definitely heavier than the former). Do you think it's too soon to divide out the work? Is there more that needs discussion?

It's not entirely clear to me yet how you see the result: an FPC frontend added to the LLVM project, or an LLVM backend added to the FPC project. I favour the latter, but a lot of what you talk about seems to be about the former. Or am I misunderstanding things?

Jonas

----- Original Message ----

-snip-

I think you are understanding things well. I'm looking to FPC as a means to an end where having a team member on the Mattathias BASIC team is creating some libraries in Delphi that I want ported to LLVM. The reason I haven't thought through the Pascal-specific portions of the task is that I am a C++ programmer.

The last time I programmed in Turbo Pascal, Delphi didn't even exist. My first C programming class in college back in 1994 was on Turbo C++. I've since gone on to do a lot of stuff in C and a little stuff in C++. I still have my Turbo Pascal textbook but it's so out-of-date with regards to OOP that it is useless to me. I am unable to parse half the stuff I see when I look at modern FPC source.

Perhaps rather than discuss what code we will use from which project, FPC or LLVM, we should be talking about who wants to do what. If I am to do most of the work, then the packed array of booleans will be implemented with an LLVM bitvector template class on a thin Pascal frontend accessing most of its code through C wrappers of C++. If you do it, we'll be using a Pascal array of packed booleans properly in Pascal and I wouldn't know what difference it would make. If packed arrays apply to non-boolean types, and I'd be none-the-wiser as to how it worked apart from my 1994-vintage Turbo Pascal programming knowledge and my knowledge of programming in 68020 Assembly.

As far as code-maintenance is concerned, I favor LLVM code over FPC code because as long as I watch the mailing list and rewrite portions of the code, it will be an out-of-sync version of Pascal that would do what I needed it to do in much the same way as FPC is out-of-sync with my old Amiga computer's backend. I don't plan on bringing the Amiga backend up-to-date but I do plan on writing a backend for the 680x0 series processors on LLVM for use with Mattathias BASIC.

In summary, herein lies the source of our disagreement: I want to work in C++ because it is what I know. You want to work in Pascal because it is what you know. We need to divide out the work so that we each know what is expected of us.

Does that answer your questions?

--Sam Crow

Delphi-style OOP is really quite similar to Java OOP at the heart (single inheritance, interfaces, exceptions), with some extra things such as properties (http://en.wikipedia.org/wiki/Property_ (programming) ) thrown in.

I think there's a misunderstanding here: I did not mean that the compiler uses bitpacked arrays of boolean (I don't think it does), but that to-be-compiled Pascal code may contain them. Or do you really mean that you would compile such code down to a call to a C routine that calls LLVM classes? (so you'd include LLVM-compiler code in the compiled binary -- or is LLVM's bitvector code is some support library that's always usable for including in compiled client programs?)

Bitpacked arrays can indeed be of any ordinal or enumerated type. In terms of memory layout and accessing, they're basically identical to bitpacked records, just that all fields are exactly the same size.

Yes, it does, thanks. The main problem I see is that my approach would require a lot of initial work from my part (creating the "high level code generator" that could call through to LLVM libraries), so a) you'd be pretty much blocked until that work were finished if you'd go with that approach
b) I honestly have no idea how soon I could finish that (or even how soon I could seriously get going in doing that)

I'm not sure what you'd need from me to create a complete C++-based LLVM code generator for FPC though. The entire parse tree is built from Pascal class instances, which have a different internal format to C++ class instances. Code generation normally happens via the pass_generate_code method of one instance calling that same method of its children in the correct order, finally followed by generating the code for itself. Simply replacing the body of the various pass_generate_code methods with calls to your C wrappers for C++ code would not make sense, since these C++ routines would be unable to call the relevant Pascal methods to traverse the tree further (i.o.w., you'd have to implement the tree traversal in C++ anyway, so there's no use in providing methods that would be used in case the tree were traversed via Pascal code).

The only option I immediately see is that you'd need some kind of serialised parse tree (and symbol table, and type definitions), which you'd de-serialise in your C++ code and then process as you like. That's not really a task that gets me jumping up and down with excitement though, because I don't see it as something that would be really useful in the longer term as far as the FPC project is concerned (not to mention that writing serialiser code is simply boring work -- there's no full rtti available that could be used to automate this).

Jonas

----- Original Message ----

Alright Jonas,

I can work on getting Borland Fastcalls implemented in the LLVM x86 backend. By the time I'm done with that we can talk about what else needs doing. Is there a test already in the FPC repository that I can use as a test to make sure that the calling convention is working? I'll be working on a MacOSX 10.5 Intel computer if that makes a difference.

Also, as part of the Mattathias BASIC project, our team leader has almost completed the LLVM-PEG utility for implementing parsing expression grammars in LLVM Assembly. If I end up continuing the work I started on a new LLVM-FPC parser I'll be using that as my tool of choice and implementing the code sections with support routines from LLVM and Clang. (I won't have the liberty of using Clang's libraries in the BASIC compiler since BASIC requires a two-pass compilation process.)

I hope your workload permits you to work on the function calling mechenism soon. Thanks for your advice.

--Sam Crow

Samuel Crow wrote:

GCC already does provide C and C++ (this is a Linux system after all :) ) and I suppose it thus also does provide Objective C and Objective Pascal.

I would need FPC ("Delphi Language" Pascal), as I'd like to port a lot of Delphi code.

I _HOPED_ LLVM would use the GCC code generator, so that it automatically would support all GCC architectures, but obviously this is not the case :(.

Optimization is not a real issue for me.

So I suppose it's easier to do a new "native" FPC code generator, especially as same can be done in Delphi-Language, which is more common in out team.

-Michael

P.S.:

In this list, we already did discuss doing an FPC version that creates the intermediate code that can be fed to the GCC code generator. This would make available to FPC all relevant CPU architectures and supposedly the low level optimization that gcc4 does on this intermediate code is hard to beat.

But this task seems to be so difficult that trying to do this was discouraged. :(.

-Michael

Just use GPC then?

Florian Klaempfl wrote

Just use GPC then?

It does not compile the many thousands of lines of the Delphi project I want to port :(.

-Michael

Yes, because it's probably very hard to make a (Object) Pascal front end for gcc. Another backend for FPC is 4k-5k lines (pascal, no need to learn C), just make it a bounty for say 2000 Eur and I'am sure, somebody will write you a backend for the desired processor.

Florian Klaempfl wrote

I'am sure, somebody will write you a backend for the desired processor.

Yep, a team of three: I, me, and myself.

As this processor, a load/store RISC with 32 non-dedicated 32 Bit registers, is very similar to MIPS and thus a bit similar to ARM (only that MIPS-alike it does not have a status register) I suppose I should use the well-tested ARM code generator and modify it appropriately. Once I am done, someone should be able to add some definitions to have the thing create MIPS-32 code.

I just need to find the spare time (and at that time, I do hope for appropriate tips from this list to get me started,)

-Michael

It does make a difference. Unfortunately. While working on a patch to add Borland fastcall support to GDB, I discovered that there's a bug in our implementation of Borland fastcall on Mac OS X (although I'm still not sure why). I'll try to fix it tonight and create a test file that you can use.

That's quite an undertaking!

I hope your workload permits you to work on the function calling mechenism soon. Thanks for your advice.

Even once this is finished, how do you propose we pass type information from FPC to LLVM libraries? Via strings in LLVM assembler format? (since I've already written that code) Or another way?

Jonas

Hi Jonas,

Seems like it was my code for GDB under Mac OS X that was wrong, not FPC. Borland fastcall is handled fine under Mac OS X. I've attached the test I've created.

To compile:
gcc -o tcc tcc.c tgdbcc.s -arch i386

The tcc.c file already includes a test for on of the two functions (and I've verified that it passes if the Pascal functions are compiled with C calling conventinos), but not yet for the other one. They mix a bunch of different parameters, which should provide a good start. I just noticed that I forgot to add in floating point parameters. They are passed in on the stack, and returned in st(0) (regardless of the precision). 80 bits extended is passed in a 12 byte location.

The details of the parameter passing are decided in http://svn.freepascal.org/svn/fpc/branches/release_2_2_4_rc1/compiler/i386/cpupara.pas . In particular, check out ti386paramanager.push_addr_param (determines whether a parameter is passed by address or by value), ti386paramanager.create_funcretloc_info (function return location) and ti386paramanager.create_register_paraloc_info (creates the actual parameter and return value locations). The internal compiler identifier for this calling convention is pocall_register.

The Pascal test file has been compiled with DWARF debug info. It's been manually stripped down from what the compiler produced to remove dependencies on the RTL.

LLVM does most of the work with its support routines but I expect that once we get the core language support in (including the ability to extend the language), users will help with writing a few extensions themselves.

Well, just like you prefer writing in C++, most of ours prefer Pascal :) Anyway, we'll see (and it would be nice to get an idea of the kind of performance gains we could get from an LLVM backend).

Ok.

Jonas

It looks like the in-code documentation in llvm/includes/llvm-c/Core.h contains all of the documentation about using IRBuilder plus wrapping and unwrapping macros for type conversions using the C wrappers. There are other abstract datatype class templates that aren't included in the C wrappers but we'd need to write separate C wrappers for each use of them anyway. Let me know if there's anything I can find out for you.

--Sam Crow

I would love fpc to be based on LLVM backend. C bindings are ok, but wouldn't it be better to fix support for cppclass inside fpc (since it is already there) and create more proper C++ headers.

I have studied clang project a bit (as C/C++ frontend for LLVM) and it actually makes more sense to me that FPC compiler even though pascal is generally more readable as a language.
CLang SVN: http://llvm.org/svn/llvm-project/cfe/trunk/lib/ (and compare that to FPC compiler source)
Simple tutorial for new language:
http://llvm.org/docs/tutorial/LangImpl1.html.

I believe FPC should be first similarly divided into separate blocks (Lexer, Parser, AST, Codegen). It is already, just perhaps more interface based approach (or more like fpcdoc parser is).

After that people could create even backends for .NET (like Delphi does) and even perhaps Java or XML code dumps (like it was suggested some time ago on this mailing list by someone else).

Another thing is, that I would prefer LLVM backend to be more C++ API compatible than FPC/Delphi even if we loose some FPC features (like RTTI), pascal calling conventions.

Basically I don't think FPC should just go into LLVM direction, but should support LLVM efforts to create more pluggable backends.

Matej Spiller-Muys

Matej Spiller-Muys wrote on Mon, 16 Nov 2009:

I would love fpc to be based on LLVM backend. C bindings are ok, but wouldn't it be better to fix support for cppclass inside fpc (since it is already there) and create more proper C++ headers.

C interfacing works today and has been very thoroughly tested. cppclass only works, for very basic things, since 2 days ago or so. Unless someone enjoys debugging a new code generator and a new external language interfacing paradigm at the same time, that does not sound like a very good idea to me.

It doesn't weigh up to over 12 years of experience developing FPC and knowing Object Pascal (to the extent that it's used in the compiler) inside out. My C++ and STL knowledge is someone else wants to create an FPC frontend for LLVM in C++, they are of course free to do so.

Another thing is, that I would prefer LLVM backend to be more C++ API compatible than FPC/Delphi even if we loose some FPC features (like RTTI), pascal calling conventions.

Then you're no longer talking about porting FPC/Delphi, but about creating a new language inspired by FPC/Delphi. There's nothing wrong with that, but given the amount of bug reports that we already get today when implementation details are slightly different to Delphi's, I'm not sure how much success you'd have with the existing Delphi/FPC users.

You have to offer a basic "amount" of either compatibility or incredible advantages before anyone will want to use your compiler. As soon as the hurdle gets too high, many people will simply not consider it worth it and stick to what they know. The threshold is different for different people, but it's not that high in general.

Basically I don't think FPC should just go into LLVM direction, but should support LLVM efforts to create more pluggable backends.

FPC's backend is already quite plugable (otherwise we would not be able to have 6 codegenerators maintained by 3-4 people). It's just at a lower level currently than what something like LLVM requires.

Jonas

---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

C interfacing works today and has been very thoroughly tested. cppclass

only works, for very basic things, since 2 days ago or so. Unless someone enjoys

debugging a new code generator and a new external language interfacing

paradigm at the same time, that does not sound like a very good idea to me. Absolutely you have a point here. But AFAIK "C++ linking" is part of FPC roadmap (not part of LLVM discussion at all). And LLVM C++ support is at least currently more feature complete. Finish porting LLVM to C or supporting C++. The question is what is more maintainable in the end?

It doesn't weigh up to over 12 years of experience developing FPC and

knowing Object Pascal (to the extent that it's used in the compiler) inside out.

My C++ and STL knowledge is someone else wants to create

an FPC frontend for LLVM in C++, they are of course free to do so. Exactly, I prefer frontend in FPC too, reusing lexer, parser, AST. Using LLVM only as a compiled library for codegen (with C or C++ bindings).

Then you're no longer talking about porting FPC/Delphi, but about creating

a new language inspired by FPC/Delphi. There's nothing wrong with that,

but given the amount of bug reports that we already get today when

implementation details are slightly different to Delphi's, I'm not sure how much

success you'd have with the existing Delphi/FPC users.

Actually I am. I was talking about C++ ABI ( not API, that was typo :) ). Having class VMT, exceptions the same as C++ could be a huge interop benefit :) And you get the C++ linking for free. Currently LLVM is still quite evolving. And it would be easier to start with code based on CLang (with some FPC features missing) and later add more FPC features when other projects "invent" features that C++ does not have (VMKit & Reflection,RTTI). It should be the faster way to implement LLVM frontend. With currently implemented clang we could already support 90% of FPC language.

You have to offer a basic "amount" of either compatibility or incredible

advantages before anyone will want to use your compiler. As soon as the hurdle gets too

high, many people will simply not consider it worth it and stick to what

they know. The threshold is different for different people, but it's not that high in general.
True. And supporting LLVM does indeed offer a lot of incredible advantages (well not all of them right away).

FPC's backend is already quite plugable (otherwise we would not be able to

have 6 codegenerators maintained by 3-4 people).

It's just at a lower level currently than what something like LLVM

requires.
That's what I meant. Abstraction on higher level.

http://www.linux-kongress.org/2009/slides/compiler_survey_felix_von_leitner.pdf Btw how does FPC or Delphi fare in comparison to LLVM, GCC, VC++?

Matej Spiller-Muys

Matej Spiller-Muys wrote on Mon, 16 Nov 2009:

There is no real "FPC roadmap" (the website page on future plans is just some random things that people picked up on the mailing lists, or that someone at one point intended to work on). There are only things that people submit/commit patches for.

Currently, someone appears to be interesting in adding such support, so it may well be implemented in the future. Or maybe he loses interest and starts working on something else instead, and then it may not be.

And LLVM C++ support is at least currently more feature complete. Finish porting LLVM to C or supporting C++. The question is what is more maintainable in the end?

No, the question is what do people submit patches for.

Having class VMT, exceptions the same as C++ could be a huge interop benefit :) And you get the C++ linking for free.

You also get breaking backwards compatibility with a lot of existing Delphi code for free. I repeat: you really cannot underestimate the amount of implementation details that existing Delphi code depends on, and people already complain about FPC's incompatibility with those implementation details.

Currently LLVM is still quite evolving.

So is FPC.

So what is holding you back? Really, trying to convince other people to do something that you would like to see done (or even worse: to work on it in the way that you would prefer them to do it) in general just doesn't work. The only way to achieve your projects is to start working on them yourself, and hope that other people join in.

http://www.linux-kongress.org/2009/slides/compiler_survey_felix_von_leitner.pdf Btw how does FPC or Delphi fare in comparison to LLVM, GCC, VC++?

I have no idea. All I know that's slightly related is the alioth computer language benchmark game (with the stress on "game"), where you have at least both FPC and GCC results.

Jonas

---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

You also get breaking backwards compatibility with a lot of existing

Delphi code for free. I repeat: you really cannot underestimate the amount of

implementation details that existing Delphi code depends on, and people

already complain about FPC's incompatibility with those implementation details.
Just so we are clear, what implementation details are you talking about (that you would loose with C++ ABI)? C++ Builder and Delphi have the same ABI and they work quite nicely together. There are even C++ libraries being used inside delphi (like xerces and xalan). Even Delphi VCL is used inside C++ (Builder). Btw how does the wxForms for Delphi (port of wxWidgets) on Delphi work? It is commercial so I don't know how they have resolved the C++ linking problems because they also claim FPC compatibility. Another possible option is to create a Objective C LLVM wrapper, since FPC already claims to support it.

So what is holding you back? Really, trying to convince other people to do

something that you would like to see done (or even worse: to work on it in the

way that you would prefer them to do it) in general just doesn't work. The

only way to achieve your projects is to start working on them yourself, and hope

that other people join in.

Currently I am still working into getting more complete COM support into FPC (0014919) and a lot is already in SVN. What is holding me back is crashing of applications using FPC DLLs (0014861 and 0014807). I could help LLVM efforts, but couldn't really lead a LLVM port, due to lack of time. But nevertheless debating how each developer see the benefits of each solution is still good IMHO (no matter what direction the development will actually take). Maybe i'll start next on more complete support for C++. Because I need xerces/xalan and it uses only virtual functions inside C++ (so it should be simple to port). Or just create OO for libxml/libxslt or start effort to implement native FPC xslt.

No, the question is what do people submit patches for.

True, but more readable code helps :) I tried to resolve above bugs myself by studying FPC compiler a bit, even working on cleaner FPCDoc parser (0011240), but fpc compiler is just too complex and confusing (for the available time that I currently have :) ).

I have no idea. All I know that's slightly related is the alioth computer

language benchmark game (with the stress on "game"), where you have at least both

FPC and GCC results.

Yup, the FPC factor is 2x-10x slower.
http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=fpascal&lang2=gpp&box=1 LLVM that is currently 30% slower than GCC and could speed FPC quite a bit :).

Matej Spiller-Muys

Matej Spiller-Muys wrote

> I have no idea. All I know that's slightly related is the alioth computer > language benchmark game (with the stress on "game"), where you have at least > both FPC and GCC results. Yup, the FPC factor is 2x-10x slower.

That's because you are looking at x86_64 results for FPC 2.2.2, which does not support register variables on x86_64. The i386 results are 20% slower than G++ on average. And as mentioned before, it's with the stress on "game": the results on that site depend on almost as much on the implementation effort that people have put in optimizing the source code for "their" language as it does on the compiler.

Jonas

And:
If you look at CPU secs, then the factor is helps the C++ programs most is that they use 4 cores more, the fpc programs are mostly singlethreaded. LLVM doesn't help to solve this (AFAIK).

Vincent

I don't think that LLVM contains an auto paralleliser at this time (not for SIMD, nor for multiple threads). Anyway, the above is an example of what I mentioned earlier (in the quoted text at the top of this message), since it's of course perfectly possible to also write multi-threaded versions using FPC.

Jonas

Jonas Maebe wrote

On 17 Nov 2009, at 08:18, Vincent Snijders wrote: > If you look at CPU secs, then the factor is helps the C++ programs most is that they use 4 cores more, the fpc programs are mostly singlethreaded. LLVM doesn't help to solve this (AFAIK). Anyway, the above is an example of what I mentioned earlier (in the quoted text at the top of this message), since it's of course perfectly possible to also write multi-threaded versions using FPC.

Not quite, because things have to be part of the language to be able to use them. GCC programs may use OpenMP since it is implemented using pragma's which are "part" of the language. FPC would have to implement the 'parallel for' ideas and friends for them to be used.

Micha

As far as I'm concerned, that's semantics. You're right that it matters for this game though :)

Jonas

In our previous episode, Matej Spiller-Muys said:
[ Charset ISO-8859-1 unsupported, converting... ]

Afaik not, you have to declare a modifier to use BCB classes in Delphi, and a lot of directives {$externalsym} and runtime checks {$stringchecks} have to be added to Pascal code so that it can interop with C++

It is different that they control both sides, IOW they also can shift with the C++ implementation to suit the Pascal side. WHich is not the case if you try to converge e.g. FPC with MSVC or GCC or LLVM.