Hi,

The LLVM code generator is more or less ready, including Dwarf-EH-based exception handling support. It's currently only supported on Darwin/x86-64 and Linux/x86-64, but it can do a "make all" and the testsuite can be finished as well. There are still some extra failures that do not happen with the built-in code generator, but most tests work fine.

What still needs to be done:
* investigate the RTE 217 when you compile a program with an error in it
* add support for a volatile() built-in function similar to how we have unaligned(), to fix e.g. the endless loop in tests/test/tmt1 (or in general: global variables used for cross-thread synchronisation)
* add support for specifying the target LLVM version. Currently it's hard-coded to LLVM 3.9, although it works all the way up to at least LLVM 7.0. Specifying the version number would mainly be necessary (for now) to target older LLVM versions
* check some remaining test failures
* add support for more platforms. Windows will be harder (and not something I plan on working any time soon), because I have not yet added support for generating LLVM-based exception handling code that is needed for SEH-based exception handling. Other 64 bit platforms should be "reasonably" easy (the parameter handling needs to be generalised more though). 32 bit platforms will probably require either some compiler re-engineering, hacking, or a translation of the tcg64 code generator to some kind of high-level code generator as well.
* add support for generating debug info
* add support for generating more meta-information for optimizations (e.g. range information about subrange types and enums)
* add support for saving the bitcode in addition to the native object files, and subsequently using link-time optimisation
* add support for some custom calling conventions
* pass on more FPC-related code generation options to LLVM (currently, mainly -CfXXX and -Ox get passed on)
* add support for TLS-based threadvars

Known limitations:
* hardware exceptions are unsupported. They will be caught, but unwinding is hit-and-miss because the state of the registers may be completely different than what LLVM expects (higher optimisation levels = more chance it will fail). This is an LLVM limitation and not something that can be fixed in FPC. It's also a major source of extra tests failures.

The support is currently only on the
https://svn.freepascal.org/FPC/svn/fpc/branches/debug_eh branch.

To test:
* make sure you have clang on your system (the ones from Xcode work fine too) and that it's in your path. If not, you can download one from http://releases.llvm.org/download.html#7.0.0
* build FPC as follows
** Darwin (can also use a higher version than 10.6, but the built-in default --10.5--- won't work):
make LOCALOPT="-dllvm -Fullvm" OPT="-WM10.6" all
** Linux: you may also have to specify the library path to libgcc_s. E.g. on Ubuntu 16.04:
make LOCALOPT="-dllvm -Fullvm -Fl/usr/lib/gcc/x86_64-linux-gnu/5" OPT="-Fullvm -Fl/usr/lib/gcc/x86_64-linux-gnu/5" all -j 4 FPMAKEOPT="-T 4"

The LOCALOPT flags are only used while building the compiler. They're separate here because otherwise you get errors while building the IDE (the -Fullvm doesn't work there while -dllvm would still make it use llvm units). This does mean that IDE will be built without the LLVM code generator, and hence its built-in compiler won't generate working code because it won't use Dwarf-EH-based exception handling, while the RTL will only contain support for that kind of exception handling.

Note that on Linux, all generated binaries will be dynamically linked and depend on libc (and libgcc_s), unlike with the regular code generator.

Jonas

On Dec 2, 2018, at 5:26 PM, Jonas Maebe wrote: The LLVM code generator is more or less ready, including Dwarf-EH-based exception handling support. It's currently only supported on Darwin/x86-64 and Linux/x86-64, but it can do a "make all" and the testsuite can be finished as well. There are still some extra failures that do not happen with the built-in code generator, but most tests work fine.

What does this mean for end users? Do we get better debugging support in LLDB?

Regards,
Ryan Joseph

Strange. You can add NOWPOCYCLE=1 to disable the cycle with wpo.

Jonas

An option to exchange (a lot of) compilation speed for faster code.

Do we get better debugging support in LLDB?

Unlikely. That would at best be a trade-off (possibly better support as long as you consider the source as C++ instead of Pascal), and something for which you don't need an LLVM-based code generator. That mainly requires patches to LLDB to add support for Pascal-specific types and features.

Jonas

Ryan Joseph wrote

What does this mean for end users? Do we get better debugging support in LLDB?

Sorry for getting slightly off topic (replies to lazarus list pleas / or fpc other).

Lazarus 2.0 has an LLDB based debugger, that uses it's own code (fpdebug) for watches. For the OS all debugging goes via LLDB, but the IDE translates watches/locals (input expression and result)

On 2018-12-02 11:26, Jonas Maebe wrote:

The LLVM version of the code generator has been significantly improved in the mean time:
* several testsuite failures and other bugs have been fixed
* Lazarus (from the fixes_2_0 branch) can now be built (for Darwin/x86-64/Cocoa at least, have not tested other platforms) (*) and works correctly
* initial support for Linux/AArch64 has been added (which fortunately did not require many changes to the compiler)
* a few attributes have been added to slightly improve the quality of the generated code

(*) There is some code in Lazarus that implements interface methods using private methods declared in other units. Delphi rejects such code (methods that implement an interface method have to be visible in the current class), and such code also failed with the LLVM code generator while it succeeded with the other ones. I've fixed the trunk compiler so it now prints an error regardless of which code generator you use (will still be added to the User_Changes_Trunk wiki page).

Additionally:

* investigate the RTE 217 when you compile a program with an error in it

This turned out to be a bug in the clang version that's part of Xcode 8. It works fine with e.g. Clang 7.0.

* add support for a volatile() built-in function similar to how we have unaligned(), to fix e.g. the endless loop in tests/test/tmt1 (or in general: global variables used for cross-thread synchronisation)

Added.

To test: * make sure you have clang on your system (the ones from Xcode work fine too) and that it's in your path. If not, you can download one from http://releases.llvm.org/download.html#7.0.0

Check out the https://svn.freepascal.org/svn/fpc/branches/debug_eh/ branch

* build FPC as follows

This is easier now:

** Darwin (can also use a higher version than 10.6, but the built-in default --10.5--- won't work):
make OPT="-WM10.6" LLVM=1 all -j 4 FPMAKEOPT="-T 4" ** Linux: you may also have to specify the library path to libgcc_s. E.g. on Ubuntu 16.04:
make OPT="-Fl/usr/lib/gcc/x86_64-linux-gnu/5" LLVM=1 all -j 4 FPMAKEOPT="-T 4"

The previously mentioned LOCALOPT parameter is no longer necessary, and as a result building the WPO versions of the compiler also works again.

Jonas

Jonas Maebe wrote

On 2018-12-02 11:26, Jonas Maebe wrote: The LLVM version of the code generator has been significantly improved in the mean time:

Good work !

Regards,

Adriaan van Os

Jonas Maebe wrote

On 2018-12-02 11:26, Jonas Maebe wrote: The LLVM version of the code generator has been significantly improved in the mean time:

The LLVM code generator on the debug_eh branch now also works with Linux/ARM, although only with the hard float EABI for now (compiler compiled with -dFPC_ARMHF).

Obligatory vipribench results (on gcc113.fsffrance.org, an APM X-Gene Mustang board -- that's an AArch64 board, but it can also run 32 bit ARM programs):
* FPC (-O2): Time: 5107ms = 2839240 pkts/s = 4245 MB/s
* FPC+LLVM (clang 7.0 -O2): Time: 5063ms = 3357693 pkts/s = 5020 M B/s

Higher optimization levels don't make a noticeable difference in either case.

Some notes about the FPC LLVM ARM Linux version:
* You have to use -Cg when compiling dynamic libraries, or they won't link (it's not necessary for all units, just for the main library source file)
* there are still a few bugs with bitpacked records that contain 64 bit values
* the compiler does not yet generate call frame information for the entry/exit code of assembler routines, which means that exceptions will not propagate through them and instead terminate the program immediately. In particular, overflow exceptions from 64 bit multiplication/division trigger this behaviour because they're implemented via assembler helpers in the RTL

Jonas

Do you have a disassembly at hand?

Attached.

Jonas

Any new news on the LLVM code generator? I wanted to test this but the old link posted seems to be dead. Is there an updated link and instructions to build?

Regards,
Ryan Joseph

Ryan Joseph wrote

Any new news on the LLVM code generator?

Link-time optimization support is now also integrated in the compiler.

I wanted to test this but the old link posted seems to be dead. Is there an updated link and instructions to build?

Which old link? Here's the link with information and build instructions: http://wiki.freepascal.org/LLVM

Jonas