FPC JVM/Internals: what changed
Revision 0 to 1. Removed lines have a minus sign; added lines have a plus sign.
Title: FPC JVM/Internals
| Old | New | Line | |
|---|---|---|---|
| 1 | + | # Compiler | |
| 2 | + | ||
| 3 | + | ## Compile-time code generation | |
| 4 | + | ||
| 5 | + | The implementation of several language features on the JVM target require the compiler to generate wrappers of all sorts: | |
| 6 | + | ||
| 7 | + | - records must be wrapped by classes that contain several helpers (initialization, deep copy) | |
| 8 | + | - if a public property accesses a private/protected field or method, the compiler has to generate a public method that wraps these accesses since the property can be used from code that does not have access to the original symbol (resulting in class verification errors) | |
| 9 | + | - enumerations are classes in the JVM and several default fields and methods must be implemented | |
| 10 | + | - constructors are not automatically inherited in the JVM class model, so the compiler has to do that | |
| 11 | + | - procedure variables are not supported by the JVM and must be emulated by classes. These classes contain a method that boxes all primitive parameters types and then dispatches them via *java.lang.reflect.Method.invoke()*. | |
| 12 | + | - virtual constructors and virtual class methods are not supported and must be wrapped by the compiler | |
| 13 | + | - ... | |
| 14 | + | ||
| 15 | + | While it is possible to generate all the internal definitions/nodes to implement all this functionality, plain Pascal code is much quicker and easier to write and maintain. For this reason the new *symcreat* unit supports parsing Pascal code injected into the source code stream at compile time. There is currently functionality to parse both method declarations and implementations, and also for typed const declarations. | |
| 16 | + | ||
| 17 | + | Once a method declaration has been parsed or otherwise generated, you can set the corresponding *procdef*'s *synthetickind* field to any of the *tsk_** constants defined in the *symdef* unit to indicate what kind of code should be generated for it. The *symcreat.add_synthetic_method_implementations_for_struct()* routine handles calling the appropriate helper routine to generate the actual Pascal code, parse it and generate the machine/byte code. | |
| 18 | + | ||
| 19 | + | Caveats: | |
| 20 | + | ||
| 21 | + | - the automatically generated Pascal code is always parsed in *objfpc* mode, regardless of the syntax mode of the current unit | |
| 22 | + | - the automatically generated code cannot do anything that is not allowed in regular source code. This means that all used identifiers must be valid Pascal identifiers, regular scoping rules apply, etc. In particular, if you duplicate a *procdef* from another unit and give it a new implementation, some of its parameter types may actually not be visible in the current unit (and hence you cannot use those type names in the Pascal code). | |
| 23 | + | ||
| 24 | + | ## Typed constants | |
| 25 | + | ||
| 26 | + | There are no generic data sections in Java class files. As a result, data initialization must be performed using explicit assignments in the class initialization (~unit initialization). | |
| 27 | + | ||
| 28 | + | Most of the functionality of the *ptconst* unit has been split off into *ngtcon*. Generating the typed constant initialization is now handled by a *ttypedconstbuilder* class. This abstract class has two child classes. *tasmlisttypedconstbuilder* handles the typed constant initialization for native targets. *tnodetreetypedconstbuilder* handles the typed constant generation for the JVM target and possible other future targets where typed constants have to be initialized via explicit assignments. | |
| 29 | + | ||
| 30 | + | Both classes can still be further overridden by target-specific classes. Depending on whether or not the current target is in the *systems.systems_typed_constants_node_init* set, the asmlist or nodetree class is instantiated. | |
| 31 | + | ||
| 32 | + | ## Nested routines | |
| 33 | + | ||
| 34 | + | The JVM has no stack or frame pointer that can be passed around, so another way is required to support accessing local variables or parameters from an outer routine in an inner routine. This has been solved by moving such parameters and local variables in a local record in the outer routine. This record is called the *parentfpstruct*. | |
| 35 | + | ||
| 36 | + | A pointer to this *parentfpstruct* is passed as hidden parameter to any child routine, so they can access the variables and parameters that way. In case a nested routine calls a more deeply nested nested routine, the *parentfpstruct* parameter itself will also be migrated to the new *parentfpstruct*, so that via daisy chaining all *parentfpstruct*s are reachable from everywhere. | |
| 37 | + | ||
| 38 | + | If a target requires that nested routines are handled this way rather than via the default framepointer technique, all it has to do is use the *ncgnstld* (instead of *ncgld*) and *ncgnstmm* (instead of *ncgmem*) units, and/or derive from the classes defined in those units rather than those defined in their generic equivalents. | |
| 39 | + | ||
| 40 | + | ## Code generator | |
| 41 | + | ||
| 42 | + | ### Overview | |
| 43 | + | ||
| 44 | + | There is now a high level code generator class, *thlcgobj* (hlcgobj.pas). The difference with the existing code generator is that all size descriptions are *tdef*s rather *tcgsize*s. | |
| 45 | + | ||
| 46 | + | The low level and high level code generators can be seamlessly mixed for targets that have a complete implementation of the low level one (all but the JVM target for now). There is a *tcghl2ll* (hlcg2ll.pas) class that translates all high level *hlcgobj* calls into low level *tcg* calls, which is used for all non-JVM targets. | |
| 47 | + | ||
| 48 | + | Apart from *tcgsize* -\> *tdef* changes, a number of *thlcgobj* methods also received extra size parameters in case *tlocation*s are involved. The reason is that a *tlocation* has a *tcgsize* member, and replacing it with a *tdef* is not possible without messing up the layer separations in the compiler (a different kind of high level location would be required). | |
| 49 | + | ||
| 50 | + | Several ncg*.pas units have been wholly or partly converted to use the new high level code generator infrastructure, so they can be used by both the JVM target and the existing targets. | |
| 51 | + | ||
| 52 | + | ### ncgutil | |
| 53 | + | ||
| 54 | + | A number of routines from ncgutil.pas has been moved to thlcgobj.pas so they can be overridden by different targets. | |
| 55 | + | ||
| 56 | + | ### ngenutil | |
| 57 | + | ||
| 58 | + | A new unit *ngenutil* has been added that contains node-related routines from other units (mainly from *nutils* and *pmodule*) that can now be overridden with target-specific versions (e.g., *jvm/njvmutil*) | |
| 59 | + | ||
| 60 | + | ### Limitations | |
| 61 | + | ||
| 62 | + | The high level code generator currently does not support the following features due to the inability of expressing them using high level types (*tdef*s): | |
| 63 | + | ||
| 64 | + | - LOC_(C)SUBSETREF, LOC_(C)SUBSETREG: would need additional type information | |
| 65 | + | - LOC_(C)MM(X)REGISTER: the compiler currently has no way to specify vector types using tdef(s) (no difference between regular arrays and vector types) | |
| 66 | + | ||
| 67 | + | Since the JVM target does not require these features and since the other targets use the existing low level code generator for them, this can be solved in the future. | |
| 68 | + | ||
| 69 | + | # RTL | |
| 70 | + | ||
| 71 | + | ## General | |
| 72 | + | ||
| 73 | + | Much of the existing RTL code is not usable for the JVM target, because it makes heavy use of pointers to arbitrary data, move and fillchar, none of which are available on the JVM platform. | |
| 74 | + | ||
| 75 | + | Rather than immediately putting the existing RTL files full of ifdefs, several files from *rtl/inc* have been duplicated into the *rtl/java* directory and modified there. As development progressed, some became more like the original ones and in one case (generic.inc) an original file is even already used. | |
| 76 | + | ||
| 77 | + | Some details: | |
| 78 | + | ||
| 79 | + | - *inc/systemh.inc* has been split into *java/systemh_types.inc* and *javasystemh.inc*. The reason is that the function/procedure definitions from *inc/systemh.inc* require that the Java wrappers (for ShortString, AnsiString) and base types (for records, sets, procvars) are already declared. However, these wrappers and base types in turn depend on types that are declared earlier on in *inc/systemh.inc*. Hence the split. Maybe this can be done to the generic version as well over time -- it doesn't even require changes to the RTLs of other targets if *inc/systemh.inc* afterwards simply includes both files resulting from the split. There are also some other changes to those files, mostly commented out things that are not relevant or not (yet) supported on the JVM target. | |
| 80 | + | - *java/jsystem.inc* is a (not very heavily) modified version of *inc/system.inc*. This one can probably also be unified with the generic version over time without too much trouble. | |
| 81 | + | - *java/compproc.inc* mainly has a bunch of routines commented out, and some extra JVM-specific added. Over time, the Java-specific ones should be moved to a Java-specific include file and with possibly a few extra ifdefs the generic one from *inc* should be usable | |
| 82 | + | - *java/jstrings.inc* contains the implementation of the *ShortstringClass* type along with some shortstring helpers. Where possible, the JVM RTL uses the existing helpers from *inc/sstrings.inc*. The only changes where that in some cases calls to *move* were replaced with calls to helpers that are replaced with JVM-specific helpers, like in [this example](http://svn.freepascal.org/cgi-bin/viewvc.cgi/branches/jvmbackend/rtl/inc/generic.inc?r1=18907&r2=18906&pathrev=18907). | |
| 83 | + | - *java/jastrings.inc* and *java/justrings.inc* are basically the same story as *java/jstrings.inc*. | |
| 84 | + | - *java/jpvar(h).inc* contains the base class used for dispatching procedure variables | |
| 85 | + | - *java/jset(h).inc* contains the parent class for all integer-based sets and a number of set helpers | |
| 86 | + | - *java/objpas(h).inc* contains the skeleton *TObject* definition and the *TVarrec* record. Since variant records make no real sense in Java (fields cannot overlap the same memory area), *TVarrec* has been implemented by boxing the value and storing the object into a single *Value* field. Extracting the value again happens via simple accessor functions. Since you're not supposed to write into *array of const* elements, this should work fine for most code that does not rely on the overlay behaviour. | |
| 87 | + | - *java/jrec(h).inc* contains the parent class for all record types | |
| 88 | + | - *java/jdynarr(h).inc* contains the support code for dynamic arrays | |
| 89 | + | - *java/rtti.inc* contains the helpers for initializing arrays. There are more than on native targets, because e.g. arrays of records on the JVM target have to be initialized similarly to arrays of ansistring on native targets: all record instances in the array must be explicitly allocated. | |
| 90 | + | - *java/jtvar(h).inc* contains the support for *threadvar* | |
| 91 | + | ||
| 92 | + | ## Debugging | |
| 93 | + | ||
| 94 | + | Because the plain Java class format does not support line number information for include files, it is currently not possible to generate line number information for any of the code in the RTL include files. If you really need to debug some code in an include file, the only workaround right now is to temporarily copy its contents into *java/system.pp* | |
| 95 | + | ||
| 96 | + | --- | |
| 97 | + | *From the Free Pascal wiki, page "FPC JVM/Internals", as it stood on 2013-07-11 after a change by Ggeldenhuys, out of the wiki's dump of 2024-05-26. Copyright stays with the author who wrote it, according to the old wiki's [[Licensing]] page. If you have concerns about this material, [message the admin](/wiki-contact).* | |
| 98 | + | Category: JVM |