Hello together!

As you might have noticed I'm currently trying to get Pascal code working on Android using the new JVM backend.

So far I've converted the Android API in the android namespace to Pascal and managed to compile, convert (to DEX) and package a full Pascal HelloWorld application, but when started on the emulator I get the message that the application stopped working (log will be shown further down).
After some non successful tries I decided to use the Java HelloWorld application outlined on
http://developer.android.com/resources/tutorials/hello-world.html#ui (I used this as an example for the full FPC one as well) and replace the string constant by a call to a FPC class.
So I added a TTestClass derived from TObject with a method getText that returns a UnicodeString. After providing the RTL class files to the build system I invoked "ant debug" and were greeted with error messages when it tried to convert TObject.class and tmethod.class to Dalvik code (log will be posted below).
To be sure I decided to replace TObject by JLObject, I built successfully and sent the package to the emulator. And look at the screenshot, I got presented by my choosen text :D

So it is definitely possible to run FPC JVM code on Dalvik, it's just not working perfectly currently...

Here are the logs:

When running the full FPC application (or "activity"):

=== begin run log ===

I/ActivityManager( 59): Start proc org.freepascal.helloworld for activity org.freepascal.helloworld/.THelloWorld: pid=601 uid=10035 gids={1015}
D/dalvikvm( 601): newInstance failed: p0 i0 [0 a1 D/AndroidRuntime( 601): Shutting down VM
W/dalvikvm( 601): threadid=1: thread exiting with uncaught exception (group=0x40015560)
E/AndroidRuntime( 601): FATAL EXCEPTION: main
E/AndroidRuntime( 601): java.lang.RuntimeException: Unable to instantiate activity
ComponentInfo{org.freepascal.helloworld/org.freepascal.helloworld.THelloWorld}: java.lang.InstantiationException: org.freepascal.helloworld.THelloWorld E/AndroidRuntime( 601): at
android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1569) E/AndroidRuntime( 601): at
android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:1663) E/AndroidRuntime( 601): at
android.app.ActivityThread.access$1500(ActivityThread.java:117) E/AndroidRuntime( 601): at
android.app.ActivityThread$H.handleMessage(ActivityThread.java:931) E/AndroidRuntime( 601): at
android.os.Handler.dispatchMessage(Handler.java:99) E/AndroidRuntime( 601): at android.os.Looper.loop(Looper.java:123) E/AndroidRuntime( 601): at
android.app.ActivityThread.main(ActivityThread.java:3683) E/AndroidRuntime( 601): at
java.lang.reflect.Method.invokeNative(Native Method) E/AndroidRuntime( 601): at
java.lang.reflect.Method.invoke(Method.java:507)
E/AndroidRuntime( 601): at
com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:839) E/AndroidRuntime( 601): at
com.android.internal.os.ZygoteInit.main(ZygoteInit.java:597) E/AndroidRuntime( 601): at dalvik.system.NativeStart.main(Native Method) E/AndroidRuntime( 601): Caused by: java.lang.InstantiationException: org.freepascal.helloworld.THelloWorld
E/AndroidRuntime( 601): at java.lang.Class.newInstanceImpl(Native Method) E/AndroidRuntime( 601): at java.lang.Class.newInstance(Class.java:1409) E/AndroidRuntime( 601): at
android.app.Instrumentation.newActivity(Instrumentation.java:1021) E/AndroidRuntime( 601): at
android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1561) E/AndroidRuntime( 601): ... 11 more
W/ActivityManager( 59): Force finishing activity org.freepascal.helloworld/.THelloWorld
W/ActivityManager( 59): Activity pause timeout for HistoryRecord{406869e8 org.freepascal.helloworld/.THelloWorld}

=== end run log ===

And here the log entry when building the Java based example with a class that derives from TObject:

=== begin build log ===

[echo] Converting compiled files and external libraries into /home/sven/projects/android/helloandroid/bin/classes.dex... [apply]
[apply] UNEXPECTED TOP-LEVEL EXCEPTION:
[apply] com.android.dx.cf.code.SimException: local variable type mismatch: attempt to set or access a value of type java.lang.Object using a local variable of type boolean. This is symptomatic of .class transformation tools that ignore local variable information. [apply] at
com.android.dx.cf.code.BaseMachine.throwLocalMismatch(BaseMachine.java:550) [apply] at
com.android.dx.cf.code.Simulator$SimVisitor.visitLocal(Simulator.java:570) [apply] at
com.android.dx.cf.code.BytecodeArray.parseInstruction(BytecodeArray.java:481) [apply] at
com.android.dx.cf.code.Simulator.simulate(Simulator.java:99) [apply] at com.android.dx.cf.code.Ropper.processBlock(Ropper.java:684) [apply] at com.android.dx.cf.code.Ropper.doit(Ropper.java:639) [apply] at com.android.dx.cf.code.Ropper.convert(Ropper.java:252) [apply] at
com.android.dx.dex.cf.CfTranslator.processMethods(CfTranslator.java:252) [apply] at
com.android.dx.dex.cf.CfTranslator.translate0(CfTranslator.java:131) [apply] at
com.android.dx.dex.cf.CfTranslator.translate(CfTranslator.java:85) [apply] at
com.android.dx.command.dexer.Main.processClass(Main.java:369) [apply] at
com.android.dx.command.dexer.Main.processFileBytes(Main.java:346) [apply] at com.android.dx.command.dexer.Main.access$400(Main.java:59) [apply] at
com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:294) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:134) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processDirectory(ClassPathOpener.java:190) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:122) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processDirectory(ClassPathOpener.java:190) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:122) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processDirectory(ClassPathOpener.java:190) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:122) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processDirectory(ClassPathOpener.java:190) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:122) [apply] at
com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:108) [apply] at com.android.dx.command.dexer.Main.processOne(Main.java:313) [apply] at
com.android.dx.command.dexer.Main.processAllFiles(Main.java:233) [apply] at com.android.dx.command.dexer.Main.run(Main.java:185) [apply] at com.android.dx.command.dexer.Main.main(Main.java:166) [apply] at com.android.dx.command.Main.main(Main.java:90) [apply] ...at bytecode offset 00000005

     [apply] locals[0000]: Lorg/freepascal/rtl/FpcBitSet;
     [apply] locals[0001]: Lorg/freepascal/rtl/FpcBitSet;
     [apply] locals[0002]: <invalid>
     [apply] locals[0003]: <invalid>
     [apply] locals[0004]: <invalid>
     [apply] locals[0005]: <invalid>
     [apply] locals[0006]: <invalid>
     [apply] locals[0007]: <invalid>
     [apply] stack[top0]: Ljava/lang/Object;
     [apply] ...while working on block 0005
     [apply] ...while working on method

containsAll:(Lorg/freepascal/rtl/FpcBitSet;)Z
[apply] ...while processing containsAll
(Lorg/freepascal/rtl/FpcBitSet;)Z
[apply] ...while processing org/freepascal/rtl/FpcBitSet.class [apply]
[apply]
[apply] trouble processing:
[apply] class name (org/freepascal/rtl/TMethod) does not match path (org/freepascal/rtl/tmethod.class)
[apply] ...while parsing org/freepascal/rtl/tmethod.class [apply] ...while processing org/freepascal/rtl/tmethod.class [apply]
[apply] UNEXPECTED TOP-LEVEL EXCEPTION:
[apply] com.android.dx.cf.code.SimException: local variable type mismatch: attempt to set or access a value of type java.lang.Object using a local variable of type boolean. This is symptomatic of .class transformation tools that ignore local variable information. [apply] at
com.android.dx.cf.code.BaseMachine.throwLocalMismatch(BaseMachine.java:550) [apply] at
com.android.dx.cf.code.Simulator$SimVisitor.visitLocal(Simulator.java:570) [apply] at
com.android.dx.cf.code.BytecodeArray.parseInstruction(BytecodeArray.java:481) [apply] at
com.android.dx.cf.code.Simulator.simulate(Simulator.java:99) [apply] at com.android.dx.cf.code.Ropper.processBlock(Ropper.java:684) [apply] at com.android.dx.cf.code.Ropper.doit(Ropper.java:639) [apply] at com.android.dx.cf.code.Ropper.convert(Ropper.java:252) [apply] at
com.android.dx.dex.cf.CfTranslator.processMethods(CfTranslator.java:252) [apply] at
com.android.dx.dex.cf.CfTranslator.translate0(CfTranslator.java:131) [apply] at
com.android.dx.dex.cf.CfTranslator.translate(CfTranslator.java:85) [apply] at
com.android.dx.command.dexer.Main.processClass(Main.java:369) [apply] at
com.android.dx.command.dexer.Main.processFileBytes(Main.java:346) [apply] at com.android.dx.command.dexer.Main.access$400(Main.java:59) [apply] at
com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:294) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:134) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processDirectory(ClassPathOpener.java:190) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:122) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processDirectory(ClassPathOpener.java:190) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:122) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processDirectory(ClassPathOpener.java:190) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:122) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processDirectory(ClassPathOpener.java:190) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:122) [apply] at
com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:108) [apply] at com.android.dx.command.dexer.Main.processOne(Main.java:313) [apply] at
com.android.dx.command.dexer.Main.processAllFiles(Main.java:233) [apply] at com.android.dx.command.dexer.Main.run(Main.java:185) [apply] at com.android.dx.command.dexer.Main.main(Main.java:166) [apply] at com.android.dx.command.Main.main(Main.java:90) [apply] ...at bytecode offset 00000009

     [apply] locals[0000]: Lorg/freepascal/rtl/FpcBaseNestedProcVarType;
     [apply] locals[0001]: [Ljava/lang/Object;
     [apply] locals[0002]: <invalid>
     [apply] locals[0003]: <invalid>
     [apply] locals[0004]: <invalid>
     [apply] locals[0005]: <invalid>
     [apply] locals[0006]: <invalid>
     [apply] locals[0007]: <invalid>
     [apply] stack[0001]: Lorg/freepascal/rtl/FpcBaseNestedProcVarType;
     [apply] stack[top0]: [Ljava/lang/Object;
     [apply] ...while working on block 0009
     [apply] ...while working on method

invokeBooleanFunc:([Ljava/lang/Object;)Z
[apply] ...while processing invokeBooleanFunc ([Ljava/lang/Object;)Z [apply] ...while processing
org/freepascal/rtl/FpcBaseNestedProcVarType.class
[apply]
[apply]
[apply] UNEXPECTED TOP-LEVEL EXCEPTION:
[apply] com.android.dx.cf.code.SimException: local variable type mismatch: attempt to set or access a value of type java.lang.Object using a local variable of type int. This is symptomatic of .class transformation tools that ignore local variable information. [apply] at
com.android.dx.cf.code.BaseMachine.throwLocalMismatch(BaseMachine.java:550) [apply] at
com.android.dx.cf.code.Simulator$SimVisitor.visitLocal(Simulator.java:570) [apply] at
com.android.dx.cf.code.BytecodeArray.parseInstruction(BytecodeArray.java:481) [apply] at
com.android.dx.cf.code.Simulator.simulate(Simulator.java:99) [apply] at com.android.dx.cf.code.Ropper.processBlock(Ropper.java:684) [apply] at com.android.dx.cf.code.Ropper.doit(Ropper.java:639) [apply] at com.android.dx.cf.code.Ropper.convert(Ropper.java:252) [apply] at
com.android.dx.dex.cf.CfTranslator.processMethods(CfTranslator.java:252) [apply] at
com.android.dx.dex.cf.CfTranslator.translate0(CfTranslator.java:131) [apply] at
com.android.dx.dex.cf.CfTranslator.translate(CfTranslator.java:85) [apply] at
com.android.dx.command.dexer.Main.processClass(Main.java:369) [apply] at
com.android.dx.command.dexer.Main.processFileBytes(Main.java:346) [apply] at com.android.dx.command.dexer.Main.access$400(Main.java:59) [apply] at
com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:294) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:134) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processDirectory(ClassPathOpener.java:190) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:122) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processDirectory(ClassPathOpener.java:190) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:122) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processDirectory(ClassPathOpener.java:190) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:122) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processDirectory(ClassPathOpener.java:190) [apply] at
com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:122) [apply] at
com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:108) [apply] at com.android.dx.command.dexer.Main.processOne(Main.java:313) [apply] at
com.android.dx.command.dexer.Main.processAllFiles(Main.java:233) [apply] at com.android.dx.command.dexer.Main.run(Main.java:185) [apply] at com.android.dx.command.dexer.Main.main(Main.java:166) [apply] at com.android.dx.command.Main.main(Main.java:90) [apply] ...at bytecode offset 00000010

     [apply] locals[0000]: [Ljava/lang/Object;
     [apply] locals[0001]: [Ljava/lang/Object;
     [apply] locals[0002]: I
     [apply] locals[0003]: I
     [apply] locals[0004]: I
     [apply] locals[0005]: [Ljava/lang/Object;
     [apply] locals[0006]: <invalid>
     [apply] locals[0007]: <invalid>
     [apply] locals[0008]: <invalid>
     [apply] locals[0009]: <invalid>
     [apply] locals[000a]: <invalid>
     [apply] locals[000b]: <invalid>
     [apply] locals[000c]: <invalid>
     [apply] locals[000d]: <invalid>
     [apply] locals[000e]: <invalid>
     [apply] locals[000f]: <invalid>
     [apply] stack[top0]: Ljava/lang/Object;
     [apply] ...while working on block 0010
     [apply] ...while working on method

fpc_setlength_dynarr_multidim:([Ljava/lang/Object;[Ljava/lang/Object;ZIC)[Ljava/lang/Object; [apply] ...while processing fpc_setlength_dynarr_multidim ([Ljava/lang/Object;[Ljava/lang/Object;ZIC)[Ljava/lang/Object; [apply] ...while processing org/freepascal/rtl/system.class [apply]
[apply] 1 warning
[apply] 3 errors; aborting

=== end build log ===

Regards,
Sven

Is this an expected exception?

Can you run the bytecode verifier on the class files generated for your program? See http://wiki.freepascal.org/FPC_JVM/Debugging#Bytecode_verification for the details. You may also have to add the android.jar to the class path for the verifier to work (although I'm not 100% certain whether that will work in case the android versions of java.lang.* start replacing the JDK versions of the same classes for running the verifier itself)

<first-fpc-dalvik.png>

Nice :)

Jonas

I'd say that it isn't an expected one... at least the converted class doesn't contain a "throws" comment at its constructor...

I don't think using the verifier on my own class files will be of much use, because if I understand the log correct it complains about the code in FpcBitSet.containsAll (mentioned in the last few lines before the next empty one). Also - regarding your android.jar comment - I'm not even using any Android classes in that example... just TObject (as a base class) and UnicodeString (it tried to convert all RTL classes, because I put the complete RTL in the projects bin directory, in which all contained .class files are converted).

Regards,
SVen

The org.freepascal.rtl.system class verifies fine here, and the FpcBitSet stuff also works fine. Maybe the methods of java.util.BitSet (on which org.freepascal.rtl.FpcBitSet is based) have different signatures in the Android version compared to the Java version. Since the FPC RTL shipped with the snapshot is compiled against the JDK bindings, that would explain the problems.

You could try to regenerate rtl/java/java_sys(h).inc using the first invocation listed at http://wiki.freepascal.org/FPC_JVM/Building#Building_javapp (but adding "-classpath android.jar" so it will use the android version of all those classes) and then rebuild the RTL (if it still compiles with those different signatures).

Also - regarding your android.jar comment - I'm not even using any Android classes in that example... just TObject (as a base class) and UnicodeString (it tried to convert all RTL classes, because I put the complete RTL in the projects bin directory, in which all contained .class files are converted).

The RTL uses android/JDK classes (such as several classes from java.lang and java.util).

Jonas

Jonas Maebe wrote

The org.freepascal.rtl.system class verifies fine here, and the FpcBitSet stuff also works fine. Maybe the methods of java.util.BitSet (on which org.freepascal.rtl.FpcBitSet is based) have different signatures in the Android version compared to the Java version. Since the FPC RTL shipped with the snapshot is compiled against the JDK bindings, that would explain the problems.

Now I see: the Android VM apparently also looks at the debug information and checks whether it's appropriate everywhere. The compiler currently says for each local variable and parameter that it's located in a particular stack slot from start to finish of the method. However, if it's only used in part of the method, the stack slot may be reused by temporary values with different types (or because of SSA, it may migrate from one stack slot to another, because virtual registers are also mapped to stack slots). This is what the Android VM verifier is complaining about.

You may be able to work around the issue for now by compiling without debugging information (-g-), but you'll also have to recompile the RTL without debug information. I believe Florian has been working recently on tracking the location of variables/parameters throughout the code of a procedure, so it may be possible to fix it with the help of that information.

Jonas

Sven Barth wrote

So it is definitely possible to run FPC JVM code on Dalvik, it's just not working perfectly currently...

Sorry if this is a stupid question from still-JAVA-ignorant:

How did you manage to move from the 8 bit SUN Java bytecode to the 16 bit Android /Dalvik code ?

Thanks,
-Michael

The same way as every other Android-JAVA developer does: Using the tool supplied by the SDK.

The point is this: All available classes of the SDK are in the normal bytecode format, so that you can use "javac" to compile your code. After you've compiled everything you use the tool "dx" to convert all your ".class" files to one ".dex" file. It's part of the normal Android development workflow. ;)

Regards,
Sven

Michael Schnell wrote

How did you manage to move from the 8 bit SUN Java bytecode to the 16 bit Android /Dalvik code ?

You are thinking of 8 and 16 bit in the wrong way. Those are simply details of the respective byte code instruction formats. For example, it's about whether an "add" instruction is encoded using 1 byte or using 2 bytes. It has nothing to do with the width of the executed operations (the JVM byte code instruction set includes 64 bit integer and floating point operations). Also note that this is only about the instruction opcodes: if you include the operands then several JVM byte code instructions are wider than 8 bit.

Jonas

But there is a significant difference between the JVM bytecode and the Dalvik one and perhaps Michael thought of that when writing his question: The JVM bytecode is a pure stackbased one while the Dalvik one uses registers.

Regards,
Sven

Of course I did understand this. I suppose the virtual "Java Processor" in fact uses 32 or 64 bit data. But the instruction coding might (or might not) be quite different when doing a byte-encoded instruction set (like a X86 processor) or using another instruction width (like ARM).

-Michael

Yep.

That is why I am astonished that converting Java-Bytecode to Dalvik code should be an easy task.

I do know a stack based 32-Bit Hardware engine using such a byte code ("ZPU") Same is done to use as little FPGA resources as possible. It do know that a GNU C compiler for this engine exists. Using it's Assembler is highly depreciated, as watching the stack locations is close to impossible for humans.

-Michael

Michael Schnell wrote

That is why I am astonished that converting Java-Bytecode to Dalvik code should be an easy task.

Sven is correct. I didn't know this until now, but all Android applications are first compiled using javac and then dex converts the bytecode. Usually you don't have to care about this, because the Java part is provided in source code and native code is compiled differently, but in any Android standard build log you will see:

compile:
[javac] Compiling 1 source file to
/home/felipe/Programas/lazarus-ccr/examples/androidlcl/android/bin/classes

-dex:
[echo] Converting compiled files and external libraries into /home/felipe/Programas/lazarus-ccr/examples/androidlcl/android/bin/classes.dex...

(striped of some lines)

Thus the "dex-converter" seems to include a kind of optimizer that allows for holding some very frequently used variables in registers instead of on the stack for a relevant code sequence.

Nice :)

-Michael

I was confused by the explicit mentioning of 8 vs 16 bit, since I didn't see how it mattered if you only meant stack-based vs register based.

That is why I am astonished that converting Java-Bytecode to Dalvik code should be an easy task.

JIT compilers for Java byte code have existed for over a decade now. They also map the Java stack machine to register-based architectures.

Jonas

Jonas Maebe wrote

JIT compilers for Java byte code have existed for over a decade now. They also map the Java stack machine to register-based architectures.

Of course they do.

Seemingly the Dalvik "to-dex-converter" is supposed to do some static pre-processor work on that behalf. Otherwise it would not make any sense to allow for defining registers in the virtual processor's instructions.

-Michael

A stackbased internal or intermediate representation is the most general one, from which a compiler can decide which registers to use for expression evaluation etc. FPC itself uses registers only for the actual code generation, depending on how many registers are available on the target machine.

DoDi

Without debug information it works. Next step: creating one of the Android classes from within FPC compiled code. :)

Regards,
Sven

Sven Barth wrote

Without debug information it works. Next step: creating one of the Android classes from within FPC compiled code. :)

Ok... creating an Android class (in this case android.widget.TextView) from within my FPC class works, but the full FPC variation still doesn't want to start... *sigh* this will take some time.
Maybe as a next step I'll try to instance an android.widget.Activity class (the base class of the failing case) and then I'll try to instance a FPC child class of that class...

Regards,
Sven

What kind of error do you get?

Jonas

The same as before, so the debug information wasn't the solution there...

Regards,
Sven

I don't see what you mean here. While of course FPC does acknowledge the count and kind of registers with targets like ARM or X86 or X86-64, Sven reported that his port for Java targets creates register-ignorant (stack based) code for standard (SUN) virtual Java machines and the "dex-converter" by Google converts this to 16-bit-wide Dalvik code which introduces Register notation.

-Michael

Honor to whom honor is due: the port is completely Jonas' work. I haven't modified a single line of code to get code working on Android. I just recompiled it once to disable debug information :D

Regards,
Sven

Maybe you have to explicitly add "-g-" to the OPT string when compiling the RTL (and your programs), since the default config file shipped with the snapshot enables debug information.

Jonas

I have done that already, because I'm calling ppcjvm with "-n" option. Or does the compiler look in its own directory first before it uses the user config file?

The problem seems to be something else and I'll find it sooner or later...

Regards,
Sven

He means that converting a stack based program representation (whether it's compiler-internal or Java byte code is irrelevant) of a program to a flat register file based program representation has been a solved problem since a long time ago. I tried to explain the same thing with my example of JIT compilers, which perform exactly the same task starting from exactly the same Java byte code.

Jonas

No, -n prevents loading any default configuration files.

The problem seems to be something else and I'll find it sooner or later...

There's a byte code verifier for dex files at http://dedexer.sourceforge.net/ (search for "-r"), maybe it can give a clearer diagnostic.

Also, at http://www.netmite.com/android/mydroid/dalvik/docs/verifier.html I noticed one forbidden thing that FPC currently does: "The Dalvik verifier is more restrictive than other VMs in one area: type safety on sub-32-bit integer widths. These additional restrictions should make it impossible to, say, pass a value outside the range [-128, 127] to a function that takes a byte as an argument."

FPC won't sign extend byte values when passing them to functions (the JVM itself only supports signed types). I'll fix that. It shouldn't affect the code you are getting an error about though, since that is about boolean vs int.

Jonas

The FPC internal representation (code tree nodes) has no notion of registers. This is (also) due to the fact, that a register allocator can work reasonably only after the entire code of a subroutine is known. When the target is a registerless VM, the register allocator never is invoked.

DoDi

Of course I see and know. I just stated that the converter obviously in fact does this.

Thanks again,
-Michael

I see. Seemingly, if the target is Dalvik, the non-FPC-internal "to-dex-converter" does this on it's own.

-Michael

I meant without the "-n", the default variant. Sorry for being unclear here ^^

I'll take a look at the verifier.

In the meantime I've done my previously mentioned additional testing:
1. instantiate an android.app.Activity class inside Pascal code: works
2. derive a Pascal class from android.app.Activity (as is the case with the full Pascal example) and instantiate that: fails

The error message still happens during instantiation of the class, but the message looks a bit different then the other one:

=== log begin ===

I/ActivityManager( 62): Start proc com.example.helloandroid for activity com.example.helloandroid/.HelloAndroid: pid=375 uid=10034 gids={1015}
W/dalvikvm( 375): VFY: new-instance on interface or abstract class Lorg/freepascal/android/TTestActivity;
D/dalvikvm( 375): VFY: replacing opcode 0x22 at 0x0000 D/dalvikvm( 375): VFY: dead code 0x0002-0018 in
Lorg/freepascal/android/TTestClass;.getTextView
(Landroid/app/Activity;)Landroid/widget/TextView;
D/AndroidRuntime( 375): Shutting down VM
W/dalvikvm( 375): threadid=1: thread exiting with uncaught exception (group=0x40015560)
E/AndroidRuntime( 375): FATAL EXCEPTION: main
E/AndroidRuntime( 375): java.lang.InstantiationError: org.freepascal.android.TTestActivity
E/AndroidRuntime( 375): at
org.freepascal.android.TTestClass.getTextView(androidfpc.pas) E/AndroidRuntime( 375): at
com.example.helloandroid.HelloAndroid.onCreate(HelloAndroid.java:16) E/AndroidRuntime( 375): at
android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1047) E/AndroidRuntime( 375): at
android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1611) E/AndroidRuntime( 375): at
android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:1663) E/AndroidRuntime( 375): at
android.app.ActivityThread.access$1500(ActivityThread.java:117) E/AndroidRuntime( 375): at
android.app.ActivityThread$H.handleMessage(ActivityThread.java:931) E/AndroidRuntime( 375): at
android.os.Handler.dispatchMessage(Handler.java:99) E/AndroidRuntime( 375): at android.os.Looper.loop(Looper.java:123) E/AndroidRuntime( 375): at
android.app.ActivityThread.main(ActivityThread.java:3683) E/AndroidRuntime( 375): at
java.lang.reflect.Method.invokeNative(Native Method) E/AndroidRuntime( 375): at
java.lang.reflect.Method.invoke(Method.java:507)
E/AndroidRuntime( 375): at
com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:839) E/AndroidRuntime( 375): at
com.android.internal.os.ZygoteInit.main(ZygoteInit.java:597) E/AndroidRuntime( 375): at dalvik.system.NativeStart.main(Native Method) W/ActivityManager( 62): Force finishing activity com.example.helloandroid/.HelloAndroid
W/ActivityManager( 62): Activity pause timeout for HistoryRecord{406742f0 com.example.helloandroid/.HelloAndroid} I/Process ( 375): Sending signal. PID: 375 SIG: 9 I/ActivityManager( 62): Process com.example.helloandroid (pid 375) has died.

=== log end ===

The derived class is (on the Pascal side) a completely empty class. Just the class declaration is there. I'll test with a declared constructor.

Regards,
Sven

Sven Barth wrote

I/ActivityManager( 62): Start proc com.example.helloandroid for activity com.example.helloandroid/.HelloAndroid: pid=375 uid=10034 gids={1015} W/dalvikvm( 375): VFY: new-instance on interface or abstract class Lorg/freepascal/android/TTestActivity;

It says that the VM is instantiating an abstract class. Does the TTestActivity.j file contain "abstract" in the ".class" line near the top? The compiler will automatically add this keyword if a class contains one or more (possibly inherited) abstract methods (it's required by the JVM).

Jonas

Indeed it does. In the .j as well as the disassembled .dex file:

=== part of TTestActivity.j begin ===

.bytecode 49.0
.source androidfpc.pas
.class abstract public org/freepascal/android/TTestActivity .super android/app/Activity

=== part of TTestActivity.j end ===

=== part of TTestActivity.dex begin ===

.class public abstract org/freepascal/android/TTestActivity .super android/app/Activity
.source androidfpc.pas

=== part of TTestActivity.dex end ===

I've also found the class that defines the abstract methods. It's four classes above android.app.Activity in the inheritance tree (android.common.Context). I yet need to check whether all methods are overridden correctly by the subclasses (and crosscheck that with the documentation).

Regards,
Sven

Sven Barth wrote

I've also found the class that defines the abstract methods. It's four classes above android.app.Activity in the inheritance tree (android.common.Context). I yet need to check whether all methods are overridden correctly by the subclasses (and crosscheck that with the documentation).

If you create an instance of that class in FPC code and compile with -vw, the compiler should list the abstract methods.

Jonas

There seems to be none :(

=== output begin ===

helloandroid]$ ppcjvm -n
-Fu/mnt/data/subversion/fpc-jvm/rtl/units/jvm-java/ -FEbin/classes -Fu../../androidlib/lib
-FD/home/sven/downloads/dev/freepascal/fpcjvmlinux-snapshot1/bin -XP -vw src/androidfpc.pas
Generated: bin/classes/org/freepascal/android/TTestClass.class Generated: bin/classes/org/freepascal/android/androidfpc.class Generated: bin/classes/org/freepascal/android/TTestActivity.class

=== output end ===

The unit looks like this:

=== source begin ===

unit androidfpc;
{$mode objfpc}{$H+}
{$modeswitch unicodestrings}
{$namespace org.freepascal.android}
interface
uses
   android;
type
   TTestActivity = class(AAActivity)
   public
     constructor create;
   end;
   TTestClass = class(TObject)//(JLObject)
     function getText: UnicodeString;
     function getTextView(aActivity: AAActivity): AWTextView;
   end;
implementation
constructor TTestActivity.create;
begin
   inherited create;
end;
function TTestClass.getText: UnicodeString;
begin
   Result := 'Hello World through FPC TObject';
end;
function TTestClass.getTextView(aActivity: AAActivity): AWTextView;
var
   js: JLString;
   test: AAActivity;
begin
   test := TTestActivity.Create;
   if test.isTaskRoot then
     js := 'Hello World! Activity is task root'
   else
     js := 'Hello World! Activity is not task root';
   Result := AWTextView.Create(aActivity);
   Result.setText(js);
end;
end.

=== source end ===

Can you make the android unit available for download somewhere?

Jonas

Jonas Maebe wrote

Also, at http://www.netmite.com/android/mydroid/dalvik/docs/verifier.html I noticed one forbidden thing that FPC currently does: "The Dalvik verifier is more restrictive than other VMs in one area: type safety on sub-32-bit integer widths. These additional restrictions should make it impossible to, say, pass a value outside the range [-128, 127] to a function that takes a byte as an argument." FPC won't sign extend byte values when passing them to functions (the JVM itself only supports signed types). I'll fix that.

Fixed.

Jonas

Fixed in svn, there was a bug in the abstract method accounting.

Jonas

Jonas Maebe wrote

Fixed in svn, there was a bug in the abstract method accounting.

When compiling the RTL (make RELEASE=1 clean all) for i386 I get the following error:

=== output begin ===

make[1]: Entering directory `/mnt/data/subversion/fpc-jvm/rtl/linux' /mnt/data/applications/fpc/2.4.4/bin/ppc386 -Ur -Ur -Xs -O2 -n -Fi../inc -Fi../i386 -Fi../unix -Fii386 -FE. -FU../../rtl/units/i386-linux -di386 -dRELEASE -Us -Sg system.pp
thread.inc(414,10) Warning: Function result does not seem to be set i386.inc(1657,10) Error: Forward declaration not solved "fpc_truely_ansistr_unique(var Pointer):^untyped;" system.pp(384) Fatal: There were 1 errors compiling module, stopping Fatal: Compilation aborted

=== output end ===

Compiler version is 2.4.4

Regards,
Sven

Sven Barth wrote

On 31.08.2011 21:21, Jonas Maebe wrote: > Fixed in svn, there was a bug in the abstract method accounting. When compiling the RTL (make RELEASE=1 clean all) for i386 I get the following error:

Fixed.

Jonas

Thanks. The next problem is when compiling the Java RTL:

=== output begin ===

astrings.inc(734,14) Error: Identifier not found "PAnsiRec"
astrings.inc(734,34) Error: Identifier not found "FirstOff"
astrings.inc(736,8) Error: Identifier not found "Move"
astrings.inc(736,20) Error: Illegal expression
astrings.inc(736,26) Error: Illegal expression
astrings.inc(737,11) Error: Identifier not found "PAnsiRec"
astrings.inc(737,25) Error: Identifier not found "FirstOff"

=== output end ===

Regards,
Sven

Forgot to commit a file, sorry.

Jonas

Nobody is perfect :)

But there seems to be another problem. When assembling the system unit I get the following error:

=== output begin ===

Assembling system
../../rtl/units/jvm-java/$system$$_fpc_nestedvars$486 -d ../../rtl/units/jvm-java/: file not found

=== output end ===

There seems to be missing a "4.j" at the end of the filename... (I might be wrong, but that seems to be the largest filename so far)

Regards,
Sven

That's strange, I've never seen an error like that. I cannot reproduce that problem:

$ make FPC=ppcjvm60 OPT="-O2 -al -g" clean all
/bin/rm -f ../../rtl/units/jvm-java/system.ppu ../../rtl/units/jvm-java/uuchar.ppu ../../rtl/units/jvm-java/objpas.ppu ../../rtl/units/jvm-java/jdk15.ppu /bin/rm -f fpcmade.jvm-java Package.fpc ppas.sh script.res link.res /bin/rm -f *.s *_ppas.bat
ppcjvm60 @rtl.cfg -Tjava -Pjvm -Fi../inc -Fi../jvm -FE. -FU../../rtl/units/jvm-java -O2 -al -g -djvm -Us -Sg system.pp Free Pascal Compiler version 2.7.1 [2011/08/29] for jvm Copyright (c) 1993-2011 by Florian Klaempfl and others Target OS: Java Virtual Machine
Compiling system.pp
Assembling system
Generated: ../../rtl/units/jvm-java/org/freepascal/rtl/$system$$_fpc_nestedvars$4864.class Generated: ../../rtl/units/jvm-java/org/freepascal/rtl/system.class Generated: ../../rtl/units/jvm-java/org/freepascal/rtl/$methodpointer.class Generated: ../../rtl/units/jvm-java/org/freepascal/rtl/FpcEnumValueObtainable.class [etc]

You can try executing this:

/path/to/ppcjvm @rtl.cfg -Tjava -Pjvm -Fi../inc -Fi../jvm -FE. -FU../../rtl/units/jvm-java -O2 -al -g -s -djvm -Us -Sg system.pp

and have a look at the generated ppas.sh

Jonas

No, the end is missing as well.
If I change the unit output path to something like "output" (something short) though, then the "4.j" is printed. Besides that the content of the ppas file is completely different in both cases... it nearly looks as if some RAM garbage is printed... I've run it a third time and the ppas.sh is again different.
Didn't you change something in the string handling?

Regards,
Sven

Sven Barth wrote

No, the end is missing as well. If I change the unit output path to something like "output" (something short) though, then the "4.j" is printed. Besides that the content of the ppas file is completely different in both cases... it nearly looks as if some RAM garbage is printed... I've run it a third time and the ppas.sh is again different. Didn't you change something in the string handling?

Ah, yes, it's possible that something is now broken in the non-JVM RTLs regarding string handling, I didn't test those thoroughly yet (only make fullcyle). I'm using a ppcjvm that's built using a simple "make all PPC_TARGET=jvm" in the compiler dir, so it's compiled using the default compiler's RTL rather than using the RTL in of the fpc-jvm branch.

I'll do a testsuite run to see whether I introduced any bugs in the string handling, but to test the Android stuff you can also use a compiler compiled against another RTL for now.

Jonas

As the saying goes: a picture says more than thousand words. :D

Thank you for your efforts, Jonas.

I'll try to improve the unit names of the android unit and its dependencies a bit and then it might become the first package for FPC-JVM ;)

Regards,
Sven

Sven Barth wrote

On 31.08.2011 22:59, Jonas Maebe wrote: > > I'll do a testsuite run to see whether I introduced any bugs in the string handling, but to test the Android stuff you can also use a compiler compiled against another RTL for now.

Testsuite didn't pick up anything, I'll debug it in the compiler tomorrow.

As the saying goes: a picture says more than thousand words. :D

Nice :)

I'll try to improve the unit names of the android unit and its dependencies a bit and then it might become the first package for FPC-JVM ;)

Great! Note that using the jdk15 unit in the android unit is not a good idea, since it probably also exports JDK api's that are not available on android.

Jonas

Sven Barth wrote

I'll try to improve the unit names of the android unit and its dependencies a bit and then it might become the first package for FPC-JVM ;)

Sven, thanks for your tests. Adding hwfpo (Hello World From Pascal Only) and simple step-by-step instructions how to compile it also would be nice :)

Max

Jonas Maebe wrote

> I'll try to improve the unit names of the android unit and its dependencies a bit and then it might become the first package for FPC-JVM ;) Great! Note that using the jdk15 unit in the android unit is not a good idea, since it probably also exports JDK api's that are not available on android.

If I remember correctly, then the android.jar contains the "JDK" classes as well, so I'll just generate a "jdkandroid" out of them... in the end I'll need to convert all other namespaces in that jar nevertheless.

Regards,
Sven

I'll do that ^^ (I hope this weekend)

Regards,
Sven

Fixed.

Jonas

Yes, it compiles now. Thank you.

Two more questions regarding compilation:
1. is it normal that the compiler is compiled to compiler/ppcjvm/ instead of compiler/jvm/ when using "make PPC_TARGET=jvm all"?
2. when calling the compiler without "-XP " then it always tries to find jvm-java-java. Didn't you prohibit that? At least I thought I had read that in your commit log...

Regards,
Sven

Sven Barth wrote

Two more questions regarding compilation: 1. is it normal that the compiler is compiled to compiler/ppcjvm/ instead of compiler/jvm/ when using "make PPC_TARGET=jvm all"?

I don't see that, all object and unit files are stored under compiler/ jvm/units/<fpchostplatform> here. I also don't understand how that can happen, because the name of the generated compiler binary is also "ppcjvm" and you can't have a file and a directory with the same name in a single directory.

2. when calling the compiler without "-XP " then it always tries to find jvm-java-java. Didn't you prohibit that? At least I thought I had read that in your commit log...

I changed fpcmake so that the generated makefiles never pass -XP when targeting the JVM. However, if you compile normally and your /etc/ fpc.cfg is used, its standard logic to set the -XP parameter based on the fact that you are cross-compiling will still be triggered. I haven't modified samplecfg/fpcmkcfg yet to prevent that from happening in newly generated config files though.

Jonas

I'll recheck that. Perhaps it's because I once called make by accident with "PPC_TARGET=ppcjvm".

When using ppcjvm I compile using "-n" and when I compile the Java RTL using make I also need to pass "-XP " in OPT. For the second part: Can it be that I need to regenerate the makefiles?

Regards,
Sven

That would indeed cause the makefile to create a directory called "ppcjvm". And the compiler binary is always called "pp" and then renamed (moved) to "ppc<arch>", so that would indeed put the "pp" binary in that ppcjvm directory rather than renaming it to "ppcjvm".

Strange:

~/fpcjvm/rtl/java$ make FPC=ppcjvm2 OPT="-n -O2 -al" clean all /bin/rm -f ../../rtl/units/jvm-java/system.ppu ../../rtl/units/jvm- java/uuchar.ppu ../../rtl/units/jvm-java/objpas.ppu ../../rtl/units/ jvm-java/jdk15.ppu
/bin/rm -f fpcmade.jvm-java Package.fpc ppas.sh script.res link.res /bin/rm -f *.s *_ppas.bat
ppcjvm2 @rtl.cfg -Tjava -Pjvm -Fi../inc -Fi../jvm -FE. -FU../../rtl/ units/jvm-java -n -O2 -al -djvm -Us -Sg system.pp Generated: ../../rtl/units/jvm-java/org/freepascal/rtl/$system$ $_fpc_nestedvars$4864.class
Generated: ../../rtl/units/jvm-java/org/freepascal/rtl/system.class

For the second part: Can it be that I need to regenerate the makefiles?

No, but maybe you did regenerate the makefile in rtl/java already with an fpcmake from trunk? The patch to prevent BINUTILSPREFIX being set in rtl/java/Makefile was applied in r18351

Jonas

Jonas Maebe wrote

That would indeed cause the makefile to create a directory called "ppcjvm". And the compiler binary is always called "pp" and then renamed (moved) to "ppc<arch>", so that would indeed put the "pp" binary in that ppcjvm directory rather than renaming it to "ppcjvm".

Ok, then I'll remove that directory and check again ^^

I'll check what I get as output by make and report that.

I didn't regenerate the makefiles till now in that checkout...

Regards,
Sven

I just tried to reproduce that at work on Windows, but I encountered an error when compiling the RTL. The problematic part are the dependencies of "uuchar".
The problem is that Win32 does not define the SYSTEMUNIT variable (and SYSTEMPPU is defined using "system" instead of "SYSTEMUNIT"). This applies to WinCE and maybe other RTLs as well (not Win64 though ^^).

Regards,
Sven

Just in case you can't follow me: I'm talking about the Makefile.fpc in rtl\win32.

Regards,
Sven

Sven Barth wrote

I just tried to reproduce that at work on Windows, but I encountered an error when compiling the RTL. The problematic part are the dependencies of "uuchar". The problem is that Win32 does not define the SYSTEMUNIT variable (and SYSTEMPPU is defined using "system" instead of "SYSTEMUNIT"). This applies to WinCE and maybe other RTLs as well (not Win64 though ^^).

Fixed (it was also for win64 btw)

Jonas

Hmm... you are right... I must have looked at the wrong file ^^

Thank you.

Regards,
Sven