Hi,

There is a new branch in svn (branches/jvmbackend) that contains support for compiling Pascal code into Java virtual machine bytecode. While it does not support all language constructs and while there are some differences in some features that are supported, in general the result is fairly close to "normal Pascal" you'd write.

Note however that this is a pure compiler port at this time. That means that while most of the language support is fairly complete, pretty much no standard Pascal units are available. For all practical intents and purposes there is only the system unit, and even that one misses many routines (including helpers required for basic functions such as read(ln)/write(ln), val, str, delete, pos, ...).

On the other hand, a unit that contains the entire public interface of the JDK 1.5 is available. Over time, it should be possible to provide most of the standard functionality of the system unit (and other units) implemented on top of that unit, or rewritten in Pascal code that does not depend on untyped pointer accesses.

Download information, usage information etc is available at http://wiki.freepascal.org/FPC_JVM

Regarding Android: the compiler and the system unit both use a bunch of standard functionality from the JDK to implement various features. So Android support would probably require quite a bit of extra work. I have not looked into that.

Enjoy!

Jonas

PS: no, this does not mean that FPC development from now on will only, or even primarily, focus on managed language support. And the infrastructure added for JVM support will make an LLVM port a lot more feasible in the future, because there is now a code generator layer that uses high level type information.

Hello,

Does this port generate Java Bytecode directly or does it generate first Java Assembler and then feeds that into an internal assembler?

I ask because Android does not use Java Bytecode, it has it's own, completely different bytecode.

Felipe Monteiro de Carvalho wrote

Does this port generate Java Bytecode directly or does it generate first Java Assembler and then feeds that into an internal assembler?

It generates Java assembler and then feeds it into an external assembler (Jasmin).

I ask because Android does not use Java Bytecode, it has it's own, completely different bytecode.

I thought it was possible to convert Java class files into Dalvik class files?

Jonas

AFAIK android SDK works just so. It converts lava`s .class files to dalvik .dex with "dx" utility.

Felipe Monteiro de Carvalho wrote

I ask because Android does not use Java Bytecode, it has it's own, completely different bytecode.

Why did Google do that? Why not stay with the standardized Java and get the benefits of all existing Java and Mobile Java apps?

Sometimes these software vendors can be really brain dead! They just love proprietary or incompatible stuff. Java was designed to be compatible on all platforms, now Android calls it "java", but it's not compatible with Sun/Oracle's Java. This sounds just like what Microsoft did years ago - luckily Sun put a stop to that.

I don't know much about Android, but just when I thought it sounded great, I hear things like this. :-(

Why don't you google yourself?

http://lmgtfy.com/?q=android+byte+code

Why not stay with the standardized Java and get the benefits of all existing Java and Mobile Java apps?

What benefits do you miss?

Sometimes these software vendors can be really brain dead!

Let's summarize. You

1. do not understand something
2. ask this list
3. do not wait for the answer
4. call it "brain dead"

That is not very clever, is it?
I guess some people would call this "brain dead".

Java is just a language. Normally you need the jdk and some third party libs. And here you got the x-platform problems. I know some projects that had too many problems porting their java projects and switched to fpc.

A jvm backend for fpc is great. I hope it is not too much work to maintain.

Mattias

Great new indeed! Just curious, is there an example program in pascal for jvm that can be compiled and seen in action? I checked out tests, but looking at random ones, they looks like console mode ones testing different rtl aspects. I'm talking about maybe a simplest one for example drawing a rectangle at the center of the window.

Max

No, I have not created any gui test programs.

Jonas

20.08.2011 19:02, Graeme Geldenhuys:

IIRC Google was unhappy with java's licensing and/or management and did not happen to arrange it all with Sun in peacefull and mutually acceptable manner so they (Google) then opted to step away and create a separate new product formally unrelated of java and not bearing Sun's copyrights (which still didn't prevent subsequent patent harassment from Oracle however)

Now, regarding mobile java, you might notice that while you still can do a /svn/phoneme~svn (and it will still mostly
work), the project looks and feels like essentially abandoned (no bugfixes/cleanups around, links bringing you to 404 pages etc etc) Of course there might be some development going on at Oracle internally, but there is no visible sign of it AFAICS.

Nikolai

Hi,
20.08.2011 12:49, Jonas Maebe:

Hi, There is a new branch in svn (branches/jvmbackend) that contains support for compiling Pascal code into Java virtual machine bytecode.

This is really amazing, even if functionality is somewhat limited at the moment.

Thanks for the great work!

Nikolai

Jonas Maebe wrote

No, I have not created any gui test programs.

There's also the problem mentioned at http://wiki.freepascal.org/FPC_JVM/Language#Using_JDK_functionality :

"The javapp utility cannot handle circular references involving nested classes, because they cannot be expressed in Pascal. There is one such circular reference in the standard JDK, between java.awt.Window and java.awt.Dialog. This is currently worked around declaring java.awt.Dialog as a formal class, which means that none of its methods, fields or nested classes are available."

Dialogs are generally fairly common in GUI programs...

The problem is that the following circular dependency: java.awt.Dialog is a subclass of java.awt.Window and hence java.awt.Window has to be declared first. However, java.awt.Window has a field (modalExclusionType) whose type is java.awt.Dialog.ModalExclusionType (i.e., a nested class of java.awt.Dialog, so java.awt.Dialog has to be declared first).

Normally, issues of this second kind are resolved by creating a forward declaration of the involed class (type JADialog.ModalExclusionType = class;), but you can't do that for nested classes outside their enclosing class. I could extend the javapp utility to enable specifying also nested classes as formal classes so that you'd only lose the definition for java.awt.Dialog.ModalExclusionType rather than for all of java.awt.Dialog, but that still would only partially work around the basic problem.

Jonas

Hans-Peter Diettrich wrote

A .NET or Dalvik backend is feasable as easy as a JVM backend. Question is the use of such an code generator, without an RTL, FCL, debugger etc.

As 'easy'? Do you have any idea how much work Jonas, one of the most experienced developers who are working on fpc, has done to accomplish this?

I've seen a lot of mails on this mailing list about all sort of features, including a JVM target. But no-one who actually started to work seriously on it.

I know that you did not mean to degrade Jonas' work with this comment. But please think about that before you send emails. I think you could expressed yourself differently without the unmentioned negative tone.

I'm really happy with Jonas hard work on this target, and want to thank him for all this (and other) work he did and does for fpc.

A jvm backend for fpc is great. I hope it is not too much work to maintain.

A .NET or Dalvik backend is feasable as easy as a JVM backend. Question is the use of such an code generator, without an RTL, FCL, debugger etc.

DoDi

Joost van der Sluis wrote

On Sun, 2011-08-21 at 00:11 +0200, Hans-Peter Diettrich wrote: > A .NET or Dalvik backend is feasable as easy as a JVM backend. Question > is the use of such an code generator, without an RTL, FCL, debugger etc.

Debuggers exist (any debugger that can deal with Java byte code should work). It's true that the usefulness will largely depend on people interested in fleshing out the RTL, and possibly also an FCL. A lot of functionality can be replicated by wrapping JDK classes though.

I think you could expressed yourself differently without the unmentioned negative tone.

No harm done.

I'm really happy with Jonas hard work on this target, and want to thank him for all this (and other) work he did and does for fpc.

To make sure that credit is given where credit is due: I want to especially thank Paul Ishenin for his work on implementing Delphi-style "advanced records" and extending the class support with nested classes, class constants, final classes etc, because without him I would have had to implement all of that myself for this port :) And Florian/Peter for the code generator design, that made it possible to gradually add a high level code generator on top of it rather than having to completely rewrite it.

Jonas

Jonas Maebe wrote

And Florian/Peter for the code generator design, that made it possible to gradually add a high level code generator on top of it rather than having to completely rewrite it.

So there're actually 2 code generators right now. One is used for JVM and another is used for native machine code?

thanks,
Dmitry

Sad that "units with dots" feature was blocked previosly - it would allow you to write java classes as is with namespace prefix instead of use abbrevations.

Best regards,
Paul Ishenin

'As' easy!

DoDi

21.08.2011 7:03, Paul Ishenin пишет:

It`s a time to unblock it, I think.

dmitry boyarintsev wrote

So there're actually 2 code generators right now. One is used for JVM and another is used for native machine code?

The JVM target only uses the high level code generator. The other targets use a combination of both.

Jonas

Paul Ishenin wrote

Sad that "units with dots" feature was blocked previosly - it would allow you to write java classes as is with namespace prefix instead of use abbrevations.

I don't think that would be usable, unless we'd also allow something like Java's "import java.xml.*" or propagating uses. There are classes from 216 different packages in the jdk15 unit, so you could easily get enormous uses clauses without such a feature. I'm also not sure how it would work for the classes that are required in the system unit, since the system unit cannot depend on other units (even if it could for one special unit, that other unit would require system unit types such as "jint" or even "longint"). Even importing them under a different name wouldn't work, since e.g. TObject is derived from java.lang.Object and the compiler should allow you to typecast these types to each other. But if you have a type called "Object" in a unit called "java.lang" and a separate type "JLObject" from which TObject derives in the system unit, that wouldn't work.

And dotted unit names would still introduce the ambiguities that caused the feature to be blocked.

Jonas

Mattias Gaertner wrote

> > Why did Google do that? Why don't you google yourself?

Lets see if I understand this correctly. Dalvik was designed to be suitable for systems that are constrained in terms of memory and processor speed. (Quoted from Wikipedia) Yet today's smart phones and tablets are more powerful than the desktops of just a few years ago, or even the computers that first sent man to the moon. Now lets not even talk amount of memory. Most mobile devices have 512-1GB memory these days - again more than most desktops had just a few years ago.

So Google creating a incompatible Java for this supposed limitation of processor speed and low memory. Yeah Sun's Java ran fine on desktops of 10-15 years ago, and even on PDA's of that time.

Google is doing with Android just the opposite of what Sun's Java stands for - compile once, run everywhere.
If Google wanted something different for their Android platform, they should not have called it Java then.

What benefits do you miss?

Compile Java (SE, ME etc) apps once, run everywhere.

Java is just a language.

No, Java is also known as a platform (like Java Virtual Machine). Google (like Microsoft before it) is now tainting the Java platform by making it incompatible with existing Java. Like I said, if they wanted something different, they should have named it differently too.

Anyway, I never said I'm an expert in Android. I'll research the subject further to get a clearer picture.

A jvm backend for fpc is great. I hope it is not too much work to maintain.

I have no issues with FPC supporting the Java platform as a target - I think this is great news indeed.

Mobile platforms are all about power consumption in order to make batteries last as long as possible. Generally: faster = less power usage.

No, Java is also known as a platform (like Java Virtual Machine).

There are three separate things:
* Java the programming language
* the JVM platform/specification
* the JDK

There are indeed commonly called together "the Java platform", but they are not indivisible. That's why it's possible to e.g. compile Pascal code into JVM byte code that makes use of the JDK.

Google (like Microsoft before it) is now tainting the Java platform by making it incompatible with existing Java.

Google is doing several things:
* they kept the Java programming language
* they replaced the JVM with Dalvik, to make it faster/more power efficient (by doing a bunch of analyses and transformations that are normally done at run time at compile time). This by itself does not introduce incompatibilities, since Java class files can be translated into Dalvik class files
* they replaced the JDK with their own class library, the Android SDK

Do you think it would have been better if they had said "you program in the Android language, which is identical to the Java language"?

Jonas

One thing which I am curious: What is the main use case which led to this work?

I can imagine it would be useful for example to have a work focused on j2me in order to target phones which run only Java and nothing else (although in reality j2me is each time more obsolete with the spread of smartphones), but for desktop platforms which can already run native applications without any fuss I wonder what the use case of a Pascal to Java compiler is.

Even for Android, I'm not sure such a Java port is much better then JNI.

I'm not saying it isn't great to have this new option, because it clearly is, I'm just curious.

Jonas Maebe <jonas. wrote

Google is doing several things: * they kept the Java programming language * they replaced the JVM with Dalvik, to make it faster/more power

efficient (by doing a bunch of analyses and transformations that are normally done at run time at compile time). This by itself does not introduce incompatibilities, since Java class files can be translated into Dalvik class files

* they replaced the JDK with their own class library, the Android SDK

Ok, thanks for explaining (without the hostile tone like Mattias). From the Dalvik Wikipedia page, converting Sun Java class files to Google Java is not always possible. It seems to be a hit and miss case. But I guess recompiling the source code might have better sucsess.

Do you think it would have been better if they had said "you program in

the Android language, which is identical to the Java language"?

Yes, not identical though, just similar syntax.

Graeme.

And for gnu gcj to compile java into native machine code. No bytecode involved.

Ludo

Graeme Geldenhuys wrote

Why did Google do that? Why not stay with the standardized Java and get the benefits of all existing Java and Mobile Java apps?

Java mobile (j2me) is almost completely different from usual Java APIs, even more then the Android API is different from JDK I'd say, and the j2me APIs are horrible, very slow, and they also work widely different across various phones. If anyone beliaves that j2me is write once and run everywhere without debugging in the hundreds of available phones, he clearly never wrote software on it =D I am happy j2me is getting obsolete.

I don't know much about Android, but just when I thought it sounded great, I hear things like this. :-(

I don't think this makes any difference, my main problem with Android is that they do not allow users to run purely native executables. If native apps work fine for the iPhone, they should have worked fine in Android too, no need for Java. Their current architecture really isn't the best IMHO, but it also not that bad either, compared to the competition, it is reasonably workable.

Android is much better then BlackBerry or Windows Phone. You can actually run native Pascal code in Android, which means that you can use all those libraries which we have without any code changes, while in those other 2 the answer is just: no, no, no

Felipe Monteiro de Carvalho wrote

One thing which I am curious: What is the main use case which led to this work?

Application servers (usually web server backends). And the fact that it prepares the compiler for an LLVM backend: Java byte code is less strict about types than LLVM byte code (e.g. Java byte code does not specify full type information for temporary values, while LLVM byte code does), so it was a good first step to finally get things moving.

Jonas

Some people from Russian community tried to start to enjoy but failed :) http://www.freepascal.ru/forum/viewtopic.php?t=7325#p55875

<http://www.freepascal.ru/forum/viewtopic.php?t=7325&view=unread#unread>Best regards,
Paul Ishenin

The problems are
a) he is adding the '.class' extension when trying to run the program. You have to specify the name of the class to run, not the name of the class file (so leave out the '.class') b) he should specify the base path to the FPC rtl classes via the -cp parameter, as explained at http://wiki.freepascal.org/FPC_JVM/Usage#Running_compiled_programs

I've added a quick start example at http://wiki.freepascal.org/FPC_JVM#Quick_start

Jonas

Are there a plans to merge branches (trunk and jvm branch) together? or is it actually the next planned step?
What about changing the native code generator to become more "high level" as JVM?

Jonas Maebe wrote

> Some people from Russian community tried to start to enjoy but failed :) http://www.freepascal.ru/forum/viewtopic.php?t=7325#p55875

sorry, didn't get to translation of Usage page yet.

thanks,
Dmitry

I followed the instruction and again it failed to run the application. However, it's unsaid that location of the ".class" file should also be listed in "-cp" option. At least it worked fine for me. I tried it in Windows, so I'm not sure if it's required for Unix OSes, but it appears that working directory isn't used by "java" for classpath searching.
Jonas, I'd suspect you have environment variable CLASSPATH setup in your system or something like that, that doesn't require you specify the path to .class file.

In the end, it has nothing to do with ppcvmj, but rather than usage of "java" application :)
I added a small note to Quick Start feel free to delete it anyway.

thanks,
Dmitry

Is it good idea to write client application work on internet browser?

Zaher Dirkey

Yes, once other developers have had the chance to review the jvm branch and comment about it. Other requirements are that there are no regressions for non-JVM targets anymore (although I think I fixed all of those today) and that all new memory leaks are fixed (there are a few in the new symcreat unit).

What about changing the native code generator to become more "high level" as JVM?

That doesn't really offer any advantages, so I'm not sure whether it's worth the trouble. Code generation has two parts in the compiler:
* all parse tree nodes have a method called pass_generate_code that calls general code generator routines
* the code generator returns emit assembler statements

In svn trunk, all pass_generate_code routines called the low level code generation routines. In the jvm branch, several of those pass_generate_code routines have been converted to call the high level code generator. For native targets, there is a special high level code generator implementation that generally just calls through to the low level code generator.

As long as new implementations of pass_generate_code methods use the high level code generator (which has almost the same interface as the low level code generator, only the "size" parameters are high level definitions instead of only things like "4 byte unsigned value"), and if new code generator methods are added to the high level instead of low level code generator classes, there should be no problem keeping everything in sync without duplicating any work (and without making things harder for anyone).

Jonas

No, but my test scripts did have "." in the -cp path.

In the end, it has nothing to do with ppcvmj, but rather than usage of "java" application :) I added a small note to Quick Start feel free to delete it anyway.

Thanks, I've updated it to use "." (that's easier to type).

Jonas

Lets see if I understand this correctly. Dalvik was designed to be suitable for systems that are constrained in terms of memory and processor speed.

IMO more important is avoiding the US patents on Java. For the curious: the Java patents typically claim "Method And Apparatus For ...", where the Apparatus is obviously aimed to outwit the restriction, that most countries do not accept pure software patents.

DoDi

Felipe Monteiro de Carvalho wrote

I ask because Android does not use Java Bytecode, it has it's own, completely different bytecode.

Ooops. Dalvik in fact completely off-standard on that behalf !

Wikipedia sais:

Standard Java bytecode <http://en.wikipedia.org/wiki/Java_bytecode> executes 8-bit stack instructions. Local variables must be copied to or from the operand stack by separate instructions. Dalvik instead uses its own 16-bit instruction set that works directly on local variables. The local variable is commonly picked by a 4-bit 'virtual register' field. This lowers Dalvik's instruction count and raises its interpreter speed.

-Michael

Jonas Maebe wrote

It generates Java assembler and then feeds it into an external assembler (Jasmin).

Is writing Java assembler code in the pascal source possible (or planned) ?

-Michael

Jonas Maebe wrote

...

GREAT !!!

Regarding Android: the compiler and the system unit both use a bunch of standard functionality from the JDK to implement various features. So Android support would probably require quite a bit of extra work. I have not looked into that.

I suppose the Android add-on will make this _really_ interesting indeed !

-Michael

Michael Schnell wrote

Is writing Java assembler code in the pascal source possible

Anything is possible.

(or planned) ?

Not at this time.

Jonas

Graeme Geldenhuys wrote

So Google creating a incompatible Java for this supposed limitation of processor speed and low memory.

Modern ARM processors (used for by far most Android devices) provide Jazelle DBX (Direct Byte code eXecution). Supposedly this does not help any more with Dalvik Java code. Funny that the 16 Bit version is supposed to be faster, though...

-Michael

Felipe Monteiro de Carvalho wrote

my main problem with Android is that they do not allow users to run purely native executables.

AFAIK, this has been overcome with the latest versions of Android..

-Michael

Michael Schnell wrote

> my main problem with Android > is that they do not allow users to run purely native executables. AFAIK, this has been overcome with the latest versions of Android..

Not true. What 2.3 added is not really useful for Lazarus. It still requires native code to be inside a library and the only way to call Android APIs is still to call some Java code which will do the real call. 2.3 just adds some extra stuff for people writing native OpenGL games, not really useful for normal GUI applications.

Jonas Maebe wrote

Regarding Android: the compiler and the system unit both use a bunch of standard functionality from the JDK to implement various features. So Android support would probably require quite a bit of extra work. I have not looked into that. Enjoy!

I've taken a look at the Android SDK: it seems the java.lang and java.util namespaces and their classes are available so it might be rather simple to get an Android application compiling and running.

But for now I have the problem to convert the class files provided by the SDK into Pascal units using javapp. I have unpacked the android.jar to $somedir/android and also copied javapp.jar there (just to be sure that there are no path problems ^^). I cd'd into that directory and issued the following command to convert the android.app namespace:

java -jar javapp.jar -protected javapp.jar android.app.

The output is the following:

default charset: UTF-8
Determining short Pascal class names...
Indexing classes under android/app/Activity...
Indexing classes under javapp/jar...
Printing classes...
Done!

The result is a "java.pas" and a "java.inc". The first includes the latter and the latter is empty.

So what am I doing wrong? (The same happens if I try to use the android.jar instead)

Regards,
Sven

You have to pass "-classpath path/to/android.jar" (or "-classpath ." if you extracted android.jar in the current directory) as parameter to javapp (not as parameter to java!). I've added this fact to http://wiki.freepascal.org/FPC_JVM/Building#Building_javapp

So:
java -jar javapp.jar -classpath android.jar -protected javapp.jar android.app.

Jonas

Jonas Maebe wrote

So: java -jar javapp.jar -classpath android.jar -protected javapp.jar android.app.

Without the extra javapp.jar, obviously:
java -jar javapp.jar -classpath android.jar -protected android.app.

Jonas

Ok, I needed to test it quite a few times (perhaps I had some paths or something wrong each time), but now I've managed to generate the file for android.app. Yeah! ^^

Now I just need to convert all other necessary classes for an Android hello world and then we'll see :D

Regards,
Sven