There are now 2 projects working on the same HTML component, one is HtmlPort by Phil Hess, one is HtmlViewer maintained in Google Code. I am trying to figure out the right way to modernize the code. There is an idea to support all string types in old Delphis and new Delphis and 3rd party Delphi Unicode libs and then FPC/Lazarus. The code is already full of IFDEFs and it would be even more so. Not good!

This component would really benefit from the Delphi2009 default string type. All string encoding problems would be solved automatically. The next logical idea is to wait until FPC has the Delphi compatible unicode string, which is being developed in a branch (was it cpstrnew?). No IFDEFs would be needed because of string types.

Now a question: is there any idea when the new string is ready for testing and experimenting by application programmers?
I understood there are some fundamental design problems that need to be solved.
Once they are solved and the code moves to a bug-hunting mode, the time is right for application testing.

Note: I am not asking when the feature is ready and published, I am asking when it can be tested with "real" programs.
The HTML rendering component would be a good test case when cleaning bugs of the unicode feature.

Juha

That is simply not correct.

I am in the process of converting a D7 application to a D2010 application. There are lots of string encoding problems, and none of them is solved "automatically".

Furthermore, if you need to support both the old and new string type (i.e. keep it compilable with older Delphi/FPC versions) you'll always need IFDEFS in your code.

Michael.

kirjoitti tiistai 04 tammikuu 2011 12:42:44:

What kind of problems did you have?

I used Delphi 2009 for a while at my former workplace. At least it worked with my existing Delphi programs amazingly well. The compiler sometimes warned about automatic conversion between string types but still it worked.

The string uses UTF-16 encoding and converts automatically from other string types. It pretty much solves the unicode encoding problems automatically, IMO. If there is again some detail of unicode that I don't understand, I am willing to learn more.

My guess is the problems come with old pascal code with shortstrings and pointers. (?)

Furthermore, if you need to support both the old and new string type (i.e. keep it compilable with older Delphi/FPC versions) you'll always need IFDEFS in your code.

No, my whole idea is to modernize the code and NOT support old versions. There are already 2 maintained versions that support old compilers and old string types.
I am often questioning myself if it makes sense to use this historical language.
As you know most programmers consider it a dead or at least a dying language. However it is advancing to a right direction so I am still here. The problem seems to be more the old code base and the programmers' attitude against using the modern features.

If I told to a Java programmer that Object Pascal features match Java features, how would I prove it? By showing him the HTML component code?

IFDEFs and self-made conversion funcs just to support different string types. The Java guy would ask if there is any common unicode string type. I would say "Not yet but it's coming, but we can't use it because we must support the old compiler versions".

The code also has lots of containers and thousands of typecasts to access their elements which could be eliminated by using generics containers. I would explain to the Java person: "Object Pascal already has generics but we can't use them because we must support the historical Pascal syntax".

I would also laugh if I was the Java developer.
So, the minimum requirements for the modernized HTML component would be Delphi 2009 / FPC with the new unicode support.

Juha

Juha Manninen wrote

Now a question: is there any idea when the new string is ready for testing and experimenting by application programmers?

No.

I understood there are some fundamental design problems that need to be solved.

I'm not aware of any fundamental design problems, only of a lack-of- somebody-working-on-it problem.

Jonas

Not so here. Off the top of my mind:
- Explicit encodings run awry
- Some Components that used explicit Windows DoSomeThingA calls no longer functioned. (TProcess for instance suffers)
- Database fields change type.
- Strings were saved to stream. All instances where that happened, String had to be changed to AnsiString explicitly

Further, many standard Delphi components had changed their interface. IBExpress, Midas etc. jump to mind. They may not have anything to do with the strings issue, but they add to the confusion.

In short: Migrating a 1.2 million line app from D7 to Delphi 2010 is not a trivial task at all, and a substantial part is due to the strings issue.

Do not understand me wrong, I am all for the new string type.

But I simply do not think it is as painless as you seem to think it is. Secondly, not everyone wants to jump to the latest and greatest version, so I thought you might want to keep the old version. If you don't, fine.

And for the record: Java is almost as old as Object Pascal. I remember well the first attempts to run Java on a Alpha DecStation back in the whereabouts of 1994.

Michael.

On Tue, Jan 4, 2011 at 10:20 AM, Juha Manninen

I would also laugh if I was the Java developer.

Then just wait until the guy needs to use an ifdef and ooooops. There are no ifdefs in Java. Your turn to laugh has come as he tries to hack around this very serious short coming.

Java's string type is roughly equivalent to FPC's UnicodeString: it always uses UTF-16. If your source material uses a different encoding, you have to load it into an array of bytes and then create a new string based on the data in that array and the encoding that you explicitly specify. Afterwards, the string remains UTF-16 and if you want the original (or some other) encoding back, you again have to explicitly convert it to an array of bytes with an explicitly specified encoding. There are no automatic string encoding conversions in Java.

Also, in general discussions based on "how can I prove that language X is at least as good as, or better than, language Y" end in flame wars, so please don't go there.

Jonas

In our previous episode, Juha Manninen said:

First, D2009 only solves labelling stringtypes with encoding. This means that all inputs must be correctly labeled, or the system falls apart. And all code must be sufficiently highlevel enough (you can still mutilate utf8-strings with simple delete operations)

Any lowlevel operations must also be checked.

I used Delphi 2009 for a while at my former workplace. At least it worked with my existing Delphi programs amazingly well. The compiler sometimes warned about automatic conversion between string types but still it worked.

The conversion was non trivial, and the improvements must not be exaggerated. It is a small addition to better unicode support, not the big one. THat must come from cleaning up the code, making the codepaths safe and labelling sources of strings outside the compilers control (OS/GUI/e.g. streamed files) properly.

The string uses UTF-16 encoding and converts automatically from other string types.

_IF_ those are properly labelled. But widestring:=ansisstring; does the same in older versions too. It was just that there was only one "label" (the current ANSI encoding) Now there is a chance at several.

And what if it doesn't work? That is the whole problem. You can't as with other versions first migrate the code to roughly work on the new delphi (fix components etc), validate stability, and only then start using new functionality.

D2009 is a watershed, and a painful one (even if the feature itself can be considered good, which I do)

I am often questioning myself if it makes sense to use this historical language.

Me too. Trouble is that I don't see anything else in which I'm half as productive. C++ makes sense, but the only halfwhat productive one is from the same small company as Delphi.

As you know most programmers consider it a dead or at least a dying language.

True. But I don't get paid by other programmers if I follow their choices. I get paid for delivering apps to clients.

becuase it is a tool with codebases dating back to 1995.

If I told to a Java programmer that Object Pascal features match Java features, how would I prove it? By showing him the HTML component code?

I would not argue features, but delivery time.

If a Java developer ever pulls that argument, slap him in the face. Just look at some of the common Java libraries ... for example SWT. It is probably still compatible to Java 1.4 or even 1.3 (hell it could even be 1.2). For that reason it doesn't use generics and not even enums (which came with Java 1.5 ... yeah, no real type safety in that regard until then). It also took until Java 1.5 to have a for-each loop, so even in that regard FPC isn't really behind.

But Java is a bad example anyway ... if you read the fine "Execution in the Kingdom of Nouns"[1] by Steve Yegge, FPC will get a pretty good look ;-) (Although he doesn't mention FPC, the points he makes with C++ match FPC just as well).

Best Regards,
Andreas.

[1]
http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom-of-nouns.html

kirjoitti tiistai 04 tammikuu 2011 15:08:01:

I see. My programs were much smaller and were quite clean VCL programs, meaning they had only few 3rd party components and no Windows API calls.

For streams I also had to define a couple of strings as AnsiString. The automatic conversion took care of the rest. So, in my case it was easier than I expected.

Juha

Marco van de Voort kirjoitti tiistai 04 tammikuu 2011 17:57:43:

Yes, string encoding in the HTML component I referred to is a challenge. I haven't done such programs myself and now I try to figure out the best way for a multi-platform Delphi + FPC version.

There are now 2 versions which will be maintained also in future. They are not going away soon. Both projects have plans to modernize the code and then it makes sense to do it in a collective new version. At the same go I would like to make the code as modern as the tools allow.

Ideas for good string types are welcome.
The code should support both Delphi and FPC.

That is my experience, too.

Juha