Hi,

Why reinvent the wheel with all these different string types and round-about discussions etc... Can we learn something from other multi-platform frameworks that solved these problems ages ago! I Google'd and couldn't even find any message thread about programmers complaining about unicode and string issues in Qt or Java.

Take Qt as an example:
* Qt is one of the most successful programming frameworks and toolkits around
* it uses a language that doesn't even have a String type so it had to implement it's own.
* it seems to handle conversions just find - i haven't seen a single msg about developers complaining of such an issue.
* it runs on multiple platforms (big and small). So different system encoding are handled just fine.

Take Java as an example:
* Also a hugely popular language and framework (despite what Microsoft wants you to believe).
* No developers seem to complain about unicode or conversion issues.
* it runs on multiple platforms - again big or small. So different system encodings are handled just fine.

So would it maybe help if we took a peek at what those frameworks have done. Clearly they managed to do it right as no developers or users are complaining! So I guess Free Pascal could learn from them and don't have to go through the whole R&D process. Study what those two frameworks did and apply the same thing to Free Pascal!

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

In our previous episode, Graeme Geldenhuys said:

Why reinvent the wheel with all these different string types and round-about discussions etc... Can we learn something from other multi-platform frameworks that solved these problems ages ago!

Who said they did? Who says their solution is more optimal than ours?

If we can't think of anything better then them, why are you using FPC instead of Java?

Unfortunately what those frameworks did isn't directly applicable to FPC. First, as I understand, they have only one (UTF-16) string type. This is good because it avoids whatever mess related to conversions, but not an option for FPC (because FPC requires compatibility with existing code).

Second, their string type is an object, therefore it is toolkit-specific rather than language-specific. It has very little, if any, to do with the compiler.

Third, QT is a shared library (and Java framework may also be treated as such), and it implements huge Unicode conversion tables without putting them into every executable.

Regards,
Sergei

Well, who said they didn't? Like I mentioned in my previous message, I google'd around and couldn't find any complaints. Compare that to the messages flying around in this mailing list! Plus here it seems like everybody has there own idea and can't agree on a single design.

Who says their solution is more optimal than ours?

Well knowing Trolltech and Qt, everything is optimal! Going with Trolltech's pass, they seem write very optimal code from graphics to base framework classes.

As for Java. They would might be slow, but the language seems pretty good. Plus have have a 10-15 years head start (however long Java has been available) to Free Pascal in the Unicode department. So it will be damn stupid if we don't look at what they have done and accomplished in that time.

If we can't think of anything better then them, why are you using FPC instead of Java?

Because my boss employed me for my Object Pascal skills! ;-) I'm just saying that both those frameworks have solved what Free Pascal is trying to do now. Plus they solved it years ago, so any issue would have been detected and resolved long ago. Not using there experiences to help us implement Unicode support would be dumb.

I quickly put together the following test program. Compiled it and ran it on Linux and Windows. The results where the same and worked perfectly. That's just one example of UpperCase usage already solved.

---------[ StringDemo.java ]---------------
public class StringDemo {
public static void main(String[] args) {

        String s;
        String n;

        s = "abc";
        n = s.toUpperCase();
        System.out.println(s + " --> " + n);
        s = "Fahrvergnügen";
        n = s.toUpperCase();
        System.out.println(s + " --> " + n);
        s = "ਊ";
        n = s.toUpperCase();
        System.out.println(s + " --> " + n);
        s = "ḛ";
        n = s.toUpperCase();
        System.out.println(s + " --> " + n);
    }
}
------------------------

-------------[ output ]---------------
java StringDemo
abc --> ABC
Fahrvergnügen --> FAHRVERGNÜGEN
ਊ --> ਊ
ḛ --> Ḛ
---------------------------------

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Sergei Gorelkin wrote

Third, QT is a shared library (and Java framework may also be treated as such), and it implements huge Unicode conversion tables without putting them into every executable.

Not every executable. Only if you want to use unicode, you can stay with ansistring.

Or you can try to convert the RTL and the LCL into libraries. There was some work in this direction.

Then for backward compatibility use UTF-8 as the only string type. That's what UTF-8 was designed for - backward compatibility. That's also the reason LCL uses UTF-8.

Second, their string type is an object,

So why can't one flatten that object structure into procedures.

therefore it is toolkit-specific rather than language-specific. It has very little, if any, to do with the compiler.

Hence the reason I mentioned Qt. It uses the C compiler, which doesn't even have a String type. I think it will still be handy to look at the Qt toolkit, to see how they accomplished the unicode string issues.

Third, QT is a shared library (and Java framework may also be treated as such), and it implements huge Unicode conversion tables without putting them into every executable.

Well can't one make that a separate library for Free Pascal applications as well. If you don't need it, or just simply want ASCII upper/lower case support, you don't ship the conversion table dll/so file.

How else was FPC thinking of handling Unicode conversion tables then? Simply ignoring the problem is not going to make it go away.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

.
.

As already stated several times, UTF-8 is not backwards compatible to the various character set encodings in active use (even with FPC), UTF-8 is only backwards compatible to plain 7-bit ASCII.

Tomas

Graeme Geldenhuys wrote

Then for backward compatibility use UTF-8 as the only string type. That's what UTF-8 was designed for - backward compatibility. That's also the reason LCL uses UTF-8.

You cannot use *only* utf-8, the existing ShortString, AnsiString and Wide/UnicodeString types have to be maintained.

Hence the reason I mentioned Qt. It uses the C compiler, which doesn't even have a String type. I think it will still be handy to look at the Qt toolkit, to see how they accomplished the unicode string issues.

Well, it isn't C but C++ which allows to overload "everything but a kitchen sink". By overloading the assignment operators, you can e.g. make any class refcounted.
But it is easier to use C++ directly instead of trying to have all its features in FPC.

That makes me wonder, too - why all these discussions are ultimately about putting everything into the compiler.

Regards,
Sergei

In our previous episode, Sergei Gorelkin said:

FPC also doesn't have the "isolation from the system" philosophy in the magnitude that Java has.

I can only guess about QT's reasons. In Kylix times I once heard a rumour that QT was mostly modeled after NT's api philosophy, to make it programmers easier to migrate. (it was said that was also why CG chose it instead of GTK, though I can imagine GTK bugs also could have been a (more) major reason)

In our previous episode, Graeme Geldenhuys said:

I didn't drag them into the conversation.

I quickly scanned the rest of the message, and still didn't find anything about the design requirements of these, except that uppercase works after ten years.

I'm sure that whatever we choose, in ten years unicode uppercase will work.

So, to cut this part of the discussion off, please come with real design criteria from these platforms, and show that they apply to FPC, or forget about it, because this is a total waste of time in a discussion that is already complicated enough.

Marco van de Voort wrote

I can only guess about QT's reasons. In Kylix times I once heard a rumour that QT was mostly modeled after NT's api philosophy, to make it programmers easier to migrate.

I have never heard that and can't think that's it's true. The Qt API is quite different. Plus the original designers didn't come from a Windows platform.

Take this quotation for example: "The toolkit was called Qt because the letter Q looked appealing in Haavard's Emacs font, and "t" was inspired by Xt, the X toolkit." ― "A Brief History of Qt" at http://safari.oreilly.com/0131872494/pref04

Plus Qt was first announced on comp.os.linux.announce which from the name you can guess is not Windows related.

So based on the whole history of Qt article, I don't see anything Windows related in there... ;-)

(it was said that was also why CG chose it instead of GTK, though I can imagine GTK bugs also could have been a (more) major reason)

I heard the Qt model was easer to apply to VCL than the GTK model. And the quote below seems to agree with that. And you might have a point about the GTK bugs. ;-)

"Much analysis and debate was had over GTK vs QT as the underlying drawing and widget lib because unlike Windows, Linux had/has a fractured UI community and there certainly wasn't a GDI or native controls on Linux, unless you used Wine which we decided to use for porting parts of our IDE, but not for development or end user apps. There are good reasons for going either way, in the end we chose QT because we liked where KDE was going and felt it was closer to the Win32/GDI direction and therefore cleaner to wire up to VCL." ― http://blog.marcocantu.com/blog/kylix_inside.html

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/