Hi

What I have understood so far:

WideString uses UTF16 encoding. It is managed by Windows and so is not reference counted by Delphi or FPC under Windows. However it is reference counted by FPC on other platforms.
Then FPC has UnicodeString which is identical with WideString except that it is always reference counted.

Delphi since v.2009 has UnicodeString as default. It also uses UTF16 encoding.
Now there is a new branch for Delphi compatible unicode strings in FPC.

What is the difference between the current FPC's UnicodeString and Delphi's UnicodeString? I am confused with all these string types and encodings.

What do you people say about Open XML: http://www.philo.de/xml/ While searching the net I found this:

"For handling other character encodings, you want to check out *Open XML*<http://www.philo.de/xml/>,
a free XML library for Delphi. As part of its XML handling, it has support for converting between 70 different encodings."

Juha Manninen

Juha Manninen wrote

What is the difference between the current FPC's UnicodeString and Delphi's UnicodeString? I am confused with all these string types and encodings.

Delphi's UnicodeString supports holding strings with arbitrary encodings (not just UTF-16), along with transparent re-coding of strings when assigning them to unicodestrings whose codepage has been statically defined at compile time.

Jonas

Jonas Maebe kirjoitti tiistai 28 joulukuu 2010 12:31:08:

Delphi's UnicodeString supports holding strings with arbitrary encodings (not just UTF-16), along with transparent re-coding of strings when assigning them to unicodestrings whose codepage has been statically defined at compile time.

Thanks.

Juha

In our previous episode, Jonas Maebe said:

This is afaik not true, or ill formulated. The D2009+ unicodestring type is much like FPC 2.4's Unicode string or Kylix and older FPC's non COM Widestring (refcounted 2-byte type) , it is the ansistring type (1-byte type) that gets codepage support.

Marco van de Voort kirjoitti perjantai 31 joulukuu 2010 13:04:27:

UTF-16 needs codepages, too.
I think only the 4-byte char type (is it UTF-32) would solve all encoding problems. All characters of all languages fit into 2^32 space.

Juha

2010/12/31 Juha Manninen :

UTF-16 needs codepages, too.

I don't think so. What would a code page be used for with UTF-16.

UTF-16 itself is already an encoding, implictly stating one and only one code page, if it can be named a code page.

Vincent

In our previous episode, Juha Manninen said:

codepageencoding

All characters of all languages fit into 2^32 space.

charactercodepoint.

Anyway surrogates etc is a different problem of processing true unicode spec, and the bits that UTF32 solves are the lesser ones. (and at the expense of speed and memory)

The document is a bit messy (since it was conceived before Delphi 2009 came out, and later updates for that), but

http://www.stack.nl/~marcov/unicode.pdf

has some details.

Vincent Snijders kirjoitti perjantai 31 joulukuu 2010 14:09:25:

Ok, right.
I think I wanted to write: UTF-16 needs variable length encoding, too. It is not guaranteed to be 2 bytes long although in most cases it is.

Juha

In our previous episode, Juha Manninen said:

Correct.

Now there is a new branch for Delphi compatible unicode strings in FPC. What is the difference between the current FPC's UnicodeString and Delphi's UnicodeString? I am confused with all these string types and encodings.

Afaik none, except in Delphi it is default. In FPC it is a beta feature, waiting for the codepage ansistring (which is being worked (*) in the branch that you name), before the (very massive!) rtl changes, and the option for unicodestring to be the default type become reality.

The actual form of the RTL changes and default types has not even been decided. There are various opinions, but no decision has been taken.

What do you people say about Open XML: http://www.philo.de/xml/

Never used it. I use FPC's XML routines, even with Delphi.