Hi,

Are there any rtl functions to compute the length (in characters) of a widestring and to iterate over its characters?
In my system (Mac OS X/fpc 2.1.3), a widestring is implemented as a UTF-8 encoded string, as I found out.

Thanks

Op Mon, 7 May 2007, schreef Christos Chryssochoidis:

Hi, Are there any rtl functions to compute the length (in characters) of a widestring and to iterate over its characters?

If you treat it as UCS-2: length(widestring_variable)

In my system (Mac OS X/fpc 2.1.3), a widestring is implemented as a UTF-8 encoded string, as I found out.

Not possible, a widestring is UCS-2/UTF-16.

Daniël

UTF-8

> encoded string, as I found out. Not possible, a widestring is UCS-2/UTF-16.

I defined a widestring with 7 characters (code points), and the length() function returned the value 15. Of the 7 code points of that widestring only one of them was greater than $07FF (the maximum code point which can be encoded in 2 bytes under UTF-8). When I changed that character with another one with code not greater than $07FF, length() returned value 14... I also printed the byte values of one of the widestring's widechars, and the values printed indicated UTF-8 encoding.

Thanks,
Christos

Op Mon, 7 May 2007, schreef Christos Chryssochoidis:

Yes, the program output is utf-8 on OS-X, because this is the native encoding for OS-X. However, widestrings are not utf-8. Can you show your code?

Daniël

length()

> function returned the value 15. Of the 7 code points of that

widestring only

> one of them was greater than $07FF (the maximum code point which can be > encoded in 2 bytes under UTF-8). When I changed that character with

another

> one with code not greater than $07FF, length() returned value 14...

I also

> printed the byte values of one of the widestring's widechars, and

the values

OK, I figured out what happened. The source file was saved in UTF-8 encoding, but I hadn't put in my source file the compiler directive {$CODEPAGE UTF8}. After including this directive in my code almost everything worked fine: length() was returning the right number of unicode characters, and subscripting the widestring returned the right character. But as the widechar and widestring encoding is, as you said, UTF-16, while my Mac OS X console uses UTF-8 encoding, for the output results to be displayed correctly I had to wrap the individual widechars or the whole widestring with the function utf8encode(), prior to output them with write()...

Thanks for your help,

Christos

Op Wed, 9 May 2007, schreef Christos Chryssochoidis:

This means no widestring manager was used; the default dumb widestring manager converts to iso-8859-1. Use the "cwstring" unit will use the system encoding when reading/writing widestrings.

Daniël

Daniël Mantione wrote

Op Wed, 9 May 2007, schreef Christos Chryssochoidis: > OK, I figured out what happened. The source file was saved in

UTF-8 encoding,

> but I hadn't put in my source file the compiler directive

{$CODEPAGE UTF8}.

> After including this directive in my code almost everything

worked fine:

> length() was returning the right number of unicode characters, and > subscripting the widestring returned the right character. But as

the widechar

> and widestring encoding is, as you said, UTF-16, while my Mac OS

X console

> uses UTF-8 encoding, for the output results to be displayed

correctly I had to

> wrap the individual widechars or the whole widestring with the

function

> utf8encode(), prior to output them with write()... This means no widestring manager was used; the default dumb

widestring

------------------------------------------------------------------------

I called setCWideStringManager() from my code, but it did not had any effect. A call to utf8encode() is still needed from proper output.

Op Wed, 9 May 2007, schreef Christos Chryssochoidis:

I called setCWideStringManager() from my code, but it did not had any effect. A call to utf8encode() is still needed from proper output.

It probably means the widestring manager thinks your system is iso-8859-1 encoded. If you think this is wrong, please report a bug.

Daniël