I never really used the UnicodeString (or WideString for that matter) - I've always used AnsiString with UTF-8 content. I also have my own UTF8 functions Copy(), Length() etc.

Looking at UnicodeString - with FPC 2.6.4 I seem a bit confused. :-/

Take the following code:

============================
{$mode objfpc}{$h+}
{--- $codepage utf8} // disabled

var
  S: UTF8String; // for FPC 2.6.4 this is an alias for AnsiString
  U: UnicodeString;
begin
  S := 'Tiburón';
  WriteLn(Length(S));
  U := 'Tiburón';
  WriteLn(Length(U));
============================

On my 64-bit FreeBSD system that outputs the following:

==========
10
8
==========

Length() returns the number of bytes, correct?

So why isn't the result 8 and 14? The letter o with acute is 2-bytes in UTF8 ($C3 & $B4). For Unicode (UTF-16), where a "character" is a word size (2-bytes), thus 2 bytes * 7 characters = 14 bytes. But Length() returns totally different values to what I expected.

Enabling the {$codepage utf8} made no difference to the results shown above.

Could anybody explain this please?

Regards,
- Graeme -

Graeme Geldenhuys wrote

Length() returns the number of bytes, correct?

It returns the number of ansi/widechars. In case of ansichars, that equals the number of bytes.

So why isn't the result 8 and 14? The letter o with acute is 2-bytes in UTF8 ($C3 & $B4).

That depends on whether the character is composed or decomposed (this depends on your text editor and its settings, not on the compiler). If it's decomposed, then you get (an) extra character(s) for the decomposed "´" following the "o" (2 bytes in utf-8, 1 widechar in utf-16).

Jonas

Jonas Maebe wrote

It returns the number of ansi/widechars. In case of ansichars, that equals the number of bytes.

Ah, not bytes, so Length returns the number of elements [like in an array]. Thanks for clearing that up.

Regards,
- Graeme -