There was a discussion in the fpc -pascal mailing list about a question a user (tobiasgiesen) asked (among other things) about storing strings of a certain encoding brand in a TStringList.

Here Juha recommended to read this page ->
http://wiki.freepascal.org/Better_Unicode_Support_in_Lazarus

Now I did some test and asked ob that list and found that the default code page for the type "String" and with that the definition of of TStrings and with that the working of TStringList.Add and friends depends on the setting of "DefaultSystemcodepage". I understand that DefaultSystemcodepage is set when compiling the compiler (e.g. to UTF-8 on Linux and (supposedly) to UTF-16 in Windows).

So I suppose Lazarus can't do anything about that. Or are there plans to use different compilers /RTL variant to allow for
"Better_Unicode_Support_in_Lazarus" (project depending potentially better performance <with UTF16) or less memory resource requirement <with UTF8> or better portability <using unaware source code in Windows and Linux, explicitly using e.g. native pos() results instead of introducing CodePointPos(), ...) ?

-Michael

BTW. according to the said wiki page (at the end of the page) I am wrong assuming that DefaultSystemCodePage is a constant introduced by the compiler.

Now I still don't know whether/how the default encoding for the type "String (which is different from DefaultSystemCodePage according to the wiki) is depending on the arch/OS the compiler is built for. (I only tested on Linux and here it rather obviously is UTF8. I assume on Windows it's UTF16 for Delphi compatibility).

-Michael

Michael Schnell wrote

I assume on Windows it's UTF16 for Delphi compatibility

No, that is not correct, at least for {$mode objfpc}. Maybe {$mode delphi} is different, I'm not sure. The system code page varies based on locale settings. I just went through this exercise with fcl-pdf.

Regards,
- Graeme -

[...] http://wiki.freepascal.org/Better_Unicode_Support_in_Lazarus

Good page.

Now I did some test and asked ob that list and found that the default code page for the type "String" and with that the definition of of TStrings and with that the working of TStringList.Add and friends depends on the setting of "DefaultSystemcodepage".

The "type" of String does not change.

I understand that DefaultSystemcodepage is set when compiling the compiler (e.g. to UTF-8 on Linux and (supposedly) to UTF-16 in Windows).

No. DefaultSystemcodepage is the RTL and the
widestring manager.
Lazarus unit LazUTF8 sets it to CP_UTF8.

Mattias

"Michael Schnell" wrote

BTW. according to the said wiki page (at the end of the page) I am wrong

assuming that DefaultSystemCodePage is a constant introduced by the compiler.

Now I still don't know whether/how the default encoding for the type

"String (which is different from DefaultSystemCodePage according to the wiki) is depending on the arch/OS the compiler is built for. (I only tested on Linux and here it rather obviously is UTF8. I assume on Windows it's UTF16 for Delphi compatibility).

The code pages that are relevant here are only single byte code pages (e.g. CP1252) or UTF-8, *never* UTF-16 as a AnsiString can not store UTF-16 data.

And the DefaultSystemCodePage is determined upon startup from the system.

Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20160413/cb6d9d25/attachment.html>

I see. And using 8 bit encoding as the brand for not the explicitly user-defind "String" type does make sense for a compiler that is supposed to create executables for multiple OSes does make prefect sense.

But AFAIK, Unicode aware Delphi uses UTF16 for the not the explicitly user-defind "String" type. (While AFAIK, not Unicode aware Delphi can't handle UTF8.)

So this is not compatible to either of them. In fact I am not asking for such compatibility (let alone that the two Delphi variants are even more incompatible to each other), but we need to be aware of the issue.

-Michael

StringCodePage(s) with an unqualified String return 0 (which is "CP_ACP", and seemingly means "Default").

But how to determine what encoding this default is, if (as we found) it can't be DefaultSystemCodePage and can be UTF16, which is dynamic, while the default for unqualified strings is static and *never* UTF-16 ?

Michael

It means DefaultSystemCodePage.

But how to determine what encoding this default is, if (as we found) it can't be DefaultSystemCodePage and can be UTF16,

Why do you think that?

which is dynamic, while the default for unqualified strings is static and *never* UTF-16 ?

Mattias

Mattias Gaertner wrote

Why do you think that?

If that would be the way the function StringCodePage would need to check if the encodeing imposed on the string (e.g. a String constant) (i.e. the final meaning of CP_ACP) would be the same as the value of the variable DefaultSystemCodePage which has been set at this start of the program.

I looked at the code of StringCodePage, and it does not do so.

-Michael