Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2016-09

UTF8 issue

5 messages · 2 people ·

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Giuliano Colla

I'm unsure whether this is a Lazarus issue, an fpc issue, or if I'm doing something wrong.

I'm in Linux environment, where everything is supposed to be UTF8. LANG environment is it_IT_.UTF-8, and I do not declare anything about codepages, leaving Lazarus and fpc work with their defaults.

1) Under Lazarus 1.6 and fpc 3.0

If I declare

   CamelStrings: array [0..5] of String =(
   ....
   'Entry éèàù');

and I load the strings in a ComboBox, the ComboBox displays

Entry éèàù

as expected

But declaring

   CamelStrings: array [0..5] of UTF8String =(
   ....
   'Entry éèàù');

then the ComboBox displays

Entry éèà ù

2) Under Lazarus 1.4.5 and fpc 2.6.4

Declaring either String or UTF8String doesn't make any difference, and the ComboBox always displays the proper string.

What I'm missing? Is that an "improvement" generated by fpc 3.0.0 or I'm doing something wrong?

Giuliano

Reply by email

Mattias Gaertner
to Giuliano Colla
Show quoted text 8 lines
[...] CamelStrings: array [0..5] of UTF8String =( .... 'Entry éèàù'); then the ComboBox displays Entry éèà ù

FPC does not use your LANG variable. Instead it always assumes ISO-8859-1. See here:
http://wiki.lazarus.freepascal.org/FPC_Unicode_support#String_constants

Same for WideString and UnicodeString.

Show quoted text 7 lines
2) Under Lazarus 1.4.5 and fpc 2.6.4 Declaring either String or UTF8String doesn't make any difference, and the ComboBox always displays the proper string. What I'm missing? Is that an "improvement" generated by fpc 3.0.0 or I'm doing something wrong?

Mattias

Reply by email

Giuliano Colla
to Mattias Gaertner

Il 07/09/2016 15:08, Mattias Gaertner ha scritto:

FPC does not use your LANG variable. Instead it always assumes ISO-8859-1. See here:

I could swallow it more easily if it were the opposite, i.e. the String type is misinterpreted and UTF8String is properly handled.

But what happens is that the String type is handled correctly, while UTF8String is not. IOW the compiler is told what to expect from a string and it messes it up.

Moreover calling SetMultiByteConversionCodePage(CP_UTF8) at program startup doesn't change a thing: String type is correct, UTF8String type is wrong.

fpc bug?

Giuliano

Reply by email

Mattias Gaertner
to Giuliano Colla
Show quoted text 16 lines
Il 07/09/2016 15:08, Mattias Gaertner ha scritto: > FPC does not use your LANG variable. Instead it always assumes > ISO-8859-1. See here: I could swallow it more easily if it were the opposite, i.e. the String type is misinterpreted and UTF8String is properly handled. But what happens is that the String type is handled correctly, while UTF8String is not. IOW the compiler is told what to expect from a string and it messes it up. Moreover calling SetMultiByteConversionCodePage(CP_UTF8) at program startup doesn't change a thing: String type is correct, UTF8String type is wrong. fpc bug?

This has become a FAQ:

http://wiki.lazarus.freepascal.org/Better_Unicode_Support_in_Lazarus#Why_does_UTF8String_show_strange_characters_and_String_works

Mattias

Reply by email