Show quoted text 15 lines
On Saturday 14 April 2012 22:36:02 Marcos Douglas wrote:
>
> Well, works if I change this line:
> fname := 'c:\? b ?\?.txt';
> to this:
> fname := UTF8Decode('c:\? b ?\?.txt');
>
> And doesn't matter if fname is UnicodeString or string -- well, the
> debug hint to 'UnicodeString' is more beautiful than 'string' because
> the compiler translate.
>
Add {$codepage utf8} to the unit header or compile with -Fcutf8, this is the
default setting in MSEide+MSEgui for automatic Unicode handling.
Warning: most likely this setting will break Lazarus on FPC 2.6.0. I don't
know if Lazarus is fully tested with cpstrnew and -Fcutf8 already.
cpstrnew is part of fpc 2.7.1 and Lazarus runs fine with it since two months.
The -Fcutf8 and {$codepage utf8} exists since ages.
There are some traps with -Fcutf8 and {$codepage utf8}. It only works if the RTL DefaultSystemCodePage is CP_UTF8. Otherwise your strings are converted by the compiler.
For example under Linux the RTL default is CP_ACP, which defaults to ISO_8859-1. The RTL does *not* read your environment language on its own. So the the default is ISO_8859-1. This means your UTF-8 string constants are converted by the compiler:
Compile this with -Fcutf8 and run it on a Linux with LANG set to utf-8:
program project1;
{$mode objfpc}{$H+}
begin
writeln(DefaultSystemCodePage,' ',CP_UTF8);
writeln('?');
end.
This results in
0 65001
??
The LCL uses a widestringmanager (at the moment cwstring), which sets the DefaultSystemCodePage. You can do the same in your non LCL programs:
program project1;
{$mode objfpc}{$H+}
uses cwstring;
begin
writeln(DefaultSystemCodePage,' ',CP_UTF8);
writeln('?');
end.
This results in
65001 65001
?
The above is a lie for kids.
See the program below:
program project1;
{$mode objfpc}{$H+}
uses cwstring;
var
a,b,c: string;
begin
writeln(ord(DefaultSystemCodePage),' ',CP_UTF8);
a:='?'; b:='='#$C3#$A4; // #$C3#$A4 is UTF-8 for ?
c:= '?='#$C3#$A4;
writeln(a,b); // writes ?=?
writeln(c); // writes ?=??
end.
You can see that an UTF-8 string constant works, a string constant with UTF-8 codes works too, but the combination does not work. The above was compiled with -Fcutf8 and uses cwstring to set the DefaultSystemCodePage to CP_UTF8. So what went wrong? The compiler treats any non ascii string constant (here: the ?) as widestring (not UTF-16).
You can not fool the compiler with '?='+#$C3#$A4. You must define two separate string constants.
Using any character outside the UCS-2 range results in Fatal: illegal character "'?'" ($F0)
You can specify them with UTF-16 codes: #$D834#$DD1E Yes, you read right. Specifying the codepage with -Fcutf8 or {$codepage utf8} actually defines a mix of UTF-8 and UTF-16.
Now compile the above without -Fcutf8:
65001 65001
?=?
?=?
Wow, everything looks as expected. You can even mix ascii and non ascii string constants. Without the codepage the compiler stores string constants as byte sequences. That's what UTF-8 is.
That's why the LCL applications do not use the codepage flags.
msegui has implemented an ecosystem of widestrings, so it works better with a codepage.
Mattias
