Now I could compile FPC 2.7.1 and perform some tests :-) Unfortunately I couldn't submit an bug report, so it comes here:

When compiled with Delphi XE, the code below shows 1252 (my DefaultSystemCodePage) for all strings, be empty or non-empty.

Compiled with FPC 2.7.1, the StringCodePage(r) is reported as zero after the first assignment (r := s;).
This is incompatible with Delphi :-(

program rawtest;
var
   s: AnsiString; //UTF8String;
   r: RawByteString;
begin
   s := 'abc';
   r := s;
   WriteLn('s CP: ',StringCodePage(s)); //1252
   WriteLn('r CP: ',StringCodePage(r)); //XE: 1252, FPC: 0
   s := r;
   WriteLn('s CP: ',StringCodePage(s)); //XE: 1252, FPC: 0
end.

This difference may be due to an error in StringCodePage, more probably in RawByteString handling in general.

Another one:
Delphi XE does not export the CP_xxx encoding constants from System.pas. This means that the encoding constants are not available in (compatible) user code.
CP_NONE is declared in Windows.pas for clipping, as: CP_NONE = 0; { No clipping of output } different from the CP_NONE encoding ($FFFF).

DoDi

"Hans-Peter Diettrich" wrote

Another one: Delphi XE does not export the CP_xxx encoding constants from System.pas.

This means that the encoding constants are not available in (compatible) user code.

It's not the first and likely not the last we export from a different unit than Delphi. There will *always* be differences which already starts with Integer that is declared as an alias to LongInt in the ObjPas unit in FPC.

CP_NONE is declared in Windows.pas for clipping, as: CP_NONE = 0; { No clipping of output } different from the CP_NONE encoding ($FFFF).

Do they really not have a CP_NONE constant in System?

Regards,
Sven

Hans-Peter Diettrich wrote on ma, 01 dec 2014:

To get behaviour that is compatible with Delphi2009+, compile with -Mdelphiunicode or {$modeswitch delphiunicode}. This particular difference is also documented in
http://wiki.freepascal.org/FPC_Unicode_support#String_constants (search for delphiunicode or systemcodepage)

Jonas

Hans-Peter Diettrich wrote on ma, 01 dec 2014:

I forgot to mention that the above also prints 0 in FPC if you don't compile in mode delphiunicode or with the systemcodepage modeswitch (by design).

WriteLn('r CP: ',StringCodePage(r)); //XE: 1252, FPC: 0 s := r; WriteLn('s CP: ',StringCodePage(s)); //XE: 1252, FPC: 0 end.

Jonas

Well, Integer (and String...) are generic types, adjustable to the "best" overall performance on every target - whatever "best" will mean.

No, not even in the definition of RawByteString, and not in any other standard (RTL...) source file (except Windows.pas, see above).

DoDi

The compiler option (-M) works, but the $modeswitch is not accepted by the compiler (2.7.1): Illegal compiler switch "DELPHIUNICODE". The same for {$mode ObjPas} - what else did I miss?

When I use Lazarus and set the compiler to the new 2.7.1, the modeswitch does not cause an error.

This particular difference is also documented in http://wiki.freepascal.org/FPC_Unicode_support#String_constants (search for delphiunicode or systemcodepage)

Thanks, that explains at least the FPC handling of literals.

But where can I find information about all the differences caused by above compiler option/modeswitch? Does it affect implicit AnsiString encoding conversions?

BTW it's nice that FPC console Write/Ln (mostly) converts AnsiStrings to the console codepage, while Delphi (XE) doesn't convert :-)

But I found a somewhat strange result with generic String variables, tested with:

var A: AnsiString; S: String;
begin
   S := ' äöü';
   A := S;
   //S := A; //changes nothing
   WriteLn('A CP: ',StringCodePage(A), A); //always shows ' äöü'
   WriteLn('S CP: ',StringCodePage(S), S); //letters differ
end.

When String is UnicodeString (DelphiUnicode), the output is correctly converted for both strings (CP 1200,1252). But when String is not UnicodeString, AnsiString and String should be the same type, no? The console however shows different letters for the generic String and AnsiString variable (both CP 1252). The output doesn't change when A is assigned back to S. How that?

When A and S are echanged:

   A := ' äöü';
   S := A;
   //A := S; //CP changes

the encoding of A is shown as zero. Now it makes a difference when S is assigned back to A, but only the codepage of A then also is shown as 1252, while the letters still differ. Obviously String is not equivalent to AnsiString now, and string literals should be assigned to AnsiString variables only, not to String variables?

Very confused
DoDi

As I said, it's merely beginning there. For example we also export resource related functions from System and with further work on packages functions like LoadLibrary will also be moved to System.

Regards,
Sven

To get behaviour that is compatible with Delphi2009+, compile with -Mdelphiunicode or {$modeswitch delphiunicode}.

Do you mean {$mode delphiunicode}?

Now I wonder about I compile a console program on the commandline, most strings are readable in the console (see previous answer). But when I compile using Lazarus, all strings (including UnicodeString!) are shown in unreadable UTF-8 encoding, regardless of $mode :-(

What causes this difference, and how to make strings readable in a (Lazarus compiled) console application?

Forgot to mention: everything on WinXP.

DoDi

Probably best to ask about the wrong behaviour with Lazarus on a Lazarus list? Otherwise: In what format (encoding) is your source file? Unless it's a UTF-8 with BOM, FPC decodes it according to the -Fc parameter and Lazarus may pass a different setting of this option. In addition, it might be related to Lazarus playing with Default*SystemCodePage which may not work well with console using a different encoding, but that is just a wild guess which would need to be checked by someone really knowing what Lazarus does there...

Tomas

You can set String=UnicodeString with
-Mdelphiunicode
or {$Mode delphiunicode}
or {$ModeSwitch unicodestrings}

All require 2.7.1.

Mattias

The DefaultSystemCodePage is by default not changed by Lazarus.

Lazarus by default saves files as UTF-8 without BOM. You can use other encodings as well.

Mattias

[...] BTW it's nice that FPC console Write/Ln (mostly) converts AnsiStrings to the console codepage, while Delphi (XE) doesn't convert :-)

+1

Although it confuses people who don't know about "console codepage" (OEM codepage).

What codepage is the source file?

Many things affect string literals. Source codepage, system codepage, string type, defaultsystemcodepage, library, compiler version.

I started a table for UTF-8 literals:
http://wiki.lazarus.freepascal.org/Character_and_string_types#String_constants

Mattias

It really seems to be a Lazarus problem. Compiled from an PAS file, the behaviour is equal to FPC. The bad encoding is used when compiled from an LPR file (LPI project).

Thanks
DoDi

Thanks, after some reading I changed the sourcefile encoding, and both UTF8bom and Ansi provide correct results. The Lazarus default (UTF-8 without BOM) is not usable on Windows :-(

DoDi

You need to add conversions.

With the new DefaultSystemCodePage many of them are no longer needed.

Mattias