Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2016-04

Problem with non-ascii chars fpc-3.0.0 vs fpc-3.1.1 under Windows

6 messages · 3 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

zeljko

Hi,
Just run into problem with fpc-3.0.0, OS Windows-10 64bit, 32bit application, lazarus trunk qtlcl and win32 ws both have same problem.

fpc-3.1.1 (18.01.2016) is correct - both running from lazarus or as standalone.

fpc-3.0.0 is not correct in both cases (even worse if running from lazarus, then GetEnvironmentVariableUTF8 returns garbage instead of letter ?.

fpc-2.6.4 - works just fine.

My project doesn't use anything special - pure empty project with one form and TMemo - no extra settings like -FcUTF8 or similar, using mode delphi ....

Problematic part is line with CACHEDIR= (see screenshoots) which is correct with 3.1.1 and it returns correct path from this function:

function GetCacheDir_: String;
{$IFDEF MSWINDOWS}
var
   s: string;
   s1: string;
{$ENDIF}
begin
   {$IFDEF UNIX}
   Result := GetEnvironmentVariable('HOME') + PathDelim + '.holobit';
   {$ELSE}
   // fpc-3.1.1 returns correct values here, while fpc-3.0.0 NOT
   // if environmentvariable contains non-ascii chars
   s := GetEnvironmentVariable('HOMEDRIVE');
   // added s := s+ to fit into mail ...
   s := s + GetEnvironmentVariable('HOMEPATH')+PathDelim+'.holobit';
   SetLength(s1, length(s));
   OEMToAnsiBuff(@S[1], @S1[1], length(S1));
   Result := S1;
   {$ENDIF}
end;

So, my question would be: is it known bug in 3.0.0 and will be fixed in 3.0.2 or it's already merged in fixes_3_0 or I can sit and cry...or I'm missing something ?

Thanks

zeljko

-------------- next part --------------
A non-text attachment was scrubbed...
Name: shot_3_0_0.png
Type: image/png
Size: 9419 bytes
Desc: not available
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20160427/b2f17c59/attachment-0002.png> -------------- next part --------------
A non-text attachment was scrubbed...
Name: shot_3_1_1.png
Type: image/png
Size: 10180 bytes
Desc: not available
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20160427/b2f17c59/attachment-0003.png>

Reply by email

Mattias Gaertner
to zeljko
Show quoted text 38 lines
Hi, Just run into problem with fpc-3.0.0, OS Windows-10 64bit, 32bit application, lazarus trunk qtlcl and win32 ws both have same problem. fpc-3.1.1 (18.01.2016) is correct - both running from lazarus or as standalone. fpc-3.0.0 is not correct in both cases (even worse if running from lazarus, then GetEnvironmentVariableUTF8 returns garbage instead of letter ?. fpc-2.6.4 - works just fine. My project doesn't use anything special - pure empty project with one form and TMemo - no extra settings like -FcUTF8 or similar, using mode delphi .... Problematic part is line with CACHEDIR= (see screenshoots) which is correct with 3.1.1 and it returns correct path from this function: function GetCacheDir_: String; {$IFDEF MSWINDOWS} var s: string; s1: string; {$ENDIF} begin {$IFDEF UNIX} Result := GetEnvironmentVariable('HOME') + PathDelim + '.holobit'; {$ELSE} // fpc-3.1.1 returns correct values here, while fpc-3.0.0 NOT // if environmentvariable contains non-ascii chars s := GetEnvironmentVariable('HOMEDRIVE'); // added s := s+ to fit into mail ... s := s + GetEnvironmentVariable('HOMEPATH')+PathDelim+'.holobit'; SetLength(s1, length(s)); OEMToAnsiBuff(@S[1], @S1[1], length(S1));

OEMToAnsiBuff expects a string in DOS codepage and returns a string in Windows codepage.
GetEnvironmentVariableUTF8 returns an UTF-8 encoded string.

What encoding do you need for the function result?

Show quoted text 7 lines
Result := S1; {$ENDIF} end; So, my question would be: is it known bug in 3.0.0 and will be fixed in 3.0.2 or it's already merged in fixes_3_0 or I can sit and cry...or I'm missing something ?

Mattias

Reply by email

zeljko
to Mattias Gaertner
Mattias Gaertner wrote 44 lines
On Wed, 27 Apr 2016 20:30:43 +0200 zeljko wrote: > Hi, > Just run into problem with fpc-3.0.0, OS Windows-10 64bit, 32bit > application, lazarus trunk qtlcl and win32 ws both have same problem. > > fpc-3.1.1 (18.01.2016) is correct - both running from lazarus or as > standalone. > > fpc-3.0.0 is not correct in both cases (even worse if running from > lazarus, then GetEnvironmentVariableUTF8 returns garbage instead of > letter ?. > > fpc-2.6.4 - works just fine. > > > My project doesn't use anything special - pure empty project with one > form and TMemo - no extra settings like -FcUTF8 or similar, using mode > delphi .... > > Problematic part is line with CACHEDIR= (see screenshoots) which is > correct with 3.1.1 and it returns correct path from this function: > > function GetCacheDir_: String; > {$IFDEF MSWINDOWS} > var > s: string; > s1: string; > {$ENDIF} > begin > {$IFDEF UNIX} > Result := GetEnvironmentVariable('HOME') + PathDelim + '.holobit'; > {$ELSE} > // fpc-3.1.1 returns correct values here, while fpc-3.0.0 NOT > // if environmentvariable contains non-ascii chars > s := GetEnvironmentVariable('HOMEDRIVE'); > // added s := s+ to fit into mail ... > s := s + GetEnvironmentVariable('HOMEPATH')+PathDelim+'.holobit'; > SetLength(s1, length(s)); > OEMToAnsiBuff(@S[1], @S1[1], length(S1)); OEMToAnsiBuff expects a string in DOS codepage and returns a string in Windows codepage.

Yes, I know that ,but that's not problem...look below...

GetEnvironmentVariableUTF8 returns an UTF-8 encoded string. What encoding do you need for the function result?

UTF-8, but question is why 3.1.1 (and 2.6.4) returns correct result but 3.0.0 NOT with same project and same settings.

Another thing is that project when running from IDE with fpc-3.0.0 doesn't get correct result even for GetEnvironmentVariableUTF8(), while 3.1.1 is correct again.

zeljko

Reply by email

zeljko
to Mattias Gaertner: On Wed, 27 Apr 2016 20:30:43 +0200 OEMToAnsiBuff expects a string in DOS codepage and…
Mattias Gaertner wrote
On Wed, 27 Apr 2016 20:30:43 +0200 zeljko wrote:
GetEnvironmentVariableUTF8 returns an UTF-8 encoded string.

This is not true under windows 10 + fpc-3.0.0 when user name is ?eljko. CurrentDir & CurrentDirUTF8 returns C:\Users\?eljko\test GetAppConfigDirUTF8(False) returns C:\Users\?eljko\test

GetEnvironmentVariable &
GetEnvironmentVariableUTF8
returns C:\Users\L"eljko <----- looks like wrong.

What encoding do you need for the function result?

Same result as from eg GetAppConfigDirUTF8(False), so UTF8 but only homedir part (C:\Users\?eljko)

zeljko

Reply by email

Bart
to zeljko
zeljko wrote 7 lines
This is not true under windows 10 + fpc-3.0.0 when user name is ?eljko. CurrentDir & CurrentDirUTF8 returns C:\Users\?eljko\test GetAppConfigDirUTF8(False) returns C:\Users\?eljko\test GetEnvironmentVariable & GetEnvironmentVariableUTF8 returns C:\Users\L"eljko <----- looks like wrong.

On Win7 (Lazarus trunk, fpc 3.0 32-bit) at least for unicode characters inside my codepage GetEnvironmentVariableUTF8 retruns correct string, encoded as UTF8, however StringCodePage is CP_ACP. Not sure it this matters at all for your usecase, but GetAppConfigDir returns a variable in UTF8 encoding where StringCodePage is CP_UTF8.

Example with
set zeljko=???
Result of StringCodePage in sqaure brackets:

GetEnvironmentVariableUtf8: zeljko = $C3 $A4 $C3 $AB $C3 $AF [0] GetEnvironmentVariable with UnicodeString: zeljko = $00E4 $00EB $00EF [1200] UTF16ToUTF8 on result of previous line: zeljko = $C3 $A4 $C3 $AB $C3 $AF [65001] GetAppConfigDirUtf8: .... [65001]

As a workaround you may try to use the Unicode variant of GetEnvironmentVariable and convert it to Utf8 with UTF16ToUtf8(). The resulting string (on my system) then has stringCodePage = CP_UTF8 again.

Bart

Reply by email

zeljko
to Bart
Bart wrote 12 lines
Example with set zeljko=??? Result of StringCodePage in sqaure brackets: GetEnvironmentVariableUtf8: zeljko = $C3 $A4 $C3 $AB $C3 $AF [0] GetEnvironmentVariable with UnicodeString: zeljko = $00E4 $00EB $00EF [1200] UTF16ToUTF8 on result of previous line: zeljko = $C3 $A4 $C3 $AB $C3 $AF [65001] GetAppConfigDirUtf8: .... [65001] As a workaround you may try to use the Unicode variant of GetEnvironmentVariable and convert it to Utf8 with UTF16ToUtf8(). The resulting string (on my system) then has stringCodePage = CP_UTF8 again.

Thanks, that workaround works, but with 2.6.4 have another type of problem: 1.Home dir is C:\Users\?eljko
2.DirectoryExistsUTF8, any file inside $HOME FileExistsUTF8 and other file routines from LazFileUtils works like a charm. 3.IniFiles - TMemIniFile uses UpdateFile, and of course eg. C:\Users\?eljko\myinifile.ini fails to update because UpdateFile inifiles uses ForceDirectories() ...
4.I said ok, let's create it with:
a) UTF8ToSys(AFileName) - does not work
b) UTF8ToConsole(AFileName) - does not work
c) UTF8ToAnsi(AFIleName) - does not work
d) LConvEncoding.UTF8ToCp1250(AFIleName) WORKS

So, have I missed something about UTF8ToSys() and others in case of
fpc-2.6.4 ?

NOTE: I'm talking about fpc-2.6.4 ! fpc-3.0.0 works correct in this case

zeljko

Reply by email