Hi,

I just enabled WindowsUnicodeSupport in the svn version for Lazarus (revison 14883), if fpc 2.2.1 or later is used. This effects all applications compiled with the win32 interface.

The main difference is how the win32 interface expects that strings from the LCL are encoded, the old way is was the system char set, in the new way is all strings are treated as UTF-8. This is already the default for most other widget sets, so this change improves portability of LCL applications.

As a consequence source files must be encoded as UTF-8 too, otherwise they cannot be shown in the Lazarus editor.

We invite all windows users to test this and report bugs to the bug tracker. If no major bugs are found, then this will be the default for the next Lazarus release too. Lazarus 0.9.26 is scheduled a couple of weeks after the fpc 2.2.2 release because of a widestring bug in fpc 2.2.0.

For more information about LCL and Unicode support, see: http://wiki.lazarus.freepascal.org/LCL_Unicode_Support#Unicode-enabling_the_win32_interface

To go back to the old situation compile the LCL with -dDisableWindowsUnicodeSupport.

Vincent

<snipped>

Great work, Vincent.

Will there not be a Windows installer released until the 0.9.26 release I assume? I've tried several time to build lazarus on windows from source, but I can never seem to get it to work. Linux on the other hand seems to be a cinch.

Also, other than encoding existing source files to UTF-8, are there any changes to the user of strings in general or is this abstracted behind LCL?

Thanks,

There are snapshots:
http://wiki.lazarus.freepascal.org/Lazarus_Snapshots_Downloads or
ftp://ftp.hu.freepascal.org/pub/lazarus

Also, other than encoding existing source files to UTF-8, are there any changes to the user of strings in general or is this abstracted behind LCL?

Take care in passing file names to the RTL and the fileutils unit. Strings in the LCL are generally encoded in UTF8, but RTL and fileutils are the system char set.

Vincent

Thanks, I didn't know those existed in binary form. I just thought they were only soruce.

Sorry to be so ignorant on the subject, are you saying the calls to RTL fileutils need to have the "strings" cast to UTF-8 somehow?

Thanks again,

Lee Jenkins wrote

Sorry to be so ignorant on the subject, are you saying the calls to RTL fileutils need to have the "strings" cast to UTF-8 somehow?

I just made a small wiki guide:

http://wiki.lazarus.freepascal.org/LCL_Unicode_Support#Instructions_for_users

Felipe Monteiro de Carvalho wrote

I just made a small wiki guide: http://wiki.lazarus.freepascal.org/LCL_Unicode_Support#Instructions_for_users

Thanks.

Vincent

Correct me when i am wrong, when declare a variable as AnsiString and assign it to Utf8String (or Widestring) it automatically converted without using AnsiToUTF8

var
 MyString: AnsiString;
begin
 MyString := SomeRTLRoutine; //this routine return AnsiString
 MyTEdit.Text := MyString;
end;

Zaher Dirkey wrote

Correct me when i am wrong, when declare a variable as AnsiString and assign it to Utf8String (or Widestring) it automatically converted without using AnsiToUTF8

Maybe it will work like that in the future, but not today.

Today ansistring = utf8string

Excellent entry, Felipe. Just to be sure that I am not off base, the following would have to the case?

var
   lString1, lString2: string;
   lDate: TDateTime;
begin
   lDate := Now;
   lString1 := 'My %s string';
   lString2 := Format(UTF8ToAnsi(lString1), UTF8ToAnsi('Sample'));
end;

Won't that break Delphi compatibility in many places? Again, my apologies as I try to grasp the concepts here.

Thanks for your patience.

Sorry, missed the brackets in format function:

lString2 := Format(UTF8ToAnsi(lString1), [UTF8ToAnsi('Sample')]);

Lee Jenkins wrote

Won't that break Delphi compatibility in many places? Again, my apologies as I try to grasp the concepts here.

Yes. Present and previous Delphi versions have no unicode support.

We cannot have a full cross-platform unicode support and still be 100% delphi compatible.

If you attein to purely ascii software, we are still delphi compatible. If you use other encodings, you will need to convert your source code to utf-8, and as a bonus you win a consistent unicode support.

thanks,

->If you use other encodings, you will need to convert your source code to utf-8,

That break my project, because i use my language in source code and compile it in both Lazarus for WinCE and Delphi For Win32, but every project have own Forms, and all my other code shared between the both projects.

The best idea now for me, i must make PO files and put my language inside it.

OK, now you've got me thinking. I've restricted my resourcestring declarations to a single file so with the exception of that, I should be OK?

Thanks again for responde, Felipe. I'm going to have to do some more reading up on Unicode. Personally, I don't mind using utf-8, I just am still unsure as to what specific actions I need to take in order to be compliant.

En/na Vincent Snijders ha escrit:

As a consequence source files must be encoded as UTF-8 too, otherwise they cannot be shown in the Lazarus editor.

I've been using lazarus under windows with unicode for a while (as you say, to ease compatibility with other platforms), and I used iconv to convert my files:

http://gnuwin32.sourceforge.net/packages/libiconv.htm

The only minor problem I had is that I have to use utf8decode/utf8encode to interface with some rtl functions (e.g., xmlconfig, while using utf8 as a file encoding, it doesn't work if the strings used with it are already utf8).
No bugs with the LCL itself, at least for me.

Bye

En/na Felipe Monteiro de Carvalho ha escrit:

Mmmh, so I guess my using utf8encode/utf8decode instead of Utf8ToAnsi/AnsiToUtf8 is wrong?

Bye

En/na Luca Olivetti ha escrit:

Answering to myself: in my copy of lazarus ("ancient" snapshot 2008-06-23)/fpc those functions are defined in wstringh.inc, *not* in LCLIntf as the wiki says, and they simply call utf8encode/utf8decode

Bye

There are other solutions. In the Virtual Magnifying Glass I have isolated all strings inside a string managing class which is capable of translating the strings. All strings are stored as string literals in utf-8. Under Delphi I convert all strings from utf-8 to ansi. Take a look at this file:

http://magnifier.svn.sourceforge.net/viewvc/magnifier/trunk/translationsvmg.pas?view=markup

The Virtual Magnifying Glass is the best example of why this conversion that may initially seam annoying is absolutely needed.

The VMG has been translated to several languages, including Russian. If you compile the VMG in Delphi you cannot see the russian translation, unless you are using a russian Windows. And if you are in russian windows you can't see the other languages.

If you use the Lazarus unicode support you can see the any translation anywhere, and that's the kind of high quality unicode support that I'm talking about here.

Luca Olivetti wrote

Answering to myself: in my copy of lazarus ("ancient" snapshot 2008-06-23)/fpc those functions are defined in wstringh.inc, *not* in LCLIntf as the wiki says, and they simply call utf8encode/utf8decode

You are right. They were previously in LCLIntf, but were added to the RTL.

UTF8Encode converts from utf-16 to utf-8

utf8decode converts from utf-8 to utf-16

You can use them if you are using widestrings.

En/na Felipe Monteiro de Carvalho ha escrit:

And AnsiToUtf8/Utf8ToAnsi simply call Utf8Encode/Utf8Decode, so they end up doing the same ?!?!
Maybe the difference lies in the automatic typecasting from widestring to ansistring?

You can use them if you are using widestrings.

I'm not (knowingly) using widestrings, but I think the system function/methods I'm calling have an overloaded version that accepts widestrings.

Bye

AnsiToUtf8 := Ansi -> WideString + WideString -> UTF-8.

For Ansi -> WideString the conversion utilities of the widestring manager are used. For WideString -> UTF-8 UTF8Encode is used.

Vincent

Yes. A string given to widestring calls internally the widestring manager and converts ansi to UCS2/UTF-16.
A string conversion manager with less overhead is planned.