The topic has been raised several times, but I have looked through the sources and documentation and there still are issues for discussion. First, I'm impressed by the Lazarus project which supports Unicode right now. It uses utf-8 encoding for this but according to the sources still uses simple AnsiString or just String types for storage. I'm very concerned of the tight and simple integration of LCL and RTL/FCL and see the page in wiki which tells to use Utf8ToAnsi convertion function for calls to RTL, to link properly the LCL and RTL. So, calling each time a conversion function is inconvenient.

Second, i haven't found any WideString or UTF8String typed standard file IO functions in the RTL that makes impossible to deal with files whose names contain national symbols not represented by the ANSI code page set in the Win32 for *A functions. (e.g. saving a french web page containing symbols with accents under a cyrillic-localized version of WinNT as a file whose name is the title of the page)

SUMMARY: Even if LCL supports unicode display, the RTL cannot deal with unicode filenames under Win32.

As far as i know, the problem concerns Win32 and WinCE only, i.e. Linux and MacOS use UTF- 8 as system encoding nowadays.

PROPOSAL: Make UTF-8 the default encoding (i.e. emulate it) for the FPC RTL under Win32 (NT-based versions) and make conversion at place where unicode-enabled system calls are needed, e.g. FileOpen/Create, FindFirst etc. No separate NT-only RTL is required for this.

Any ideas? I would like to do some work towards this if the solution is approved by masses.

Also any help is appreciated: 1) the list of string-related functions to change; 2) next char, previous char functions to skip 3-4 bytes of UTF-8 encoded character; 3) are there any locale calendar/date-time functions which return the name of the month, day?

fpGUI and MSEgui has custom wrapper functions that handle file/directory handling correctly. I don't know what Lazarus LCL does.

As far as i know, the problem concerns Win32 and WinCE only, i.e. Linux and MacOS use UTF- 8 as system encoding nowadays.

Win32, Win64 and WinCE uses UTF-16 since Win2000 days, not ANSI. Win9x uses ANSI as far as I know.

encoded character; 3) are there any locale calendar/date-time functions which return the name of the month, day?

For Windows this already works. I just had a discussion in this mailing list, highlighting a possible solution for *BSD and Linux systems.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Graeme Geldenhuys wrote

fpGUI and MSEgui has custom wrapper functions that handle file/directory handling correctly. I don't know what Lazarus LCL does.

Well, i'd like RTL be completely unicode-aware, so that any command-line tool for file management
would work well, not just to do with Lazarus.

Win32, Win64 and WinCE uses UTF-16 since Win2000 days, not ANSI.

Win9x uses ANSI as far as I know. Yes, but there are different library calls, you know, and FPC uses those

that accept ANSI strings not Wide ones (UTF-16)

I just want everything be consistent and solid as in Java and not just on per-library basis.

I think this would be really excellent and a reliable solution that can be quickly implemented, but based on previous discussions I believe most core developers are against it.

The most popular solution was the one proposed by Florian of having a string without defined encoding (it would be the native one, whatever it is, even utf-16 or utf-89), and one can have a call to force it's encoding into the desired one.

On Wed, Aug 6, 2008 at 4:42 AM, Felipe Monteiro de Carvalho <

So until the most recent Delphi release is out we have to use some workarounds as it is not 100% clear how it will be implemented in Tiburon :(. I will probably write all file, calendar whatever functions dependent on unicode support as a separate unit which will use unicode functions when under WinNT and fall back to RTL analogues on other systems. I will go for utf8 strings to be transparent to LCL. If the unit will be ok then it could be then included as part of the rtl in the future or somehow helpful anyway. Or is there such unit already available?

Converting the usual string to utf8 is problematic because it breaks compatibility with existing non-visual Pascal software. Because of this I initially proposed to create a new set of routines using widestring, which would be a utf-16 routines set. To use it with the LCL we can call conversion routines.

Most core developers insist that this would be too ineficient, but this is almost never an issue to convert the filenames and similars which are not the most time consuming task, unless your application only deals with filenames but do nothing with them. And specially in GUI software this conversion is no big deal.

I beliave there are no unicode alternative units in the RTL at the moment.

A big problem of the variable encoding as I already mentioned in the past is that it is more complex to implement and it´s not certain which problems and inconveniences it may pose to developers.