Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2005-04

natural unicode support for WinNT via UTF-8

4 messages · 3 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Yury B.

Hello fpc-devel,

If there were full unicode support (TCHAR analogue) the life under Win32 would be simpler $) Two RTLs should be for Win32, one compiled for ANSI/DBCS and the other for UNICODE UCS-2 (WinNT), where a special type,
say TUniString is AnsiString in the first version and UnicodeString in the second version.
What do you think of it?

I need a unicode GUI and command line processing in my Win32 programms for they deal with a number of languages and intended for WinNT/2k/XP with natural unicode support.

When I say UNICODE support I don't mean automatic translation between 8-bit and 16-bit strings but calling *W API functions and using Unicode strings in parameters to file io, parameters etc. the rest functions like ToUTF8(), FromUTF8() can be provided as a separate unit.

Under Linux the life is easier because one can set UTF-8 for system encoding and use AnsiStrings or char* in their applications but under WinXX one have to use wchar_t and *W variants of Win32 API functions :( and maintain two versions of libraries...

Even better solution - to treat AnsiStrings as UTF-8 when UNICODE_UTF8 is defined, but convert UTF8 to WideString on system calls (Win API). But this can cause performance hits...

Though to treat AnsiStrings as UTF-8 is the best solution regarding compatibility, i.e. no need to rewrite FCL etc. as AnsiString or String types can be used to store UTF-8 strings.

As soon as I finish downloading FPC 1.9.8 sources I'll try to edit system unit for Win32 to understand UNICODE_UTF8 define and to build a unit using *W (widechar Win32 functions) and Utf8 conversion routines (MultiByteToUnicode() etc.)... So that file/console IO and parameters (paramstr) functions deal with Win32 API widechar functions and convert between UTF8 and UCS-2...

Does anybody work on this issue too? Please comment on it, maybe there's already something in the sources that I am not aware of?

Best regards, Yury

Reply by email

DrDiettrich
to Yury B.
"Yury B." wrote 5 lines
Two RTLs should be for Win32, one compiled for ANSI/DBCS and the other for UNICODE UCS-2 (WinNT), where a special type, say TUniString is AnsiString in the first version and UnicodeString in the second version. What do you think of it?

IMO there exists no need for separate libraries. The A and W versions of WinAPI functions are (? can be) overloaded, so that it's simply up to the coder to provide the appropriately typed parameters.

You can define types wchar_t etc. and set these to AnsiChar or WideChar, as desired.

Did I miss something?

DoDi

Reply by email

Florian Klaempfl
to DrDiettrich
DrDiettrich wrote 18 lines
"Yury B." wrote: > Two RTLs should be for Win32, one compiled for ANSI/DBCS and the other for > UNICODE UCS-2 (WinNT), where a special type, > say TUniString is AnsiString in the first version and > UnicodeString in the second version. > What do you think of it? IMO there exists no need for separate libraries. The A and W versions of WinAPI functions are (? can be) overloaded, so that it's simply up to the coder to provide the appropriately typed parameters. You can define types wchar_t etc. and set these to AnsiChar or WideChar, as desired. Did I miss something?

Probably ;) E.g. the whole file handling stuff uses the "A-versions" so it would require to make here at least overloaded versions. More complicated would be the file records, they store the names as char arrays.

Show quoted text 8 lines
DoDi _______________________________________________ fpc-devel maillist - http://lists.freepascal.org/mailman/listinfo/fpc-devel

Reply by email

Yury B.
to Florian Klaempfl

Hello Florian,

you wrote
> You can define types wchar_t etc. and set these to AnsiChar or WideChar, > as desired.

FK> the whole file handling stuff uses the "A-versions" so FK> it would require to make here at least overloaded versions. More FK> complicated would be the file records, they store the names as char arrays.

I looked through the sources of RTL and I think I won't do any changes to add unicode
because it will involve a lot of work and I have also looked in MSDN about console, it's more natural to use ANSI codepages than unicode because UNICODE in win32 needs special functions, e.g. WriteConsole instead of WriteFile ...

So unicode support under WinNT is the task of a specific app. developer.

Best regards, Yury

Reply by email