Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2009-08

Set of encoding conversion routines

8 messages · 4 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Felipe Monteiro de Carvalho

Hello,

Does anyone know if there is a set of encoding conversion routines? If there isn't, how should we add one? New directory in fpc/packages?

I just needed UTF-8 to Latin 1 ISO but I can't find a cross-platform solution. Remembering that UTF8ToAnsi doesn't solve the problem because it converts to the system encoding and not Latin 1 ISO.

There do is iconv, but does it already come installed in Windows? I couldnt find any iconv.dll or something like that.

thanks,

Felipe Monteiro de Carvalho

Reply by email

Marco van de Voort
to Felipe Monteiro de Carvalho

In our previous episode, Felipe Monteiro de Carvalho said:

Does anyone know if there is a set of encoding conversion routines? If there isn't, how should we add one? New directory in fpc/packages?

Cross platform, with a library?

The lowest level is unit charset in the rtl, Unix style iconv is supported by package iconvenc.

Show quoted text 6 lines
I just needed UTF-8 to Latin 1 ISO but I can't find a cross-platform solution. Remembering that UTF8ToAnsi doesn't solve the problem because it converts to the system encoding and not Latin 1 ISO. There do is iconv, but does it already come installed in Windows? I couldnt find any iconv.dll or something like that.

No. Non *nix OSes have their own api. There is an abstraction, lconvencoding, but that is in the LCL afaik.

Reply by email

theo
to Felipe Monteiro de Carvalho: Does anyone know if there is a set of encoding conversion routines? If there isn't, how…
Show quoted text 5 lines
I just needed UTF-8 to Latin 1 ISO but I can't find a cross-platform solution. Remembering that UTF8ToAnsi doesn't solve the problem because it converts to the system encoding and not Latin 1 ISO.

A simpe option might be:

wides:=UTF8Decode(utf8s);
;
ansis:=wides;

Maybe set back widestringmanager.Wide2AnsiMoveProc to the original proc after that.

Reply by email

Felipe Monteiro de Carvalho
to Marco van de Voort: In our previous episode, Felipe Monteiro de Carvalho said: Cross platform, with a library?
Marco van de Voort wrote
The lowest level is unit charset in the rtl, Unix style iconv is supported by package iconvenc.

The charset unit doesn't look complete or usable.

No. Non *nix OSes have their own api. There is an abstraction, lconvencoding, but that is in the LCL afaik.

In the latest lazarus svn lconvencoding has real encoding conversion tables! I don't know if they all work 100%, but I managed to strip LCL dependencies and leave only the encoding I want and the conversion from UTF-8 to Latin 1 ISO worked perfectly.

Felipe Monteiro de Carvalho

Reply by email

Marco van de Voort
to Felipe Monteiro de Carvalho: The charset unit doesn't look complete or usable. In the latest lazarus svn lconvencoding…

In our previous episode, Felipe Monteiro de Carvalho said:

Show quoted text 5 lines
On Tue, Aug 11, 2009 at 12:43 PM, Marco van de Voort wrote: > The lowest level is unit charset in the rtl, Unix style iconv is supported > by package iconvenc. The charset unit doesn't look complete or usable.

Afaik it is. The unit is just the main conversion, and can be plugged with units that are generated directly from unicode.org tables.

Our own cache of those tables are in ucmaps, and the tool to make units out of them is utils/creumap.pp. Examples for such units are compiler/cp*

The only problem is that it uses UTF16 internally iirc.

Show quoted text 7 lines
> No. Non *nix OSes have their own api. There is an abstraction, > lconvencoding, but that is in the LCL afaik. In the latest lazarus svn lconvencoding has real encoding conversion tables! I don't know if they all work 100%, but I managed to strip LCL dependencies and leave only the encoding I want and the conversion from UTF-8 to Latin 1 ISO worked perfectly.

Maybe expanding charset with a utf16<=>utf8 conversion, and making a package with units with some common encodings would do it also.

Reply by email