Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2006-04

XML and encoding problems

4 messages · 2 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Alexander Todorov

Hello,
I am developing an application in Lazarus that is multi platform. It has very strong usage of xml that contains contains cyrillic text (Bulgarian language). All xml feeds are UTF-8 encoding and is converted internaly using iconv (UTF-8 -> CP1251 and vice versa) when needed. On Linux this is working very well, but on Windows it is not. I have iconv.pas and link to iconv.dll but the problem is that when a xml document is read from file / stream after that the contents of the document are not UTF-8 encoded and iconv fails.
Attached is a simple program that reads a file and prints xml values.

On Linux the result is :
./project1
user=Тихомир Руменов Тотев

This is correct. Tested with gnome-terminal on native Linux system and with Bitvise Tunelier ssh client for Windows.

On Windows the result is :
E:\TMP\utf>project1.exe
user=????N???????N? ?аN????╡?????? ????N??╡??

This is not correct. I tried debugging this but couldn't find the problem. Compiled with Lazarus 0.9.14 / FPC 2.0.2.

Please give some help / hints / workaround. This is very cruical for me ATM.

Thanks.

Reply by email

Bogusław Brandys
to Alexander Todorov
Alexander Todorov wrote 29 lines
Hello, I am developing an application in Lazarus that is multi platform. It has very strong usage of xml that contains contains cyrillic text (Bulgarian language). All xml feeds are UTF-8 encoding and is converted internaly using iconv (UTF-8 -> CP1251 and vice versa) when needed. On Linux this is working very well, but on Windows it is not. I have iconv.pas and link to iconv.dll but the problem is that when a xml document is read from file / stream after that the contents of the document are not UTF-8 encoded and iconv fails. Attached is a simple program that reads a file and prints xml values. On Linux the result is : ./project1 user=Тихомир Руменов Тотев This is correct. Tested with gnome-terminal on native Linux system and with Bitvise Tunelier ssh client for Windows. On Windows the result is : E:\TMP\utf>project1.exe user=????N???????N? ?аN????╡?????? ????N??╡?? This is not correct. I tried debugging this but couldn't find the problem. Compiled with Lazarus 0.9.14 / FPC 2.0.2. Please give some help / hints / workaround. This is very cruical for me ATM. Thanks.

Result printed on console are not sufficient.Compare binary chunks, then later check for system locale support.

Regards
Boguslaw

Reply by email

Alexander Todorov
to Bogusław Brandys
Bogusław Brandys wrote
Result printed on console are not sufficient.Compare binary chunks,

C:\>fc -B save.txt.windows save.txt.linux
Comparing files save.txt.windows and save.txt.linux 00000005: 3F D0
00000006: 3F A2
00000007: 3F D0
00000008: 3F B8
00000009: 4E D1
0000000A: 3F 85
0000000B: 3F D0
0000000C: 3F BE
0000000D: 3F D0
0000000E: 3F BC
0000000F: 3F D0
00000010: 3F B8
00000011: 4E D1
00000012: 3F 80
00000014: 3F D0
00000016: 4E D1
00000017: 3F 83
00000018: 3F D0
00000019: 3F BC
0000001A: 3F D0
0000001C: 3F D0
0000001D: 3F BD
0000001E: 3F D0
0000001F: 3F BE
00000020: 3F D0
00000021: 3F B2
00000023: 3F D0
00000024: 3F A2
00000025: 3F D0
00000026: 3F BE
00000027: 4E D1
00000028: 3F 82
00000029: 3F D0
0000002B: 3F D0
0000002C: 3F B2
0000002D: 0D 0A
FC: save.txt.windows longer than save.txt.linux

then later check for system locale support.

How do I do that?
On Linux:
locale
LANG=bg_BG.UTF-8
LANGUAGE=bg,mk
LC_CTYPE="bg_BG.UTF-8"
LC_NUMERIC="bg_BG.UTF-8"
LC_TIME="bg_BG.UTF-8"
LC_COLLATE="bg_BG.UTF-8"
LC_MONETARY="bg_BG.UTF-8"
LC_MESSAGES="bg_BG.UTF-8"
LC_PAPER="bg_BG.UTF-8"
LC_NAME="bg_BG.UTF-8"
LC_ADDRESS="bg_BG.UTF-8"
LC_TELEPHONE="bg_BG.UTF-8"
LC_MEASUREMENT="bg_BG.UTF-8"
LC_IDENTIFICATION="bg_BG.UTF-8"
LC_ALL=

How to check on Windows?

Is this enough to report as a bug?

Reply by email