Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2009-04

DOM and GTK 1&2: how to set encoding?

5 messages · 3 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Marc Santhoff

Hi,

I have a DOM model read from a xml file that has no encoding named inside it. The var TXMLDocument.Encoding is empty after the file is loaded.

WHen compiling the program for GTK 1 the german umlauts are shown correctly, I assume GTK 1 defaults to system encoding (or so ;). The files have been written in a GUI using the same system encoding (ISO-8859-1).

When compiled for GTK 2 the umlauts are not shown corrctly and on the console messages like this appear when handing over strings to gtk widgets:

Pango-WARNING **: Invalid UTF-8 string passed to pango_layout_set_t ext()

When the texts are not used as caption but as text inside a memo, the warning differs:

Gtk-CRITICAL **: gtk_text_buffer_emit_insert: assertion `g_utf8_val idate (text, len, NULL)' failed

and the strings in question are left out up to the next newline.

The encoding inside the file l?ooks like this:

Stichwörter:

for the Letter "?".

How can make GTK 2 show those chars as in GTK 1?

Regards,

Marc Santhoff

Reply by email

Mattias Gaertner
to Marc Santhoff
Show quoted text 5 lines
[...] I have a DOM model read from a xml file that has no encoding named inside it. The var TXMLDocument.Encoding is empty after the file is loaded. [...]> How can make GTK 2 show those chars as in GTK 1?

convert the xml file to UTF-8

OR

convert the xml file before passing it to the xml reader

OR

convert the strings before passing them to the LCL

See unit lconvencoding GuessEncoding and ConvertEncoding.

Mattias

Reply by email

Sergei Gorelkin
to Marc Santhoff: I have a DOM model read from a xml file that has no encoding named inside it. The var…
Marc Santhoff wrote 6 lines
Hi, I have a DOM model read from a xml file that has no encoding named inside it. The var TXMLDocument.Encoding is empty after the file is loaded.

TXMLDocument.Encoding is currently not functional. The XML reader assumes that files without specified encoding are UTF-8 (or UTF-16 if a BOM is present).
In your case, however, it appears that the file does not contain non-ASCII characters in literal (they are 'escaped' by character entities). The encoding is not relevant in this case, because character entities are directly converted to Unicode code points.

WHen compiling the program for GTK 1 the german umlauts are shown correctly, I assume GTK 1 defaults to system encoding (or so ;). The files have been written in a GUI using the same system encoding (ISO-8859-1).

The DOM is based on WideStrings (UTF-16 encoding). When you assign DOM properties to AnsiStrings, the compiler inserts implicit conversions into system encoding.

Show quoted text 14 lines
When compiled for GTK 2 the umlauts are not shown corrctly and on the console messages like this appear when handing over strings to gtk widgets: Pango-WARNING **: Invalid UTF-8 string passed to pango_layout_set_t ext() When the texts are not used as caption but as text inside a memo, the warning differs: Gtk-CRITICAL **: gtk_text_buffer_emit_insert: assertion `g_utf8_val idate (text, len, NULL)' failed and the strings in question are left out up to the next newline.

GTK 2 requires UTF-8 everywhere, so implicit conversions inserted by compiler will work correctly only when your system locale is UTF-8. Otherwise, you'll need to manually convert strings using UTF8Encode() and UTF8Decode().

Show quoted text 8 lines
The encoding inside the file l?ooks like this: Stichwörter: for the Letter "?". How can make GTK 2 show those chars as in GTK 1?

Regards,
Sergei

Reply by email

Marc Santhoff
to Mattias Gaertner: On Fri, 17 Apr 2009 15:13:37 +0200 convert the xml file to UTF-8 OR convert the xml file…
Mattias Gaertner wrote 20 lines
On Fri, 17 Apr 2009 15:13:37 +0200 Marc Santhoff wrote: >[...] > I have a DOM model read from a xml file that has no encoding named > inside it. The var TXMLDocument.Encoding is empty after the file is > loaded. >[...]> How can make GTK 2 show those chars as in GTK 1? convert the xml file to UTF-8 OR convert the xml file before passing it to the xml reader OR convert the strings before passing them to the LCL See unit lconvencoding GuessEncoding and ConvertEncoding.

I like cookbook-style compressed receipes. :)

Many Thanks,

Marc Santhoff

Reply by email

Marc Santhoff
to Sergei Gorelkin: TXMLDocument.Encoding is currently not functional. The XML reader assumes that files…
Sergei Gorelkin wrote 14 lines
Marc Santhoff wrote: > Hi, > > I have a DOM model read from a xml file that has no encoding named > inside it. The var TXMLDocument.Encoding is empty after the file is > loaded. > TXMLDocument.Encoding is currently not functional. The XML reader assumes that files without specified encoding are UTF-8 (or UTF-16 if a BOM is present). In your case, however, it appears that the file does not contain non-ASCII characters in literal (they are 'escaped' by character entities). The encoding is not relevant in this case, because character entities are directly converted to Unicode code points.

I see, hopefully I'll get used to this stuff really soon ...

Show quoted text 27 lines
> WHen compiling the program for GTK 1 the german umlauts are shown > correctly, I assume GTK 1 defaults to system encoding (or so ;). The > files have been written in a GUI using the same system encoding > (ISO-8859-1). The DOM is based on WideStrings (UTF-16 encoding). When you assign DOM properties to AnsiStrings, the compiler inserts implicit conversions into system encoding. > When compiled for GTK 2 the umlauts are not shown corrctly and on the > console messages like this appear when handing over strings to gtk > widgets: > > Pango-WARNING **: Invalid UTF-8 string passed to pango_layout_set_t ext() > > When the texts are not used as caption but as text inside a memo, the > warning differs: > > Gtk-CRITICAL **: gtk_text_buffer_emit_insert: assertion `g_utf8_val > idate (text, len, NULL)' failed > > and the strings in question are left out up to the next newline. > GTK 2 requires UTF-8 everywhere, so implicit conversions inserted by compiler will work correctly only when your system locale is UTF-8. Otherwise, you'll need to manually convert strings using UTF8Encode() and UTF8Decode().

One more question on this topic:

How do other OS handle encodings?

I have to decide whether to put the encoding routine calls inside the data classes, that would mean any OSes GUI is handed over UTF8 strings, or if I have to do it in the GUI glue code differently on any platform.

Show quoted text 10 lines
> The encoding inside the file l?ooks like this: > > Stichwörter: > > for the Letter "?". > > How can make GTK 2 show those chars as in GTK 1? > Regards, Sergei

That are very helpful explanations, thanks a lot!

Marc Santhoff

Reply by email