Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2013-04

PAnsiChar to UTF8 issues

15 messages · 7 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Leonardo M. Ramé

Hi, I'm working with a "C" shared library that returns a pointer to char. It's all ok when not using special characters such as "?" or accents, but when I get these, they are replaced by "?" signs.

For example, if I receive an "?", the debugger shows this:

lStr := 'A'#209'oranza';

The correct string is:

lStr := 'A?oranza';

What can I do to show accented characters?.

Regards,

Reply by email

Martin
to Leonardo M. Ramé
Leonardo M. Ram? wrote 14 lines
Hi, I'm working with a "C" shared library that returns a pointer to char. It's all ok when not using special characters such as "?" or accents, but when I get these, they are replaced by "?" signs. For example, if I receive an "?", the debugger shows this: lStr := 'A'#209'oranza'; The correct string is: lStr := 'A?oranza'; What can I do to show accented characters?.

It's a known problem and already reported. here is no fix available yet. The debugger does can not yet display this.

Actually it is 2 problems.
1) is the "?"
2) the other is #209 etc

Reply by email

Leonardo M. Ramé
to Martin
Martin wrote 23 lines
On 09/04/2013 15:11, Leonardo M. Ram? wrote: >Hi, I'm working with a "C" shared library that returns a pointer to >char. It's all ok when not using special characters such as "?" or >accents, but when I get these, they are replaced by "?" signs. > >For example, if I receive an "?", the debugger shows this: > >lStr := 'A'#209'oranza'; > >The correct string is: > >lStr := 'A?oranza'; > >What can I do to show accented characters?. > It's a known problem and already reported. here is no fix available yet. The debugger does can not yet display this. Actually it is 2 problems. 1) is the "?" 2) the other is #209 etc

Well, I can live with the debugger problem, but is there a solution for the "?" sign end displaying in my app?.

Reply by email

Martin
to Leonardo M. Ramé
Leonardo M. Ram? wrote 26 lines
On 2013-04-09 15:20:33 +0100, Martin wrote: > On 09/04/2013 15:11, Leonardo M. Ram? wrote: >> Hi, I'm working with a "C" shared library that returns a pointer to >> char. It's all ok when not using special characters such as "?" or >> accents, but when I get these, they are replaced by "?" signs. >> >> For example, if I receive an "?", the debugger shows this: >> >> lStr := 'A'#209'oranza'; >> >> The correct string is: >> >> lStr := 'A?oranza'; >> >> What can I do to show accented characters?. >> > It's a known problem and already reported. here is no fix available > yet. The debugger does can not yet display this. > > Actually it is 2 problems. > 1) is the "?" > 2) the other is #209 etc > Well, I can live with the debugger problem, but is there a solution for the "?" sign end displaying in my app?.

If you get an ansi string, you must convert it to utf8 before giving it to the LCL

need to find the exact function

label.caption :- ansitoutf8(s) // assuming it is ansi in the system codepage

Reply by email

Leonardo M. Ramé
to Martin
Martin wrote 34 lines
On 09/04/2013 15:22, Leonardo M. Ram? wrote: >On 2013-04-09 15:20:33 +0100, Martin wrote: >>On 09/04/2013 15:11, Leonardo M. Ram? wrote: >>>Hi, I'm working with a "C" shared library that returns a pointer to >>>char. It's all ok when not using special characters such as "?" or >>>accents, but when I get these, they are replaced by "?" signs. >>> >>>For example, if I receive an "?", the debugger shows this: >>> >>>lStr := 'A'#209'oranza'; >>> >>>The correct string is: >>> >>>lStr := 'A?oranza'; >>> >>>What can I do to show accented characters?. >>> >>It's a known problem and already reported. here is no fix available >>yet. The debugger does can not yet display this. >> >>Actually it is 2 problems. >>1) is the "?" >>2) the other is #209 etc >> >Well, I can live with the debugger problem, but is there a solution for >the "?" sign end displaying in my app?. > If you get an ansi string, you must convert it to utf8 before giving it to the LCL need to find the exact function label.caption :- ansitoutf8(s) // assuming it is ansi in the system codepage

Yes, I've tried that, but it shows a box with an X inside where the ? should appear...

My codepage is es_AR.UTF-8.

Reply by email

Anton Kavalenka
to Leonardo M. Ramé
Leonardo M. Ram? wrote 40 lines
On 2013-04-09 15:28:56 +0100, Martin wrote: > On 09/04/2013 15:22, Leonardo M. Ram? wrote: >> On 2013-04-09 15:20:33 +0100, Martin wrote: >>> On 09/04/2013 15:11, Leonardo M. Ram? wrote: >>>> Hi, I'm working with a "C" shared library that returns a pointer to >>>> char. It's all ok when not using special characters such as "?" or >>>> accents, but when I get these, they are replaced by "?" signs. >>>> >>>> For example, if I receive an "?", the debugger shows this: >>>> >>>> lStr := 'A'#209'oranza'; >>>> >>>> The correct string is: >>>> >>>> lStr := 'A?oranza'; >>>> >>>> What can I do to show accented characters?. >>>> >>> It's a known problem and already reported. here is no fix available >>> yet. The debugger does can not yet display this. >>> >>> Actually it is 2 problems. >>> 1) is the "?" >>> 2) the other is #209 etc >>> >> Well, I can live with the debugger problem, but is there a solution for >> the "?" sign end displaying in my app?. >> > If you get an ansi string, you must convert it to utf8 before giving > it to the LCL > > need to find the exact function > > label.caption :- ansitoutf8(s) // assuming it is ansi in the system codepage > Yes, I've tried that, but it shows a box with an X inside where the ? should appear... My codepage is es_AR.UTF-8.
u:unicodestring;
u:=Putf8Char(buf); // or PAnsiChar depends on what your C DLL returns
label.caption:=u; // implicit cast

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20130409/dd34b626/attachment.html>

Reply by email

Leonardo M. Ramé
to Anton Kavalenka
Anton Kavalenka wrote 47 lines
On 09.04.2013 18:07, Leonardo M. Ram? wrote: >On 2013-04-09 15:28:56 +0100, Martin wrote: >>On 09/04/2013 15:22, Leonardo M. Ram? wrote: >>>On 2013-04-09 15:20:33 +0100, Martin wrote: >>>>On 09/04/2013 15:11, Leonardo M. Ram? wrote: >>>>>Hi, I'm working with a "C" shared library that returns a pointer to >>>>>char. It's all ok when not using special characters such as "?" or >>>>>accents, but when I get these, they are replaced by "?" signs. >>>>> >>>>>For example, if I receive an "?", the debugger shows this: >>>>> >>>>>lStr := 'A'#209'oranza'; >>>>> >>>>>The correct string is: >>>>> >>>>>lStr := 'A?oranza'; >>>>> >>>>>What can I do to show accented characters?. >>>>> >>>>It's a known problem and already reported. here is no fix available >>>>yet. The debugger does can not yet display this. >>>> >>>>Actually it is 2 problems. >>>>1) is the "?" >>>>2) the other is #209 etc >>>> >>>Well, I can live with the debugger problem, but is there a solution for >>>the "?" sign end displaying in my app?. >>> >>If you get an ansi string, you must convert it to utf8 before giving >>it to the LCL >> >>need to find the exact function >> >>label.caption :- ansitoutf8(s) // assuming it is ansi in the system codepage >> >Yes, I've tried that, but it shows a box with an X inside where the ? >should appear... > >My codepage is es_AR.UTF-8. > u:unicodestring; u:=Putf8Char(buf); // or PAnsiChar depends on what your C DLL returns label.caption:=u; // implicit cast

No, I've tried all kinds of combinations of ansitoutf8, unicodestring, etc, but nothing works, all returns "?" or the box with the X in the place of "?" or accents.

Reply by email

Howard Page-Clark
to Leonardo M. Ramé
Leonardo M. Ram? wrote 25 lines
>>>> Well, I can live with the debugger problem, but is there a solution for >>>> the "?" sign end displaying in my app?. >>>> >>> If you get an ansi string, you must convert it to utf8 before giving >>> it to the LCL >>> >>> need to find the exact function >>> >>> label.caption :- ansitoutf8(s) // assuming it is ansi in the system codepage >>> >> Yes, I've tried that, but it shows a box with an X inside where the ? >> should appear... >> >> My codepage is es_AR.UTF-8. >> > u:unicodestring; > > u:=Putf8Char(buf); // or PAnsiChar depends on what your C DLL returns > > label.caption:=u; // implicit cast > No, I've tried all kinds of combinations of ansitoutf8, unicodestring, etc, but nothing works, all returns "?" or the box with the X in the place of "?" or accents.

Impertinent question really, but do you know for certain that the font assigned to your label has glyphs corresponding to each unicode entity your string contains?

Reply by email

Leonardo M. Ramé
to Leonardo M. Ramé: No, I've tried all kinds of combinations of ansitoutf8, unicodestring, etc, but nothing…

I'm finding some weird results.

If I execute my program from an XFCE4-Terminal, I get the diamond with a "?" inside, but if I redirect the output of my program to vim (executed from the same xfce4-terminal) I can see the "?" (ASCII 209). Please see the attached screenshot, the upper part shows the console output, the lower part shows vim running on the same console.

So, aparently the text is ASCII and not UTF8 as I thought.

I hope this data helps to find a solution...

Leonardo M. Ram? http://leonardorame.blogspot.com -------------- next part -------------- A non-text attachment was scrubbed... Name: ascii.jpg Type: image/jpeg Size: 62030 bytes Desc: not available URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20130409/fef521b5/attachment-0001.jpg>

Reply by email

waldo kitty
to Howard Page-Clark: Impertinent question really, but do you know for certain that the font assigned to your…
Howard Page-Clark wrote 8 lines
On 09/04/13 6:35, Leonardo M. Ram? wrote: > > No, I've tried all kinds of combinations of ansitoutf8, unicodestring, > etc, but nothing works, all returns "?" or the box with the X in the > place of "?" or accents. Impertinent question really, but do you know for certain that the font assigned to your label has glyphs corresponding to each unicode entity your string contains?

i was wondering the same thing but couldn't get it to phrase properly... the font table is what i was having trouble with expressing... thanks for clearly pointing out another possible problem area for UTF-8 glyph rendering... hopefully this will be documented as a possible pitfall in the UTF functions' documentation or basic explanation of operations...

Reply by email

Leonardo M. Ramé
to waldo kitty
waldo kitty wrote 18 lines
On 4/9/2013 14:42, Howard Page-Clark wrote: >On 09/04/13 6:35, Leonardo M. Ram? wrote: >> >>No, I've tried all kinds of combinations of ansitoutf8, unicodestring, >>etc, but nothing works, all returns "?" or the box with the X in the >>place of "?" or accents. > >Impertinent question really, but do you know for certain that the font assigned >to your label has glyphs corresponding to each unicode entity your string contains? i was wondering the same thing but couldn't get it to phrase properly... the font table is what i was having trouble with expressing... thanks for clearly pointing out another possible problem area for UTF-8 glyph rendering... hopefully this will be documented as a possible pitfall in the UTF functions' documentation or basic explanation of operations...

I really don't know, I'm using the same font as my Terminal (where the ? is shown correctly). Apart from it, I checked with other different fonts, and Charsets, but got the same result.

Reply by email

waldo kitty
to Leonardo M. Ramé
Leonardo M. Ram? wrote 21 lines
On 2013-04-09 15:35:18 -0400, waldo kitty wrote: > On 4/9/2013 14:42, Howard Page-Clark wrote: >> On 09/04/13 6:35, Leonardo M. Ram? wrote: >>> >>> No, I've tried all kinds of combinations of ansitoutf8, unicodestring, >>> etc, but nothing works, all returns "?" or the box with the X in the >>> place of "?" or accents. >> >> Impertinent question really, but do you know for certain that the font assigned >> to your label has glyphs corresponding to each unicode entity your string contains? > > i was wondering the same thing but couldn't get it to phrase > properly... the font table is what i was having trouble with > expressing... thanks for clearly pointing out another possible > problem area for UTF-8 glyph rendering... hopefully this will be > documented as a possible pitfall in the UTF functions' documentation > or basic explanation of operations... I really don't know, I'm using the same font as my Terminal (where the ? is shown correctly). Apart from it, I checked with other different fonts, and Charsets, but got the same result.

maybe your terminal is having problems? have you tried another terminal package in place of FCEwhateveritwas?

Reply by email

José Mejuto
to Leonardo M. Ramé: No, I've tried all kinds of combinations of ansitoutf8, unicodestring, etc, but nothing…

El 09/04/2013 19:35, Leonardo M. Ram? escribi?:

Show quoted text 5 lines
No, I've tried all kinds of combinations of ansitoutf8, unicodestring, etc, but nothing works, all returns "?" or the box with the X in the place of "?" or accents.

Hello,

Maybe you have an ANSI string inside a string type declared as UTF8 coded, so try this:

var
   ansivar: ansistring;
begin
   ansivar:=lStr; //Your lStr
   SetCodePage(RawByteString(ansivar),CP_ACP,false);
   //Display the ansivar instead lStr to check.
end;

Reply by email