Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2010-03

Problems with accented characters

15 messages · 6 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Silvio Clecio

Hi friends,

I'm trying to create a function to replace accented characters by single characters, e.g:

? = a
? = e
? = c
? = n

I'm trying to use OnKeyDown and OnUTF8KeyPress, but is not working, so I made some tests and saw that the problem is in the component:

Code:
---------------------------------
procedure TForm1.Button1Click(Sender: TObject);
begin
  Edit2.Text := Edit1.text[1]
end;
---------------------------------
When: Edit1.Text = 'A', Edit2.Text = 'A';
When: Edit1.Text = '?', Edit2.Text = '', empty;

What can this be?

I started a thread here:
http://www.lazarus.freepascal.org/index.php/topic,8816.msg42665.html#msg42665

[]'s Silvio Cl?cio -- programmer ObjectPascal - Blog: http://blog.silvioprog.com.br programmer ObjectPascal - Site: http://www.silvioprog.com.br programmer ObjectPascal - Twitter: http://twitter.com/silvioprog -- grupo Lazarus-BR no Google - http://groups.google.com/group/lazarus-br/ -- Mensagem assinada com PGP Public Key, Ver.: GnuPG v1.4.9 (GNU/Linux)

Reply by email

Felipe Monteiro de Carvalho
to Silvio Clecio
Silvio Clecio wrote
I'm trying to create a function to replace accented characters by single characters, e.g:

Maybe you should show how your code for this is.

The easiest way is making a large if block.

Show quoted text 7 lines
Code: --------------------------------- procedure TForm1.Button1Click(Sender: TObject); begin ?Edit2.Text := Edit1.text[1] end; ---------------------------------

There is no problem here, Lazarus encodes strings using UTF-8. This code is simply completely flawed. The [] operator refers to bytes of the string, so you are only taking the first byte of the string and assigning it to the text of the TEdit, which obviously only works for single byte characters, but UTF-8 chars usually have 1, 2 or 3 bytes.

I suggest that you read more on UTF-8: http://en.wikipedia.org/wiki/UTF-8

Felipe Monteiro de Carvalho

Reply by email

Silvio Clecio
to Felipe Monteiro de Carvalho: Maybe you should show how your code for this is. The easiest way is making a large if…

Hi Felipe

I know this link on the wiki, but it did not really help you solve the problem in TCustomEdit.

See this demo, I explain it better how to reproduce the error:

http://groups.google.com.br/group/lazarus-brasil/web/be_bug_tests.zip?hl=pt-BR

Thanks.

Felipe Monteiro de Carvalho wrote 23 lines
On Sun, Mar 7, 2010 at 7:29 PM, Silvio Clecio wrote: > I'm trying to create a function to replace accented characters by single > characters, e.g: Maybe you should show how your code for this is. The easiest way is making a large if block. > Code: > --------------------------------- > procedure TForm1.Button1Click(Sender: TObject); > begin > Edit2.Text := Edit1.text[1] > end; > --------------------------------- There is no problem here, Lazarus encodes strings using UTF-8. This code is simply completely flawed. The [] operator refers to bytes of the string, so you are only taking the first byte of the string and assigning it to the text of the TEdit, which obviously only works for single byte characters, but UTF-8 chars usually have 1, 2 or 3 bytes. I suggest that you read more on UTF-8: http://en.wikipedia.org/wiki/UTF-8
[]'s Silvio Cl?cio -- programmer ObjectPascal - Blog: http://blog.silvioprog.com.br programmer ObjectPascal - Site: http://www.silvioprog.com.br programmer ObjectPascal - Twitter: http://twitter.com/silvioprog -- grupo Lazarus-BR no Google - http://groups.google.com/group/lazarus-br/ -- Mensagem assinada com PGP Public Key, Ver.: GnuPG v1.4.9 (GNU/Linux)

Reply by email

Silvio Clecio
to Marco Alvarado: Hello Silvio! Try the UTF8-Tools units: http://wiki.lazarus.freepascal.org/Theodp At the…

Hello Marco, thanks for the response :)

I'll test it, I give a return in a moment.

Will send the demos that I'm trying to do, please see how the error appears to be in TCustomEdit.

Sending (WithoutAccent-pt_BR) in a moment...

Marco Alvarado wrote 15 lines
Hello Silvio! > On Sun, Mar 7, 2010 at 7:29 PM, Silvio Clecio wrote: >> I'm trying to create a function to replace accented characters by single >> characters, e.g: Try the UTF8-Tools units: http://wiki.lazarus.freepascal.org/Theodp At the end of the page there is an example "Case demo" on replacing accented characters. Best regards! -Marco
[]'s Silvio Cl?cio -- programmer ObjectPascal - Blog: http://blog.silvioprog.com.br programmer ObjectPascal - Site: http://www.silvioprog.com.br programmer ObjectPascal - Twitter: http://twitter.com/silvioprog -- grupo Lazarus-BR no Google - http://groups.google.com/group/lazarus-br/ -- Mensagem assinada com PGP Public Key, Ver.: GnuPG v1.4.9 (GNU/Linux)

Reply by email

JoshyFun
to Silvio Clecio: Hi Felipe I know this link on the wiki, but it did not really help you solve the problem…

Hello Lazarus-List,

Sunday, March 7, 2010, 11:57:45 PM, you wrote:

SC> I know this link on the wiki, but it did not really help you solve the SC> problem in TCustomEdit.
SC> See this demo, I explain it better how to reproduce the error: SC> http://groups.google.com.br/group/lazarus-brasil/web/be_bug_tests.zip?hl=pt-BR

So your problem is that UTF8Key is not "realtime" modificable as "Key" in KeyPress event ?

As UTF8Key appears as "var" parameter it looks like a bug to me.

Meanwhile you can use KeyPress to perform the task converting key to UTF8, perform the changes needed and them convert back to ansi mode. This way when UTF8KeyPress accepts changed UTF8Key (if it should do it) you should simply change the routine position and remove the 2 conversion lines.

Best regards, JoshyFun

Reply by email

Silvio Clecio
to Marco Alvarado: Hello Silvio! Try the UTF8-Tools units: http://wiki.lazarus.freepascal.org/Theodp At the…

Marco, I tested now, is really an very good package :)

You have some example to replace the text in real time using this package?

Please see this, I'm trying to do and only works right with OnChange:

http://lazarus-brasil.googlegroups.com/web/be_bug_tests_II.zip (or: http://bit.ly/aknuUm)

I need to avoid the OnChange; up of the 500 characters this function slows.

Thanks again :)

Marco Alvarado wrote 15 lines
Hello Silvio! > On Sun, Mar 7, 2010 at 7:29 PM, Silvio Clecio wrote: >> I'm trying to create a function to replace accented characters by single >> characters, e.g: Try the UTF8-Tools units: http://wiki.lazarus.freepascal.org/Theodp At the end of the page there is an example "Case demo" on replacing accented characters. Best regards! -Marco
[]'s Silvio Cl?cio -- programmer ObjectPascal - Blog: http://blog.silvioprog.com.br programmer ObjectPascal - Site: http://www.silvioprog.com.br programmer ObjectPascal - Twitter: http://twitter.com/silvioprog -- grupo Lazarus-BR no Google - http://groups.google.com/group/lazarus-br/ -- Mensagem assinada com PGP Public Key, Ver.: GnuPG v1.4.9 (GNU/Linux)

Reply by email

Marco Alvarado
to Silvio Clecio
You have some example to replace the text in real time using this package?

Sorry, I don't have an example like that, but the example listed on the page seems simple enough:

 s := TUTF8Scanner.Create(Memo1.text);
 s.FindChars := '???';
 repeat
   case s.FindIndex(s.Next) of
 {?} 0: s.Replace('oe');
 {?} 1: s.Replace('ae');
 {?} 2: s.Replace('ue');
   end;
 until s.Done;
 Memo1.Text := s.UTF8String;
 s.free;

Hope it helps!
-Marco

Reply by email

Silvio Clecio
to JoshyFun: SC> I know this link on the wiki, but it did not really help you solve the SC> problem in…

Hellow JoshyFun,

JoshyFun wrote
So your problem is that UTF8Key is not "realtime" modificable as "Key" in KeyPress event ? As UTF8Key appears as "var" parameter it looks like a bug to me.

Yes. For me too. And here em my PC, OnKeyPress works normal only for Windows (XP). :(

Show quoted text 6 lines
Meanwhile you can use KeyPress to perform the task converting key to UTF8, perform the changes needed and them convert back to ansi mode. This way when UTF8KeyPress accepts changed UTF8Key (if it should do it) you should simply change the routine position and remove the 2 conversion lines.

In a very simple test, I'm trying this:

[code]
const
CWithAccent: array [0..46] of string = ('?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?',
'?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?', '?',

    '?', '?', '?', '?', '?', '?', '?', '?', '?', '?');
  CWithoutAccent: array [0..46] of string = ('a', 'a', 'a', 'a', 'a',

'e', 'e',
'e', 'e', 'i', 'i', 'i', 'i', 'o', 'o', 'o', 'o', 'o', 'u', 'u', 'u', 'u',
'A', 'A', 'A', 'A', 'A', 'E', 'E', 'E', 'E', 'I', 'I', 'I', 'O', 'O', 'O',

    'O', 'O', 'U', 'U', 'U', 'U', 'c', 'C', 'n', 'N');
uses
  StrUtils;
procedure TForm1.Edit1KeyPress(Sender: TObject; var Key: char);
begin
  Key := StringsReplace(AnsiToUtf8(Key), CWithAccent, CWithoutAccent,
[rfReplaceAll])[1];
end;
[/code]
(Please, see well here: http://pastebin.com/qmu80j74)

But unfortunately is working only on Windows. :(

[]'s Silvio Cl?cio -- programmer ObjectPascal - Blog: http://blog.silvioprog.com.br programmer ObjectPascal - Site: http://www.silvioprog.com.br programmer ObjectPascal - Twitter: http://twitter.com/silvioprog -- grupo Lazarus-BR no Google - http://groups.google.com/group/lazarus-br/ -- Mensagem assinada com PGP Public Key, Ver.: GnuPG v1.4.9 (GNU/Linux)

Reply by email

Silvio Clecio
to Marco Alvarado: Sorry, I don't have an example like that, but the example listed on the page seems simple…

Hello Marco, no problems friend :)

I'm trying with TUTF8Scanner + OnKeyPress, and TUTF8Scanner + OnUTF8KeyPress, but not works. :(

Another test that fails:

procedure TForm1.Edit1UTF8KeyPress(Sender: TObject; var UTF8Key: TUTF8Char);
begin
  UTF8Key := '?';
end;

Appear the caracters of the current typing in Edit, and not '?'. :(

Marco Alvarado wrote 19 lines
> You have some example to replace the text in real time using this package? Sorry, I don't have an example like that, but the example listed on the page seems simple enough: s := TUTF8Scanner.Create(Memo1.text); s.FindChars := '???'; repeat case s.FindIndex(s.Next) of {?} 0: s.Replace('oe'); {?} 1: s.Replace('ae'); {?} 2: s.Replace('ue'); end; until s.Done; Memo1.Text := s.UTF8String; s.free; Hope it helps! -Marco
[]'s Silvio Cl?cio -- programmer ObjectPascal - Blog: http://blog.silvioprog.com.br programmer ObjectPascal - Site: http://www.silvioprog.com.br programmer ObjectPascal - Twitter: http://twitter.com/silvioprog -- grupo Lazarus-BR no Google - http://groups.google.com/group/lazarus-br/ -- Mensagem assinada com PGP Public Key, Ver.: GnuPG v1.4.9 (GNU/Linux)

Reply by email

Vladimir Zhirov
to Silvio Clecio
Silvio Clecio wrote 11 lines
I'm trying with TUTF8Scanner + OnKeyPress, and TUTF8Scanner + OnUTF8KeyPress, but not works. :( Another test that fails: procedure TForm1.Edit1UTF8KeyPress(Sender: TObject; var UTF8Key: TUTF8Char); begin UTF8Key := '?'; end; Appear the caracters of the current typing in Edit, and not '?'. :(

It seems to be a bug in OnUTF8KeyPress, the similar behaviour was reported at Russian Free Pascal forum. Would you please create a bug report?

Reply by email

Silvio Clecio
to Vladimir Zhirov
Vladimir Zhirov wrote
It seems to be a bug in OnUTF8KeyPress, the similar behaviour was reported at Russian Free Pascal forum. Would you please create a bug report?

Hi friend,

Yes. Please see here:

http://bugs.freepascal.org/view.php?id=15932

[]'s Silvio Cl?cio -- programmer ObjectPascal - Blog: http://blog.silvioprog.com.br programmer ObjectPascal - Site: http://www.silvioprog.com.br programmer ObjectPascal - Twitter: http://twitter.com/silvioprog -- grupo Lazarus-BR no Google - http://groups.google.com/group/lazarus-br/ -- Mensagem assinada com PGP Public Key, Ver.: GnuPG v1.4.9 (GNU/Linux)

Reply by email

Antônio
to Silvio Clecio

Chances are that this code works.

procedure TForm1.Edit1Change(Sender: TObject);
var
  s :string;
  p :integer;
begin
  s := edit1.text;
  while utf8pos('?', s) > 0 do
  begin
    p := utf8pos('?', s);
    utf8delete(s, p, utf8length('?'));
    utf8insert('a', s, p);
  end;
  edit1.selstart := 100;
  edit1.text := s;
end;

2010/3/8 Silvio Clecio

Show quoted text 30 lines
Vladimir Zhirov escreveu: > It seems to be a bug in OnUTF8KeyPress, the similar behaviour was > reported at Russian Free Pascal forum. Would you please create > a bug report? Hi friend, Yes. Please see here: http://bugs.freepascal.org/view.php?id=15932 -- []'s Silvio Cl?cio -- programmer ObjectPascal - Blog: http://blog.silvioprog.com.br programmer ObjectPascal - Site: http://www.silvioprog.com.br programmer ObjectPascal - Twitter: http://twitter.com/silvioprog -- grupo Lazarus-BR no Google - http://groups.google.com/group/lazarus-br/ -- Mensagem assinada com PGP Public Key, Ver.: GnuPG v1.4.9 (GNU/Linux) -- _______________________________________________ Lazarus mailing list http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20100309/548459a1/attachment.html>

Reply by email

Antônio
to Antônio
uses lclproc;

2010/3/9 Ant?nio

Show quoted text 52 lines
Chances are that this code works. procedure TForm1.Edit1Change(Sender: TObject); var s :string; p :integer; begin s := edit1.text; while utf8pos('?', s) > 0 do begin p := utf8pos('?', s); utf8delete(s, p, utf8length('?')); utf8insert('a', s, p); end; edit1.selstart := 100; edit1.text := s; end; 2010/3/8 Silvio Clecio Vladimir Zhirov escreveu: > > It seems to be a bug in OnUTF8KeyPress, the similar behaviour was > > reported at Russian Free Pascal forum. Would you please create > > a bug report? > > Hi friend, > > Yes. Please see here: > > http://bugs.freepascal.org/view.php?id=15932 > > -- > []'s > > Silvio Cl?cio > -- > programmer ObjectPascal - Blog: http://blog.silvioprog.com.br > programmer ObjectPascal - Site: http://www.silvioprog.com.br > programmer ObjectPascal - Twitter: http://twitter.com/silvioprog > -- > grupo Lazarus-BR no Google - http://groups.google.com/group/lazarus-br/ > -- > Mensagem assinada com PGP Public Key, Ver.: GnuPG v1.4.9 (GNU/Linux) > > -- > _______________________________________________ > Lazarus mailing list > > http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus >

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20100309/e9425ec5/attachment.html>

Reply by email

Silvio Clecio
to Antônio: Chances are that this code works. procedure TForm1.Edit1Change(Sender: TObject); var s…
Ant?nio wrote 17 lines
Chances are that this code works. procedure TForm1.Edit1Change(Sender: TObject); var s :string; p :integer; begin s := edit1.text; while utf8pos('?', s) > 0 do begin p := utf8pos('?', s); utf8delete(s, p, utf8length('?')); utf8insert('a', s, p); end; edit1.selstart := 100; edit1.text := s; end;

Friend, thanks for the dedication in helping me :)

Your function works for these characters that you presented. And thank you again for your efforts to find a solution. At the moment I gave up of the On(UTF8) KeyPress, I will use OnChange.

I was avoiding OnChange that it becomes slow with more than 500 characters, but I found a way to solve with the help of you. See a full example with the solution here:

<http://www.silvioprog.com.br/downloads> File: "WithoutAccent-pt_BR.zip".

In my country we have many problems with accented characters in Lazarus, I to do an article (in my language) showing how to reduce the gaps especially in DB, and publish here:

<http://blog.silvioprog.com.br>

Thanks to everyone who contributed in this question ;D

[]'s Silvio Cl?cio -- programmer ObjectPascal - Blog: http://blog.silvioprog.com.br programmer ObjectPascal - Site: http://www.silvioprog.com.br programmer ObjectPascal - Twitter: http://twitter.com/silvioprog -- grupo Lazarus-BR no Google - http://groups.google.com/group/lazarus-br/ -- Mensagem assinada com PGP Public Key, Ver.: GnuPG v1.4.9 (GNU/Linux)

Reply by email