Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2016-04

Iterator for Unicode encoded strings

4 messages · 3 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Graeme Geldenhuys

Hi,

This question really applies for all Unicode encodings (UTF-8, UTF-16 and UTF-32). In the modern world you simply can't use indexed access into Object Pascal strings to retrieve a "character" (loosely what you see on the screen - ignoring combining diacritics). In my own projects I have custom written Iterators for various data types like StringList, ObjectLists etc, and that includes strings.

I know FPC RTL supports the Iterator concept (not as advanced as what I have implemented), but still useful. As far as I know there are some standard iterator classes already implemented. Does FPC include such iterators for the various string types?

Regards,
Graeme

fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp
Michael Van Canneyt
to Graeme Geldenhuys
Graeme Geldenhuys wrote 13 lines
Hi, This question really applies for all Unicode encodings (UTF-8, UTF-16 and UTF-32). In the modern world you simply can't use indexed access into Object Pascal strings to retrieve a "character" (loosely what you see on the screen - ignoring combining diacritics). In my own projects I have custom written Iterators for various data types like StringList, ObjectLists etc, and that includes strings. I know FPC RTL supports the Iterator concept (not as advanced as what I have implemented), but still useful. As far as I know there are some standard iterator classes already implemented. Does FPC include such iterators for the various string types?

Simply said: No. The character iterator

var
  C : Char;
  S : String; // or unicode string
begin
   for c in S do
end.

Will iterate bytes or words, not characters.

Michael.

Graeme Geldenhuys
to Michael Van Canneyt
Michael Van Canneyt wrote
Simply said: No. The character iterator Will iterate bytes or words, not characters.

OK thanks. So I definitely need the IsSurrogatePair() like function then
- in the case of UTF-16 encoded (UnicodeString) text. Do you know if such a function exists in the RTL? If not, I'll browse unicode.org to see what is required for such an implementation.

Regards,
Graeme

fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp