Hello,

I love the helper functions from SysUtils' `TStringHelper`. Unfortunately, they're only available for `String`, but not for other types like `Utf8String` or `RawByteString`.

Is it safe to cast to `String()` and then invoking the helper, e.g. `String(AUtf8String).IndexOf('foo')`? Or does this casting involve a codepage conversion?

I've tried with my FPC installation:

program Project1;
uses sysutils;
var S: Utf8string;
begin
  S := 'äfoo';
  writeln (String(S).IndexOf('f')); // prints "2", as expected
end.

But can I rely on it?

Are there any plans of making string helpers available for `Utf8String` and `RawByteString`?

Kind regards,
Thomas

S := 'äfoo'; writeln (String(S).IndexOf('f')); // prints "2", as expected

As expected?
In the Utf8String 'f' will be at index 3, since the 'ä' will take up 2 bytes in UTF8-encoding.

ummm... is it in that writeln, though? looks to me like the UTF8 is cast to an old school mono-byte String type ;)

As expected? In the Utf8String 'f' will be at index 3, since the 'ä' will take up 2 bytes in UTF8-encoding.

Seems I'm wrong.
IndexOf() is zero-based, so 2 is indeed the correct value.

I would have expected/feared that casting to String() would trigger a codepage conversion to CP_ACP, which is a 1-byte encoding by default on
Bart

I've extended the little test program:

```
program Project1;
{$mode objfpc}
{$longstrings on}
uses sysutils;
var S: Utf8string;
begin
  S := 'äfoo';
  Writeln(PtrUInt(Pointer(S)));
  Writeln(PtrUInt(Pointer(String(S))));
  Writeln('DefaultSystemCodePage = ', DefaultSystemCodePage);
  Writeln('S codepage             = ', StringCodePage(S));
  Writeln('String(S) codepage     = ', StringCodePage(String(S)));
  Writeln('Index                   = ', String(S).IndexOf('f'));
end.
```

Thank you for pointing me to the shortstring/longstring issue. The string pointers are equal and there is no indication for a codepage conversion as part of the cast.

However, the remaining questions are:

1) Is the explicit cast guaranteed to preserve the underlying string and its dynamic codepage without conversion, or is this only an implementation detail?

2) Are there any plans for making the string helpers available with Utf8String and RawByteString, too?

Thanks,
Thomas

Thomas Kurz wrote

However, the remaining questions are: 1) Is the explicit cast guaranteed to preserve the underlying string and its dynamic codepage without conversion, or is this only an implementation detail?

It's very well documented:
https://www.freepascal.org/daily/doc/ref/refsu9.html#x32-39000

Yes - no conversion is guaranteed.

2) Are there any plans for making the string helpers available with Utf8String and RawByteString, too?

I can't answer this question.

Ondrej

I would recommend using the Lazarus solution where String uses CP_UTF8. Then no typecasts or other tweaks are needed.

Juha

That is interesting and exactly what I'd like to have. How can I do that?

Thomas

Thomas Kurz wrote

That is interesting and exactly what I'd like to have. How can I do that?

It is enabled automatically in LCL GUI apps. In a console program you must add unit *LazUTF8* to uses section. That's all.

See: https://wiki.lazarus.freepascal.org/Unicode_Support_in_Lazarus

for details.

Regards Juha

You don't need LazUTF8 anymore with recent FPC versions. FPC has everything built-in. (I don't like Lazarus dependencies in console applications.)

Set these 3 CodePages to CP_UTF8:
   SetMultiByteConversionCodePage(CP_UTF8);
   SetMultiByteRTLFileSystemCodePage(CP_UTF8);
   SetMultiByteFileSystemCodePage(CP_UTF8);

and you are good to go.

Ondrej

Op 30-8-2026 om 19:40 schreef Ondrej Pokorny via fpc-pascal:

For a recent Windows 10  or 11 for a non lazarus program simply add "winmanutf8lfn"  to the uses line of the main program. This works in FPC 3.2.4-rc2 and trunk

For a lazarus compiled program (also console) simply enable the utf8 and lfn ticks in the application manifest, which does the same as the unit, but also works with 3.2.2.

The manifest changes make the whole winapi accept utf8, effectively working as an native UTF-8 system, with as only downside win10+ (and 1805 or 1905+.

At the same time, the LFN tick (or the unit) enables support for file paths  > 260 characters.

Thank you all for your help.

Thomas

Ondrej Pokorny via fpc-pascal kirjoitti 30.8.2026 klo 20.40:

LazUTF8 also sets callbacks on Windows for AnsiCompareStr(), AnsiCompareText() and similar functions.

I didn't know about the "winmanutf8lfn", I guess it does the same thing.

BTW, the Ansi...() functions have a misleading name. They actually work with Unicode.

Juha

If I am not mistaken, they are not needed anymore. FPC 3.x handles it. A lot of the UTF-8 stuff in Lazarus could be cleaned up as it was written for FPC 2.x.

Ondrej

Op 31-8-2026 om 11:47 schreef Juha Manninen via fpc-pascal:

No, it does the same as the manifest options, which works at the windows level, not at the RTL level. So at startup of the exe the system encoding is directly utf8, for everything, so also the windows api.

The only downside is that it requires windows 10 (actually a 201903 version of windows 10, but older versions than that are long unsupported).

This means for supported windows versions, anything "ansi", no manual conversions are needed any more, and you can pass a ansistring containing utf8 directly to the windows API with pchar(thestring).

The only place where manual conversions are needed are for file and wire formats with specific encoding requirements, and maybe some special api (console/OEM ?)