Hi List,

I'm writing an API interface that passes #0-terminated cstrings with encoding UTF8. What data type should be used to declare these parameters so that I may be able to use as much of 3.0+'s automatic encoding conversion as possible?

Some example declarations would look like:
  procedure SetUserID(const NewValue: PUTF8Char);
  function GetUserID(const Buf: PUTF8Char; const BufLength: UInt32): UInt32;

If I read the wiki correctly, PAnsiChar would not be clear as it is always assumed to be CP_ACP, causing the compiler to generate conversions to DefaultSystemCodePage. I'm posting this to the Lazarus list instead of fpc-pascal because I already use LazUTF8 so CP_ACP really is CP_UTF8, but I want to be sure that the header always works whether LazUTF8 is used or not.

Is there a good way to do what I want, or would it be easier to use PUnicodeChar and pass the strings as UTF-16? How well would other languages work with that?

Thank you for any hints,

Martok

PAnsiChar is usually a PChar.
They are different when using $mode delphiunicode, in which case PChar becomes PWideChar.
So PAnsiChar is always a pointer to a CP_ACP char. Thus assigning a PAnsiChar (=PChar) to a String, AnsiString, RawbyteString or ShortString does not add conversion code and therefore does no conversion with or without LazUTF8.
Assigning it to another string type (UnicodeString, UTF8String or AnsiString[cp]) will add conversion code. With LazUTF8 this means your PChar will be treated as CP_UTF8, without it will be treated as the runtime system codepage.

The other way round - assigning a string to a PChar - is not supported by FPC. So only with LazUTF8 you can use a simple type cast. Without LazUTF8 you must convert the string first, before type casting.

If your header should work whether LazUTF8 is used or not then you can provide a helper function:

procedure SetUserIDUTF8(const NewValue: PChar);
begin
  ...
end;
procedure SetUserID(const NewValue: UTF8String);
begin
  SetUserIDUTF8(PChar(NewValue));
end;

Alternatively you can use a more optimized version in case LazUTF8 is used:

procedure SetUserID(const NewValue: AnsiString);
var
  uValue: String;
begin
  if (DefaultSystemCodePage=CP_UTF8) then
    SetUserIDUTF8(PChar(NewValue))
  else begin
    uValue:=AnsiToUTF8(NewValue);
    SetUserIDUTF8(PChar(uValue));
  end;
end;

Is there a good way to do what I want, or would it be easier to use PUnicodeChar and pass the strings as UTF-16? How well would other languages work with that?

Whether it's easier totally depends on the other language.

Mattias

Okay, that's pretty much what I thought, thanks.

However:

The other way round - assigning a string to a PChar - is not supported by FPC. So only with LazUTF8 you can use a simple type cast. Without LazUTF8 you must convert the string first, before type casting.

This does not work either, assigning a UnicodeString to a PChar simply does nothing, as if PChar was CP_NONE...

This works as expected and is effectively the same as calling UnicodeToUTF8() directly:

  pc:= PChar(UTF8String(UniStr));

If there is no conversion at all that would be okay too, but if not I would have to define PUTF8Char = PByte or something just to be safe.

Alternatively you can use a more optimized version in case LazUTF8 is used: procedure SetUserID(const NewValue: AnsiString);

AnsiString? Why would that help?

> Is there a good way to do what I want, or would it be easier to use PUnicodeChar > and pass the strings as UTF-16? How well would other languages work with that? Whether it's easier totally depends on the other language.

Well, people with Windows experience will probably expect a wchar_t * as UCS-2 or UTF-16, but in contrast Rust would work far better with UTF-8 (because that's their native string type).

From your code examples I think I didn't mention one thing: the internal fields

are all UnicodeString. This question is just about the external (non-pascal-facing) interface and how to design it without having to use explicit Utf8ToUnicode everytime.

Regards,

Martok