Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2019-04

WideString vs UnicodeString / len in char vs byte / *nix vs windows

5 messages · 5 people ·

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Martin

Looking at https://bugs.freepascal.org/view.php?id=35359 and the latest comments (see end of mail)

Is it correct, that "widestring" here has 2 meanings?
- the "widestring" that has length in bytes (windows bstr)
- "widestring" as a typename / alias

And apparently then on *nix, "widestring" is just an alias (to unicodestring).
So using the type with the name of "widestring" on *nix will compile to a type that has length in chars?

At least my test app, seems to imply this.

Martin:

Show quoted text 6 lines
Is the len on widestring always in bytes? This prints the dwords in front of each string. On Windows, it indeed prints 10 for a widestring of len 5. But on linux, it prints 5 for all the strings. Tested 64 bit.

^^ test source with wide and uni string of len 5, and memory dump of string header

Anyway the patch seems to work for linux too (only tested 64 bit) widestring seems to be an alias for unicodestring in this case.

Serge Anvarov:

Show quoted text 5 lines
Yes, the length field for WideString is specified in bytes. For compatible with Delphi (http://docwiki.embarcadero.com/RADStudio/Rio/en/Internal_Data_Formats_%28Delphi%29#Wide_String_Types) and Windows BSTR type (https://msdn.microsoft.com/en-us/windows/desktop/1B2D7D2C-47AF-4389-A6B6-B01B7E915228)

Reply by email

Sven Barth
to Martin
Martin wrote 13 lines
Looking at https://bugs.freepascal.org/view.php?id=35359 and the latest comments (see end of mail) Is it correct, that "widestring" here has 2 meanings? - the "widestring" that has length in bytes (windows bstr) - "widestring" as a typename / alias And apparently then on *nix, "widestring" is just an alias (to unicodestring). So using the type with the name of "widestring" on *nix will compile to a type that has length in chars? At least my test app, seems to imply this.

Correct. On non-Windows platforms WideString is always an alias to UnicodeString. The distinctive WideString type only exists on Windows platforms as it relies on Windows specific functionality.

Regards,
Sven

Reply by email

Michael Van Canneyt
to Sven Barth
Sven Barth wrote 17 lines
Am 25.04.2019 um 15:41 schrieb Martin: > Looking at https://bugs.freepascal.org/view.php?id=35359 > and the latest comments (see end of mail) > > Is it correct, that "widestring" here has 2 meanings? > - the "widestring" that has length in bytes (windows bstr) > - "widestring" as a typename / alias > > And apparently then on *nix, "widestring" is just an alias (to > unicodestring). > So using the type with the name of "widestring" on *nix will compile > to a type that has length in chars? > > At least my test app, seems to imply this. Correct. On non-Windows platforms WideString is always an alias to UnicodeString. The distinctive WideString type only exists on Windows platforms as it relies on Windows specific functionality.

This is only about generated debug information, I suppose ?

because the following prints 3 in delphi:

Var
   A : Widestring;
begin
     A:='ABC';
     Writeln(Length(A));
end.

Michael.

Reply by email

Martin Frb
to Michael Van Canneyt
Michael Van Canneyt wrote 23 lines
On Thu, 25 Apr 2019, Sven Barth via fpc-devel wrote: > Am 25.04.2019 um 15:41 schrieb Martin: >> Looking at https://bugs.freepascal.org/view.php?id=35359 >> and the latest comments (see end of mail) >> >> Is it correct, that "widestring" here has 2 meanings? >> - the "widestring" that has length in bytes (windows bstr) >> - "widestring" as a typename / alias >> >> And apparently then on *nix, "widestring" is just an alias (to >> unicodestring). >> So using the type with the name of "widestring" on *nix will compile >> to a type that has length in chars? >> >> At least my test app, seems to imply this. > Correct. On non-Windows platforms WideString is always an alias to > UnicodeString. The distinctive WideString type only exists on Windows > platforms as it relies on Windows specific functionality. This is only about generated debug information, I suppose ?

Yes.

There for the debugger the bytes have to be "div 2" or "shr 1", which leads to above mentioned issue, that (if dwarf 3 is used) any debugger (gdb, fpdebug) will show the first half of any unicodestring. (because the unicodestrings length in char is also divided by 2)

More strangeness though.
The code that adds the "shr 1" is actually not used for windows-like widestrings (only tested dwarf-3, not tested dwarf-4 / dwarf 2 does not use it at all; not supported in dwarf 2).

And the commit that adds the "shr 1"

Show quoted text 14 lines
Revision: 12444 Author: jonas Date: 27 December 2008 22:43:45 Message:   + append_block1() to add extra block DW_FORM_block1 attributes to     a dwarf entry   - (dwarf3) removed stride size from dynamic array entries because     it's not required there (the stride always equals the element size)   + (dwarf3) added "allocated" attribute for dynamic arrays   + (dwarf3) added generic implementation of string support for dwarf3,     which no longer depends on the hacked fake record type in gdb's     Pascal support. Includes support for all string types, except for     winlike widestrings (because I don't know how to extract the     length from them)

actually states, that it excludes them: "except for winlike widestrings " (or at least that what I guess this means)
However, that commit actually also adds the test for "if (chardef.size=1) then" to decide if "shr 1" should be added.

I do not know, if widestrings actually ever were handled by this code (maybe at the time of above commit), or what other reasons the "shr 1" had at the time.

-------------------------
In any case the patch (fixed patch) available on the mantis issue does fix the problem for unicodestring (and for the widestring alias on *nix).

As the code is currently not used for winlike widestring (or at least I could not trigger it), I can not tell if it would affect such.

Reply by email

Marco van de Voort
to Michael Van Canneyt: This is only about generated debug information, I suppose ? because the following prints…

Op 2019-04-25 om 17:37 schreef Michael Van Canneyt:

Show quoted text 12 lines
>> And apparently then on *nix, "widestring" is just an alias (to >> unicodestring). >> So using the type with the name of "widestring" on *nix will compile >> to a type that has length in chars? >> >> At least my test app, seems to imply this. > Correct. On non-Windows platforms WideString is always an alias to > UnicodeString. The distinctive WideString type only exists on Windows > platforms as it relies on Windows specific functionality. This is only about generated debug information, I suppose ?

I see you also picked up on this, I also picked it up  (on mantis) and ran some tests and then filed https://bugs.freepascal.org/view.php?id=35444

Reply by email