Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2007-09

more questions on storage

21 messages · 6 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Marc Santhoff

Hi,

having this definition:

{$mode objfpc}
{$longstrings on}
type
	cmpstr = record
		ID: integer;
		Name: string;
		Value: single;
	end;
var
	buf: array [0..MAX] of cmpstr;

How does the memory layout of the buffer look like?

I tried to get there with ddd but I couln't find out how the string inbetween is put into memory. For atomic types (integer, single) it's pretty clear, but longstrings do have a variable size. How is the buffer set up?

TIA,
Marc

Michael Van Canneyt
to Marc Santhoff
Marc Santhoff wrote 23 lines
Hi, having this definition: {$mode objfpc} {$longstrings on} type cmpstr = record ID: integer; Name: string; Value: single; end; var buf: array [0..MAX] of cmpstr; How does the memory layout of the buffer look like? I tried to get there with ddd but I couln't find out how the string inbetween is put into memory. For atomic types (integer, single) it's pretty clear, but longstrings do have a variable size. How is the buffer set up?

Longstrings are not working correctly. You should use ansistrings or shortstrings.

Michael.

Marc Santhoff
to Michael Van Canneyt
Michael Van Canneyt wrote 29 lines
On Wed, 26 Sep 2007, Marc Santhoff wrote: > Hi, > > having this definition: > > > {$mode objfpc} > {$longstrings on} > type > cmpstr = record > ID: integer; > Name: string; > Value: single; > end; > > var > buf: array [0..MAX] of cmpstr; > > How does the memory layout of the buffer look like? > > I tried to get there with ddd but I couln't find out how the string > inbetween is put into memory. For atomic types (integer, single) it's > pretty clear, but longstrings do have a variable size. How is the buffer > set up? Longstrings are not working correctly. You should use ansistrings or shortstrings.

Okay, good to know. Besides I'm using fpc 2.0.4, what is it that's not working?

And back to the original question: how is the memory layout using AnsiString then?

Thanks,
Marc

Marco van de Voort
to Marc Santhoff
Show quoted text 10 lines
> > set up? > > Longstrings are not working correctly. You should use ansistrings or > shortstrings. Okay, good to know. Besides I'm using fpc 2.0.4, what is it that's not working? And back to the original question: how is the memory layout using AnsiString then?

The field is a pointer to the ansistring. Which is basically a pchar with a TAnsirec record on negative offset of the ptr value.

Jonas Maebe
to Marc Santhoff: Okay, good to know. Besides I'm using fpc 2.0.4, what is it that's not working? And back…
Marc Santhoff wrote 5 lines
> Longstrings are not working correctly. You should use ansistrings or > shortstrings. Okay, good to know. Besides I'm using fpc 2.0.4, what is it that's not working?

Longstrings were never completely implemented nor tested, so no one really knows what works and what doesn't. It would be better if the compiler simply gave an error for them at compile time.

Jonas

Marc Santhoff
to Jonas Maebe
Jonas Maebe wrote 11 lines
On 26 Sep 2007, at 11:18, Marc Santhoff wrote: >> Longstrings are not working correctly. You should use ansistrings or >> shortstrings. > > Okay, good to know. Besides I'm using fpc 2.0.4, what is it that's not > working? Longstrings were never completely implemented nor tested, so no one really knows what works and what doesn't. It would be better if the compiler simply gave an error for them at compile time.

That's irritating, because "rtl.pdf" from the 2.0.4 release says on page 19:

1.1.20 $H or $LONGSTRINGS : Use AnsiStrings
If {$LONGSTRINGS ON} is specified, the keyword String (no length specifier) will be treated
as AnsiString, and the compiler will treat the corresponding variable as an ansistring, and will
generate corresponding code.

I read it as there is no difference between the two (in that release) and the compiler switches can be exchanged.

FWIW, I have this sitch in a couple of source files for a while now and never had any (noticable) problems.

Marc

Marc Santhoff
to Marco van de Voort: The field is a pointer to the ansistring. Which is basically a pchar with a TAnsirec…
Marco van de Voort wrote 13 lines
> > > set up? > > > > Longstrings are not working correctly. You should use ansistrings or > > shortstrings. > > Okay, good to know. Besides I'm using fpc 2.0.4, what is it that's not > working? > > And back to the original question: how is the memory layout using > AnsiString then? The field is a pointer to the ansistring. Which is basically a pchar with a TAnsirec record on negative offset of the ptr value.

Thanks, that's what I assumed and could not proof.

Is this the case for any occurences of AnsiString, the type name itself is a pointer behind the scenes or only in this special case of an array (would make less sense)?

Marc

Marco van de Voort
to Marc Santhoff
Show quoted text 9 lines
Am Mittwoch, den 26.09.2007, 11:34 +0200 schrieb Marco van de Voort: > The field is a pointer to the ansistring. Which is basically a pchar with a > TAnsirec record on negative offset of the ptr value. Thanks, that's what I assumed and could not proof. Is this the case for any occurences of AnsiString, the type name itself is a pointer behind the scenes or only in this special case of an array (would make less sense)?

In general, and can be nil when empty.

Jonas Maebe
to Marc Santhoff: That's irritating, because "rtl.pdf" from the 2.0.4 release says on page 19: 1.1.20 $H or…
Marc Santhoff wrote 18 lines
> Longstrings were never completely implemented nor tested, so no one > really knows what works and what doesn't. It would be better if the > compiler simply gave an error for them at compile time. That's irritating, because "rtl.pdf" from the 2.0.4 release says on page 19: 1.1.20 $H or $LONGSTRINGS : Use AnsiStrings If {$LONGSTRINGS ON} is specified, the keyword String (no length specifier) will be treated as AnsiString, and the compiler will treat the corresponding variable as an ansistring, and will generate corresponding code. I read it as there is no difference between the two (in that release) and the compiler switches can be exchanged.

It seems there's some confusion in terminology. {$longstrings on} maps to {$h+}, which indeed means that string becomes an alias for ansistring rather than for shortstring. Separately from that there is also a type called "longstring" which is basically a shortstring but with a 4 byte length field. It's this longstring which has not been implemented properly.

Ansistrings (and thus {$longstrings on}) work fine.

Jonas

Jonas Maebe
to Marco van de Voort: In general, and can be nil when empty.
Marco van de Voort wrote 14 lines
> Am Mittwoch, den 26.09.2007, 11:34 +0200 schrieb Marco van de Voort: >> The field is a pointer to the ansistring. Which is basically a >> pchar with a >> TAnsirec record on negative offset of the ptr value. > > Thanks, that's what I assumed and could not proof. > > Is this the case for any occurences of AnsiString, the type name > itself > is a pointer behind the scenes or only in this special case of an > array > (would make less sense)? In general, and can be nil when empty.

And it's reference counted, so any record containing an ansistring has initialization rtti (and when declaring a record of this type, or allocating memory for it using new(), the compiler initialises the ansistring fields with nil).

Jonas

Marc Santhoff
to Jonas Maebe: It seems there's some confusion in terminology. {$longstrings on} maps to {$h+}, which…
Jonas Maebe wrote 29 lines
On 26 Sep 2007, at 11:56, Marc Santhoff wrote: >> Longstrings were never completely implemented nor tested, so no one >> really knows what works and what doesn't. It would be better if the >> compiler simply gave an error for them at compile time. > > That's irritating, because "rtl.pdf" from the 2.0.4 release says on > page > 19: > > 1.1.20 $H or $LONGSTRINGS : Use AnsiStrings > If {$LONGSTRINGS ON} is specified, the keyword String (no length > specifier) will be treated > as AnsiString, and the compiler will treat the corresponding > variable as > an ansistring, and will > generate corresponding code. > > I read it as there is no difference between the two (in that release) > and the compiler switches can be exchanged. It seems there's some confusion in terminology. {$longstrings on} maps to {$h+}, which indeed means that string becomes an alias for ansistring rather than for shortstring. Separately from that there is also a type called "longstring" which is basically a shortstring but with a 4 byte length field. It's this longstring which has not been implemented properly. Ansistrings (and thus {$longstrings on}) work fine.

Ah, okay. The Fog is lifting fastly. :)

And AnsiStrings are pretty much compatible to C's char*. Very handy.

Many thanks,
Marc

Marco van de Voort
to Marc Santhoff
Show quoted text 11 lines
> On 26 Sep 2007, at 11:56, Marc Santhoff wrote: > ansistring rather than for shortstring. Separately from that there is > also a type called "longstring" which is basically a shortstring but > with a 4 byte length field. It's this longstring which has not been > implemented properly. > > Ansistrings (and thus {$longstrings on}) work fine. Ah, okay. The Fog is lifting fastly. :) And AnsiStrings are pretty much compatible to C's char*. Very handy.

Yes, though ansistrings may contain #0's.

Marc Santhoff
to Marco van de Voort
Marco van de Voort wrote 13 lines
> > On 26 Sep 2007, at 11:56, Marc Santhoff wrote: > > ansistring rather than for shortstring. Separately from that there is > > also a type called "longstring" which is basically a shortstring but > > with a 4 byte length field. It's this longstring which has not been > > implemented properly. > > > > Ansistrings (and thus {$longstrings on}) work fine. > > Ah, okay. The Fog is lifting fastly. :) > > And AnsiStrings are pretty much compatible to C's char*. Very handy. Yes, though ansistrings may contain #0's.

Interesting, how will a cast handle this case?

Will it change the intermediate #0 to something else or will the resulting pchars be cut at first occurrence?

I'd never use #0 in the middle of a string myself, but to be sure if a user manages to do so ...

Marc

Jonas Maebe
to Marc Santhoff
Marc Santhoff wrote 8 lines
>> And AnsiStrings are pretty much compatible to C's char*. Very handy. > > Yes, though ansistrings may contain #0's. Interesting, how will a cast handle this case? Will it change the intermediate #0 to something else or will the resulting pchars be cut at first occurrence?

You just get a pointer to the string data as-is, without any substitutions or pointer juggling. If whatever you pass the data to interprets #0 as the end of the string, it will see that embedded #0 as the end of the string.

Jonas

Marco van de Voort
to Marc Santhoff: Interesting, how will a cast handle this case? Will it change the intermediate #0 to…
Show quoted text 14 lines
Am Mittwoch, den 26.09.2007, 13:24 +0200 schrieb Marco van de Voort: > > > > > > Ansistrings (and thus {$longstrings on}) work fine. > > > > Ah, okay. The Fog is lifting fastly. :) > > > > And AnsiStrings are pretty much compatible to C's char*. Very handy. > > Yes, though ansistrings may contain #0's. Interesting, how will a cast handle this case? Will it change the intermediate #0 to something else or will the resulting pchars be cut at first occurrence?

A cast is really a cast. IOW the cast pchar(ansistring) is mostly a no-op. Traditional C code then usually treats the #0 as end of string.

It's mostly important when ansisstring is abused as general purpose buffer, but generally it is better to use dyn arrays for that.

Mattias Gaertner
to Marco van de Voort
Show quoted text 19 lines
> Am Mittwoch, den 26.09.2007, 13:24 +0200 schrieb Marco van de Voort: > > > > > > > > Ansistrings (and thus {$longstrings on}) work fine. > > > > > > Ah, okay. The Fog is lifting fastly. :) > > > > > > And AnsiStrings are pretty much compatible to C's char*. Very > > > handy. > > > > Yes, though ansistrings may contain #0's. > > Interesting, how will a cast handle this case? > Will it change the intermediate #0 to something else or will the > resulting pchars be cut at first occurrence? A cast is really a cast. IOW the cast pchar(ansistring) is mostly a no-op. Traditional C code then usually treats the #0 as end of string.

PChar(AnsiString) was a no-op typecast in the past and is nowadays a function. It checks whether the AnsiString is nil and if yes returns a pointer to a string containing one character: #0.

That means:
Pointer(AnsiString) Pointer(PChar(AnsiString))

To get a no-op typecase you can use:
MyPChar:=PChar(Pointer(AnsiString));
It's mostly important when ansisstring is abused as general purpose buffer, but generally it is better to use dyn arrays for that.

Mattias

Marco van de Voort
to Mattias Gaertner
Show quoted text 14 lines
(Marco van de Voort) wrote: > A cast is really a cast. IOW the cast pchar(ansistring) is mostly a > no-op. Traditional C code then usually treats the #0 as end of > string. PChar(AnsiString) was a no-op typecast in the past and is nowadays a function. It checks whether the AnsiString is nil and if yes returns a pointer to a string containing one character: #0. That means: Pointer(AnsiString) Pointer(PChar(AnsiString)) To get a no-op typecase you can use: MyPChar:=PChar(Pointer(AnsiString));

Is this delphi compat, and if yes, what is this exactly for?

Michael Van Canneyt
to Marco van de Voort
Marco van de Voort wrote 16 lines
> (Marco van de Voort) wrote: > > A cast is really a cast. IOW the cast pchar(ansistring) is mostly a > > no-op. Traditional C code then usually treats the #0 as end of > > string. > > PChar(AnsiString) was a no-op typecast in the past and is nowadays a > function. It checks whether the AnsiString is nil and if yes returns a > pointer to a string containing one character: #0. > > That means: > Pointer(AnsiString) Pointer(PChar(AnsiString)) > > To get a no-op typecase you can use: > MyPChar:=PChar(Pointer(AnsiString)); Is this delphi compat, and if yes, what is this exactly for?

It is Delphi compatible.

PChar(AnsiString)

Will always point to a null-terminated string. If the string is empty (which means the pointer value is Nil), a pointer to a pre-defined null-terminated string is returned. In other words: you always get a non-nil value.

On the other hand:

Pointer(AnsiString)

Returns Nil if the string is empty.

Michael.

Mattias Gaertner
to Marco van de Voort: Is this delphi compat, and if yes, what is this exactly for?
Show quoted text 16 lines
> (Marco van de Voort) wrote: > > A cast is really a cast. IOW the cast pchar(ansistring) is mostly > > a no-op. Traditional C code then usually treats the #0 as end of > > string. > > PChar(AnsiString) was a no-op typecast in the past and is nowadays a > function. It checks whether the AnsiString is nil and if yes > returns a pointer to a string containing one character: #0. > > That means: > Pointer(AnsiString) Pointer(PChar(AnsiString)) > > To get a no-op typecase you can use: > MyPChar:=PChar(Pointer(AnsiString)); Is this delphi compat, and if yes, what is this exactly for?

I don't know.

The change broke some lazarus code and now it is polluted by the ugly double type casts, just to get some no-op type casts. Because this fact is not widely known or often forgotten, many use the PChar() as simple type cast. That's why the lcl interfaces already got into troubles several times, so everytime we get strange errors I first search for these typecasts.
Maybe we should mention this fact at a more prominent place in the wiki and docs.

Mattias

Marc Weustink
to Mattias Gaertner
Mattias Gaertner wrote 28 lines
On Thu, 27 Sep 2007 12:38:13 +0200 (CEST) (Marco van de Voort) wrote: >> (Marco van de Voort) wrote: >>> A cast is really a cast. IOW the cast pchar(ansistring) is mostly >>> a no-op. Traditional C code then usually treats the #0 as end of >>> string. >> PChar(AnsiString) was a no-op typecast in the past and is nowadays a >> function. It checks whether the AnsiString is nil and if yes >> returns a pointer to a string containing one character: #0. >> >> That means: >> Pointer(AnsiString) Pointer(PChar(AnsiString)) >> >> To get a no-op typecase you can use: >> MyPChar:=PChar(Pointer(AnsiString)); > Is this delphi compat, and if yes, what is this exactly for? I don't know. The change broke some lazarus code and now it is polluted by the ugly double type casts, just to get some no-op type casts. Because this fact is not widely known or often forgotten, many use the PChar() as simple type cast. That's why the lcl interfaces already got into troubles several times, so everytime we get strange errors I first search for these typecasts. Maybe we should mention this fact at a more prominent place in the wiki and docs.

Maybe also add another "type" to get the OldTypecase behaviour.

I allways thought that casting through Pointer() did return the real allocated data (thus including reference and length)

Marc

Marco van de Voort
to Mattias Gaertner: On Thu, 27 Sep 2007 12:38:13 +0200 (CEST) I don't know. The change broke some lazarus…
Show quoted text 18 lines
On Thu, 27 Sep 2007 12:38:13 +0200 (CEST) > > Pointer(AnsiString) Pointer(PChar(AnsiString)) > > > > To get a no-op typecase you can use: > > MyPChar:=PChar(Pointer(AnsiString)); > > Is this delphi compat, and if yes, what is this exactly for? I don't know. The change broke some lazarus code and now it is polluted by the ugly double type casts, just to get some no-op type casts. Because this fact is not widely known or often forgotten, many use the PChar() as simple type cast. That's why the lcl interfaces already got into troubles several times, so everytime we get strange errors I first search for these typecasts. Maybe we should mention this fact at a more prominent place in the wiki and docs.

Maybe yes. I quickly checked dateutils and strutils and found a few cases that were easy to "fix". (all using a NIL checking routine immediately after)

Sysutils however had 39, and I suspect several would crash when changed, since few routines have empty string early-outs like e.g. strutils has.

I'm still investigating those.