I’m making some string helper functions and using ansistring as inputs (in case there are longer strings than 255 chars). What happens when you pass a short string as an ansistring? Does the compiler have to allocate a new ansistring or can it do some smart optimization?

Regards,
Ryan Joseph

Ryan Joseph wrote

I’m making some string helper functions and using ansistring as inputs (in case there are longer strings than 255 chars). What happens when you pass a short string as an ansistring? Does the compiler have to allocate a new ansistring or can it do some smart optimization?

It has to allocate a new ansistring.

Jonas

Ouch. That means I need to make lots of overloads. This is a prime candidate for generics then but I’m still waiting for my “implicitfunctionspecializations” modeswitch patch to be accepted.

Regards,
Ryan Joseph

On Jun 25, 2019, at 2:16 PM, Jonas Maebe wrote: It has to allocate a new ansistring.

Another question: if I do "s += c” will the string reallocate memory for 1 character or is there an exponential growing function? If it does grow by one character at a time how can I reserve a certain capacity in advance?

Regards,
Ryan Joseph

Ryan Joseph wrote

Another question: if I do "s += c” will the string reallocate memory for 1 character or is there an exponential growing function? If it does grow by one character at a time how can I reserve a certain capacity in advance?

Strings always contain the exact amount of memory required for their length.

If you need to do a lot of concatenations, you can use the TStringBuilder class from sysutils for better performance.

Jonas

Op 2019-06-25 om 21:12 schreef Ryan Joseph:

An exponential growing function applied to all strings would have many worse case behaviours, where it would eat up heaps of memory for nothing. This is why it is better to have the programmer indicate it is a growing string some how (or have a separate type, like TStringBuilder mentioned by Jonas)

On Jun 25, 2019, at 3:55 PM, Marco van de Voort wrote: An exponential growing function applied to all strings would have many worse case behaviours, where it would eat up heaps of memory for nothing. This is why it is better to have the programmer indicate it is a growing string some how (or have a separate type, like TStringBuilder mentioned by Jonas)

It would be nice if ansistring had a SetCapacity function to accompany SetLength but in leu of that I think making a new type is the best idea.

Regards,
Ryan Joseph

Op 25/06/2019 om 21:58 schreef Ryan Joseph:

An exponential growing function applied to all strings would have many worse case behaviours, where it would eat up heaps of memory for nothing. This is why it is better to have the programmer indicate it is a growing string some how (or have a separate type, like TStringBuilder mentioned by Jonas) It would be nice if ansistring had a SetCapacity function to accompany SetLength but in leu of that I think making a new type is the best idea.

I think we have enough string types
http://www.stack.nl/~marcov/delphistringtypes.txt

TStringbuilder is fine . Not every problem needs new syntax or types to solve.

I second that.

Michael.

AM Michael Van Canneyt wrote

I second that. Michael.

I think Ryan probably meant his own custom types. And certainly, you can do some interesting stuff with operator overloading that mostly avoids the normal AnsiString overhead. Here's an example.
<https://godbolt.org/#g:!((g:!((g:!((h:codeEditor,i:(j:1,lang:pascal,source:'unit+output%3B%0A%0A%7B$mode+ObjFPC%7D%7B$H%2B%7D%0A%7B$modeswitch+AdvancedRecords%7D%0A%7B$ImplicitExceptions+Off%7D%0A%0Ainterface%0A%0Atype+%0A++TPCharEx+%3D+record%0A++++P:+PChar%3B%0A++++L:+SizeInt%3B%0A++++class+operator+:%3D(constref+LHS:+AnsiString):+TPCharEx%3B+inline%3B%0A++++class+operator+:%3D(constref+LHS:+ShortString):+TPCharEx%3B+inline%3B%0A++end%3B%0A%0Afunction+Reversed(constref+P:+TPCharEx):+AnsiString%3B%0Aprocedure+ImplicitConversionTest%3B+inline%3B%0A%0Aimplementation%0A%0Aclass+operator+TPCharEx.:%3D(constref+LHS:+AnsiString)(LHS)%3B%0A++end%3B%0Aend%3B%0A%0Aclass+operator+TPCharEx.:%3D(constref+LHS:+ShortString)(LHS%5B0%5D)%3B%0A++end%3B%0Aend%3B%0A%0Afunction+Reversed(constref+P:+TPCharEx):+AnsiString%3B%0Avar+Old,+New:+PChar%3B%0Abegin%0A++SetLength(Result,+P.L+%2B+1)(New)%3B%0A++++Dec(Old)%3B%0A++end%3B%0Aend%3B%0A%0Aprocedure+ImplicitConversionTest%3B%0Abegin%0A++WriteLn(Reversed(!'ABCD!'))%3B%0Aend%3B%0A%0Aend.%0A'),l:'5',n:'0',o:'Pascal+source+%231',t:'0')),k:54.64556642679632,l:'4',n:'0',o:'',s:0,t:'0'),(g:!((h:compiler,i:(compiler:fpc304,filters:(b:'0',binary:'1',commentOnly:'0',demangle:'0',directives:'0',execute:'0',intel:'0',libraryCode:'1',trim:'1'),lang:pascal,libs:!(),options:'-O3+-Ci-+-Cr-+-g-+-CX+-XXs',source:1),l:'5',n:'0',o:'x86-64+fpc+3.0.4+(Editor+%231,+Compiler+%231)+Pascal',t:'0')),k:44.015665176626996,l:'4',n:'0',o:'',s:0,t:'0'),(g:!((h:output,i:(compiler:1,editor:1,wrap:'1'),l:'5',n:'0',o:'%231+with+x86-64+fpc+3.0.4',t:'0')),k:1.3387683965766675,l:'4',n:'0',o:'',s:0,t:'0')),l:'2',n:'0',o:'',t:'0')),version:4>

Messed up the "Reversed" function a little bit in that example. Fixed version.
<https://godbolt.org/#g:!((g:!((g:!((h:codeEditor,i:(j:1,lang:pascal,source:'unit+output%3B%0A%0A%7B$mode+ObjFPC%7D%7B$H%2B%7D%0A%7B$modeswitch+AdvancedRecords%7D%0A%7B$ImplicitExceptions+Off%7D%0A%0Ainterface%0A%0Atype+%0A++TPCharEx+%3D+record%0A++++P:+PChar%3B%0A++++L:+SizeInt%3B%0A++++class+operator+:%3D(constref+LHS:+AnsiString):+TPCharEx%3B+inline%3B%0A++++class+operator+:%3D(constref+LHS:+ShortString):+TPCharEx%3B+inline%3B%0A++end%3B%0A%0Afunction+Reversed(constref+P:+TPCharEx):+AnsiString%3B%0Aprocedure+ImplicitConversionTest%3B+inline%3B%0A%0Aimplementation%0A%0Aclass+operator+TPCharEx.:%3D(constref+LHS:+AnsiString)(LHS)%3B%0A++end%3B%0Aend%3B%0A%0Aclass+operator+TPCharEx.:%3D(constref+LHS:+ShortString)(LHS%5B0%5D)%3B%0A++end%3B%0Aend%3B%0A%0Afunction+Reversed(constref+P:+TPCharEx):+AnsiString%3B%0Avar+Old,+New:+PChar%3B%0Abegin%0A++SetLength(Result,+P.L)(New)%3B%0A++++Dec(Old)%3B%0A++end%3B%0Aend%3B%0A%0Aprocedure+ImplicitConversionTest%3B%0Abegin%0A++WriteLn(Reversed(!'ABCD!'))%3B%0Aend%3B%0A%0Aend.%0A'),l:'5',n:'0',o:'Pascal+source+%231',t:'0')),k:54.64556642679632,l:'4',n:'0',o:'',s:0,t:'0'),(g:!((h:compiler,i:(compiler:fpc304,filters:(b:'0',binary:'1',commentOnly:'0',demangle:'0',directives:'0',execute:'0',intel:'0',libraryCode:'1',trim:'1'),lang:pascal,libs:!(),options:'-O3+-Ci-+-Cr-+-g-+-CX+-XXs',source:1),l:'5',n:'0',o:'x86-64+fpc+3.0.4+(Editor+%231,+Compiler+%231)+Pascal',t:'0')),k:44.015665176626996,l:'4',n:'0',o:'',s:0,t:'0'),(g:!((h:output,i:(compiler:1,editor:1,wrap:'1'),l:'5',n:'0',o:'%231+with+x86-64+fpc+3.0.4',t:'0')),k:1.3387683965766675,l:'4',n:'0',o:'',s:0,t:'0')),l:'2',n:'0',o:'',t:'0')),version:4>

On Jun 26, 2019, at 3:28 PM, Ben Grasset wrote: I think Ryan probably meant his own custom types. And certainly, you can do some interesting stuff with operator overloading that mostly avoids the normal AnsiString overhead. Here's an example.

Yes indeed. FPC already has an overwhelming amount of string types. As I said though a SetCapacity option for growing would be nice (for dynamic arrays also) because += is such a common operation. As it stands I often don’t use dynamic arrays (and now ansistring) because of growing limitations, which is a shame really.

Regards,
Ryan Joseph

Well, I think the idea is that stuff like capacity is what you'd have / do have in actual containers built *around* strings and / or dynamic arrays, as they're both just built in primitives.

No. That would mean an additional entry in the metadata of strings and arrays and those are already big enough in my opinion compared to the data they're handling.
If one needs capacity based handling for these types then one should use helper types.

Regards,
Sven

Note: with which I mean types like TStringBuilder, not type helpers.

Regards,
Sven

Yes indeed. FPC already has an overwhelming amount of string types. As I said though a SetCapacity option for growing would be nice (for dynamic arrays also) because += is such a common operation. As it stands I often don’t use dynamic arrays (and now ansistring) because of growing limitations, which is a shame really.

Just chiming in: this should be handled by the memory manager. "A chunk of memory gets grown repeatedly" is actually a fairly common workload, not just for arrays, but also things like MemoryStreams or List classes.

So, I would expect (and FastMM has codepaths for that), that repeated reallocations cause some form of "over-allocating" growth and most of the individual "+1" reallocs will be essentially no-ops.

Interesting. How could we test this to see if it’s true?

Regards,
Ryan Joseph

From FastMM4's readme:

https://github.com/pleriche/FastMM4/blob/master/README.md

"Intelligent reallocations. Avoids slow memory move operations through not performing unneccesary downsizes and by having a minimum percentage block size growth factor when an in-place block upsize is not possible."

Allocations within certain ranges are all lumped together based on fixed block sizes (e.g., 15-24 bytes allocations are all rounded up to 24 bytes). It both reduces fragmentation and helps with resize performance (there is other resize-specific logic as well).

String allocations and performance factored heavily into the benchmarking for the FastCode competition that lead to FastMM4. You should do your own benchmarking vs the system allocator though, to see if it provides an improvement, especially if you're doing extensive multithreading. We've been using FastMM with FPC on macOS and Linux for about 4 years now and are very happy with it.