currently there is some demand for a fast widestring type on windows and i suspect there will be requirements for more string types in the future.

So i make a proposal for a new syntax that will allow string types to be created easilly in libraries or user code.

my proposal is a statement like

type
  tmystring=string(elementtype,allocator,deallocator,copier,uniquer);

the allocator,deallocator,copier and uniquer would generally be inline functions

the elementtype may be any type that doesn't use any compiler magic the allocator allocates a new string of a given number of elements and returns a pointer to the first element, length refcount

the allocator and deallocators jobs are obvious to create new strings and reduce the reference count of/dispose of old ones.

the copiers job is to increase the reference count if the string is reference counted or copy the string if not

the uniquers job is to copy the contents of the string if it is refcounted and its reference count is greater than 1.

indexing of the string would be handled in the same way as current strings, that is normally direct access but call the uniquer automatically before starting writes

if my understanding is correct this should allow all the widestrings stuff to be moved to the library and people to implement thier own string types as required (for example maybe a widestring counterpart for some system that is similar to com)

Op Sun, 4 Feb 2007, schreef peter green:

my proposal is a statement like type tmystring=string(elementtype,allocator,deallocator,copier,uniquer);

And how should you generate debug information for such a user defined string?

Daniël

if we require user defined string types to keep the conventions that the underlying pointer points to the first character (with metadata at negative offsets) and that there is a null after the last character in memory then at least for simple element types debugging tools should be able to handle them in the same way they currently do (by treating them as C strings).

from a low level perspective such a string is just a pointer to an array plus some compiler magic just like current ansistrings, debugging tools shouldn't need to be aware of the magic any more than they need to be aware of it for current ansistrings.

peter green wrote

currently there is some demand for a fast widestring type on windows and i suspect there will be requirements for more string types in the future.

During the creation of our project, Technetium (www.technetium.dk), we designed a string library that would incorporate support for not only widestrings, but also ansistrings, utf-8 strings, mbcs, etc - and not by compiling to different types, but at the /same/ time.

My programming partner, Ivo Steinmann, has done a truly excellent job implementing and perfecting this string library. If used right, it's also faster than FPC's internal strings.

We'll discuss licensing options, and try to see if we can make ready a public source collection for testing.

The source is directly compatible with FPC in objfpc mode, and should even be /somewhat/ compatible with delphi, if you remove the operator overloading. These overloaded operators does make the library /much/ easier to use, though.

Is anyone interested in testing this?

This is a very good idea.
There should be two additional functions, one to set string constants (literals) in the current locale encoding and one in UTF-8 encoding if the compiler runs with -Fcutf8.
MSEgui is completely done with widestrings, it suffers from the switch to not reference counted widestrings on windows.
Also wanted: a possibility to have wide-resourcestrings.

Martin

Agreed. To show ansi and widestrings in MSEide watch window correct I must read the string data as binary anyway.

Martin

if my understanding is correct this should allow all the widestrings stuff to be moved to the library and people to implement thier own string types as required (for example maybe a widestring counterpart for some system that is similar to com)

What's wrong with

type
   tmywidestring = type array of widechar;

?

- Null based index.
- mywidestring:= 'abcde'; not possible.
- No terminating null character.
- Lower performance due to nulling memory by setlength.
- No unique check by assigning to character.

Martin

You can ignore the char 0?

- mywidestring:= 'abcde'; not possible.

Why not? Overload the := operator?

- No terminating null character.

This is a matter of implementation, i.e. the helper routines.

- Lower performance due to nulling memory by setlength.

Overload setlength.

- No unique check by assigning to character.

True. But maybe we should simply allow to overload [] then.

Op Sun, 4 Feb 2007, schreef Florian Klaempfl:

> - No unique check by assigning to character. True. But maybe we should simply allow to overload [] then.

Not necessary, wrap it into an object and define a default property.

Daniël

Then you need to implement a conversion for pwidechar(x).

- Object is not refcounted, which is why dyn array is used.
- performance was the reason for this :-)

What's wrong with type tmywidestring = type array of widechar;

a number of things

1: afaict it lacks the automatic refcounting/copy on write semantics of strings 2: it doesn't allow specification of the allocators/deallocators which causes problems for its safe passing to dlls etc

Op Sun, 4 Feb 2007, schreef Marco van de Voort:

I meant a dynarray wrapped in an object.

Daniël

Then you can't take the address anymore of individual characters, as Thorsten mentioned on irc last night. E.g. this will not have the desired effect:

move(mystringobj[1],chararray,mystringobj.len)

But in general I dislike allowing to overload everything but the kitchen sink, otherwise we'll end up with C++ (which does allow you to overload pretty much every operator/token, meaning that you always have to read each and every header and carefully look at each type before you have any idea at all about what a certain expression could mean).

Jonas

Florian Klaempfl wrote

> - Null based index. You can ignore the char 0?

[...]

Do you think it is possible with this methods to achieve the same or better performance as with ansi strings or refcounted widestrings?

Martin

Op Sun, 4 Feb 2007, schreef Martin Schreiber:

No "user" string type can match the performance of built in string types. It is best if the language itself provides all the necessary string types.

Daniël