Hello,

As happy user of mORMot (ORM framework) and FPC, I am encountering some difficulties with the (more or less recent) string changes. It's hard at the moment to get string handling for Delphi and FPC lined-up.

So, I have this (hypothetical) question !

In the (near) future, I am still a very happy user of FPC. And mORMot. And sometimes some version of Delphi > XE2.
Lets say its 2017. And I am using FPC 3.2.0. Or FPC 4.0.

What mode will I be using when writing programs/frameworks for both Delphi and FPC ?
* is -Mdelphiunicode the way to go ?
* do we have a 100% UTF8 RTL.
* UTF16 / UTF32 ?
* ......

The question could also be: how to make a brand new start that is more or less future proof and compatible with Delphi unicode.

The answer to this question is important for making decisions at this very right moment.
Some hacks are (still) needed to get strings working for the mORMot framework in both Delphi and FPC.
And I (we) would like make hacks that can be dismissed when using future versions of FPC.

I know the question is a bit strange, but I would appreciate some answers, if possible.

Mmmm ... not that many answers ...

Let me try another question: does the link below correctly state the current use and especially future of FPC / Lazarus ?

http://wiki.freepascal.org/Better_Unicode_Support_in_Lazarus

Especially (quotes):
* The goal of FPC project is to create a Delphi compatible UnicodeString (UTF-16) based solution.
* If you want to use the new UTF-8 strings in your non LCL programs, add LazUTF8 unit in the uses section of the main program file.
* UTF8String becomes useful when eventually there is an UTF-16 FCL.

In addition to the above: how do I know how far FPC trunk is on the roadmap as described in the link.

This is not very clear. Where do you see a roadmap in this page?

And to the best of my knowledge, FPC doesn't have a roadmap where strings are concerned.

Michael.

I was assuming that a goal (first quote: UTF16) would be accompanied by (some sort of) a roadmap.
Perhaps I am wrong.

But, again, a comment on the remaining quotes would be welcome. And also a comment on the use of {$mode delphiunicode}.

The -Mdelphiunicode is the first step towards Delphi XE compatibility. The next step is to provide a compatible UTF-16 UnicodeString RTL/FCL and then third party packages like the LCL can be adapted. This would be the most compatible solution. I don't know who is working on the UTF-16 RTL/FCL.

* do we have a 100% UTF8 RTL.

Not 100%. For example the ThousandSeparator is defined as char. And under Windows there are some units still using the Windows A functions, so they only support the system codepage.

* UTF16 / UTF32 ?

There are no plans for UTF32 strings.

* ...... The question could also be: how to make a brand new start that is more or less future proof and compatible with Delphi unicode.

Option 1: Help creating the UTF16 RTL/FCL and then use mode delphiunicode.
Option 2: Use mode Delphi with IFDEFs and UTF-8 or system codepage.

Mattias

Alfred wrote

I was assuming that a goal (first quote: UTF16) would be accompanied by (some sort of) a roadmap.

Opinions are divided, hence there is no roadmap.

The main point is what to do with backwards compatibility. Most people do not want to give this up.

There is also the question of what to do with use of dotted units and namespaces.

There are IMHO 2 options:

1. Making String = Widestring (as Delphi did) will require work, and will definitely break backwards compatibility.

2. Provide 2 sets of units:
a) one ansistring based. (basically what we have now) b) one widestring based.
People wishing to continue with backwards compatible code, will use a), others will use b)

My personal preference is 2, with the following 'addition':

1. Provide units as they are now, where string = ansistring.

2. Provide Delphi-compatible dotted units, with string = widestring.

Basically, the user has then 2 choices:
1. is the pre-delphi 2009 option,
2. is the Delphi 2009 and higher option

Both units can be created using a single codebase quite easily. I take as example the sysutils unit:

unit sysutils.pp contains {$IFDEF UNICODESTRING} to distinguish between unicode and non-unicode code.

the dotter unit system.sysutils.pp is then the following simple file:

{$mode delphiunicode}
{$define UNICODESTRING}
{$I sysutils.pp}

The above is my personal preference, but other people in the core group may (and do) have different preferences.

Michael.

No-one.
Jonas made sure that all OS System interfaces can be done with both Ansi and Widestring. That's it.

The problem starts with classes.
Methods could be overloaded, that is not a problem (except lots of duplicated code).

But is impossible to have TComponent.Name have type AnsiString and UnicodeString at the same time. Same for all other string properties, and TStrings in particular.

Michael.

In our previous episode, Michael Van Canneyt said:

I usually use Delphi XE2+ with namespace prefix, and many with me. IMHO requiring to change Delphi sources to dotted units is a nono.

In our previous episode, Michael Van Canneyt said:

Very low prio. I only have fragments of time.

I do have a patch for executeprocess new style somehwere, and had started on TProcess.

[...] I usually use Delphi XE2+ with namespace prefix, and many with me. IMHO requiring to change Delphi sources to dotted units is a nono.

What do you mean with Delphi sources? The Classes unit or the user's unit?

Do you mean the dotted unitname will not be needed when the compiler is extended to support namespace prefixes (project and command line switch)?

Mattias

In our previous episode, Mattias Gaertner said:

User code that is dual use Delphi XE2+ and FPC. So Delphi XE2+ code without all units dotted, except the designer generated ones (too lazy to rename that all the time)

Do you mean the dotted unitname will not be needed when the compiler is extended to support namespace prefixes (project and command line switch)?

I assume not since that would be Delphi incompatible. Under delphi this is possible.

You lost me here.
Why would implementing the Delphi features be Delphi incompatible?

Mattias

In our previous episode, Mattias Gaertner said:

D7+ have dotted units support. DXE2 rename the RTL units to dotted units.

DXE2+ also allow to introduce the scopeprefix so that you don't need to use dotted units (iow if you uses sysutils, then System.sysutils.dcu is found etc).

So I use

@dcc32 "-NSSystem;System.Win;WinAPI;Vcl;Vcl.Imaging;Data;VclTee" %1 %2 %3 %4 %5 %6

in a batchfile and I can use all unit names without scopenames.

However in Michael's scheme with Sysutils using Ansi and System.Sysutils using unicodestring this will fail.

Worse, one mistake in your sources (e.g. a non qualified forms) and you get bunches of "Incompatible Type Forms.TForm, expected VCL.Forms.TForm" style errors.

I think some solution with alternative -Fu settings is to be prefered and not abuse an unrelated feature (dotted unitnames) to have both ansi an unicodestrings in one install.

Why would this fail ? All we need to do is introduce -NS ?

And obviously the 2 RTLs cannot be mixed.

The alternative is 4 rtls.

Dotted, ansi
not dotted, ansi

Dotted, unicode
not dotted, unicode.

which is clear nonsense.

Michael.

may

> (and do) have different preferences. I usually use Delphi XE2+ with namespace prefix, and many with me. IMHO requiring to change Delphi sources to dotted units is a nono.

Of course we are going to implement the namespace prefix as well...

Regards,
Sven

In our previous episode, Michael Van Canneyt said:

If you have a mix of generations (as is currently possible with Delphi), how do you avoid linking in two different sysutils?

And obviously the 2 RTLs cannot be mixed.

Correct. And if they only differ in name, dotted or not, and you can set namespaces to override the dots, how do you keep them apart?

No, two rtls, both dotted, with namespace if needed. Which is the same as what you propose, just kept physically apart.

We probably agree that trying to mix both approaches (dotted, non-dotted) is a recipe for disaster.

So, different directories, and different compiler configs must be used, obviously.

I never had the intention of having all this in 1 directory.

i.e. we distribute

/usr/local/lib/fpc/4.0.0/x86_64-linux/XYZ

/usr/local/lib/fpc/4.0.0/x86_64-linux/dotted/XYZ

But only one is referenced in fpc.cfg :

#IFDEF NAMESPACED
/usr/local/lib/fpc/4.0.0/x86_64-linux/dotted/XYZ
#ELSE
/usr/local/lib/fpc/4.0.0/x86_64-linux/XYZ
#ENDIF

where the -NS switch defines NAMESPACED

(or whatever)

If the above fails to satisfy you, no problem, all for it. But I don't see the use of the dotted for old code.

Lets not forget that Delphi tries to hide a lot of the complexity by always writing a complete config file and using that file when compiling.

I don't see why we cannot do the same.

Michael.

I hope somebody implements this in fpmake :) My "subarch" directory approach needed by several targets is already on hold for years as I do not owe to touch fpmake regarding this.

I also do not see any use in split directories for dotted and non-dotted: people messing with the default namespaces can also mess with -Fu. And messing with two variables (default namespaces and -Fu) is normally even worse than having only one variable.

Florian Klämpfl wrote

I hope somebody implements this in fpmake :) My "subarch" directory approach needed by several targets is already on hold for years as I do not owe to touch fpmake regarding this.

I don't understand why ?
Why would you not touch fpmake for this ? It's just adding a level of directories ?

I also do not see any use in split directories for dotted and non-dotted: people messing with the default namespaces can also mess with -Fu. And messing with two variables (default namespaces and -Fu) is normally even worse than having only one variable.

Technically, you can of course throw everything in 1 directory.

But having 2 trees keeps things clear, tidy and separate. In case of problems, the cause will be much easier to determine.

I have a build theory I could clean the directories on startup, build and be done with it. In practice, I create a directory per version of the project at build time, and build in that. Works fine, all clear, no stray files. If there is a problem, it's usually quickly determined.

I do the same for FPC, btw.

Michael.

In our previous episode, Michael Van Canneyt said:

That is correct if the namespace switches was introduced with XE2. It would fail for sources that used dotted notation and namespace, but don't assume it for the RTL, iow pre XE2.

That was just for simplicity's case.

Lets not forget that Delphi tries to hide a lot of the complexity by always writing a complete config file and using that file when compiling.

I think a good first step would be building without storing the generated files in the same dir as the source.

So a bit like configure that allows ../configure and works even when the source dir is read-only. If you start building sources multiple times in one run, that is a good way.

This is a non-argument, since that wouldn't work today either. We need to draw a line somewhere, no need to be holier than the pope.

Lets try to target e.g. D7 and XE2+

A matter of configuration, I would think.

Michael.

System.Sysutils

how

is a recipe for disaster.

So, different directories, and different compiler configs must be used,

obviously.

That shouldn't be something that depends on the namespace switch (the proper FPC parameter would probably be -FN by the way if it isn't used yet). What about people that want to use the ANSI RTL, but also use dotted names, because they exist in 3.0.0 and also want to use a namespace prefix? Or other combinations? No, it should be explicitely determined by the user whether she wants Unicode or not (even if it should just be a -dUNICODERTL that is handled by the fpc.cfg).

Regards,
Sven

same as

compiling.

one

run, that is a good way.

That's part of the build system. E.g. our RTL and packages are already built into separate directories. It's however not the task of the compiler to enforce a build directory (there's no "one size fits it all" after all).

Regards,
Sven

In our previous episode, Sven Barth said:

Yes it is, and that is what I meant.