Hello,

Posting here as this was requested in the bug report: http://bugs.freepascal.org/view.php?id=20473

I think it is very simple: Having such a directive would save a lot of work when porting Lazarus-based projects to FPC 2.7

1. Currently there is this huge work going on to get trunk to a reasonable and usable shape again and until that is done (and maybe RTL changes as well) it is (in my opinion (!) ) not reasonable to think about changing the meaning of "string" at all.
2. As Marco wrote in this mail
http://lists.freepascal.org/lists/fpc-devel/2011-October/026667.html it is not free of problems to just change the default meaning of "string" using a define, switch, compiler option or whatever.

So, please give the FPC team time to solve the remaining issues in current trunk and to think about how FPC's legacy and the compatibility to Delphi 2009+ can be put together with as less problems as possible for all users.

Regards,
Sven

13.10.2011 15:29, Felipe Monteiro de Carvalho пишет:

Wrote to that issue and writing here.

When String = AnsiString you only need the way to set codepage of ansistring constants to UTF8 and default codepage for conversions.

The first can be made either by {$codepage UTF8} directive or by -FcUTF8 directive.

The later can be made by call SetMultiByteConversionCodePage(CP_UTF8) at the program start.

Best regards,
Paul Ishenin.

Sven Barth wrote

1. Currently there is this huge work going on to get trunk to a reasonable and usable shape again and until that is done (and maybe RTL changes as well) it is (in my opinion (!) ) not reasonable to think about changing the meaning of "string" at all.

It does not change the meaning of string everywhere, just in 1 source file at a time.

2. As Marco wrote in this mail http://lists.freepascal.org/lists/fpc-devel/2011-October/026667.html it is not free of problems to just change the default meaning of "string" using a define, switch, compiler option or whatever.

I would say that people that use the define then can deal with the problems which it causes. For me it causes much less work then to rename everywhere with "string"

There is no easy way out, but I think that the directive solution would work well for Lazarus apps.

For inheritance we could define in system something like:

RTLString = string;

It would not need to be used anywhere in the RTL, but files which use the utf8string directive can use this string type to reffer to the RTL string type.

Paul Ishenin wrote

The later can be made by call SetMultiByteConversionCodePage(CP_UTF8) at the program start. From my comment on the bug report:

Won't SetMultyByteConversionCodePage inadvertedly affect 3rd party libraries used by the program which, for example, execute something like:

myansistring := mywidestring;

And then expect myansistring to be in the system encoding?

In our previous episode, Felipe Monteiro de Carvalho said:

How do you override classes that are declared in RTLString which might not be equal to utf8string?

Marco van de Voort wrote

How do you override classes that are declared in RTLString which might not be equal to utf8string?

I dont fully understand your question, but lets suppose this RTL file:

unit system;
{$mode string = banana}
type RTLString = string;
unit classes;
{$mode string = banana}
type
  TWhaeverClass = class
  procedure Do(Param: string); virtual;
  end;
unit mylazarusunit;
{$mode string = utf8string}
type
  TWhaeverClassOverride = class(TWhaeverClass)
  procedure Do(Param: RTLString); override;
  end;

I think that this would work, because the signatures of the procedures match perfectly.

It will affect as well as compiler directive you suggested to add.

Best regards,
Paul Ishenin.

Paul Ishenin wrote

It will affect as well as compiler directive you suggested to add.

No, the directive is per source code file. 3rd party libraries do not need to use it.

+1

BTW.: is there a chance to compile Lazarus and Lazarus based projects using the trunk FPC any time soon ? This would help trying out new Strings provided there is a possibility to define some of the variables to be new strings in user code.

The last FPC version I installed was "Free Pascal Compiler version 2.7.1 [2011/08/26] for i386". "/usr/bin/ppc383" points to "../lib/fpc/2.7.1/ppc386".

So, when installing a Unicode enabled it might be a good idea to do this to a separate subdirectory in usr/lib/fpc to be able to switch to and fro. Maybe it would be nice to provide this by changing the version ID.

-Michael

When installing you can always select a custom directory though you can not change the version-id. E.g. you could install to "/lib/fpc-cpstring/2.7.1".
Also as far as I see the version of the PPU files is different (though not because of the string changes) between your version and current trunk, so Lazarus should trigger recompilations if you switch between your older 2.7.1 and a current one.

Regards,
Sven

In our previous episode, Felipe Monteiro de Carvalho said:

I think so, but I haven't tried.

But I don't consider this as solution. You have to put each string type in the entire libraries on trial, and keep it up.

Moreover, it gets funky when string is banana on Windows and banana2 on unix. It keeps working, but if people make mistakes that "happen" to work on one platform, the other people might have a problem etc.

Also, dual maintenance of components with Delphi is next to impossible, since you would force that scheme also onto them, and all new code first needs to be lazarused.

Sven Barth wrote

When installing you can always select a custom directory though you can not change the version-id. E.g. you could install to "/lib/fpc-cpstring/2.7.1".

Of course I do know how to install FPC in another directory or how to preserve the current installation by other means. I just wanted to suggest that changing the version ID could help to avoid nasty hazards, as you need to manually change the link in /usr/bin to activate a modified version, plus the old version is automatically preserved while installing a version with a new ID (on Linux).

Also as far as I see the version of the PPU files is different (though not because of the string changes) between your version and current trunk, so Lazarus should trigger recompilations if you switch between your older 2.7.1 and a current one.

Yep, but AFAIK, currently Lazarus can't be compiled with the trunk version and recompiling a user project with same (which forces recompiling the LCL as you state) does not work either.

-Michael

Marco van de Voort wrote

But I don't consider this as solution. You have to put each string type in the entire libraries on trial, and keep it up.

Could you explain more? Which libraries are you talking about? As I showed, nothing would need to change in the RTL or FCL classes, they can use string as usual, or whatever else.

If the problem is in utf8string based libraries, then it only affects people that use such libraries and authors of such libraries would take care of solving the issues.

Moreover, it gets funky when string is banana on Windows and banana2 on unix. It keeps working, but if people make mistakes that "happen" to work on one platform, the other people might have a problem etc.

That's a general problem of having a string type which varies across platforms, which is something I was always against.

Also, dual maintenance of components with Delphi is next to impossible, since you would force that scheme also onto them, and all new code first needs to be lazarused.

It will not be worse then maintaining Delphi 7 and Unicode Delphi at the same time, and yet most components support both at the same time.

Plus, isn't Embarcadero pushing FireMonkey anyway?

I think he ment that if such a feature is introduced it would be a natural conclusion to define "string = unicodestring" on Windows and "string = utf8string" for Unix in the RTL and the FCL (and maybe "string = ansistring" for DOS and OS/2). Thus those two libraries need to be tested intensively that they can cope with that.

Regards,
Sven

Sven Barth wrote

I think he ment that if such a feature is introduced it would be a natural conclusion to define "string = unicodestring" on Windows and "string = utf8string" for Unix in the RTL and the FCL

? I am totally lost as to what this has to do with my proposal. My proposal is to add a minimum facility to port UTF-8 based code which makes heavy use of the "string" keyword. It says nothing about which string the RTL should use.

In short, you want a directive to say

"In this unit, 'string' means MyNiceStringType"

Which is in fact an extension of the current {$H} to a more general case ?

Michael.

In our previous episode, Felipe Monteiro de Carvalho said: [ Charset ISO-8859-1 unsupported, converting... ]

Pretty much everything that inherits from a class in classes, or passes to VAR parameters in the RTL. (classes or not)

showed, nothing would need to change in the RTL or FCL classes, they can use string as usual, or whatever else.

I'm talking from the usage viewpoint. That includes Lazarus. I don't think this is a sane solution for Lazarus either.

If the problem is in utf8string based libraries, then it only affects people that use such libraries and authors of such libraries would take care of solving the issues.

Assuming that Lazarus stays utf8 only long term. That is still the party policy, but I have some doubts there.

True. But you are crafting a system out of a patchwork where these things differ from block of code to another.

A very informed person can make this work, since he has a lot of the overview already memorized.

But for an uninformed (read user) this kind of stuff is nearly undoable, since they don't know the exact string properties of each module that contributes to the greater Lazarus system. (as in Lazarus + most used components)

And of course I don't like having to modify Delphi code on a massive scale. We didn't have to till now, and I see no reason to start.

If it comes to such solutions, I give up all my resistance, and go with 1:1 D2009 compatibility on all platforms all the way. So string=unicodestring, and screw the rest.

Having to modify Delphi sources before use, more than just uses clauses is an absolute nono for me, which goes over everything.

Since all other sources (FPC backwards compat, Lazarus being UTF8), might change, but Delphi isn't going to.

I disagree. In D7 everything is the same "string", and so is in Delphi unicode. They are just not the same. This is worse, much worse.

Plus, isn't Embarcadero pushing FireMonkey anyway?

I don't see it that way. I see FM as a Delphi PHP/3rd Rail/Prism like product, not as serious VCL contender. If I understood it correctly, it is mostly a sideproduct of internal efforts to port Embarcadero's non-Borland originating lines.

In short, you want a directive to say  "In this unit, 'string' means MyNiceStringType" Which is in fact an extension of the current {$H} to a more general case ?

yes

Actually I don't necessarely need a general case, just {$useutf8string} or whatever else only for utf-8 would be just as fine.

Well, if we add such a beast, it won't be just for UTF-8.

Lazarus happens to be UTF-8 centric, but that doesn't mean we should limit the use of such a feature to UTF-8...

Michael.

Well, if we add such a beast, it won't be just for UTF-8. Lazarus happens to be UTF-8 centric, but that doesn't mean we should limit the use of such a feature to UTF-8...

An addendum to that: We also need the normal PChar. Changing PChar was *very* nasty.

So, the define would activate:

string = utf8string
PChar = PAnsiChar or PUTF8Byte or whatever which gives a byte array.

In one of your mails you had the example of the units "system" and "classes" which define "string" as aliased to the string type "banana". So if such a feature is added (and as Michael wrote, if it's added it's added the powerful way) then it seems likely that it will be used in the RTL and the FCL as well to (hopefully) simplyfy the problem of having UTF-16 on Windows and UTF-8 on Windows and OEM/Ansi on OS/2 and DOS. Thus those two libraries (RTL and FCL) need to be able to cope with a different string type.

So it has nothing to do directly with the proposal of that feature or Lazarus, but with the consequences or further usecases of this feature.

Regards,
Sven

Sven Barth wrote

In one of your mails you had the example of the units "system" and "classes" which define "string" as aliased to the string type "banana"

Ah, that was not ment to be interpreted like that ... I just wanted to say that in this RTL mode or whatever we have string=banana as the standard case which is used by the RTL.

13.10.2011 13:34, пишет:

another alternative. in {$H-} no changes string=shortstring, in {$h+} string = ansistring or unocodestring in delphiunicode mode) by default but can be redefined directly in code (or this definition is set in ObjPas unit) "type string = MyNiceStringType". and this redefinition is inherited from used unit if all of them compiled in {$h+} mode. In other words if unit has directive {$h-} string=shortstring in any case, but if unit has {$h+} it use "string" type from used unit as it is regular type.

Out of curiosity: how did you come to that conclusion? According to what I read in rtl/inc/systemh.inc "PChar" is still defined as an "^Char" and further down is also an alias "AnsiChar = Char", so "PChar" should still be and behave the same?!

Regards,
Sven

In our previous episode, Sven Barth said:

No, worse even, both on both platforms. So win32-utf8 and win32-utf16, where the utf* designates the type of the default string type. And the same for unix. Maybe win32-ansi too.

Some of these are for the transition period and legacy only. I don't think "ansi" versions for non windows platforms are necessary in the long run, and maybe not even initially. I haven't really made up my mind yet if the 1-byte target for unix should be utf8 specifically, or native encoding.

This means 2 RTLs for all major platforms (say
linux/freebsd/windows/darwin), and at least for a time an extra one (ansi, compat with old FPC and Delphi) on windows. The others are up to the maintainer.

The default stringtype is set on compiler startup, but can be modified inside procedures with $H like behaviour.

So first you pick the general encoding first, depending on your plans and where your code originates from. The whole FPC distro will be compiled using that type as much as possible.

If you need escape from it, you can workaround it in two ways:

1. (best choice) change the routines that need escape to proper typing. preferably update the code to narrow the spot that is encoding dependent, and make the rest "string"
2. Try a blanket {$H} to make the default stringtype what you want, and fix problems (e.g. overrides of methods, passing to var params).

Some of the reasons for this:
- ability to use Delphi unicode code without changes. Forever, on all platforms.
- ability to use current lazarus code (cleaned up a bit to use 2.7.1 string types), by using utf8 rtls on all platforms)
- ability for lazarus to change opinions later (e.g. go to utf16 on windows)

Advantages:
- nobody gets left out. Saves a lot of discussion that will never fully die out.
- possibility to run a system in mostly one encoding, without micromanaging sctring types in the packages (RTL, classes, packages, lazarus + parts, external code) and finding some why to reconcile them on all systems.
- A scheme that is easy to understand and explain. "take the utf16 version, it is like D2009, period". No rewrite rules, minimal changes to the modes etc.
- Fully Delphi 2009 compatible options on a source level.
- Fully Delphi-old compatible on windows only. (and unix too if we change 1-byte to be default encoding) option
- the multiple choices give people options during the migration period. First pick the nearest (utf8 in Lazarus' case), then start testing and migrating to ideal situation.
- If problems turn up (and they will), there is a policy to deal with them, and deal with them properly. No workarounds, rewrites. IOW by separating this, problems that I can't overview now get a place. There are less chances on roadblocks and discussion points.

Problems:
1 obviously the release engineering effort.
2 Not all details fleshed out. Specially wrt modes. But this goes for all suggestions till now.
3 needs a lot of commitment from everybody. It is revolutionary, and not evolutionary, and cannot be presented that way. We cannot pretend nothing much changes now, and then assume everything works out, or we will find workarounds (with a bit rewrite .....) lateron.

I think problem 3 is bigger than the other two.

Defining the "string" type in a unit (e.g. the ObjPas unit) is a bad idea if you want to keep the capability to change the "string" type on a per unit base as you can't change the compiled ObjPas unit. Also you'd need a unit for every string type that should be supported. The proposal where "string" is changed based on some compiler option/define is better (though the question about its use or consequences is still not solved).

Regards,
Sven

I think from an implementation point of view, the directive approach is easier.

From discussions on the core mailing list, I understood that the

implementation is rather hairy.

In each case, this feature is not there yet, and there is also no guarantee that it will be there. I assume that Paul Ishenin can shed more light on the feasibility of this feature.

Michael.

Ok... that you want multiple distributions per platform is clear now. But you would not oppose (in the case of having multiple distributions) a feature like Felipe suggested, do I understand that correctl?

The default stringtype is set on compiler startup, but can be modified inside procedures with $H like behaviour.

You would go as far as allowing a change of "string" on a per-procedure base?

Regards,
Sven

The blanket {$H} seems like the way to go for most packages.

If we insert {$H ANSISTRING}, then most packages should remain functional, allowing us to tackle the packages one by one.

Codepage conversions should happen automatically, except for the case of
Var params, but these need to be sorted out anyway.

But I would like to hear from Florian, Jonas or Paul whether they think this is possible. If I remember correctly, Florian proposed something similar some years ago.

Michael.

I think such a modifier should be global.

Michael.

In our previous episode, Sven Barth said:

Oh sure. If you read the text, my proposal already assumes it. But base it on $H. {$H ansi/utf8/utf16} is good enough.

The difference is more that I don't think it will solve as much as people think, and using this to stitch together code from different origins will fail miserably or be unworkable.

That is in the case of multiple distributions. If the direction goes to something unworkable, I rather forget about it, and make everything D2009+ compat.

Sorry, units. That was a typo.

Lazarus happens to be UTF-8 centric

I understand, this is because it had been impossible for them to support as well UTF-8 for Linux as UTF-16 for Windows,

With the new code page aware Strings it should be possible to compile the same source for both flavors.

-Michael

I thought so, but wanted to be sure.

The difference is more that I don't think it will solve as much as people think, and using this to stitch together code from different origins will fail miserably or be unworkable.

For non-inheritance-related code that is encoding-agnostic this should be not that much of a big problem, because the compiler will add conversions then. The problametic parts - as you yourself metioned - will be inheritance (when you need to "guess" the correct string for your override to work) and - I assume - loading and saving of resources (DFM/LFM).

Regards,
Sven

In our previous episode, said:

The more procedural packages yes. The OOP ones not, since their stringtype must match classes for overrides.

And the OOP fcl- packages are the ones that are used the most I think.

Codepage conversions should happen automatically, except for the case of Var params, but these need to be sorted out anyway.

Var (/const/out?) parameters, inheritance, events (procedure variables) with strings as parameters. There will probably be more found when conversion really starts.

But I would like to hear from Florian, Jonas or Paul whether they think this is possible. If I remember correctly, Florian proposed something similar some years ago.

Afaik the only thing I heard from them is that they wanted to have an D2009 mode that would also change stringtype. (string=unicodestring)

In our previous episode, Sven Barth said:

That's fine as a temporary solution.

But keep in mind that such conversions are different from calling e.g. a RTL function (except string functions) that also forces a conversion. This because whatever the RTL does will most likely be relatively slow, while redundant conversions between businesscode->(some fcl class) -> (base classes) can hurt horribly.

They are fine for the transition period, but long term you want to get rid of unnecessary conversions.

The problametic parts - as you yourself metioned - will be inheritance (when you need to "guess" the correct string for your override to work) and - I assume - loading and saving of resources (DFM/LFM).

And var parameters, procedure/function types with string parameters used across units etc. And that is just from memory, when testing probably more will be found.

Sven Barth wrote

Out of curiosity: how did you come to that conclusion? According to what I read in rtl/inc/systemh.inc "PChar" is still defined as an "^Char" and further down is also an alias "AnsiChar = Char", so "PChar" should still be and behave the same?!

Delphi changed PChar to mean PWideChar, so I simply supposed that FPC will do the same. Or won't it?

In our previous episode, Felipe Monteiro de Carvalho said:

I would say that pchar and char match whatever string is. So IMHO they should change with $H too.

Yes, but
1. this is the smallest group, so I don't consider that a problem.
2. I doubt there is a lot of code in these classes that relies on sizeof(char)=1.

It's time to cut the gordian knot and take a decision. All pros and contras have been discussed at great length meanwhile.

Michael.

In our previous episode, said:

Only in number of packages.

2. I doubt there is a lot of code in these classes that relies on sizeof(char)=1.

They override methods.

It's time to cut the gordian knot and take a decision. All pros and contras have been discussed at great length meanwhile.

Exactly. That's why a decision that is futureproof should be made, and not just start hacking, and see where we end up.

Suggestion:
Let it be as it is in fixes_2_6, add support for Unicode resource strings and invest power into development of Delphi like packages for example. Reading the mentioned thread in Embarcadero forums it seems that there are plans to simplify strings in Delphi so in some time FPC needs to catch up again.

Martin

In our previous episode, Martin Schreiber said:

Suggestion: Let it be as it is in fixes_2_6, add support for Unicode resource strings and invest power into development of Delphi like packages for example.

That is already not an option anymore, with the newstr branch merged in.

And while that is an option for you, it doesn't satisfy anybody else. Not even the ones that want as close to D2009 compatibility as possible, since they also need the new ansistring type here and there.

Reading the mentioned thread in Embarcadero forums it seems that there are plans to simplify strings in Delphi so in some time FPC needs to catch up again.

Without details, such info is useless, since it can't be decided if we support a superset or not. We can't put off 2 years development on the basis of a rumour.

It might be so abrupt that we don't decide to follow.

Ok, so after summing in all of this problems which we are facing, I will withdraw all of my previous ideas and agree with you Marco. If FPC can produce a RTL mode where string=utf8string then I agree Lazarus should try to use this UTF-8 RTL.

Or improved optimization or improved debugging or...

That is already not an option anymore, with the newstr branch merged in.

Ehm, SVN probably can revert? Otherwise I am sure Graeme will help to convert to git, do the reverting and convert back to SVN. ;-)

And while that is an option for you, it doesn't satisfy anybody else. Not even the ones that want as close to D2009 compatibility as possible, since they also need the new ansistring type here and there.

Hm, even Embarcadero can live without in Delphi XE2 for iOS development.

Martin

In our previous episode, Martin Schreiber said:

It was presented on the XE2 introductionary presentation (of my reseller) as an unfinished preview that was pretty much unworkable.

So that is not something that I take as metric for FPC development direction.

Felipe Monteiro de Carvalho wrote

On Thu, Oct 13, 2011 at 10:26 AM, Paul Ishenin wrote: > It will affect as well as compiler directive you suggested to add. No, the directive is per source code file. 3rd party libraries do not need to use it.

Then use {$codepage UTF8} only. This directive treats all ansistring
constants in UTF8 codepage. So the next code:
{$codepage UTF8}
var
   s: ansistring;
begin
   s := 'utf8 string';
   WriteLn(StringCodePage(s));
end.

will output CP_UTF8 constant.

Best regards,
Paul Ishenin.

This I don't really understand. Shouldn't the constant be converted at run time from UTF-8 to the DefaultSystemCodePage to make sure that an ansistring(0) variable always contains strings encoded in the DefaultSystemCodePage? If you assign e.g. a string(866) variable to a plain ansistring variable, then such a conversion is also done, no?

I think the only case in which the source code encoding should be kept is in case you assign the string constant to a rawbytestring.

Jonas

Yes, this is my understanding.

It should read
   s: string;

DoDi

Why would you think that? FireMonkey isn't mature enough to be usable yet, but it's definitely where future Delphi development will occur. After XE2 shipped Embarcadero laid off a bunch of VCL developers and announced they were growing their overseas teams (likely FireMonkey):

http://community.devexpress.com/blogs/ctodx/archive/2011/09/19/delphi-vcl-is-dead-long-live-firemonkey.aspx

ARM/iOS/Android support will require FireMonkey, and is their biggest growth opportunity. Allen Bauer has also said FireMonkey apps should be able to work with Windows 8's Metro interface, which won't be true for VCL apps.

BTW, I wanted to point out, to anyone arguing for full Delphi compatibility, that on OS X Delphi's DefaultSystemCodePage and AnsiString(0) are *not* the system encoding _or_ UTF-8. They're the Windows ANSI codepages that correspond to the system's locale (so, 1252 for US/UK versions). It's a weird gotcha, and means it's not safe to rely on AnsiString to communicate with system functions anymore.

In our previous episode, Craig Peterson said:

No backwards compatibility, no native ties to tie it down. Clearly it is meant to be fast moving by skipping details. And that doesn't appeal to the bulk of the Delphi crowd

FireMonkey isn't mature enough to be usable yet, but it's definitely where future Delphi development will occur. After XE2 shipped Embarcadero laid off a bunch of VCL developers and announced they were growing their overseas teams (likely FireMonkey):

I doubt it, but if that is true, it is the end of Delphi.

http://community.devexpress.com/blogs/ctodx/archive/2011/09/19/delphi-vcl-is-dead-long-live-firemonkey.aspx ARM/iOS/Android support will require FireMonkey, and is their biggest growth opportunity.

That is however fully true, revenue growth won't come from the traditional Delphi/VCL users, it will only wane. But that is something totally, totally else than killing off VCL and trying to convince them to go to FM.

That is the quickest way to loose that nice big comfy auto-upgrade money that is coming in every year and pays the fixed bills.

And the same story (new revenue) is true for the other products (Delphi PHP/3rd Rail and Prism) too, so I'm sticking to my view :_)

Allen Bauer has also said FireMonkey apps should be able to work with Windows 8's Metro interface, which won't be true for VCL apps.

That would be logical yes, since it is not win32 api based. I hope for E. that Metro will remain a sideshow, since I think Embarcadero would lose more Delphi/VCL guys to C# than that would go to FM, if push came to shove.

I don't understand this. Yes the ansistring system uses 1252, but the compiler/RTL can make the mappings, so I don't really understand your problem.

A numeric enumeration of codepages is more efficient (and less free form) than a stringbased one, so the Windows system isn't that bad.

Marco van de Voort wrote

No backwards compatibility, no native ties to tie it down. Clearly it is meant to be fast moving by skipping details. And that doesn't appeal to the bulk of the Delphi crowd

They're so far behind they pretty much have to run fast and loose.

I'm certainly not advocating it myself. I think it's a buggy mess, and wouldn't use it for a desktop app until it does a better job of acting native, but I do expect a lot of developers to transition to it eventually. If they don't, Delphi doesn't have a strong future anyway.

That is however fully true, revenue growth won't come from the traditional Delphi/VCL users, it will only wane. But that is something totally, totally else than killing off VCL and trying to convince them to go to FM.

They aren't killing the VCL off, but it is mature, so it doesn't need as many resources. It now supports Unicode and Win64, and can't support OS X/iOS/Android/Metro. I'm sure the new styling engine needs work, and there's plenty of other fixes, but I have a hard time seeing what other extensive development they could do with it. About the only other missing feature is fast, scalable vector graphics, and that's FireMonkey.

I wasn't commenting on using integers vs. strings to represent the codepage.

Whenever you pass a PAnsiChar string to a system function (eg, lstat on Unix, CreateFileA on Windows), the system expects a specific encoding. On Windows that encoding is one of the "ANSI" encodings like "Windows-1252". On OS X it's UTF-8. On Linux it's the "LC_CTYPE" encoding, usually either UTF-8 or one of the ISO encodings like "ISO 8859-1".

In FreePascal (currently), Kylix, and Delphi 2007 and earlier, that encoding was represented by the CP_ACP encoding, and the AnsiString type, so any AnsiString<->WideString conversions use it.

In Delphi XE2 on OS X AnsiString does *not* map to UTF-8. On an English system it maps to "Windows-1252". If you want to convert an "array of AnsiChar" to a WideString you need to cast it as a UTF8String first. By default DefaultSystemCodePage is 1252, not 65001.

Jonas Maebe wrote

This I don't really understand.

This is how delphi works according to my tests. I will retest to be 100% sure.

Best regards,
Paul Ishenin

Does this also happen if the string contains non-ASCII chars? If it doesn't, the first case seems like a fine optimization to me ;-)

-Flávio

Craig Peterson wrote

http://community.devexpress.com/blogs/ctodx/archive/2011/09/19/delphi-vcl-is-dead-long-live-firemonkey.aspx

One thing that needs to be considered about FireMonkey and similar "application internal Widget Sets" is that they don't integrate well with the (potentially customized) current Desktop layout - other than VCL or (most) LCL Widget types, that rely on the target system's widget set.

So the (cross-platform, ) application looks identical everywhere. Is this what everybody wants ?

-Michael

In Delphi XE2 on OS X AnsiString does *not* map to UTF-8. On an English system it maps to "Windows-1252". If you want to convert an "array of AnsiChar" to a WideString you need to cast it as a UTF8String first. By default DefaultSystemCodePage is 1252, not 65001.

Well, XE2 is quite a different Delphi version, which probably is not mature yet. From the FPC view it could become just another mode, that works on OS X only with that specific RTL version.

But is it worth to configure or split FPC into any number of versions, which are fully compatible with specific Delphi versions (including bugs), in addition to the objfpc etc. modes?

When we include Lazarus in our considerations, then we have no fully VCL compatbile LCL yet, what's impossible for certain non-Windows widgetsets. Going ahead and changing the behaviour of FPC on those platforms, which had been supported before in the "FPC way", would break the LCL badly.

Instead I'd expect that the current LCL should be completed, or declared as completed, before taking the next step with a D2009+ compatible RTL and LCL. And in this move we should fix the Delphi flaws, residing in the Windows-only versions up to XE.

When somebody wants something compatible with Delphi XE2, he can start immediately to modify FPC accordingly - but please without breaking other code - and start implementing a new FireMonkey component library, independent from the LCL.

But are we dreaming right now, or are we about to take the *next* step in the FPC evolution?

BTW, I just found -Mdelphi documented as "Delphi 7 compatibility mode", how many further options had to be added for newer Delphi versions???

At least one new option/mode is required for Unicode Delphi 2009+, and this mode should not break any existing code (Lazarus, mseGui, fpGUI...).

And this new mode should be designed either in collaboration with the LCL maintainers, which may better know what changes deserve what efforts and breaks/forks in the LCL and Lazarus, or it should be designed as a mere option, for the implementation of other packages, component libraries or IDEs.

It should be worth to note that the Delphi IDE is no more written in Delphi.Win32, since the Delphi.NET experiment, but instead relies on .NET heavily. Perhaps a similar separation may be required with the Lazarus IDE, which must not necessarily use the new string types internally.

BTW2, the Lazarus IDE has big problems with the lack of dynamically loadable packages, which still are *not* supplied by FPC. This will possibly make impossible above separation between the IDE implementation and FPC progress. Wouldn't it be much more important to improve the package support, before experimenting with new Delphi features?

DoDi

Hans-Peter Diettrich wrote

- and start implementing a new FireMonkey component library, independent from the LCL.

Is this really necessary ? FireMonkey is a rebranded open source library, so it should be possible just to _use_ it.

OTOH, (I don't have XE2): does XE2 come with (or did E. announce) a visual Designer for FireMonkey (if such thing is even possible) ? In this case Lazarus might be willing to invest work on something like that.

-Michael

Of course, Dephi-type runtime packages (i.e. DLLs / so's specially crafted to be only used by programs done in the same language and using the same libraries) are a desirable feature (e.g. to do "plugins" in closed source projects), I feel that it's not a big lack regarding the IDE. (At least in Linux) re-compiling the IDE is done in a few seconds, so adding/deleting/replacing static packages is no hurdle.

-Michael

Ever tried to implement a form or any function which uses dynamic types or common application elements in a dll/so?

Martin

AFAIK XE2 contains a visual designer. And in theory it should be possible to extend Lazarus with a designer by implementing a non LCL-designer (Lazarus contains code for something like that, though I've never tested it yet...).

Also one of the Embarcadero/Delphi developers said that they intend to keep FireMonkey compatible with FPC and in some time provide it again (no specifics for timeframe and possible costs given though).

Regards,
Sven

Martin Schreiber wrote

Ever tried to implement a form or any function which uses dynamic types or common application elements in a dll/so?

I understand that this is exactly what Delphi-type runtime packages are invented for.

-Michael

and Lazarus-type packages can't do.

Martin

Craig Peterson wrote

BTW, I wanted to point out, to anyone arguing for full Delphi compatibility, that on OS X Delphi's DefaultSystemCodePage and AnsiString(0) are *not* the system encoding _or_ UTF-8. They're the Windows ANSI codepages that correspond to the system's locale (so, 1252 for US/UK versions). It's a weird gotcha, and means it's not safe to rely on AnsiString to communicate with system functions anymore.

My guess is that this is because of Delphi's behaviour to assume that the source code has been saved in the locale's default code page unless specified otherwise. If they'd changed the default code page to UTF-8 for all locales on Mac OS X, lots of Delphi source code would probably not compile correctly on Mac OS X without explicitly specifying the code page.

They could of course have changed this only in their compiler rather than also for end-user programs, but a) that would be bad from a maintenance/dog food point of view b) it would have resulted in run time code page conversions for all string constants on the developer's own system, since the compiler would have stored the constants in cp1252 or the like (the specific "compiler"-determined code page) and at run time they'd all be converted into UTF-8 (the actual "ansi" code page of the system)

Jonas

Martin Schreiber wrote

> I understand that this is exactly what Delphi-type runtime packages are > invented for. and Lazarus-type packages can't do.

Obviously, AFAIK, this never has been claimed, but that there have been requests for "loadable binary plug-in" type runtime packages.

-Michael