Hi,
Are there any plans to extend FPC to support Unicode file paths under Windows?
Of course this also means the other tools, like fpcres, fpmake and fppkg.
Would this require a new flag?
Mattias
Mailing lists / fpc-devel / 2016-04
Unicode paths
20 messages · 6 people · –
- Marco van de Voort 7
- Mattias Gaertner 5
- Michael Van Canneyt 3
- Sven Barth 3
- Jonas Maebe 1
- Martin Schreiber 1
Reply by email Subscribe to this list Your subscription
Read-only archive. Use j and k to move between messages.
"Mattias Gaertner" wrote 9 lines
Hi,
Are there any plans to extend FPC to support Unicode file paths under
Windows?
Of course this also means the other tools, like fpcres, fpmake and
fppkg.
Would this require a new flag?
In the end the Windows RTL should only use the *W functions and thus fully allow for Unicode paths. Or what do you mean? (especially with your last question?)
Regards,
Sven
Show quoted text 5 lines
Am 11.04.2016 19:31 schrieb "Mattias Gaertner" :
[..]
In the end the Windows RTL should only use the *W functions and thus fully
allow for Unicode paths. Or what do you mean? (especially with your last
question?)
What about printing the paths in notes, hints, etc? What about ppu files?
Mattias
Are there any plans to extend FPC to support Unicode file paths under Windows? Of course…
Mattias Gaertner wrote 7 lines
Hi,
Are there any plans to extend FPC to support Unicode file paths under
Windows?
Of course this also means the other tools, like fpcres, fpmake and
fppkg.
Would this require a new flag?
The rtl already support this. But none of the tools currently use unicode names.
It would also require examining command-line parameters, I think, because I am not sure these are currently unicodestring (or UTF8).
Michael.
Are there any plans to extend FPC to support Unicode file paths under Windows? Of course…
In our previous episode, Mattias Gaertner said:
Show quoted text 5 lines
Are there any plans to extend FPC to support Unicode file paths under
Windows?
Of course this also means the other tools, like fpcres, fpmake and
fppkg.
Would this require a new flag?
Yes. cmd.exe with /u ? :-)
Show quoted text 8 lines
In our previous episode, Mattias Gaertner said:
> Are there any plans to extend FPC to support Unicode file paths under
> Windows?
> Of course this also means the other tools, like fpcres, fpmake and
> fppkg.
> Would this require a new flag?
Yes. cmd.exe with /u ? :-)
Is this already supported?
And when executing the compiler directly e.g. from TProcess?
Mattias
In our previous episode, Mattias Gaertner said:
Show quoted text 7 lines
> > Of course this also means the other tools, like fpcres, fpmake and
> > fppkg.
> > Would this require a new flag?
>
> Yes. cmd.exe with /u ? :-)
Is this already supported?
Well, afaik on Windows unicode commandlines are supported in $mode delphiuncode. Overloaded unicode(string) versions of paramstr handle it (unit uuchar)
And when executing the compiler directly e.g. from TProcess?
Specific binaries would be case by case basis, but I don't expect any to be upgraded all ready. In trunk I updated executeprocess to unicodestring, but for TProcess I'm waiting on decisions on TSting(s)(list). (system.classes)
Marco van de Voort wroteWell, afaik on Windows unicode commandlines are supported in $mode delphiuncode.
They are not supported, because we get the original command line data using the ansi version of the API call (see setup_arguments() in rtl/win/syswin.inc). If this is "fixed", then we also have to decide what to do with the argv p(ansi)char (a good place would be to check what Windows itself returns from the ansi API call when passing command line arguments that contain characters that cannot be represented in the ansi code page.
Another way to know it is not supported, is that it is not listed at http://wiki.freepascal.org/FPC_Unicode_support#RTL_changes (*)
Jonas
(*) I know you recently committed unicode support for executing external programs and it isn't listed there yet either, but that's because I think it is broken on *nix: encoding the command line parameters using DefaultFileSystmCodePage is wrong, because the receiving/executed applications will interpret them according to what they deduce as being DefaultSystemCodePage from the environment instead. We have a routine that can deduce this code page from the environment (GetSystemCodepage), so we should probably use that (although it may need some refactoring to be able to use it in case the environment for the executed process is specified separately from that of the invoking process)
In our previous episode, Mattias Gaertner said: Well, afaik on Windows unicode…
Show quoted text 11 lines
[...]
Well, afaik on Windows unicode commandlines are supported in $mode
delphiuncode. Overloaded unicode(string) versions of paramstr handle it
(unit uuchar)
> And when executing the compiler directly e.g. from TProcess?
Specific binaries would be case by case basis, but I don't expect any to be
upgraded all ready. In trunk I updated executeprocess to unicodestring, but
for TProcess I'm waiting on decisions on TSting(s)(list). (system.classes)
Are we still talking about the compiler?
Mattias
Are there any plans to extend FPC to support Unicode file paths under Windows? Of course…
Hi, Are there any plans to extend FPC to support Unicode file paths under Windows?
To clarify: I mean the compiler fpc.exe.
Of course this also means the other tools, like fpcres, fpmake and fppkg. Would this require a new flag?
Mattias
They are not supported, because we get the original command line data using the ansi…
In our previous episode, Jonas Maebe said:
Show quoted text 7 lines
They are not supported, because we get the original command line data
using the ansi version of the API call (see setup_arguments() in
rtl/win/syswin.inc). If this is "fixed", then we also have to decide
what to do with the argv p(ansi)char (a good place would be to check
what Windows itself returns from the ansi API call when passing command
line arguments that contain characters that cannot be represented in the
ansi code page.
The central place of posix argv/argc in the command processing (including getopts), also on nonposix systems is indeed a bit of a problem.
I guess the only way, as always, is to have two duplicate systems on windows. One wide that is for unicode, (unicodestring paramstr and D2009 compatible tapplication), one for old legacy ansi apps. (getopts etc).
That pains me however, I always liked getopts for a quick program over TCustomApplication.
Another way to know it is not supported, is that it is not listed at http://wiki.freepascal.org/FPC_Unicode_support#RTL_changes (*)
I must have taken talk about it and considered it implemented. I did look, but only the existence of the call, not its implementation :-)
Marco van de Voort wrote 16 lines
In our previous episode, Jonas Maebe said:
> They are not supported, because we get the original command line data
> using the ansi version of the API call (see setup_arguments() in
> rtl/win/syswin.inc). If this is "fixed", then we also have to decide
> what to do with the argv p(ansi)char (a good place would be to check
> what Windows itself returns from the ansi API call when passing command
> line arguments that contain characters that cannot be represented in the
> ansi code page.
The central place of posix argv/argc in the command processing (including
getopts), also on nonposix systems is indeed a bit of a problem.
I guess the only way, as always, is to have two duplicate systems on
windows. One wide that is for unicode, (unicodestring paramstr and
D2009 compatible tapplication), one for old legacy ansi apps. (getopts etc).
If argv/argc is known to be UTF8 encoded, then I see no problem with keeping getopts ?
Michael.
In our previous episode, Michael Van Canneyt said:
Show quoted text 6 lines
> I guess the only way, as always, is to have two duplicate systems on
> windows. One wide that is for unicode, (unicodestring paramstr and
> D2009 compatible tapplication), one for old legacy ansi apps. (getopts etc).
If argv/argc is known to be UTF8 encoded, then I see no problem with
keeping getopts ?
Getopts is not utf8 but shortstring, so ACS on Windows. Or do you plan to redo it using utf8string or unicodestring?
Marco van de Voort wrote 10 lines
In our previous episode, Michael Van Canneyt said:
>> I guess the only way, as always, is to have two duplicate systems on
>> windows. One wide that is for unicode, (unicodestring paramstr and
>> D2009 compatible tapplication), one for old legacy ansi apps. (getopts etc).
> If argv/argc is known to be UTF8 encoded, then I see no problem with
> keeping getopts ?
Getopts is not utf8 but shortstring, so ACS on Windows. Or do you plan to
redo it using utf8string or unicodestring?
I don't necessarily plan to do so, I was proposing it as a way out.
Michael.
In our previous episode, Michael Van Canneyt said:
Show quoted text 7 lines
>> If argv/argc is known to be UTF8 encoded, then I see no problem with
>> keeping getopts ?
>
> Getopts is not utf8 but shortstring, so ACS on Windows. Or do you plan to
> redo it using utf8string or unicodestring?
I don't necessarily plan to do so, I was proposing it as a way out.
If getopts is not leading, then the needs of the unicode T(Custom)Application classes is probably the best indicator for which way to go.
If they don't need argv and argc, just leave them as they are.
Anyway, I never liked the fact that argv and argc are exported symbols. If they weren't public, we could change them at will and/or have differing implementations and encodings depending on target. I would deprecate them (being public)
"Marco van de Voort" wrote 6 lines
In our previous episode, Michael Van Canneyt said:
> >> If argv/argc is known to be UTF8 encoded, then I see no problem with
> >> keeping getopts ?
> >
> > Getopts is not utf8 but shortstring, so ACS on Windows. Or do you
plan to
Show quoted text 5 lines
> > redo it using utf8string or unicodestring?
>
> I don't necessarily plan to do so, I was proposing it as a way out.
If getopts is not leading, then the needs of the unicode
T(Custom)Application
Show quoted text 8 lines
classes is probably the best indicator for which way to go.
If they don't need argv and argc, just leave them as they are.
Anyway, I never liked the fact that argv and argc are exported symbols. If
they weren't public, we could change them at will and/or have differing
implementations and encodings depending on target. I would deprecate them
(being public)
Agreed. They are only exported for ObjPas and that can be solved in the same way as other symbols uses there are solved.
Regards,
Sven
In our previous episode, Michael Van Canneyt said: If getopts is not leading, then the…
Marco van de Voort wrote 6 lines
Anyway, I never liked the fact that argv and argc are exported symbols. If
they weren't public, we could change them at will and/or have differing
implementations and encodings depending on target. I would deprecate them
(being public)
Is there a possibility to still get the raw data in case one wants to handle encoding himself?
Martin
In our previous episode, Martin Schreiber said:
Show quoted text 7 lines
> Anyway, I never liked the fact that argv and argc are exported symbols. If
> they weren't public, we could change them at will and/or have differing
> implementations and encodings depending on target. I would deprecate them
> (being public)
>
Is there a possibility to still get the raw data in case one wants to handle
encoding himself?
argv and argc is not really raw data, and on many platforms (like windows) processed.
If you want to use the raw data, you'll need OS specific functions (getcommandlinew for e.g. Windows), or, in case of the Unix RTL, the operatingsystem_parameter_* assembler symbols that convey the registers loaded by the loader to the RTL.
"Marco van de Voort" wroteIn our previous episode, Martin Schreiber said: > > Anyway, I never liked the fact that argv and argc are exported
symbols. If
> > they weren't public, we could change them at will and/or have
differing
> > implementations and encodings depending on target. I would deprecate
them
> > (being public) > > > Is there a possibility to still get the raw data in case one wants to
handle
> encoding himself? argv and argc is not really raw data, and on many platforms (like
windows) processed.
Show quoted text 5 lines
If you want to use the raw data, you'll need OS specific functions
(getcommandlinew for e.g. Windows), or, in case of the Unix RTL, the
operatingsystem_parameter_* assembler symbols that convey the registers
loaded by the loader to the RTL.
The usage of which I strongly discourage as they conflict with dynamic packages and thus in the future won't be used on systems that support them.
Regards,
Sven
In our previous episode, Sven Barth said:
Show quoted text 10 lines
> argv and argc is not really raw data, and on many platforms (like
windows) processed.
>
> If you want to use the raw data, you'll need OS specific functions
> (getcommandlinew for e.g. Windows), or, in case of the Unix RTL, the
> operatingsystem_parameter_* assembler symbols that convey the registers
> loaded by the loader to the RTL.
The usage of which I strongly discourage as they conflict with dynamic
packages and thus in the future won't be used on systems that support them.
Yeah, it is more logical to say "just do paramstr(unicodestring) and convert it to the encoding yourself". On unix paramstr and argv are nearly the same, and on windows they are emulated anyway.