This is related to http://bugs.freepascal.org/view.php?id=22751

- What FPC/Lazarus functions are available to parse a command line (where parameters begin with a dash)?

- What function can split an argument of the type name=value or option value

Stephano

patspiper wrote

This is related to http://bugs.freepascal.org/view.php?id=22751 - What FPC/Lazarus functions are available to parse a command line (where parameters begin with a dash)?

Depends. Split arguments for TProcess use (like in the above bugreport): afaik none. Simply keep all arguments separate and pass them to the "parameters" propery.

In case of the callee, see the unit getopts.

Keeping arguments separate may not be possible:
http://permalink.gmane.org/gmane.comp.ide.lazarus.general/64781

However, splitting the command line with the separator being an unquoted space should be good enough. This would even work with a gdb option such as the -b (baudrate).
Ex: gdb -b 9600

     Process1.Parameters.Add('-b');
     Process1.Parameters.Add('9600');

The above seems to work.

Stephano

patspiper wrote

> Depends. Split arguments for TProcess use (like in the above bugreport): > afaik none. Simply keep all arguments separate and pass them to the > "parameters" propery. Keeping arguments separate may not be possible:

Not *easily* possible. Not impossible.

http://permalink.gmane.org/gmane.comp.ide.lazarus.general/64781

That's why the old option is still available "deprecated" for a while. But no new functionality will be added. IOW it will remain compatible to how it was when it was deprecated (2.6.0 compatible).

If you can't change it out of compatibility, this will be no problem, since the current interface should be exactly compatible to 2.6.0 and before.

However, splitting the command line with the separator being an unquoted space should be good enough.

No it wouldn't, since -xx="bla" needs to be dequoted on windows (and I assume OS/2) too.

Your message is a very good example why the whole splitting and escaping approach was abandonned. Every issue was considered to be "just add this, since then it works for me now", but after multiple years of such changes, there are still corner cases left.

Martin is currently investigating this venue.

Separating based on unquoted spaces is the 1st step. The 2nd step is to remove the 1st and last matching quote from each parsed parameter (even on *nix). The 3rd step is to add each parameter using TProcess.Parameters.Add. It is kind of a middle ground though I definitely support your suggestion if doable. Parsing a command line is a can of worms!

Stephano

Might I make a suggestion? Why try and invent the wheel ourselves? There's a very good library named getopts that has been used by masses of programmers for years now. Maybe we could learn from that and re-implement the choices made in there?

Grx HdV

We have this unit (getopt, it is documented?).
But the code for that is for the receiving end, i.e. used by the program that is started. Not for the program that starts another program.

The main problem is the Windows API, which has only 1 string in which to pass all command-line options. The started program simply gets this string, and must split it into options. How this string is split (quotes, spaces, whatnot) depends on the started program and is governed by custom. It may be using getopts, or something else.
(Look at the GetParamStr routine in the system unit to see how Delphi does it).

On Unix-like systems, there is simply no problem: At the API level, Options are passed in an array of strings. Whether or not there are quotes, spaces or whatnot in this string, is irrelevant: the strings are passed on as-is and need no interpretation.

What causes confusion on unix systems is the use of a shell (bash, csh, sh): It receives commands from the user as a single line. It must split a command-line into the various elements that make up the array used in the Unix API. To be able to do so, the shell needs quotes. The API itself does not need it.

The TProcess interface has been reworked so it conforms more to the Unix API, since it is easier to convert that to something that Windows understands than vice versa.

Michael.

2012/9/10 J.A. de Vries :

Might I make a suggestion? Why try and invent the wheel ourselves? There's a very good library named getopts that has been used by masses of programmers for years now. Maybe we could learn from that and re-implement the choices made in there?

But isn't this a completely separate problem? If I understand this thread correctly then the problem here is properly emulating what the shell would do when parsing one line of text (splitting the line at unquoted spaces, removing quotes, maybe even expanding wildcards) and then *producing* the argv array while getopts on the other hand can be used to extract key-value pairs that follow a certain syntax from an already existing argv array?

We have this unit (getopt, it is documented?). But the code for that is for the receiving end, i.e. used by the program that is started. Not for the program that starts another program.

Thanks for the clear explanation of the situation. I think I understand better now. I see why getopts might not help.

On the other hand, let me check what perl's implementation does. Because I seem to remember that they do parse a single input string. But I am not sure, so I need to verify this. I'll let you know what I find.

Grx HdV

You might be right. I seem to remember that the native perl implementation of getopts parses the argumentstring as a plain string and builds from that. I could be wrong though. I'll look it up later this week, as soon as I can find the time.

Grx HdV

The main problem is the Windows API, which has only 1 string in which to pass all command-line options.

IMHO this is not a Windows problem.

There are two problems as I see them, one being that APIs only accept one cleanly normalized form (be it as an array of arguments, or with exact quoting rules), and that the "common" notation must be rewritten to those rules. This always needs some filtering and cleaning.

The second problem is that the "common" everything on one line notation is OS specific, and even within an OS is fragmented between shell and API rules. (windows too, shellexecute createprocess, though the differences are smaller)

This significantly increases the need for rewriting (one could argue that notation like --xxx="yy xx" is an unixism) that is not strictly necessary.

As already said in similar discussions on the FPC maillist: nearly _every_ FPC version had changes in quoting and new bugs/featurerequests. In 2.0.x/2.2.x times, some of those even have been a problem for lazarus.

Most of these changes also have not been minor, and quite often newly introduced bugs have been fixed just barely in time, between the RC and release.

Already in 1.9.2, executeprocess() was added that fixed this issue for the more procedural interface. All the older versions have slowly been deprecated and removed (a still ongoing process)

So in short, the tprocess maintainers (mostly Michael and me) have said enough is enough. The primary, supported way to hand parameters to tprocess is to deliver them separated, and cleansed of unnecessary quoting. This mostly fixes the second problem. The first problem is more minor. (and can often be fixed by simply stripping quoting before passing it.

We hope this will allow for a more stable interface, and I heartily recommend it for any non-trivial use.

Besides the already named problems, that library and its principles are unix-centric. Though that matters less than in the past.

J.A. de Vries wrote

You might be right. I seem to remember that the native perl implementation of getopts parses the argumentstring as a plain string and builds from that. I could be wrong though. I'll look it up later this week, as soon as I can find the time.

As promised I looked it up and as far as I can see the perl module Getopt does indeed do its own parsing.

See /usr/share/perl/5.14.2/Getopt/Long.pm (or the equivalent path under Windows) if anyone is interested to see how John Vromans has solved the problem of parsing commandline areguments without resorting to Unix-isms. Perl and this module are platform independent, so I presume the code might contain some useful hints for Lazarus too.

Grx HdV