Hi all,

the $R directive accepts resource scripts (RC) and compiles the RES file from it on demand.

However, that only happens if the RES file does not exist - if it does, the resource compiler is never invoked. Even clean-building the unit from source does not recompile the resource file. This is a bit awkward when the resource script (or any of the linked files) is changed: the only way to recreate the resource unit appears to be removing both the compiled unit and resource file.

Is there some sort of flag for this? I'd be fine with just recompiling the resource file every time the unit is compiled, windres is fast enough...

Thanks,
Martok

Martok wrote

the $R directive accepts resource scripts (RC) and compiles the RES file from it on demand.

Also note that that is only true for Windows. FPC doesn't include a resource compiler for any other platforms. So if no resource compiler exists, you will get a compilation error until you make other plans. Another FPC limitation that really should be resolved.

Regards,
Graeme

You are aware that FPC supports resources on basically all platforms? After all it was introduced for Lazarus back before 1.0 or so.

Regards,
Sven

Please report a bug.

Regards,
Sven

Sven Barth wrote

You are aware that FPC supports resources on basically all platforms?

I know that. What I meant is that FPC only ships with a Windows resource compiler - ability to compile RC files to RES files. All other platforms you have to find your own Resource Compiler. So on a clean system with a fresh FPC install, only under Windows will you have the ability to compile source code that references .rc files.

Regards,
Graeme

Because it's only Windows that we provide a setup for. If someone would provide an FPC implemented resource compiler we'd include it.

Regards,
Sven

Sven Barth wrote

Because it's only Windows that we provide a setup for.

Huh? I've been downloading the official FPC console installers for FreeBSD and Linux since 6 years ago. I never use distro packages managers for FPC.

Regards,
Graeme

I think that what Sven meant is that fpc's Windows installers include binutils, while this is not true for other platforms: that is, you need to have ld on your system if you want to use it as a linker, fpc does not provide it.

Same as for windres: if you need it, usually you can install it from mingw's binutils. In fedora I can install it from mingw32-binutils (or mingw64-binutils). I see that a package with the same name also exists in FreeBSD, so I assume it will contain windres as well.

https://www.freshports.org/devel/mingw32-binutils/

Giulio

"Giulio Bernardi" wrote

I think that what Sven meant is that fpc's Windows installers include

binutils,

while this is not true for other platforms: that is, you need to have ld

on your

system if you want to use it as a linker, fpc does not provide it.

Exactly.

Regards,
Sven

Also note that that is only true for Windows. FPC doesn't include a resource compiler for any other platforms. So if no resource compiler exists, you will get a compilation error until you make other plans.

AFAIK, windres is available for lots of other platforms. If you have to install some package, that shoud not really surprise a developer? I would assume that FPC just tries to call a resource compiler on all platforms and fails with some error message if it isn't available?

If so, I wouldn't worry too much about having our own resource compiler. The RC script language is actually quite complex, and there'd be next to no benefit to reinventing the wheel.

Sven Barth wrote

> system if you want to use it as a linker, fpc does not provide it. Exactly.

Exactly my point! Why? Why does the official FPC installer for Windows give a more workable solution out of the box (1), but official installers for other platforms don't?

And please don't tell me the usual "Delphi compatibility" for G*d sake.

1. Ability to compile source code with .RC reference and magically compile those to .RES as needed.

Regards,
Graeme

Because on non-Windows systems (especially the *nix ones) we rely on their package distributions systems and it's up to the package maintainers to set up the packages accordingly for their distribution so that they pull in the required packages. Such a system does not exist on Windows however so we bundle everything there.

Regards,
Sven

Sven Barth wrote

Please report a bug.

Well this is awkward - I tried to provide a test project for the report, and couldn't get it to *not* work. The real-world project that has this issue for me has the resource included from a file in a package, and it seems like this is somehow significant. Might be more of a Lazarus issue then.

There is still a bug in that the RES is not recompiled if the PPU is present but the RES is not (causes error 9029), but I'll report that separately, after figuring out what is going on here...

Martok wrote

If so, I wouldn't worry too much about having our own resource compiler. The RC script language is actually quite complex, and there'd be next to no benefit to reinventing the wheel.

The same could be said about implementing a competing (alternatives to C/C++ or Delphi) compiler. ;-) Yet FPC project took up that challenge.

Regards,
Graeme

Graeme Geldenhuys wrote

The same could be said about implementing a competing (alternatives to C/C++ or Delphi) compiler. ;-)

Not really though ;-)
Rewriting a resource compiler with the same input and output files as the existing one is a bit different than writing a compiler for a new language and targets...

However, it seems that for example in Debian, windres is only in the crossbinutils package? That's not quite what I expected... What we really need then is "only" a parser frontend for fpcres, fcl-res already can do all the writing we might ever need. Could someone please check if there exists an issue for it and if not, report it? Might have a poke at it, but I'll probably forget before having the time ;-)

install

some package, that shoud not really surprise a developer? I would assume that FPC just tries to call a resource compiler on all

platforms

and fails with some error message if it isn't available? If so, I wouldn't worry too much about having our own resource compiler.

The RC

script language is actually quite complex, and there'd be next to no

benefit to

reinventing the wheel.

The RC language itself isn't *that* difficult. Main difficulty would be to essentially implement a C-preprocessor-compatible preprocessor.

Regards,
Sven

Sven Barth wrote

The RC language itself isn't *that* difficult. Main difficulty would be to essentially implement a C-preprocessor-compatible preprocessor.

As I have just discovered, the installer provides a (rather antique) gcc on Windows and depends on build-essential on (all?) *nix platforms, so we don't even need to do that - piping through "gcc -E -xc" should be all we need. That just leaves a reader for fcl-res.

Hm.

"Martok" wrote

Am 25.09.2017 um 14:24 schrieb Sven Barth via fpc-pascal: > The RC language itself isn't *that* difficult. Main difficulty would be

to

> essentially implement a C-preprocessor-compatible preprocessor. As I have just discovered, the installer provides a (rather antique) gcc

on

Windows and depends on build-essential on (all?) *nix platforms, so we

don't

even need to do that - piping through "gcc -E -xc" should be all we need. That just leaves a reader for fcl-res.

The point of our own resource compiler would be to avoid external dependencies. While me might ship with a gcc on Windows I nevertheless consider it as an external dependency and thus a preprocessor written in Pascal would be preferred.

Regards,
Sven

Sven Barth wrote

The point of our own resource compiler would be to avoid external dependencies.

True. And having a general-purpose preprocessor would come in useful anyway.

Fun fact: windres also just calls gcc, which is probably why we deploy it: #define DEFAULT_PREPROCESSOR "gcc -E -xc -DRC_INVOKED"

But it would be a replacement of a somewhat obscure dependency (windres seems to be a crossbinutils thing) with just keeping a standard one.

Sven Barth wrote

The RC language itself isn't *that* difficult. Main difficulty would be to essentially implement a C-preprocessor-compatible preprocessor.

Just "a few" months later, I finally got around to checking this out.

You were completely right, the language itself is not very difficult, and can be processed into a TResource in one pass in a tply-generated parser. The preprocessor is also just a subset of C's (i.e.: no macros), so it would not be too complicated (I've done the same for idlproc before). There are dialect issues between windres and Microsoft RC, but they are minor.

The more difficult part is implementing the remaining resource access classes for fcl-res. Well, not difficult per se, but the copy-on-write-sharing thing done there adds a lot of complexity for (IMHO) very little benefit and makes it a bit awkward to work with. There's also a lot of endian conversions that could probably be factored out.

That sounds challenging. How much interest would there be in having a full resource compiler, with the appropriate changes/extensions to fcl-res?

I think a lot. There is a request in the bugtracker for a resource compiler since ages.

Michael.

In our previous episode, Michael Van Canneyt said:

> changes/extensions to fcl-res? I think a lot. There is a request in the bugtracker for a resource compiler since ages.

It comes up regularly. Many people don't even pursue it further, and just use resources generator by e.g. an old delphi project if needed.

But longterm, we need this for completeness sake.

Well, I could use it short term too :)

I imagine lazarus could use it as well to bind icons etc.

Michael.

Hi everybody, I'm just chiming in after quite some years. Just two notes:

* For lazarus, it shouldn't need to use a real resource compiler for standard things like icons, manifests etc: see programs res6 and res7 at https://www.freepascal.org/docs-html/fclres/basic%20usage.html

* For the "real" resource compiler, regarding Martok's comment:

Actually, fcl-res was designed as a library to be used by a resource compiler, and I don't think you need to implement new classes at all: once you have written a parser for .rc files you just have to read the files referred from the .rc file (icons, raw data etc), create the right kind of resource and add to the output TResources. IIRC all the standard resource types were already implemented, so once you write a parser that handles .rc files you should be done.

Giulio

Hi!

Good feedback so far. Given recent... conflicts, I wanted to ask first, just to be sure.

* For lazarus, it shouldn't need to use a real resource compiler for standard things like icons, manifests etc: see programs res6 and res7 at https://www.freepascal.org/docs-html/fclres/basic%20usage.html

That is probably enough for simple resource editor, yes. I believe this is what Lazarus already does for VERSIONINFO resources (?)

In part, yes. Particularly the really difficult types are already implemented. I would attempt to implement all remaining RT_x types, including those that are not commonly used in cross-platform targets. Things like DIALOG or MENU have a custom format in their RawData, but that should be fairly straightforward, basically doing the same as the ACCELERATOR resource and tutorial.

Hi again,

a quick update: I have a somewhat-working version, but many things are still missing.
We have: a preprocessor, full support for BITMAP, ICON, CURSOR, resource attributes (LANGUAGE etc.), all RCDATA-like resources, including their definition from inline literal statements.

What's missing:
- UnicodeString Strings (L"foo")
- various forms of escape sequences for strings
- Concatenated adjacent strings ("ABC" "DEF")
- numeric expressions (used for things like computing flags).
- complex resources: ACCELERATORS DIALOG/EX MENU/EX STRINGTABLE VERSIONINFO

The first two are not that difficult, just awkward (as all encoding-aware stringhandling is), adjacent strings are more of an issue: turns out this creates a problem in the language:
FOO BITMAP "900.bmp"
"A NAME" TEXT BEGIN "A String" END
This can't parse right: the filename and resid get concatenated to "900.bmpA NAME". While entirely correct, that's not very useful. I might leave that out until someone needs it.
Of the complex resources, STRINGTABLE and VERSIONINFO are probably the more important ones. Luckily, both already have resource types implemented, so that shouldn't be too hard either.
It's also leaking memory like a rusty bucket, I have so far completely ignored what transfers object ownership in fcl-res classes. Heaptrc will hopefully help here.

I only get to work on this on about one day a week and spend half of that trying to keep the grammar somewhat readable and with no bad reduce conflicts, but if anyone wants to play along:
<https://github.com/martok/freepascal/compare/master...fpcres-rc>

Seems like you have the main things covered, though. Nice job and keep up the good work :)

Michael.

More Progress!

I've since finished STRINGTABLE and VERSIONINFO as well as mathematical expressions.

However, there's an issue I hope somebody can help with. Binutils has this test case:

<https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blob;f=binutils/testsuite/binutils-all/windres/version_mlang.rc;hb=HEAD>

Every Resource compiler I have tried compiles this slightly differently, and Resource Hacker doesn't understand one of them. Could somebody test this with MS RC and share the res file?

Independently, TVersionResource emits one Translation block per translation instead of one containing all translations. I take it that isn't formally wrong, but it's also not optimal? Was this intentional?

In our previous episode, Martok said:

I happened to have an old win8 sdk around (vs community edition didn't seem to have "RC"). Anyway:

http://www.stack.nl/~marcov/funkyres.res

I happened to have an old win8 sdk around (vs community edition didn't seem to have "RC"). Anyway: http://www.stack.nl/~marcov/funkyres.res

Thank you!

Funky indeed, I'll mark that down as a Reshacker bug then.

Off to String stuff then.

Update time!

<https://github.com/martok/freepascal/compare/master...fpcres-rc>

- UnicodeString Strings (L"foo")Done - various forms of escape sequences for strings

Done

- Concatenated adjacent strings ("ABC" "DEF")

This is weird: windres, brcc and rc all do something different when concatenating mixed Unicode and CP strings, as in `L"a" "b"`, `"a" L"b"`. I have no idea what the correct action would be.

- complex resources: ACCELERATORS DIALOG/EX MENU/EX STRINGTABLE VERSIONINFO

Probably going to leave out the Dialog-Related stuff, since it's not super relevant for us anyway. Windows users can just use RC ;-)

It's also leaking memory like a rusty bucket, I have so far completely ignored what transfers object ownership in fcl-res classes. Heaptrc will hopefully help here.

Fixed

All of windres's tests pass now, except for the Dialog related ones. I would consider it more or less finished. Any more things to do? Otherwise, I'll stack it into some patches.

A few lessons learned:
Using TStringList as a key-value-store (#define) is SLOW. Replacing it with a TFPStringHashTable brought processing time for windows.h down from 4 seconds to some 10ms. That's just insert operations.

Codepage Strings are weird if more than one CP is involved. There is almost no way to carry a specific codepage anywhere: AnsiString[850] + Char = AnsiString[CP_UTF8]... Also, ShortStrings are often converted to AnsiStrings, so that doesn't reliably work either. Same for array of char. That made working with plex awfully complicated, and I still think it's wrong in some cases. But at least it works for most now.

Having a reentrant version of plex&pyacc would be nice. Flex and Bison can do it these days.

To a lesser degree: something like Python's argparse would be nice. Everyone builds their own argument parser, usually with very different concepts and varying amounts of code duplication.

Martok wrote

To a lesser degree: something like Python's argparse would be nice. Everyone builds their own argument parser, usually with very different concepts and varying amounts of code duplication.

Assuming you mean the command-line arguments:

What's wrong/missing with the functionality in TCustomApplication ?

Michael.

Michael Van Canneyt wrote

Assuming you mean the command-line arguments: What's wrong/missing with the functionality in TCustomApplication ?

Other than that nobody in the wild seems to fully use it? ;-) Even most of the examples in the compiler tree use homegrown parsers, or look like [1].

I'm a fan of getopts and use it (with a small wrapper) for almost everything (i. [2]), but it seems that many people are not aware it even exists. It's even POSIX compliant, so the program's users won't be surprised by some implementation particulars.
But even that is nowhere near as elegant as argparse. I just don't like retyping usage statements when they could be autogenerated ;-) And Subparsers are, while possible, always quite hacky.

Of course, if someone was to implement something now, we'd be in the https://xkcd.com/927/ situation...

[1] https://github.com/graemeg/freepascal/blob/master/utils/svn2cvs/svn2cvs.pp#L494 [2] https://github.com/martok/buildtools/blob/master/buildutil.lpr

Your count is a little biased. A total of 41 examples/tools use it. 14 utils distributed in the FPC tree use it.

home:~/fpc/packages> grep -i TCustomApplication */examples/*.* */tests/*.* ../utils/*.* ../utils/*/*.* | wc -l 41
home:~/fpc/packages> grep -i TCustomApplication ../utils/*.* ../utils/*/*.* | wc -l 14

There is a lazarus IDE wizard for a console application. Every Lazarus LCL application has it built-in. So it is used "in the wild".

The bare compiler tools do not use it because
a) They predate TCustomApplication
b) It introduces a dependency on something which is not in the RTL but in packages.

So that it's not used is simply incorrect. It's more used than the posix-y getopts unit which is also present in the fpc tree.

You simply didn't know about it, most probably.

You're welcome to suggest improvements, but as your rightly observed, we should not introduce another system.

Michael.

I was 100% expecting that sort of answer.

Cheers,

Martok

Martok wrote

I was 100% expecting that sort of answer.

My invitation to suggest improvements, you mean ?

I meant it: you're welcome to suggest improvements.

If you implement something like argpars and it can be used as backend for TCustomApplication, I'm willing to rework TCustomApplication to use it. This way we keep backwards compatibility and offer the possibility to extend it.

I'm not blind to the shortcomings of TCustomApplication argument parsing. It fully handles the unix '-a' '--aaa' scheme of doing things, but currently does not handle options in command form
prog --globalopts command [--commandopts] args

So, if you can offer something to keep current functionality and allow this as well: be my guest...

But e.g. the geptops way, which basically forces you to use a while loop is really not acceptable, which is why the TCustomApplication interface is what it is today.

Michael.

You can also try this: http://yann.merignac.free.fr/unit-cmdline.html

It's a command line parser I wrote when I needed it a few years ago.

I was inspired by these 2 articles:
- https://www.adacore.com/gems/gem-138-gnatcoll.command-line
- https://www.adacore.com/gems/gem-139-master-the-command-line-part-2

AM Martok wrote

I'm a fan of getopts and use it (with a small wrapper) for almost everything (i. [2]), but it seems that many people are not aware it even exists. It's even POSIX compliant, so the program's users won't be surprised by some implementation particulars.

GetOpts gets it wrong sometimes: see
https://bugs.freepascal.org/view.php?id=19842

Hm. I must have missed this one.
I've assigned the bug to myself, will look at it ASAP. I think getopts has changed since the bug was reported, but maybe the proposed fix is still valid.

Michael.

I wrote one, too: http://benibela.de/sources_en.html#rcmdline

Supports --linux=style and /windows style, and ' and " quotes

By changing the subject of an existing (very old thread) and post new content you have screwed up the message handling of threads so this conversation now gets stuffed into the "Resource compilation" thread dating back a year and 3 months...

Please start new threads from scratch instead.

I just changed the title for my reply to an already off-topic reply - but apparently people don't do that old newsgroup convention anymore.

My original reply post was very much supposed to be in this thread. But as nobody seems to care about that post's content anyway and instead latches on to the smallest of notes... well. Whatever.

actually, they do if they have any understanding of proper netiquette ;)

My original reply post was very much supposed to be in this thread. But as nobody seems to care about that post's content anyway and instead latches on to the smallest of notes... well. Whatever.

it is ok, martok :)

PM Michael Van Canneyt wrote

I think getopts has changed since the bug was reported, but maybe the proposed fix is still valid.

The report is from 2011.
Since then there has been a 1 change, adding 1 line, to getopts.pp

I tried the proposed fix (it should go in 5 lines from the bottom i=of Internal_getopt), and it does not fix the issue.

PM Bart wrote

I tried the proposed fix (it should go in 5 lines from the bottom i=of Internal_getopt), and it does not fix the issue.

I attached a possible fix to the bugreport.

Are there tests for the getopst unit?

I think the "old newsgroup convention" is to start a completely new thread as a pristine message without embedded history. I read all of my watched groups through a news reader....

My original reply post was very much supposed to be in this thread. But as nobody seems to care about that post's content anyway and instead latches on to the smallest of notes... well. Whatever.

Explanation:
------------
I have archived this mail list via GMANE News so I have *everything* from about Sept 2003. Very handy for local searches of issues and content.

The original thread started at 2017-09-23 and discussions on it continued a bit over 2 weeks until your last message on 2017-10-11.

Later the thread was resumed by you on 2018-08-04 and went on a few days until 2018-09-09 (your last message then).

Then again the thread was resumed by you 2019-01-13 and now after a few hours you renamed the subject to "CLI argument parser" and a fair deal of activity ensued.

The problem with this for users interfacing to the discussion via news readers is that these keep track of the posts not via the subject line but through some identity marker in the headers of the messages and this causes new messages to be inserted into the *original* thread instead of appearing at the bottom of the thread list in the reader. (Or top if you sort your threads that way)

I discovered that suddenly there were unread messages in my reader but I could not see any among the new stuff. So I had to go hunting for them and found this thread a long way up in the hierarchy...

This is what happens when one re-purposes old threads for new stuff.

I just wanted to make you aware of the readability issue, sorry if I have upset you.

I have archived this mail list via GMANE News so I have *everything* from about Sept 2003. Very handy for local searches of issues and content.

I do the exact same thing, and I know how much bad threading can ruin the usefulness of a group.

You may want to examine whose message was the first one in the recent tree that did not directly relate to progress reports on the resource compiler, and if that still bothers you, take it up with Tomas.

I just wanted to make you aware of the readability issue, sorry if I have upset you.

Don't worry, *you* haven't.

And, given that this was something of my last attempt to contribute something useful anyway, there won't be any more messages from my side that could upset you (or anyone else), at least for a long while.

Cheers,

Martok

Martok wrote

Update time! <https://github.com/martok/freepascal/compare/master...fpcres-rc>

[snip]

All of windres's tests pass now, except for the Dialog related ones. I would consider it more or less finished. Any more things to do? Otherwise, I'll stack it into some patches.

I know it's been some time, but I've now integrated this into trunk (took a bit of work, because I had to nicely integrate it into the build system) and also made sure that FPC can use fpcres for RC -> RES compilation as well. :)

Regards,
Sven