In a programming course that I teach, some students are using Lazarus+FPC instead of Delphi. I made available two precompiled units, that is, the *.ppu and *.o files. However, when the students tried to compile their programs using these units, the compiler complained that it could not find the *.pas files. When compiling with -va (verbose all), it became apparent that the compiler disliked something about the *.ppu files. (Sorry, I didn't get the exact message from the students.)

Can someone explain to me under what circumstances FPC will (attempt to) recompile a unit for which *.ppu and *.o are already available? I couldn't find this in the documentation.

Versions: I used FPC 2.0.1 to compile the units; the students used FPC 2.0.2 to compiler their programs.

Thanks,

Tom

Without the exact message it's hard to tell.

Can someone explain to me under what circumstances FPC will (attempt to) recompile a unit for which *.ppu and *.o are already available? I couldn't find this in the documentation.

There are lots of possible reasons. The main ones are

1) sources with the same name as those used in a unit are found, and they are more recent than the ppu files
2) incompatible ppu files because the internal structure changed and the internal version number was increased

Cause 1) can be solved by compiling using the -Ur switch.

Versions: I used FPC 2.0.1 to compile the units; the students used FPC 2.0.2 to compiler their programs.

It's quite possible that your 2.0.1 uses a different ppu format than the 2.0.2 release.

Jonas

Tom Verhoeff wrote

Can someone explain to me under what circumstances FPC will (attempt to) recompile a unit for which *.ppu and *.o are already available? I couldn't find this in the documentation.

Wrong target (OS) or incompatible compiler version, usually. Vote for:

http://www.freepascal.org/bugs/showrec.php3?ID=4691

:-)

Micha

It's interesting, because it was the last argument for me to uninstall Lazarus after experiments with compiling it with different compilers. I read about different OS-es, but the reason was probably different compiler version (or something absolutely different).

Maybe, if you, Micha, submit "patch" to compiler sources (i.e. point the lines you want to be changed) to fpc-devel mail list or something, after a short discussion the developers will add such a hint?

(Of course, only if it is absent in development branch tonow.)

The hint exists already. use -vu.

Michael.

The hint exists already. use -vu.

You really don't expect the average newbie (to freepascal) programmer to read the error message "Can't find unit FOO" and then expect him to *guess* he has to use -vu ?!?!

At the least, expand the error message to "Can't find unit FOO (use -vu for more information)" or something. Although I still think that then -vu will give *too much* information, especially if you're using a large number of units.

Micha

I don't agree: RTFM is *not* an excuse for insightful, to-the-point, clear, not too elaborate error messages.

Otherwise, we could remove all compiler error messages altogether, return an exitcode and then point to the manual.

Micha

Of course I do. What else do we provide -vu for ??

I expect EVERY programmer to have read the user's guide, I expect every full-time developer to have read the programmer's guide.

Otherwise, it's RTFM.

Michael.

Yes, and the 'elaborateness' is controlled by the -v setting.

The point is that "Can't find unit foo" can have several causes.

To find out the exact cause, you must use -vu. To find out even more, you use -vut. after that, it's -va...

RTFM is not an excuse, but people should also take some initiative and start looking for the cause of trouble. The first thing is to increase the verboseness of the compiler... For this, you should read the manual.

It's hard to determine the exact number/amount of messages that is useful to detect/resolve problems. To get more 'feeling' for how to resolve problems, one can and should read the manual...

Michael.

Vote where? :)
The bug reporting system is very limited, once the bug is opened or closed you can't make any additional comments.

p.s. anyone seen this?
http://www.bug-a-boo.org/
Written in pascal/easy btw. Our friends at tdbengine our going to make tdbengine accessible from FPC programs soon, not just from the EASY programming language. I have been talking to them about it.

Exactly! So: split them out, with a separate message for each of them. That's why we have separate messages anyway, right? To be helpful...

In one of the cases above, of the "several causes", you might need extra verbosity indeed, but there are at least two cases where a more helpful and to the point error message can be given.

If you do not want to prioritize this item, it's fine, but please keep it open at wishlist level.

It's hard to determine the exact number/amount of messages that is useful to detect/resolve problems. To get more 'feeling' for how to resolve problems, one can and should read the manual...

This very much sounds like the C mentality TBH: assume people know exactly what they are doing or burn them with cryptic error messages: one needs to debug it all in the aclocal/autoconf/automake/configure/... scripts and figure it out yourself.

I'm not saying people shouldn't read manuals...just saying that "Can't find unit" can be split up in 3 or so "causes", and a more helpful message given for each of them.

Micha

Vote where? :) The bug reporting system is very limited, once the bug is opened or closed you can't make any additional comments.

I know, but sometimes developers who know the system inside-out get 'out-of-touch' with the users who are just starting to use it. So 'voting' is essentially "proving" somehow that this situation needs (some) attention.

Besides, in this case, more specific, to-the-point error messages make programming more productive for all of us, including the more experienced developer. We all mess up our unit output directory or configuration file sometimes (just like that ';' before else, at least by me ;-) ).

Micha

> Vote where? :) > The bug reporting system is very limited, once the bug is opened or closed

you

Sure and that's what I meant: no way to add a comment to your bug once it was "fixed".

Here are the relevant lines from -vaa (but -vu would have provided that as well):

Unitsearch: Fig0.ppu Searching file Fig0.ppu... found PPU Loading Fig0.ppu (FIG0) PPU Name: Fig0.ppu (FIG0) PPU Source: Fig0.pas not found (FIG0) Recompiling Fig0, checksum changed for System

The target OS is correct (I have made that mistake before, and the resulting error message is clear).

The message "checksum changed for System" is not so clear. Now that I think of it (especially with System capitalized), it seems to imply that the compiled unit (Fig0.ppu) depends on the unit System, and when the unit System accessible by the compiler when finding Fig0.ppu differs from the checksum of the unit System used to compile Fig0.ppu, it triggers an automatic recompile of the unit begin loaded.

It would certainly have helped me if the word "unit" was prepended to "System". Furthermore, the checksum did not "change" (there are two separate contexts; neither of which changed), but it differs from the one used to compile the unit being loaded. Thus, a better message might be: "checksum of current unit System differs from the one used to compile unit Fig0.ppu", or "unit Fig0.ppu was compiled with an incompatible System unit".

>Can someone explain to me under what circumstances FPC will >(attempt to) >recompile a unit for which *.ppu and *.o are already available? >I couldn't find this in the documentation.

Apparently, there is a dependency on the System unit, and incompatibility between System units can cause a recompile. Any other such dependencies? Is every separately compiled unit dependent on System? If not, under what circumstances will it depend on System? And on what other units?

There are lots of possible reasons. The main ones are 1) sources with the same name as those used in a unit are found, and they are more recent than the ppu files

This was not the case.

2) incompatible ppu files because the internal structure changed and the internal version number was increased

Also this was not the case (the message would have been clearer).

Cause 1) can be solved by compiling using the -Ur switch.

Good point. I will certainly do that in the future.

That sounds rather vague. From my analysis above, it seems that updating your System unit will invalidate all (?) precompiled units (at least the ones that somehow "depend" on it). Or am I missing something here?

Well, I did learn something. I hope it was useful for others as well.

Thanks,

Tom

Apparently, there is a dependency on the System unit, and incompatibility between System units can cause a recompile. Any other such dependencies? Is every separately compiled unit dependent on System? If not, under what circumstances will it depend on System? And on what other units?

Units and programs always depend on System. It is the main unit where all the basic types are defined.

That sounds rather vague. From my analysis above, it seems that updating your System unit will invalidate all (?) precompiled units (at least the ones that somehow "depend" on it). Or am I missing something here?

You are correct. That is the reason why it is recommended to use the stable releases.

That may be harder than you think, because the routines looking for a unit source do not know WHY they are looking for this.

So they just report 'sorry, we didn't find what you requested". (you=other compiler routines)

At the moment when the compiler decides it needs to recompile a unit and starts looking for sources, it doesn't know yet whether it will find it or not, so it has no sense outputting a message then. You would get far too many messages.

Yes, you could pass on a 'reason' why a unit is searched, and then output that, but it would make the compiler slower and more complicated.

So, for this, you need to use -vu. Then you can follow the compiler's logic exactly...

Which doesn't mean we shouldn't maybe rephrase some messages, but just don't get your hopes up too high :-)

Michael.

*Now* we are getting somewhere, the compiler source structure does not lend itself well to have a more helpful message. This is a perfectly fine argument.

No, of course you don't want it to say something for every unit, that's what -va/-vu are for.

Yes, you could pass on a 'reason' why a unit is searched, and then output that, but it would make the compiler slower and more complicated.

Are you sure? Passing one flag? Checking whether a file exists is probably way more expensive than passing a flag.

Sure, I can understand some things are difficult to implement, and not high priority for whatever reason, but please do not "hide" behind manuals.

Therefore, I ask again to reopen bug report 4691. Not that it has to be fixed now, next week, or ever, for that matter; simply leave it open as a wishlist item.

As somebody else suggested, I may try myself; but I also have a TODO list like you :-).

Micha

I have added a page titled "Releasing units without source code" to the FreePascal Wiki (look under FPC).

I (once more) propose to improve the message when the compiler attempts to recompile a unit because of an incompatible System unit, i.e. the message "Recompiling ..., checksum changed for System":

It would certainly have helped me if the word "unit" was prepended to "System". Furthermore, the checksum did not "change" (there are two separate contexts; neither of which changed), but it differs from the one used to compile the unit being loaded. Thus, a better message might be: "checksum of current unit System differs from the one used to compile unit Fig0.ppu", or "unit Fig0.ppu was compiled with an incompatible System unit".

Note: The two contexts are:
* the compiler installation of the provider of the compiled unit
* the compiler installation of the user of the compiled unit

Should I put this message-improvement request in the bug reporter?

This raises the concern that precompiled units become obsolete when the user of such a unit upgrades the compiler. The compiler apparently requires that the System unit of the precompiled unit and of the using program are IDENTICAL (which is enforced by the somewhat weaker check of comparing their checksums; but in practice that will have the same effect).

Thus, there is NO "backward compatibility". I would hope that IF the newer System unit provides at least the services of (and possibly more/better services than) the System unit used for compiling the provided unit, THEN the "by-now-aged" precompiled unit can still be used without having its sources. In the current situation, the provider of a precompiled unit needs to publish *.ppu and *.o files for ALL 'relevant' System units (and Target OSs, of course).

This makes it much less attractive to release units without source code. That may be understandable from an open-source perspective, but in teaching I find it useful or even necessary to provide units without source code.

Tom

The checksum is only calculated on the interface (i.e. all exported symbols), it's not a generic binary checksum of the entire unit.

Jonas

There is nothing different compared with TP / Delphi. The internal PPU format is made for speed and contains direct references from one PPU to another PPU. Incompatible PPUs can therefor never be used together.

The release of closed source PPUs need to be done on a per stable release. Just like with delphi and tp7.

So please don't make a big issue out of this, the situation is still the same as it was for the last 20+ years.

OK. I didn't know that, and haven't found it in the documentation (or have I missed it?). I have updated the wiki page accordingly.

Tom

Peter Vreman wrote

There is nothing different compared with TP / Delphi. The internal PPU format is made for speed and contains direct references from one PPU to another PPU. Incompatible PPUs can therefor never be used together.

I could not find any of this in the documentation, but I may have looked in the wrong places. Of course, I am not reading TP / Delphi doc when using FPC :-).

Except that there were fewer releases/platforms to cope with, when using TP or Delphi.

I just happened to be bitten by it in an unexpected way. If it is properly documented, then there is no issue. I have tried to remedy that by creating the wiki page on relasing units without source code. It could be helpful if an FPC developer checks that page:

<http://www.freepascal.org/wiki/index.php/Releasing_units_without_source_code>

It might be useful to include a bit in the User's Manual as well (updated Section 3.3 on Compiling a unit).

Tom

I will check the manual.

You note somewhere that:

"(The User's Manual is not up to date on this topic, I believe. If you know more details, e.g. from which version on this changed, then please put it here.)"

Can you tell me what you mean by this ?

Michael.

I've added a remark about different compiler versions.

Jonas

The current on-line User's Guide (HTML version), at the end of Section 3.3 (Compiling a unit), contains the paragraph:

The compiler produces lowercase files, so your unit will be found, even if your source file has uppercase letters in it. Only when the compiler tries to recompile the unit, it will not find your source because of the uppercase letters.

I don't believe this is any longer the case. But I don't know for what version of the compiler this stopped to be a necessity. At some version, the compiler started preserving the casing of the source file in the generated *.ppu and *.o files.

It would be good to state explicitly in the User's Guide to what versions of the compiler the paragraph applies, and to add an explanation of what the situation is with newer versions.

It wouldn't hurt to point in Section 3.3 to the compiler option -Ur.

Tom

Tom Verhoeff ha scritto:

IN TEACHING?????

Btw. no sane commercial Delphi developer has ever considered using a component without source code at least since Delphi 2.0. Look at the offering of component vendors and you will see that they invariably offer source code licenses.

Giovanni

Why this shouting and bafflement? Here are two reasons:

* In order to teach students to use services (=units) solely based on the contract (= specification), we withhold the implementation and only provide the compiled unit. That way the implementation cannot be consulted (=abused).

* In some cases, the unit incorporates some "secrets", e.g. related to grading. Consider e.g. an assignment where one has to develop an algorithm for finding the median of an odd number of distinct integers by using only a binary comparison operator.

To enforce the use of only binary comparisons, we don't give them the set directly, but rather give them a unit, which "hides" the actual integers and provides the comparison operator. Of course, this hidden set can be made inaccessible from the outside by putting it in the implementation part. In that case, it might seem harmless to give away the source code. However, there are more complications: e.g. where does the actual set come from? It is not built into the unit, but read from a file, or generated via a random generator with a seed read from a file. The data in the file is scrambled so that the students cannot simply read it and abuse that information. Unless you want to use sophisticated encryption techniques, it is easier to use a "proprietary" method and not give them the source code.

In fact, the unit need not have a set of numbers to start with, but could construct a "worst case" scenario base on the actual sequence of operations carried out by the program (a so-called adversary). We don't want to give away the strategy of the adversary algorithm.

Btw. no sane commercial Delphi developer has ever considered using a component without source code at least since Delphi 2.0. Look at the offering of component vendors and you will see that they invariably offer source code licenses.

Most likely because the 'compatibility mechanism' is not very effective. Compatibility is one of the big problems in component-based development.

Tom

No, also to be able to maintain the component (fix bugs and look at the implementation). Both the companies I did Delphi work for professionaly, mandated full source for the components.

I would never buy or use components without sources. All components I ever bought/downloaded contained bugs, without exception, and we can't wait for the company to fix them.

Michael.