See Mantis #20786 for details.

The first problem was the use of single quotes in the generated fpdoc commandline, which are not handled properly by the Windows shell. I fixed this manually, by editing the Makefile.

Next I get the following error messages:

../rtl/unix/sockets.pp(20,15): Could not find include file 'osdefs.inc' ../rtl/unix/ipc.pp(26,15): Could not find include file 'osdefs.inc' D:\repos\fpc-2.5.trunk\rtl\linux\ptypes.inc(33,15): Could not find include file
'ctypes.inc'
../rtl/unix/baseunix.pp(24,17): Could not find include file 'aliasptp.inc' ../rtl/unix/unix.pp(24,17): Could not find include file 'aliasptp.inc' D:\repos\fpc-2.5.trunk\rtl\objpas\classes\classesh.inc(210,18): Expected "," or
":" at token "=" in file
D:\repos\fpc-2.5.trunk\rtl\objpas\classes\classesh.inc at line 210 column 18
../rtl/objpas/dateutils.pp(1,17): Could not find include file 'dateutil.inc'

I'm not sure how to fix these, e.g. 'osdefs.inc' does not exist in the rtl\win or rtl\win32 directories. The compiler options for unix/sockets.pp are:
-dver1_0 -Fi../rtl/inc -Fi../rtl/win -Fi../rtl/win32

Perhaps linux/sockets.pp should be replaced by win/sockets.pp? Or should the Makefile supply the Linux/Unix include directories, instead those of the current platform (Windows)?

Also I don't understand the problem with dateutils.pp, options -Fi../rtl/objpas -Fi../rtl/win -Fi../rtl/win32
because dateutil.inc is present in rtl/objpas.

Can somebody explain what I should try to fix the include file errors?

DoDi

In our previous episode, Hans-Peter Diettrich said:

commandline, which are not handled properly by the Windows shell. I fixed this manually, by editing the Makefile. Next I get the following error messages:

Afaik the docs are hardwired for Linux. Even if you are on host windows, you should enter paths and defines like it was Linux.

I already committed a fix so that the linux paths are appended in windows.

Michael.

The latter. The paths must be appended, but I already committed a fix for this (untested, though).

Just check out the latest version. I committed some fixes that should improve matters.

Michael.

The patch fixes most problems :-)

Some duplicate include directories could be stripped from the command line, but that's more a cosmetic change.

Found the bug in PParser, see Mantis #20791. Windows seems to be more picky than other platforms, on attempts to open an file twice with AssignFile (maybe different exclusive/shared access?)

This leads currently to memory leaks and exceptions, when an include file in the BaseDirectory is opened twice. This second attempt fails and Result (containing the first file reader) is set to Nil.

DoDi

Is this the right fix? Shouldn't it be possible to generate docs based on other targets as an option? Or possibly something like a superset for all targets (showing the differences)? The current situation is that Linux-specific extensions to common units (e.g. variables, functions, etc.) are always included in the docs, while extensions/additions existing on other targets are not documented at all (and just adding their description to the respective .xml files wouldn't make any difference to that if I understand it correctly)...

Tomas

In our previous episode, Tomas Hajny said:

Is this the right fix? Shouldn't it be possible to generate docs based on other targets as an option?

What do you expect from that, other then that a few symbols disappear ? (that are hopefully already annotated as platform specific?)

The patch changes nothing to the content of the docs, it just makes sure that the parser finds all files on Windows.

If there are any platform-specific identifiers, the docs have always contained the linux-specific ones. The patch changes nothing about this.

Michael.

Right, platform specific documentation must be written before it can be shown. OTOH it's not nice when e.g. TDateTime is undocumented.

Now that we have fpdoc projects, it's time to add platform specific documentation. That's no extra time for the FPC team, this also can be done by the users.

I just think about an extension to the fpdoc projects, listing platform specific input files and compiler options. Linux may be used as the fallback, whenever no replacments are given for the actual platform.

DoDi

If there are any platform-specific identifiers, the docs have always contained the linux-specific ones.

I'm just about to create a Windows version of the RTL docs. In contrast to the assumption, that one XML file could hold descriptions of multiple modules, I tested the opposite way: having multiple XML files for one unit. And it seems to work :-)

This means that we can document platform-specific items in additional files, which are automatically merged with the other descriptions. The description file specifications can be extended by a platform attribute, so that problems arising from multiple (platform specific) descriptions can be eliminated. E.g.:
<description file="sysutils.xml"/>
<description file="linux-sysutils.xml" platform="linux,unix"/> <description file="win-sysutils.xml" platform=windows/>

It may make sense to place the platform specific files into dedicated subdirectories. When fpdoc is extended with description directories, from which all XML files are collected automatically, it will be sufficient to provide the platform-specific directories to fpdoc, so that most explicit --descr items can be removed or replaced by: <description file="*.xml"/>
<description file="windows/*.xml"/>
or
<description directory="windows"/>

DoDi

Michael Van Canneyt wrote

On Fri, 2 Dec 2011, Tomas Hajny wrote: > On Thu, December 1, 2011 13:27, wrote: >> On Thu, 1 Dec 2011, Marco van de Voort wrote:

.
.

I understand this now, but before Marco's post I thought that the reason for the current situation was simply the fact that you always built under Linux rather than that it would never work otherwise.

Tomas

Possibility of proper documentation of extensions existing for other targets?

Tomas

Nice to hear that.

I don't see the point of this.
Why not simply add them to the description file sysutils.xml ?

<element> tags for which no source file identifier exists are discarded anyway.

If anything needs to be done at all, I see more use in adding a 'platform' tag to the element tags themselves; That would give the documentation generator useful information which it can use. In fact, the 'platform' directive should in the first place be in the sources themselves, where it will be picked up by the scanner/parser - as per the remark of Marco.

I have no plans to support directory or wildcard search in fpdoc.
That is a can of worms I will not open;

One objection is for instance that your proposal will cause all XML files to be loaded both when FCL and RTL documentation are created.

In general: "Keep it simple" is a good design strategy.

All these 'automatisms' should be delegated to separate tools. This keeps things simple, much more maintainable.

Your strategy seems to be to put everything and the kitchen sink in fpdoc. You would be far better off making separate tools that implement these automatisms. You can write a e.g. tool that creates or updates a project file based on a directory scan. I have no problem including such tools in the FPC distribution, whereas wildcard search in fpdoc itself is sure to be rejected.

Michael.

Out of curiosity: How could/would one create one documentation of source which supports multiple platforms, but where there are identifiers that are only available for some platforms? (like our RTL)

Regards,
Sven

You currently can't as long as the source files are different.

And even with the same source file it is difficult, since the parser will always parse only one branch from an IFDEF.

{$IFDEF WINDOWS}
   THANDLE = Handle;
{$ELSE}
   THANDLE = Pointer;
{$endif}

Cannot be parsed correctly. Same for library units with different calling conventions etc.

Michael.

To show that I'm serious about this, I implemented a small tool which does exactly that. (mkfpdocproj, in utils/fpdoc).

I intend to add options to it, till I end up with something that can be used to create a XML file to create the RTL documentation (possibly using various runs of the tool).

Michael.

You're right. So what's the use of --ostarget and --cpuversion, WRT common documentation?

Right.

I just implemented wildcards, and it seems to work nicely :-)

One objection is for instance that your proposal will cause all XML files to be loaded both when FCL and RTL documentation are created.

Already included: all XML files are discarded, when they don't contribute to the current package :-)

In general: "Keep it simple" is a good design strategy. All these 'automatisms' should be delegated to separate tools. This keeps things simple, much more maintainable.

Well, I found it the simplest approach to add certain features to fpdoc itself, as long as nothing else is broken.

Your strategy seems to be to put everything and the kitchen sink in fpdoc. You would be far better off making separate tools that implement these automatisms. You can write a e.g. tool that creates or updates a project file based on a directory scan.

There may remain things, which better should (and can!) reside in separate tools. Currently I feel no need for such additional tools, where the purpose would justify such efforts.

I have no problem including such tools in the FPC distribution, whereas wildcard search in fpdoc itself is sure to be rejected.

I already expected such a reaction ;-)

As long as the the fpdoc classes cannot be used and extended outside fpdoc, I see no way around inofficial patches, for private use. The number of fpdoc users is very limited, most users will be comfortable with the final documenatation, so that there exists no urgent need to improve the distributed fpdoc itself. Fortunately my work on fpdoc itself is almost finished, so that I can return to the Lazarus documentation now.

DoDi

Somehow that's what I feared based on your explanation above (Note: I don't have the need for it currently, but I believe it's a "nice to have")

Regards,
Sven

Out of curiosity: How could/would one create one documentation of source which supports multiple platforms, but where there are identifiers that are only available for some platforms? (like our RTL)

Right, that's a problem - but how many identifiers are really affected?

IMO the RTL provides a platform-independent API, with not many platform specific identifiers. The same situation exists in the LCL, with the support of multiple widgetsets.

When ever required, the platform or widgetset specific *units* deserve their own documentation (sources) in the first place. Then it's only a minor effort to create specialized fpdoc projects, which use the right units and create the documentation for them. Every user then can create his own customized documentation, with commands as simple as fpdoc -p=winrtl.xml
fpdoc -p=gtk2lcl.xml

DoDi

One workaround is to do something like is currently done in the RTL. Let fpdoc define a conditional (e.g. FPDOCSYSTEM) and add dummy identifiers to the unit:

{$IFDEF FPDOCSYSTEM}
   // Add definitions for fpdoc here which are normally not present.
{$ENDIF}

This way, they are documented, but not in the actual unit. (unless someone actually wants to cheat the compiler and compile the unit with -dFPDOCSYSTEM)

Michael.

They are for the parser, not for the description files.

> I have no plans to support directory or wildcard search in fpdoc. That is a > can of worms I will not open; I just implemented wildcards, and it seems to work nicely :-)

I do not doubt it is possible, I am just concerned about everything that may go wrong in the future. I prefer a different way.

Because of the loose coupling between XML files and source files, there is no way to know if a XML file is contributing or not, unless you impose some convention like 1 unit = 1 file with equal filenames, or parse the XML file to detect the module it documents or start moving XML files to directories.

I went ahead and improved the tool I announced earlier. The end goal is to use this tool to create the project file from the Makefiles for the docs. Currently it can:

- Add directories of source files, with file mask, parser options to be added to the found files, and recursion.
- Add directories of description files, with file mask and recursion.
- Add a list of input files (with options).
- Add a list of description files.
- Remove a list of input files.
- Remove a list of description files.
- Set/Unset engine options.
- Perform macro substitutions on a fpdoc project file.

The last is in fact probably the easiest solution for the RTL/FCL docs. We can distribute a template project on which simple macro substitution is performed prior to feeding it to fpdoc. Thanks to your patch for creating a project file from given command-line options, I now actually have such a file.

I split the tool so the class that does the actual work can be integrated in e.g. a GUI tool for quick manipulation of project files.

I expect and hope the tool will solve most - if not all - of the problems you experienced as well.

Michael.

Reading the XML files for both RTL and FCL takes almost no time, compared to the time spent in generating the documentation. Lazarus already maintains package documentation in a dedicated directory, so that it's up to the documentation maintainers what they put into these directories. You remember my question about the suggested organization of documentation?

Nice :-)

What I'm still missing, is support for documentation writers:
- process a single unit from a project, to get XML syntax and other error messages.
- more verbose output for spotting such errors.

You remember my -n and -v options?

Thanks for all that. I'll merge it with my own work, and can supply an informational patch with my remaining modifications.

DoDi

A first note:

A commandline option for creating a project is nice, but it doesn't help when fpdoc is called from a script, e.g. make rtl.chk.

That's why I let fpdoc always create a project, when it was not already invoked with -p.

DoDi

Probably none for Windows targets because the original Delphi RTL was designed for Win32. The other platforms sometimes need certain extensions in order to emulate MS Windows behaviour/APIs not supported on those platforms, or because fairly basic functionality of those platforms would be completely left out otherwise (since not being taken care of by the Delphi designers for obvious reasons).

IMO the RTL provides a platform-independent API, with not many platform specific identifiers. The same situation exists in the LCL, with the support of multiple widgetsets.

They're not / should not be many, indeed.

Sorry, could you elaborate on that a bit, please? Would that allow producing a complete rtl.inf (or rtl.chm, etc.) with all common parts together with OS/2 specific extensions (based on the common description files, obviously extended with those platform specific elements)? That's something I'd be highly interested in...

Thanks

Tomas

Yes, of course.
The question is whether the fpdoc engine should cater for specific setups. There our answers differ.

Absolutely, and I'll gladly accept patches implementing those 2 things in fpdoc. If you could separate those out from your other work, that would be much appreciated. If not, send whatever you have, and I'll try to extract it myself.

A last word:
This new tool will solve also a problem which we've been struggling with in fppkg. The tool will allow to create a "site documentation file": I envision a site-docs.xml file in a location known to fppkg. Installing a package using fppkg can now call the new tool and add the new package documentation to the site-docs.xml file as part of the installation process, and rebuild the docs.

Anyway, I'll wait for your patch with the -n and -v options.

Michael.

I've always wonder about the very verbose command line parameters used with fpdoc. Why couldn't the XML simply mention which unit(s) it documents. You could also include macros [eg: $(somemacro) ] which fpdoc could substitute when the xml file is processed. The macros could be defined in some fpdoc.ini file, or passed as command line option. I would opt for the ini file choice, but the command line param could be used to override the macro if needed.

eg:
-----------[ fpg_base.xml ]---------
<?xml version="1.0"?>
<fpdoc-descriptions>

<package name="fpGUI">

<!-- multiple files could be describe in a single xml file too --> <files>
<unit>$(fpgui_src)/corelib/fpg_base.pas</unit>
</files>

<module name="fpg_base">
<short>....</short>
<descr>...</descr>
</module>

</package>
</fpdoc-descriptions>
--------------------------------------

That way you only need to tell fpdoc what XML files to process (even my passing it a directory, instead of file by file). No project file needed.

One could even extend the <unit> tag, to include the unit name, and search path for that unit.

eg:
<unit>
<name>$(fpgui_src)/corelib/fpg_base.pas</name>
<searchpath>-Fu$(fpgui_src)/corelib/ -Fi$(fpgui_src)/corelib/</searchpath> </unit>

So the xml file is really self-describing. Containing everything it relates too.

No project file per-se required at all? No extra tools needed to generate project files etc.

In our previous episode, Graeme Geldenhuys said:

Keeping content and settings apart. Such macros would introduce assumptions about directory layout (even if just minor stuff) to the content. I don't think that is necessary.

A solution like you propose with INI is difficult since then a checkout in a random place would work without configuration first.

Strictly speaking, the XML file does say which units it documents. See the <module name=xyz"> tag. One problem is that you know this only after you have parsed the XML when you should know before starting.

I think there are several other reasons why the project file approach is more powerful.

First, it keeps the actual documentation XML more "clean" in the sense that it contains only documentation, and not 'organizational instructions'.

Just like a lazarus project file (.lpi) is different from the actual source file (lpr). You'll remember that Delphi used to try and put all project info in the .dpr file, and I think we all know what a pain that was.

I think it would also require the documentation writer to foresee organizational issues and parametrize them in the XML, which I think is something the documentation writer should not have to care about. The person generating the documentation, on the other hand, 'knows' the possible situations and can parametrize his project file for this. (and indeed, can create many different project files)

Speed is also an option. If we start allowing macros, then as an end result the macros must be processed in the whole XML file. The documentation XML is rather big; the project file is small.

In true unix philosophy, I think it is better to have many small and simple tools cooperating well than one large tool which tries to do everything and fails miserably. There you have, in a nutshell, the design philosophy of fpdoc :-)

Michael.

I'll attach an patch with my remaining changes, fpdoc-nv.patch. Here's a roadmap for you:

I've enclosed most still different parts of the code with conditional DoDi. This should allow (me) to run both versions from the same source, and helps in searching for differences.

pparser.pp is extended with a -v option, passed as commandline argument. It writes out the currently processed unit. Not perfect, of course, it would be more useful when the current unit were shown with every error message, by default.

pscanner.pp is even more nasty, I could not find an easy way to implement a -v (or -vt) option. That's why I added a conditional "Verbose", that turns on the trace of tried include directories. Perhaps you find an better way, using the settings of pparser. IMO this option is essential for the fpdoc compiler (--input) options setup in every project. I'd be happy if you can provide common package options, like
<package name="bla" options="-Fi common to all units"/> which is automatically appended to all unit options.

dGlobals.pp contains the Verbose flag, for use in fpdoc. And the help messages, of course.
AddDocFile gets an additional package name, that allows to exclude unrelated docs, and also does wildcard substitution.

dw_html.pp traces the created HTML files. Like with pparser, the element name (html file) should be shown with *every* error message instead.

fpdoc.pp obviously accepts the added commandline options. It also creates a default project whenever invoked without a project.

fpdocproj.pas Options.DryRun should be obvious.
Improved --imports handling, so that the prefix is no more required (can be synthesized). This part can be ommitted.
Moved CreateDocumentation here, to make it independent from commandline handling.

fpdocxmlopts.pas can be almost ignored, it retains my old save/load code (see DoDiSave), until I'm sure that I added all required modifications to your code. Thanks for extracting the file names from the options :-)

Have fun :-)

DoDi

Sorry, the parser logic doesn't allow for multiple platforms (with different sets of conditionals or include pathes). Multiple doc versions can be created only in multiple runs, as for the cross-compiler compilation cycle, but merging these docs will become very complicated.

When the documentation should include multiple platforms, parsing of the source files should be omitted, and instead the docs should be created from all XML elements. But without parsing the units, the index generator cannot separate e.g. classes from procedures or variables :-(

BTW I hate that multi-language documentation of e.g. Delphi (including CBuilder help) or .NET, with insufficient means to suppress unwanted branches, or popping up help for a different (unwanted) language. The same might happen with an all-in-one multi-platform documentation.

Otherwise combined help for different platforms can be generated from artificial units, which declare *all* identifiers of *all* platforms. But I don't see an simple way to create such dummy units from the actual sources. You may try to run the preprocessor for all configurations of interest, then merge the created units (removing duplicate declarations and implementations). But wait, the preprocessor is defunct right now, my according patches had been rejected for inobvious reasons :-(

DoDi

I dare to disagree, WRT the Unix philosophy. When I was used to create Delphi programs from simple project files, it was hell writing Makefiles for the same purpose. And when, according to the Unix philosophy, "make" was extended with "automake", "configure" and more bloat, the user has to learn a couple of additional languages (5 or so), and has to obfuscate the source code with options for all these tools, in order to make a project multi-platform. FPC does all that a bit better, using only "make" (or fpcmake), and a well designed directory structure :-)

WRT fpdoc itself, it IMO should work with either CLI or GUI, i.e. the worker code should be separated from the invocation. This is not possible right now, when CreateDocumenation is available only in the CLI version :-(

Otherwise it were easy to extend the functionality of fpdoc "from outside", by deriving classes and inserting additional passes between the collection of (commandline or project) options and the document generation. Bummer! I should apply that concept to my own fpdoc extensions, by making the fpdoc classes more virtual, and moving my own code into derived classes in separate units...

Would a patch be acceptable, that splits some methods into parts (new virtual methods), so that more features can be implemented without touching the fpdoc base classes further?

DoDi

First, it keeps the actual documentation XML more "clean" in the sense that it contains only documentation, and not 'organizational instructions'.

The documentation is useless unless you have the associated *.pas unit. As you even mentioned earlier. fpdoc doesn't include documentation if it can't find the associated identifiers, units etc.

So no matter how you think about it, the XML documentation is closely tied to the source code it documents. After all, it is API documentation we are talking about.

I think it would also require the documentation writer to foresee organizational issues and parametrize them in the XML, which I think is something the documentation writer should not have to care about.

API documentation writers are technical people, probably a programmer. They *will* have to look at the source code, no know how to document each API.

I think you are confusing the type of documentation. FPC Language Reference for example is a totally different type of documentation to API documentation. The latter needs a programmer or at least somebody looking at the source code to know what it does, before they can write documentation or code examples.

Speed is also an option. If we start allowing macros, then as an end result the macros must be processed in the whole XML file. The documentation XML is rather big; the project file is small.

To produce API documentation with fpdoc, you have to read all the XML files anyway. The macros I'm talking about will only be used in the <unit> take, nowhere else in the documentation. The macro will should normally point to the root of the source code directory - wherever that is on each persons computer. The structure of the source code from that point on (deeper than the root of the source directory) can be assumed to be constant (or change very very little). Even now, if the source code directory changes, the documentation scripts will not work. All the --input parameters will have to be adjusted. So the inclusion of that information in the XML file adds no extra overhead.

I'm just thinking in terms of minimizing the amount of files we need to carry around if we want to generate documentation.

The other benefit of having the unit information inside the XML file means you can extend whatever documentation tool you use, so it can for example generate documentation just for that one unit, or one XML file. As Hans-Peter mentioned, a preview to see if you documentation for a specific unit or class etc looks correct.

In true unix philosophy, I think it is better to have many small and simple tools cooperating well

Yes, I agree with this philosophy, but no matter how you look at it, API documentation is closely tied to the code it is documenting. So keeping the information together in one place seems logical too.

In our previous episode, Graeme Geldenhuys said:

True

So no matter how you think about it, the XML documentation is closely tied to the source code it documents. After all, it is API documentation we are talking about.

True.

But separating the path data from the documentation in no way invalidates those statements.

And the contents in the XML is _NOT_ dependent where exactly those files are, as long as fpdoc can find them.

Agreed. But see below. I think we are talking about different things.

All correct, but the documentation writers are not the people who create the final documentation (as in "html files", or whatever format).

It is the packagers who must produce the output, and it is the latter who, in my opinion, benefit from the use of a project file: they must 'organize' their work: from disparate XML files create a single, coherent documentation, from files located in different locations, obtained from different writers. This is the organization I was talking about. Not the organization of actually writing the documentation.

The former (the documenters) will normally work in Lazdoc or whatever tool that supplements the IDE, and the IDE knows where all relevant source files are.

Michael.

Marco van de Voort wrote

And the contents in the XML is _NOT_ dependent where exactly those files are, as long as fpdoc can find them.

Exactly my point. fpdoc cannot generate documentation without knowing where the the *.xml _and_ *.pas files are. So currently we need to keep telling fpdoc where both are. Why not combine both options into one. Simply tell it where the xml file is, it knows where the *pas file is that it describes. One less option to worry about, one less parameter to pass to fpdoc.

Yes the project file does the same as I describe, with one less option.... the preview a single documentation file as mentioned before. I guess with some effort that could be accomplished with the project file too. But my point still remains, the xml means nothing without the associated *.pas file - so I still think it seems logical to keep those *very much related* pieces of information together. It's not as if the documentation write would need to keep editing the unit information anyway - it is information that hardly ever changes.

Michael Van Canneyt wrote

The former (the documenters) will normally work in Lazdoc or whatever tool that supplements the IDE, and the IDE knows where all relevant source files are.

Not everybody uses lazdoc! I know I don't. And last time I heard, you told me you don't either. There are much better tools out there to edit XML file than lazdoc. lazdoc brings nothing "extra" or "special" to the party. I find a simple programmer editor with code-templates much more useful in editing fpGUI's API documentation, than lazdoc.

So as far as my use-case goes, I would find it much more convenient to simply do Ctrl+Home (to jump to the beginning of the XML file) so I can see what *.pas unit I am documenting. If that need ever arises. Compare that to hunting down some totally separated project file somewhere, to see what it might link the current xml file with. Keeping related information / data together is a basic programming principle. ;-)

I thought about such a simplification, too, but you'll get some objections:

How then do you want to document files for different platforms? (needs to supply different source and include directories)

How to find the source files, when an XML file documents multiple modules? (I never saw such an case, but it's possible).

The same for multiple XML files, documenting parts of an source file?

I've reduced my description file lists to an single entry: dir\*, and let my fpdoc search for all applicable XML files there :-)

This solution has already been rejected by the core developers [no comment here :-], but a similar solution --descdir could serve the same purpose, with less risks for errors in the supplied argument.

DoDi

In our previous episode, Graeme Geldenhuys said:

True.

So currently we need to keep telling fpdoc where both are.

On one system.

Why not combine both options into one.

Because then you can't have multiple projects using the same content and sources in different paths.

Simply tell it where the xml file is, it knows where the *pas file is that it describes. One less option to worry about, one less parameter to pass to fpdoc.

One more rigidity added.

I did several things:

1. Enable various log levels in parser and scanner. It uses an event handler. (writing to terminal is not possible)

2. Route all this logging through the TPasContainer. I tried to remove all direct writes from all backends and other places in fpdoc, so now it should work in a GUI app as well.

3. Create a TFPDocCreater component. It contains the CreateDocumentation logic, verbose and dry-run logic, and some more.
You should be able to re-use this component as-is, see unit mkfpdoc.pp

4. Implemented the --verbose and --dry-run options.

These 2 options are not stored in the project file, they must be given on the command-line. I do not believe they belong in the project file.

Todo: implement a --descr-dir command-line option for fpdoc application. That will scan a directory and add all XML files it finds in that dir to the project. I can see why you want such thing, but I think such logic belongs in the command-line tool itself, not in the project description itself.

Michael.

Hans-Peter Diettrich wrote

I thought about such a simplification, too, but you'll get some objections:

I believe I already have. ;-)

How then do you want to document files for different platforms? (needs to supply different source and include directories)

Macros. Just like, how does the compiler find the same file, but for different platforms. The concept of macros are well known and already used in a lot of places. fpdoc will just be another use case.

How to find the source files, when an XML file documents multiple modules? (I never saw such an case, but it's possible).

Based on my XML sample I posted a while back. The <unit> tag could maybe be moved under the <module> tag. Or, looking at my original XML sample, add a mapping (unit to module mapping) to the <unit> tag.

The same for multiple XML files, documenting parts of an source file?

See previous paragraph.

This solution has already been rejected by the core developers [no comment here :-],

Same here. Michael doesn't seem keep to my idea either, so I very much doubt it will make it into fpdoc. Core developers always trump mere contributors like us. ;-)

Great :-)

Waiting for a commit of these extensions, and for a description of your mkfpdocproj tool...

These 2 options are not stored in the project file, they must be given on the command-line. I do not believe they belong in the project file.

ACK. See my other mail about the options stored in project files.

My solution doesn't deserve an change to the project files, the expansion of --descr="dir/*" can be done in TFPDocEngine.AddDocFile, for all kinds of invocations. A search also could be started when the --descr argument does not specify an ".xml" extension, whereupon the argument can be treated as a documentation directory.

I still miss a "required package" option (variation of --import?), describing the packages whose content files are required to build a dependent package documentation. Since the location of these packages can differ on every machine, the user should be alerted when he missed to specify the locations of required content files. An error message "content file not found" exists already, but it will not be shown when fpdoc has no idea of the required content files.

Such an option IMO becomes very important, when e.g. the LCL documentation shall be created from a project file. Required packages can be handled in an according tool, but even such a tool needs informations about required packages.

Another issue are the relative directories, as used in all current scripts and project files. E.g. build_lcl_docs invokes fpdoc from the HTML directory, so that the project file will contain references relative to that directory - which does not normally exist before the first invocation of build_lcl_docs! Moving the created project file into an safe place will invalidate all contained file references :-( Should this bad situation be cured in build_lcl_docs, or can we provide other assistance?

DoDi

I definitely hate macros :-(
Where are they defined, what's the syntax for their definition and use, how are nested macros expanded, why is my macro not found...

DoDi

Everything was already committed in rev. 19755.

I prefer an explicit command, and impleented it like that, there are now 2 options --descr-dir and --input-dir. This is also in rev. 19755.

Incidentally

will append all lines in @filename as description files, the same is true for --input. This was already there since many years.

IMHO The project file should simply contain the appropriate import lines ?

Such an option IMO becomes very important, when e.g. the LCL documentation shall be created from a project file. Required packages can be handled in an according tool, but even such a tool needs informations about required packages.

There is no such thing as a 'Required' package. fpdoc should still be able to build the docs in the absence of this package. It will simply not be able to build some cross-references. So at most a warning is in order.

I think that the warnings about missing links in target ID should be plenty to alert the user; Of necessity they will start with #packagename, and will probably drown the one additional warning.

This must be cured in build_lcl_docs.

fpdoc builds documentation from available files.
Where these files are depends on the environment and should be determined by the person who created the environment.

Michael.

Hans was not right. I did implement the directory search. Just not the way he wanted/designed it.

We simply have a vision and try to remain true to it. Ideas corresponding to this vision will make it. Others not.

I dare say this is the very reason you started fpGUI. And this is not meant as a reproach, on the contrary.

Michael.

Marco van de Voort wrote

Because then you can't have multiple projects using the same content and sources in different paths.

I'm afraid I don't understand your point? fpdoc is about documenting API's. Those are rather specific to a single project. Could you supply a concrete example backup your statement?

I already mentioned that macros in the unit paths could be used to fix path issues. In the case of FPC's RTL for example. Simply switch for example a $(fpc_src) macro you can compile help for FPC 2.4.4 or 2.7.1. The unit path could even contain TargetOS and TargetCPU macros if you wanted to differentiate further.

One more rigidity added.

I see more flexibility.

Hans-Peter Diettrich wrote

Where are they defined, what's the syntax for their definition and use, how are nested macros expanded, why is my macro not found...

You are over exaggerating a bit. Macros are used all over the place with great success. FPC, Lazarus IDE, MSEide and many CLI tools to name just a few.

Where are macros define... the project file, or overridden/defined via a command line parameter. The norm would be inside the project file or ini file, or whatever file fpdoc uses to read settings - like it now does.

Hardly any time was given to discuss options and alternatives, pros and cons etc. You simply stated in one message you wanted a project file for fpdoc, then boom, the project file idea was implemented - nobody else's ideas count any more from that point on. Core developer has spoken! :-( One has to love the way the FPC team works.

Anyway, I'll move on now.

Unfortunately, Graeme, you are mistaken in your facts.

The fpdoc project file support already exists since almost a year (rev r16711). It was discussed way before that in the FPC team (Marco will back me up on this). There was even a bug tracker item for it in the meantime, half a year ago. You can examine the SVN log files if you do not take my word.

All I did now was add some options for its manipulation, based on the problems Hans was experiencing when he tried to build the FPC docs.

Michael.

> Waiting for a commit of these extensions, and for a description of > your mkfpdocproj tool... Everything was already committed in rev. 19755.

Now it has arrived here, and it really looks great :-)

One comment on ParseOption:
   else if s = '--show-private' then
     FCreator.Options.ShowPrivate := False
doesn't look correct to me.

This must be cured in build_lcl_docs.

I think that build_lcl_docs can be removed, the new directory options in fpdoc can do the same job :-)

fpdoc builds documentation from available files. Where these files are depends on the environment and should be determined by the person who created the environment.

Right. I'll check the new fpdoc options and discuss it in the Lazarus list.

DoDi

All I did now was add some options for its manipulation, based on the problems Hans was experiencing when he tried to build the FPC docs.

Right, I'm impressed that almost all of my wishes have been implemented now. I was quite pessimistic after the first reaction on my patch, but the more I'm happy now with the final result. Cooperation rulez :-)

DoDi

Can you please add the imports to the project files?

Another issue: when no --format is given, an unhandled exception occurs, instead of an error message. Obviously the old default HTML doesn't apply any more?

DoDi

In our previous episode, Michael Van Canneyt said:

Check, Michael and I discussed, and he said he would do it.

But the discussion was triggered by Hans-Peter. Somewhere in spring he was also messing with docs on windows and there were some examples of the unix-centricity of the fpcdocs setup.

Some things in the latex parts were fixed (but there is still a bit of shellscript somewhere). Wrt the fpdoc docs, I did some tests too, which lead us to conclude the long cmdlines had to go long term.

For the other (scripting) related bits I hoped on fpmake, which I hoped had progressed outside packages/ by now :_)

In our previous episode, Marco van de Voort said:

Some things in the latex parts were fixed (but there is still a bit of shellscript somewhere).

(for the people wondering, it is the for loop in Makefile.4ht)