Preface: I know how much work a good documentation requires, and honor the goodwill of all contributors :-)

Nonetheless most entries (I visited) are in poor state, not helpful and often wrong (outdated?). That's why I want to contribute myself, and want my documentation *never* overwritten automatically by SVN updates (merging of course will be acceptable).

Unfortunately the current help document (xml) management gets into the way very often. Here some problems:

- How to share documentation across multiple Lazarus installations? (including working with Lazarus/FPC branches)

- How to document IDE units? Error: not in a package...

- How to update documents, when these reside in read-only (system) directories?

Most annoying is the inability to share updated help, in multiple Lazarus branches. This could be made possible by having FPDoc files on a machine only once, independently from working copies. Effectively the docs should reside in their own SVN repository, not in every Lazarus SVN branch.

Currently the help system requires that all documented units belong to a package or project. I do not really understand this requirement, in detail when Lazarus.lpi is not detected/accepted as the owning project :-( IMO a default directory could be used to store help on source files, whose owner cannot be determined. The source filename could be interpreted as <somepath>/projectdir/unit.pp, and the help be stored in $(MyLazDocDir)/xml/projectdir/unit.xml.
When the file resides in a subdirectory of the current project, that part of the path can be used.

Any comments?

DoDi

svn does not simply overwrite modified files. It tries to merge updates. This fails if your modifications and others modifications are on the same lines. In this case svn gives you several choices. In any case your modifications are never lost (unless you did a mistake).

The current fpdoc documentation is only for sources and is only valid for the associated version. A help entry may be wrong for an older version, or an identifier might be renamed or moved to another place.

Normally the biggest part of the entries can be used for older versions and the newest version of a package contains the best fpdoc entries. Therefore it might be desirable to use an old version of a package for compiling and use the documentation of the newer version. But this works only for the biggest part of the entries, and opens a can of worms for the rest, so I have not implemented such a system yet.

- How to document IDE units? Error: not in a package...

I added the fpdoc search path docs/xml/ide to the lazarus.lpi, so that IDE sources can now be documented via fpdoc as well. I also added it to the IDE itself, so that you don't need to open the lazarus.lpi to document the IDE sources.

- How to update documents, when these reside in read-only (system) directories?

The same as the ppu files.
Install a newer version or install lazarus in a writable directory.

No, this would not work with renamed/moved entries.

Currently the help system requires that all documented units belong to a package or project.

Technically a fpdoc file is associated with a unit. The fpdoc search path is stored in the package/project. Practically you want to create one help file per package, for example for the LCL.

I do not really understand this requirement, in detail when Lazarus.lpi is not detected/accepted as the owning project :-(

I hope you understand now.

IMO a default directory could be used to store help on source files, whose owner cannot be determined.

Please check before writing.
It exists since years.

See Environment options / FPDoc editor / FPDoc files path.

The fpdoc editor does not support to create new fpdoc files for such units.

I see problems with this approach. See above.

Mattias

Problems arise with branches.

Moved or renamed items will loose their associated documentation anyhow.

Normally the biggest part of the entries can be used for older versions and the newest version of a package contains the best fpdoc entries. Therefore it might be desirable to use an old version of a package for compiling and use the documentation of the newer version.

ACK.

But this works only for the biggest part of the entries, and opens a can of worms for the rest, so I have not implemented such a system yet.

Version specific notes are desireable, but not a must. Such a feature can wait until the documentation is almost complete. In the meantime it may be more interesting to have changed features mentioned in the (unique) documentation, e.g. when upgrading projects.

Okay, I'll test that new feature.

The current model doesn't work with such entries as well.

You refer to the final (compiled) help?

Let's assume a single help directory associated with every package. Then two search strategies for help files can be used:
1) find the package containing the file, then search the help directory associtated with that package, or
2) find the help directory containing the file.

In the first case a list of packages has to be searched, then the associated help directories. Packages or units not in that list can not be found, e.g. when no package exists for the IDE project.

In the second case a list of help directories is required, that can be maintained by the user or package system. A single machine-specific xml directory may be sufficient, with subdirectories for all packages, so that no existing help file can be missed in the search. A concrete unit owner is not required, the help directory name can be used for that purpose. E.g. the complete LCL help can be created from <helpdir/xml>/lcl/*.xml.

> I do not really understand this requirement, in > detail when Lazarus.lpi is not detected/accepted as the owning project :-( I hope you understand now.

No, see above.

Dedicated help directories are somewhat self-organizing, no chance for missing help files in a search. But lists of packages and their contained files can be incomplete.

Sorry, not in my time machine :-(

See Environment options / FPDoc editor / FPDoc files path.

These pathes do not specify unit owners, so I wonder in what situations these directories are searched at all.

The fpdoc editor does not support to create new fpdoc files for such units.

That's why we should allow to create help for unowned files first, before we discuss how they can be found later again.

DoDi

For example?

Not anyhow.
When you rename an identifier the corresponding fpdoc entry must be renamed too. This could be done by the IDE. In fact I already started it.

The current model works. It is only uncomfortable when renaming.

In other words: If a package has help it is found, if a package has no help, nothing is found.

How can help files for lcl version 1.0 and lcl version 2.0 be distinguished?
How should help files for third party packages be distributed?

If the file is not listed in any package the IDE searches in the unit and src paths of all packages.

Do you have an example where the IDE can not find a fpdoc file?

IMO finding help files is the most important thing. F1 must work out of the box with minimal setup. The current model is this: I download a package from the net, open it in the IDE and can press F1 on an identifier to get help. All details are up to the package maintainer.

Mattias

> Problems arise with branches. For example?

When I work in a local branch, and find something worth to document, then this documentation is added to the branch and removed when switching back to another branch.

Fine :-)

If no package is found, *or* the package was found but does not have help...

When this matters, the user can create separate doc folders.

In fact a versioning system were nice for the documentation, where added or updated help can be applied either to common help or to version (branch...) specific help. Most entries apply to all versions, and it even does no harm to add version 2.0 specific extensions to the common lcl help - they will never be searched when working with 1.0 code.

Only the final documentation may have to know about versions, so that the proper overviews can be created for the 1.0 and 2.0 version. IMO such overviews *always* should be created from the source code of the proper version (like makeskel does), not from manually maintained lists in help or topic files. The related entries then can be collected from the separate help files, where every item can be tagged with the version that introduced (moved, renamed...) that source code item.

Since documentation is written only after the source code has been created and committed, it doesn't make sense to have both in the same versioning system. When help *on* r0815 source code is committed, the help should become visible *starting* with r0815 as well, not only when committed *as* r4711. Of course SVN doesn't allow to amend an older commit, so that separate source and help repositories IMO are the only viable solution.

How should help files for third party packages be distributed?

Usually these are distributed as part of the package. As you suggested already, the user can add a symlink to that help after installation, or the help can be placed into the user/machine specific help directory on installation of the package.

Why use search paths for packages, when it's easier to specify search paths for documentation? As already mentioned, a single LazDoc path may be sufficient, when all docs are installed into subdirectories of that directory. The package-specific location of the docs is required only at the time of installation of the package, later on the help system will know where to find all the installed docs.

Do you have an example where the IDE can not find a fpdoc file?

When the FPDoc Editor denies to create an item, and requires to add the source file to a package...

As a concrete example, I couldn't add help on widgetset-specific units like win32wscomctrls.pp. Why should help be available only for installed packages, known to the IDE, when the IDE allows to *open* source files without such a restriction?

When e.g. an index file is added to every source directory, that file could contain all the information required to find the associated help files. Such a file can be created automatically, or when not found the parent directories can be searched. Then the help file for $(LazDir)/lcl/interfaces/win32/someunit.pp could be found in $(LazDoc)[/xml]/lcl/interfaces/win32/someunit.xml, with the only required information that $(LazDir)/lcl/ is the root directory of package lcl. For projects an existing .lpi file could be used as the root-directory marker of the project, and similar conventions could apply to package directories.

I'm really confused now :-(

First FPDoc Editor requires that the source file is part of a package. Then you say that this is not (no more?) a requirement. So what's the use of added pathes?

When a help directory can be specified whenever a non-packaged file is documented, the help system (later) has no idea in which directory that help file has been created, and consequently must search the entire help path. Until here a single NonPackaged directory would be sufficient to hold all such help files. When multiple such directories are allowed, the help system still cannot know whether Unit1.xml should be loaded from directory X or Y :-(

ACK. The FPDoc Editor should also work with minimal setup, since it doesn't require to press F1 all the time, and does not require compiled (HTML) help.

The current model is this: I download a package from the net, open it in the IDE and can press F1 on an identifier to get help. All details are up to the package maintainer.

Details are not (only) up to the package maintainer, but also are up to the package installer and help system.

As mentioned already in my preface, the FPDoc Editor is of my actual interest. As I could figure out till now, everything boils down to the (fragile) determination of the owner of an source file, based on only(?) the module name. When that module name is unique, knowledge of the owner is not required, it only can speed up the first search for the according help file. My impression is that all(?) registered packages are scanned at IDE startup, so that instead the registered help directories could be scanned as well.

DoDi

Why not merge the change to the other branches?

How should that work with the above search pattern <helpdir/xml>/lcl/*.xml.
?

AFAIK fpdoc does not support such tagging.

True.
Overriding the fpdoc search path for packages is needed.

ok

I understood the question as what happens when the lpk file does not list all files of the package. The IDE will fail to associate a unit that is in a directory only used on another target. So the IDE does not know where the help is for the w32wscontrols.pas under Linux. Although with some guesswork it could. Maybe this will be added, when the problem arises. So far no one needed that.

As already mentioned, a single LazDoc path may be sufficient, when all docs are installed into subdirectories of that directory.

Of what directory? The directory of the current unit or the global override directory for fpdoc files?

The FPDoc editor is only a tool for the Lazarus way of fpdoc files. Its big brother lazde can be used to create any kind of fpdoc files.

As a concrete example, I couldn't add help on widgetset-specific units like win32wscomctrls.pp. Why should help be available only for installed packages, known to the IDE, when the IDE allows to *open* source files without such a restriction?

The IDE allows to open a win32wscomctrls.pp, but many code features only work under windows.

The unit belongs to the package LCL, which has no fpdoc path set yet. When I open the unit qtobjects.pas under Linux/QT, move cursor to an identifier and click on "create" in the fpdoc editor the IDE asks me to setup a fpdoc path first. I set it to "docs/xml/lcl" and a new fpdoc file is created. It's pretty easy.

First of all, the lpk is an index. No need to invent another one. Searching the lpk/lpi file when opening a unit is a good idea. But an lpi or lpk file is not always placed in the root directory of a package/project. In fact many ported Delphi packages/projects put the Lazarus stuff into a sub directory. A more sophisticated search is needed.
Fourth, a special solution for the packages in the Lazarus sources can be done.

I too, when I hear the reasoning of such programmers, but this universe is not that simple.

First FPDoc Editor requires that the source file is part of a package. Then you say that this is not (no more?) a requirement. So what's the use of added pathes?

When you create a new fpdoc file, the fpdoc editor requires a package. When searching a fpdoc file of a unit the IDE searches in the above path too. I know no one that uses this. You are the first wanting to document units without a package.

Yes. In most cases the search path has only one directory. In fact I don't know a case with more than one fpdoc directory.

Until here a single NonPackaged directory would be sufficient to hold all such help files. When multiple such directories are allowed, the help system still cannot know whether Unit1.xml should be loaded from directory X or Y :-(

Is this a theoretical case or do you have any real life example where this is a problem?

An option to override and use some different fpdoc files would be nice. But this is for expert users. I think the help must be done for newbies, experts come here second.

As mentioned already in my preface, the FPDoc Editor is of my actual interest. As I could figure out till now, everything boils down to the (fragile) determination of the owner of an source file, based on only(?) the module name. When that module name is unique,

It is not always unique.

Only the lpk files are loaded.

Mattias

It does. The <version> tag has the same format as the <descr> tag. It is even documented :-)

Michael.

Don't you use git? If you do, the change are carried over (unless the branch you are switch too, has different changes to the current branch). If the latter, simply use 'git stash' and 'git stash pop' to carry changes over - that if you didn't want to commit them for some or other reason.

I'm using Git in my experiments, and switch branches frequently. After a while it's hard to remember in which branch I've updated the documentation, and into which other branches the updates have not yet been merged. That's the most important argument for storing documentation apart from source code.

The helpdir is set in the Lazarus options, so that Lazarus1 uses lcl 1.0 and LazHelp1, and Lazarus2 uses lcl 2.0 and the LazHelp2 directory.

That's not a general problem - XML supports any number of tags ;-)

When can this change be expected?

@Graeme: can you split your Git repository accordingly?

Overriding the fpdoc search path for packages is needed.

It may be a good idea to move all help directories under docs/, and to turn Lazarus/docs into an SVN (sub-)project.

This is one case that can never occur when the directories are searched for the source and help files :-)

The IDE will fail to associate a unit that is in a directory only used on another target. So the IDE does not know where the help is for the w32wscontrols.pas under Linux.

The help is in the same place on all platforms.

Although with some guesswork it could. Maybe this will be added, when the problem arises. So far no one needed that.

I just found a lot of help for the gtk widgetset, so no general problem seems to exist. I only couldn't make the FPDoc Editor display content of these files (on Windows). It should not be a problem to find <lazdocdir>/xml/lcl/interfaces/gtk/gtkdef.xml for
<lazdir>/lcl/interfaces/gtk/gtkdef.pp.

When Lazarus/lcl/ (currently) is associated with Lazarus/docs/xml/lcl/, or ../docs/xml/lcl/, the remaining interfaces/gtk/ portion can be applied to that root directory.

All help files should reside in subdirectories of the global fpdoc directory, i.e. currently in the Lazarus/docs/xml/ directory. No difference to the current handling, only that the $(LazarusDir)/docs/ part should be replaced by $(LazarusDocs). Then the FPDoc search path must not contain separate entries for all the subdirectories (lcl, rtl...), as long as these reside in the same parent directory (as is). Extra entries will be required only for third-party packages, whose help resides in other directories, or a symlink to that directory can be put into the default help directory.

The FPDoc editor is only a tool for the Lazarus way of fpdoc files. Its big brother lazde can be used to create any kind of fpdoc files.

How does lazde make the created files known to the IDE?

And AFAIR lazde had the same problem, with files not belonging to a package.

It's sufficient when the IDE (FPDoc Editor or F1) also shows the help for the opened units.

Again, how is that file (or directory) made known to the IDE?

It only is a manually created and maintained index, which does not necessarily contain all files of the package (etc.).

And all this is why I suggested an independent/additional file, that can be added to every package source directory. Such a file is required only when the IDE can not otherwise locate the root directory of a package or project. The IDE can e.g. search for an '*.lp?' file, in the directory of the source file and all parent directories. Once such a file is found, its name specifies the project or package name, and the default location (directory name) of the associated documentation in the LazDoc path. In so far I don't understand why the lcl package file is named lclbase.lpk, and not lcl.lpk :-(

DoDi

> I'm really confused now :-( I too, when I hear the reasoning of such programmers, but this universe is not that simple.

Good solutions tend to be simple ;-)

Everything happens once for the first time. That's what makes coders earn a continued living ;-)

I just have increased the FPDoc path box (to alClient), because it contains a whole bunch of pathes (for the lcl, rtl...).

Fortunately I don't have an example right now, but shit happens...

Newbies will use F1, while power users will use the FPDoc editor.

What will happen in this case?

I get any number of GetFPDocFilenameForSource calls on IDE startup. Since the SrcToDocMap cache is empty at that time, SearchInPath is called until a help file can be found on disk. This way the candidate directories are scanned very often (once for every source file!), where a single scan of every directory, for all contained files, would be sufficient to fill the

I do, and whenever I switch branches, the help files are switched as well.

I don't stash changes, instead I either commit all changes or reset the old branch. Just to make sure that I don't mix up modifications to different branches.

BTW, in my FPC development I created independent local Git repos for /compiler and /docs, leaving the remaining framework to SVN. The compiler repo contains various branches, while the docs repo only contains the master branch - I only use it as a backup, in case another SVN update will touch the docs.

DoDi