Hi,

Seeing the discussions about annotating the documentation created by FPDoc, I added support for notes. (revision 20304)

In short:

At the level of the <module>, <topic> and <element> tags, you can now include a <notes> tag.

Within the <notes> tag, you can include one or more <note> tags.

They are typeset as an unordered list, which means that it can basically contain the same things as a <li> element. (so no tables, code blocks and what not).

By default, the <notes> tag is ignored.

Only if you give fpdoc the --emit-notes option, then the notes will be included in the output. (at least in HTML and all linear output).

The fpdoc XML project file also supports the emit-notes option.

I have documented all this it in the official fpdoc reference guide. (rev. 890)

As a side note:
The official FPC documentation will not contain notes in the output. I consider them for private use only, so it is useless to submit notes. But I can understand other people want such a feature, therefor I added it to fpdoc.

Michael.

Seeing the discussions about annotating the documentation created by FPDoc, I added support for notes. (revision 20304)

Thanks :-)

Why so complicated? Can you give a *reasonable* example of the intended usage of this feature?

It looks to me as if you only implemented only half of a possibly much more powerful feature. Provided that your intention is about *conditional* parts, which are formatted for inclusion into the final docs, I'm missing more detailed condition handling. Then it would be possible to e.g. provide documention specific to a platform or widgetset, wherever different implementations or behaviour suggest such separate documentation.

Apart from that, author/editor notes deserve no special structure or formatting, and should be easier to insert (what about LazDE?).

What about a less structured solution, e.g.
<notes XYZ deserves a clarification!/>
for an alternative use of the just introduced tag? (exact syntax to be specified)

In contrast to:
<notes><note>XYZ deserves a clarification.</note></notes> or
<notes>
<note subject="win32">On Win32 ...</note>
<note subject="linux">On Linux ...</note>
</notes>

The fpdoc XML project file also supports the emit-notes option. I have documented all this it in the official fpdoc reference guide. (rev. 890)

Available how/where?
<http://lazarus-ccr.sourceforge.net/fpcdoc/fpdoc/fpdoc.html> still is from 2008.

Okay, found it - but it should be accessible easier, without rebuilding all the FPC docs. Above questions/annotations still apply, examples welcome.

DoDi

Hans-Peter Diettrich wrote

What about a less structured solution, e.g. <notes XYZ deserves a clarification!/> for an alternative use of the just introduced tag? (exact syntax to be specified)

That example would be a violation of the XML specification. The following would have been sufficient:

<notes>XYZ deserves a clarification</notes>

Though I suspect that Michael had his reasons to implement it in the way that is implemented now.

Rgeards,
Sve

Yes. User notes on a website.
This website has been in the pipeline for a long time.

You must be joking ?

Let me recapitulate.

1. You were complaining that you cannot enter notes.

2. You make do with [note whatever] in current lazarus docs.

3. People seem to agree some kind of note functionality is in order.

4. Noting all this, I can see the use of author's notes, and decide to add a feature for notes, which is extensible if need be, and can be added to any editor with a minimum of effort, making them easy to manage. As a bonus, they come in handy for a long-standing idea, namely, a collaborative website for creating docs.

And now you have the incredible nerve to complain that

a) it is "too complicated"

b) "notes deserve no structure or formatting"

c) it "misses features for conditionals" in a "more powerful solution"

Disregarding the fact that I think your complaints are simply contradicting each other I am simply *apalled* at the fact you have the guts to complain.

Michael.

For this stands the (...) postscript ;-)

I also could have written
<notes DoDi="my comment"/>
or
<notes missing="XYZ should be described"/>
with the only requirement that FPDoc does not stop or AV on an unknown tag.

DoDi

Sorry, I cannot establish any connection from a website to the <notes> syntax and usage in FPC documentation :-(

Do you mean kind of a separate section, like <links> or <errors>, or tags embeddable into other text?

And now you have the incredible nerve to complain that

...your descriptions have not been very helpful so far, I'm still waiting for at least one practical example - XML, not usage :-(

DoDi

Well.

Your reply shows you have not actually read the fpdoc docs (it contains at least 2 example XML snippets in the tag reference) or checked out the testunit1.xml file that is checked in with fpdoc itself (it contains notes at various levels).

You should have noticed it was updated when you did an update. So, looking at either would have answered your question.

Michael.

Sorry that I'm too stupid to read your mind :-(

And sorry that I did realize only now, that your <notes> can not replace my [notes], inlined in the texts.

IMO this is a classical example of the different assumptions of the developers (enough said) and the users (understood nothing) :-(

DoDi

The prospect of a website dedicated to discussion of documentation seems to me to be a helpful move forward. This mailing list is not well suited to detailed discussions of the numerous issues 'documentation' and 'help' raise, and use of it for that creates a lot of noise that some find irritating.
Nevertheless it is one of the major bullets to be bitten before Lazarus can consider getting to the 1.0 stage, and so having an interactive arena dedicated to it would be excellent, and I hope would move Lazarus docs further towards the goal of being adequate if not good or excellent. FPC documentation has the advantage of being overseen by a single person who has the ability not only to produce and maintain the tools needed to write the documentation, but is punctilious in writing the docs and keeping shiny new code (however good) out of the FCL until it is properly documented. Someone who has poured countless hours into writing and correcting the existing docs - making that a priority - alongside writing much of the original code. Someone who is also good at collaborating with other core developers. Also FPC does not need to document or provide help for a complex IDE that is growing in features all the time.
I think the Lazarus team lacks an equivalent core person who combines coding expertise with the organisational, administrative and English/XML writing skills (and available time to marshal all the information/databases/people/wikis and so on) needed to produce good quality documentation, and good quality help. This is not to knock anyone - goodness knows, each does his best - but simply to point out what is true for so many open source collaborative efforts: you have to work with the people who offer and stay engaged for more than just the short term, and who make the time to contribute enough to make their contribution significant.
I hope that a new website might help us bring together the best of what is in the current wiki and has been written over the years in these mailing lists and the already published Lazarus book. A consolidation and editing exercise is needed (in addition to documenting the hitherto undocumented parts of the LCL), and I believe there are several people, myself included, who are ready to help with this task who have not done much towards it so far beyond tidying up the worst bits we stumble over in the wiki.

<snip>

Totally agreed.

Regards,
Reinier

What does my mind have to do with anything ?

You ask about practical examples. You explicitly accuse me of not being helpful in my descriptions. So, I explicitly tell you where the examples are located, when in fact there was no need to do so:

Why do you think I gave revision numbers in the announcement message ? (Of both the implementation and documentation, to boot...)

The answer is obvious:
So interested people could check out the relevant code and documentation and study them. Of course, I expect them to do so, and that includes you.

What more can I do ? Travel to Germany and hold your hand ?

Michael.

Do you mind explaining a bit what you have in mind? :)

Regards,
Sven

It's not really about discussion (fed up with that, if you can believe it ;) ), but about showing and collaborative editing of the docs.

The idea was/is the following:

* All nodes of an FPDoc description file can be put in a database; One record per element/module/whatnot, fields for short/descr/seealso and whatnot elements. I have the structure floating around here somewhere.

* FPDoc needs (or already has, don't remember?) a mode to generate a single HTML page, based on node name.

* The website can use the fpdoc engine to fetch and display the HTML 'on-demand'.

* For people with enough rights, the content can be edited.

* When creating the full docs, the XML is read from the database and written to an FPDoc file.

In fact, not so different from a WIKI, but more structured.

And now:
Since we now have notes, these can be displayed, and edited by all.

Michael.

Just sent the idea in a mail in response to Howards mail.

Michael.

Wow... this sounds nice O.O

This will allow to view and edit the documentation of the unfinished FCL units as well?

For users without enough rights it will more or less work like the current online HTML documentation just for trunk instead of the last release?

You will write the backend with FPWeb? :D

I'm looking forward to it! :)

Regards,
Sven

Sven Barth wrote

On 11.02.2012 20:35, Michael Van Canneyt wrote: > >

[snip]

It's actually a very old idea.

This will allow to view and edit the documentation of the unfinished FCL units as well?

Maybe.

For users without enough rights it will more or less work like the current online HTML documentation just for trunk instead of the last release?

More or less, I suppose.

You will write the backend with FPWeb? :D

Obviously :-)

We're getting to the point where all needed technologies are present and mature enough.

Michael.

Being old does not necessarily mean that it can't be nice ^^

If that would be possible it might speedup the completion of those units, so they can be included in the release documentation as well.

FPWeb is one of those gems of FPC that I definitely will need to spent more time with. :)

Regards,
Sven

Michael Van Canneyt wrote

Seeing the discussions about annotating the documentation created by FPDoc, I added support for notes. (revision 20304)

Thank you Michael, this should definitely come in handy for some people.

I also investigated this functionality further with INF and DocView. The INF specification has support for a "hide" function. These HIDE tags takes into account an environment variable in the OS, which tells the INF viewer if that content should be displayed or not. INF takes it one step further and even allows you to specify various "access levels" separated by a + sign (eg: set IPF_KEYS=LEVEL1+LEVEL2) to have even more fine-grained control over which notes are displayed or not.

So based on this, fpdoc notes could be displayed or hidden using a single INF file - simply by toggling the value of the IPF_KEYS environment variable.

I'll pull the latest fpdoc code and experiment with this functionality a bit more.

Not me ;-)

A better help system should allow to show or hide details based on more criteria. In Delphi and MS (.NET...) help it would be helpful to show the texts associated with a specific language. In FPC/Lazarus help the platform or widgetset details could be shown or hidden. In all cases the use case could be honored, like:
- user needing instructions for calling a method in code
- user having (known?) problems with a method
- component writer wants to override a method
- expert wants to fix or extend a method

All that can not be achieved with only a level of verbosity.

Please note that I don't ask for such a feature in FPDoc (right now). I only mention it in case somebody wants to implement a similar feature. An implementation could extend the <notes> tag with arbitrary conditions, interpreted in generating the docs, or later in a viewer. Kind of conditional compilation, guided e.g. by -dExpert etc. on the fpdoc (or viewer) commandline.

DoDi

Graeme Geldenhuys wrote

So based on this, fpdoc notes could be displayed or hidden using a single INF file - simply by toggling the value of the IPF_KEYS environment variable.

IMHO, multiple verbosity levels are a great idea and even more useful than just "Notes" that can be hidden. But I would vote for letting the user decide at any point in time to switch the verbosity level in the help viewer itself rather than use a static environment variable.

-Michael

Michael Schnell wrote

just "Notes" that can be hidden. But I would vote for letting the user decide at any point in time to switch the verbosity level in the help viewer itself rather than use a static environment variable.

That's already on my DocView todo list. :-)

But it can easily be achieved by adding attributes to each note, and reacting on these attributes. This is already implemented in revision 20335.

Things like

<note platform="win32"></note>
<note type="warning"></note>
<note author="websiteuser=laurence at arabia"></note> <note author="DoDi"></note>

Whatever. The attributes do not get in the way of the regular working of fpdoc.

You could also do something like
<note anchorpos="123"></note>
and have the note 'anchored' at position 123 in the text. (or a linenumber, it is just an example, or maybe an <link anchor=123> could be inserted in the text). This could serve as a hint for the output engine.

Since I do not know what attributes people would like to see supported, the mechanism currently always emits all notes.

People need to agree on:

1) Which attributes to support by default.
2) What values the attributes can take.
3) what to do if an attribute is absent.

As soon as an agreement is reached, I can implement the necessary logic to emit notes on a selective basis.

The documentation editors need then to be adapted to allow easy setting of the agreed attributes.

Michael.

Michael Van Canneyt wrote

> All that can not be achieved with only a level of verbosity. But it can easily be achieved by adding attributes to each note, and reacting on these attributes. This is already implemented in revision 20335.

Great stuff, but Greame stated that he is about to make the verbosity level (that already is supported by inf files) selectable on the DocView GUI. "note attributes" would nee new inf file support. I don't know if Graeme is inclined to see to that.

-Michael

Michael Schnell wrote

Great stuff, but Greame stated that he is about to make the verbosity level (that already is supported by inf files) selectable on the DocView GUI.

It is like Michael van Canneyt said, use attribute in the <NOTE> tag. I'll then modify the fpdoc IPF writer to output these notes and attributes as INF "hide" tags. The IPF_KEYS environment variable defines what attributes, verbosity levels or whatever else you want to call it, should be made visible in DocView. I'll also modify DocView so that you can modify the IPF_KEYS environment variable from inside DocView - for the duration of the running instance. The INF spec supports attributes for the HIDE tag, so they coincide nicely with the attributes Michael van Canneyt mentioned for the fpdoc NOTE tag.

IMO we have to agree in the first place about the *intended use* for the notes.

The first implementation looks like an "Errata" sheet, referring to the entire document.

The mentioned verbosity level suggests expandable parts in the docs, e.g. represented by "[more]" or "[+]" insertions.

A "footnote" like usage would come close to my needs, but should be accompanied by points-of-reference in the text, like "[1]". These may be implemented as clickable links, which lead to the note's text, or display the note in kind of a an overlaid hint window - depending on the viewer. Or the entire referenced word or phrase may be represented as a link, without additional insertions into the text - again heavily depending on viewer capabilites.

We should keep in mind how the notes can be represented in the final docs - linear formats have more restrictions than e.g. HTML pages. This is where I'd draw a line between Documentation (linear) and Help (random pages).

One solution could show the available attributes in a headline or menu bar, as currently implemented for "[index] ... (#lcl)", with the selected items being expanded when selected. Two such lines can be inserted, for common (agreed) attributes (like platform, widgetset), or document-specific attributes. The expansion can be implemented as a different page for every selected attribute, or by scripting features of the server or viewer. Scripting would allow to activate (expand) multiple attributes at the same time, without a need to store every single version as a new document.

For the editors I'd suggest multiple panes, for the original (smallest) content, and another pane for notes. A command "Insert Note" would place a link mark into the text, and connect it to whatever is written in the Notes pane. This also may be a solution for the HTML display of the final docs, where the Notes pane can reside in another window. This secondary window then can contain means to select the available and currently activated attribute(s). Different solutions may be required for small screens (of portables).

DoDi

This is exactly one of the intended uses.

Let us not forget that DOCView's INF format is just one of the possible output formats. Which means a FPDoc solution for notes must be generally applicable, and not taylored to the INF specs.

I'm sure Graeme will be one of the first people to implement support for attributes in INF as soon as a set of attributes is agreed upon.

Michael.

I fully understand that. I was simply describing how the new fpdoc functionality matches existing functionality of INF. A coincidence yes, but a nice one. :)

I'm sure Graeme will be one of the first people to implement support for attributes in INF as soon as a set of attributes is agreed upon.

Indeed. A lot of working is currently being done to DocView to improve it even more. This will just be another one of those improvements. ;-)

Great Stuff indeed !

So I think DocView finally is qualified as the most desired candidate for the default help viewer for Lazarus (not excluding that other help viewers might be selected instead or additionally).

-Michael

I'm sure you'll have wonderful discussions with the Lazarus team.

The first implementation looks like an "Errata" sheet, referring to the entire document.

If by "document" you mean package or module, I think you understood it wrong then.

It is implemented on a topic level. i.e you can associate notes with each <element>.

This means that for HTML, notes are emitted on the level of a single HTML page.

For linear, it is at the end of the current identifier section. [[sub]sub]section, in LaTeX.

The mentioned verbosity level suggests expandable parts in the docs, e.g. represented by "[more]" or "[+]" insertions.

[snip]

By the looks of it, you are going to have a field day with this.

Everybody is eagerly awaiting your patches for the documentation editors...

Michael.

Despite DocViews obvious intrinsic qualities:

I seriously doubt that, for the simple reason it is not implemented using the LCL.

Hence it cannot be compiled as part of Lazarus, and therefor not distributed either.

Michael.

Why me?

The document here is the entity, referred to in the notes.

It is implemented on a topic level. i.e you can associate notes with each <element>.

Then the entire <element> is the document/context.

Everybody is eagerly awaiting your patches for the documentation editors...

I thought that we are discussing the roadmap, before any (premature) implementation?

And before you ask for patches: please document the code to be patched, so that somebody else has a reasonable chance when putting his hands on it :-]

DoDi

Because I know where I want to go with FPDoc.
I've explained on the mailing lists what I still intend to do.

Contrary to what you seem to think, there was nothing premature about the implementation. I just executed a bit earlier what was on my todo anyway (cfr. the website idea, which is more than 7 years old), so the lazarus team can benefit from it.

The "notes" discussion on this mailing list highlighted the need, and I went ahead, that's all.

Now the lazarus team needs to decide how it intends to use the new feature. When a decision has been reached, I'll see how I can help (if any help is still needed).

Michael.

Now the lazarus team needs to decide how it intends to use the new feature. When a decision has been reached, I'll see how I can help (if any help is still needed).

This reminds me on: Here's the solution, where's the problem?

DoDi

i wanted to also add to this that, as i mentioned to another participant in this discussion, the original notes were (apparently?) a problem for one developer, and the notes were (apparently?) intended to lead others to update, correct, or add to the documentation but (and here's my question) what about the possibility of "local" notes... notes that are only stored on the local machine?

in other words, there are notes, like the "[?]" and "[what?]" ones that were removed, which appear to have been done for publishing to all so that the docs could be fixed up... so, i was also thinking that those notes might have been for "local" purposes so that the one who created them would have something as a reminder to go back to for the completion...

so, basically, i'm working on the documentation and write a "local" note that only belongs on my machine so that it doesn't "pollute" the documentation... then i would find the answer and complete the documentation or alter the note so that it is no longer limited to my local machine and thus would bring the situation to others' attention...

is this something that would be beneficial or no? i can see where the notes would be stored in possibly additional files that either traveled with the documentation files or were included in those files for all to see (if configured for such)... i really likes graeme's description of how he uses annotations and the like but i have not yet gotten to the point where i can see or use his method(s) but the question still came to my mind about local-only annotations, notes, updates, etc... and how they might be handled or if this possibility was even considered...

It is more 'I had this planned, it can also be used to solve your problem'.

But if it's not wanted, hey, suit yourself...

Michael.

waldo kitty wrote

so, basically, i'm working on the documentation and write a "local" note that only belongs on my machine so that it doesn't "pollute" the documentation...

This is exactly how DocView's annotations work, and this is exactly how I improve FPC and fpGUI documentation. I don't always have time to immediately edit the XML documentation files (because I'm busy with another task). But I often see a spelling or grammar mistake in the docs, or note that something could be extended. I then quickly add an annotation using DocView (seeing that it is already open and viewing the topic in question). I can then later revisit all these annotations and fix the real documentation.

As I mentioned before, I also use annotation for adding other personal notes to extend the existing documentation so it makes it better for myself - those are personal notes.

All annotations in DocView are stored in an external *.notes file that travels with the *.inf files. I normally keep my *.inf files on a USB flash drive because that always travels with me from computer to computer. So my help files and personal notes travel with me.

Graeme Geldenhuys wrote

On 14 February 2012 04:54, waldo kitty wrote: > so, basically, i'm working on the documentation and write a "local" note > that only belongs on my machine so that it doesn't "pollute" the > documentation...

This is something that does not belong in FPDoc.
Such functionality belongs in the viewer one uses for the output of FPDoc.

PDF readers (okular and acrobat) have the same functionality.

Michael.

<michael.van.....> wrote

It is more 'I had this planned, it can also be used to solve your problem'. But if it's not wanted, hey, suit yourself...

+1

@DoDi
No offence, but Michael is trying to be accommodating here. But with every since code change he makes, you seem to find more and more "problems" (those are debatable). There is a saying in South Africa: "Don't make a mountain out of a molehill".

Michael Van Canneyt wrote

Despite DocViews obvious intrinsic qualities: I seriously doubt that, for the simple reason it is not implemented using the LCL.

Is the CHM viewer based on the LCL ?

I don't know if/how it is possible to create an installable Lazarus Package as a wrapper for (non-LCL) DocView to allow for using it as a help viewer decently integrated with the IDE.

-Michael

Yes, it is.

I don't know if/how it is possible to create an installable Lazarus Package as a wrapper for (non-LCL) DocView to allow for using it as a help viewer decently integrated with the IDE.

It should be possible as the package for the CHM viewer is basically doing the same (the lhelp application is independant of the IDE). The only thing needed for DocView is the possibility to communicate using IPC (@Graeme?).

Please note: it's not a technical problem that DocView is not using the LCL, but a distribution problem.

Regards,
Sven

Michael Schnell wrote

> Despite DocViews obvious intrinsic qualities: > > I seriously doubt that, for the simple reason it is not implemented using > the LCL.

I don't really see this as a problem. After all, you need to download the INF or CHM or HTML or PDF offline help anyway. Not many people want to go the "build your own docs" route. A DocView binary is available for download (a small download at that), just like the pre-generated documentation.

Is the CHM viewer based on the LCL ?

LHelp is, but in its current state I would not call it a help viewer. It can only display CHM files (barely), but not much else.

Either way, it seems the conversation is dying down and the Lazarus Developers once again can't make a clear cut decision on what help format or help viewer to use. I guess you all are stuck with the WIKI help for the next 10 years. Good luck with that.

Sven Barth wrote

The only thing needed for DocView is the possibility to communicate using IPC (@Graeme?).

That is already on my docview todo list, and should be one of the next things I implement (in a week or two).

Graeme Geldenhuys wrote

>> I seriously doubt that, for the simple reason it is not implemented using >> the LCL. I don't really see this as a problem.

But it is the main issue.

That's the current state but will not be like that in the next release. The next release will come with everything for offline LCL help pre-configured, including LHelp pre-compiled.

Either way, it seems the conversation is dying down and the Lazarus Developers once again can't make a clear cut decision on what help format or help viewer to use. I guess you all are stuck with the WIKI help for the next 10 years. Good luck with that.

It will be CHM + LHelp. No other option is know to me.

Graeme Geldenhuys wrote

That is already on my docview todo list, and should be one of the next things I implement (in a week or two).

Sounds Great ! Looking forward to testing this.

-Michael

No offence, but Michael is trying to be accommodating here.

Appreciated :-)

But with every since code change he makes, you seem to find more and more "problems" (those are debatable).

What problems?

I only tried to give some *possible* use cases for an intended feature. It should not be understood as a wish list.

DoDi

Graeme Geldenhuys wrote

I don't really see this as a problem. After all, you need to download the INF or CHM or HTML or PDF offline help anyway.

The "distribution" (version 1.0) might include those.

Not many people want to go the "build your own docs" route.

But please don't forget them ! A dummy-aware way to create all of them is necessary on the svn.

A DocView binary is available for download (a small download at that), just like the pre-generated documentation.

Hmm. I would prefer a complete "version 1.0" distribution including an LCL binary with a pre-installed offline help viewer (including an external binary if necessary) and including the necessary package file(s) that allow recompiling the IDE and for installing a different help viewer if the user likes and to re-install the default help viewer.

Regarding you last lines: I feel that for "version 1.0", a decent offline help viewer *needs* to be included in the user-distribution, and as this discussion showed, DocView is the only candidate that is (1) versatile enough, (2) has powerful supporter(s) and (3) can be brought in a form valid for distribution in an appropriately short amount of time.

-Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20120214/fef27976/attachment.html>

question) what about the possibility of "local" notes... notes that are only stored on the local machine?

You can do with the downloaded doc sources whatever you like.

Create and supply an patch, then your update may (or may not) be added to the official docs.

DoDi