I've been working on writing Javascript to generate lexical language diagrams for the getlazarus.org learning center portal.

With my script I'm able to render these diagrams which will be appear under the Free Pascal language Lexical topic:

http://cache.getlazarus.org/images/lexical-diagrams.png

Hopefully they are clear enough in their schematic design such they're understandable. I'll be adding code examples and brief summaries to each diagram which will likely be of aide.

Also, I am considering rendering hover over tooltips with popup lexical diagrams on pages similar to the Language overview page.

http://www.getlazarus.org/learn/language/overview/

Anthony Walter wrote

I've been working on writing Javascript to generate lexical language diagrams for the getlazarus.org learning center portal. With my script I'm able to render these diagrams which will be appear under the Free Pascal language Lexical topic: http://cache.getlazarus.org/images/lexical-diagrams.png Hopefully they are clear enough in their schematic design such they're understandable. I'll be adding code examples and brief summaries to each diagram which will likely be of aide.

The look of the diagrams is very good.

Some things that I think can be improved:

1.) Direction of lines:
The lines should have arrows to denote the direction in which a diagram has to be followed. This would allow recursive definitions. Example: http://i.stack.imgur.com/SOq3x.gif
It would also avoid the somewhat irritating use of words like [repeat] and [open symbol] which are only needed because of the missing arrows.

2.) \n
I think \n is not used in Pascal (it's more a C convention). Instead it should be named "line ending" or something like that. Also, especially new users may not know what \n means (unless they come from C).

3.) String literal:
Why does (char literal) appear twice? There can be any number of (char literals) so I think it would be clearer to have a simple recursive loop:

/>-'[any character other than "line ending"]'->-|
->->-------------(char literal)----------------->-|->- \----------------------<------------------------/

4.) Comment:
There should only be simply 3 kinds of comments:
/->-(//)----[line ending]->-\
->--->-({)---------------(})->-|->-
\->-((*)--------------(*))--/

5.) Identifier:
Same here, instead of the [repeat] word arrows should be used.

I'll consider arrow heads, though I'm unlikely to add them. My layout design is based on the JSON http://json.org/ style with which I believe plenty of people are familiar.

Here's an updated diagram with some of the changes you've noted including some new additions and thinner lines and fonts.

http://cache.getlazarus.org/images/lexical-diagrams-2.png

Nice.

Which tool do you use to make these charts? :-)

A finalization section can exist without initialization.

Mattias

What's wrong with those found in the documentation? (for example: http://www.freepascal.org/docs-html/ref/refse105.html ) IMO those provide more clarity.

Mattias,

Noted. Thank you.

Ewald,

There was/is a big discussion on the forums about the terrible state of the documentation. I already am hosting easy to setup bundles (for example my OSX bundle installed CodeTools and offers you the choice between Homebrew and Macports) which include Free Pascal terminal shortcuts (perfect for following along with the language guide examples). I also host a much improved documentation portal, a few instructional videos (more to come in the tutorials section), and in my opinion aesthetically pleasing page layouts with breadcrumb navigation (perfect for navigating nested topics).

With this in mind, given my penchant for design, I believe hosting new hyperlinked language and IDE guides is a natural progression. When I add in brief summaries to go along with each diagram as well as syntax highlighted examples I think there will be even more clarity. And I neglected to mentioned that diagram lexical sub paths can hyperlink link to other diagrams or page anchors. That is, on the actual pages users receive visual feedback for certain diagram elements (hover over effects) to indicate they hyperlink to other diagrams.

I summary, what I am producing will be an improvement, again in my opinion, over other disparate resources.

Anthony Walter wrote

There was/is a big discussion on the forums about the terrible state of the documentation.

Two things:
- I don't think the state of the documentation is terrible. It has in fact improved vastly over the years.
- By creating a secondary set of documentation the "terrible" state won't improve, now will it?

Ewald,

If you read the threads on the Lazarus forums you'll see the points people are making while complaining about the documentation, tutorials, and application developer guides, including complaints from some of the top contributors. In a nutshell it has been said that the wikimedia engine is bad for these things. One complaint is that due to it's everyone can edit/create pages there is a general lack of cohesion. Many people have vastly different writing skills, topics are not ell organized, contributors and moderators spend time fighting vandalism or bickering rather than on creating quality content, there are not enough people in out community interested in taking on their work of maintain a decent wiki. Many examples were given, and I personally prefer a nested structure with folders which the wikimedia engine does not provide.

If you are okay with the state of everything then move along.

However if you feel like addressing the issues people are raising, then I believe the points brought up in my last post bring forth one possible balanced solution. And yes, if treated comprehensively the terrible state will be improved.

Contributions to the documentation have been scarce. It is basically a 1-man job for the last 20(!) years.

Writing coherent and comprehensive documentation is a huge effort.

Making it look nice is simply beside the point.
So the proposal of Walter (with all due respect for the efforts made) is just a waste of Walter's time. It is again a scattering of information.

Although well-meant, the current effort is not factually helping. The effort would be better spent giving feedback on the current documentation, so it can be improved where it is lacking. I have never refused critique or ignored remarks.

I realize it may come over as not very nice on my part, or even vindictive, seeing that I am the author, but after more than 20 years I have become very sceptical.

Michael.

Instead of discussing, the time would be better spent on reporting issues about the documentation. I always respond very quickly to documentation issues.

Michael.

Wikimedia is NOT documentation. It is a knowledgebase.

Michael.

Anthony Walter wrote

Ewald, If you read the threads on the Lazarus forums

I'm not subscribed to that forum as I do not use lazarus. I use freepascal without an IDE. So by all means: enlighten me :-)

You are talking about the wiki here. I was talking about the programmers guide and language reference. The names may be off, but if you would have clicked that link I sent in my previous message, you would have noticed that that page is no wiki at all.

If you are okay with the state of everything then move along.

Read the pages. What is wrong with them? If something is wrong, then
file a bug!

Now, since something terrible has gone wrong in pointing you to the two documents I am talking about, I'll walk through the procedure with thee:
- Go to www.freepascal.org
- Click documentation (6th entry under general)
- Select one of the docuemts, humour me and pick the language reference.
- You are now here:
http://www.freepascal.org/docs-html/current/ref/ref.html

Now tell me (or file a report), what is so "terrible" about this?

Michael,

My portal is titled 'Learning Center'.

Regarding the label Documenation, I am following along with this topic:

Documentation / tutorials need improvement (was: Object Pascal decline?) http://forum.lazarus.freepascal.org/index.php/topic,31272.0.html

Where most people are referencing the wiki, as that's the place where they feel tutorials and guides should be, but in the ends it seems quite disparate. This entry was given as an example in the prior topic:

http://wiki.freepascal.org/Pascal_Tutorial

Ewald,

Maybe you should go to the forums and enlighten everyone. I provided the link in my last reply.

Anthony Walter wrote

Ewald, Maybe you should go to the forums and enlighten everyone. I provided the link in my last reply.

You missed the point. It is not my intention to shove my opinion down anyone's throat.

I do however think that reinventing documentation (with all the bugs related to new projects) is a waste of time. Especially if the only thing that changes is the form factor. It is the content that matters. That content already exists, but needs some work here and there. So I am trying to convince you to collaborate to the existing documentation instead of writing your own. Because your own documentation will also be a one-man project, will also have its flaws and then in about five years a similar thread on some forum I'll probably not be subscribed to will state that the getlazarus documentation is in a terrible state. And voila, same discussion all over again.

All that effort down the drain. And what for?

I can only concur with what Ewald writes.

Michael.

On Sat, Anthony Walter wrote:
[...]

[...]

Found <https://github.com/tabatkins/railroad-diagrams>.

Sorry for ask! ^^'

Nice. Added to my bookmarks :-)
I may actually use this to redo the official ones as PNG images.

Michael.

On Sat, Jan 30, 2016 at 5:23 PM, Michael Van Canneyt <

Me too. :-)

I didn't know that this tool existed. I'll use this one to generate a graph of my basic properties lexer.

Thanks Anthony for share that picture. :-)

Without wanting to go too far OT, I wonder whether I could make a couple of observations.

When I was with Burroughs in the early 80s, these things were referred to internally as "railroad diagrams". However,
http://www.ianjoyner.name/Files/Waychoff.pdf (page 8) tells the story of the invention (by Burroughs) of this sort of illustration, and while much of that document is suspect I think that bit can be believed because some of it involves ACM publications etc.

Waychoff refers to them as "syntax charts", Perlis described the ALGOL one as "The ALGOL Roadmap" and it was in "Communications Of The ACM [...] September, 1961. It was undoubtedly the first centerfold for any magazine other than “Playboy”".

So hopefully we can drop the nasty "railroad" Americanism, since it wasn't the original name. And hopefully we can use "syntax" in the name, since that's what they describe. Hence "syntax chart", "syntax diagram" or something similar. Please :-)

Michael Van Canneyt wrote

Wikimedia is NOT documentation. It is a knowledgebase.

+1000

The Lazarus project just doesn't seem to get that. The Lazarus wiki has so much broken or outdated information, no offline support, no documentation per Lazarus release. The list off "broken" things are getting longer and longer.

Regards,
- Graeme -

Ewald wrote

So I am trying to convince you to collaborate to the existing documentation instead of writing your own. Because your own documentation will also be a one-man project,

+1

And he will be 15+ years behind Michael van Canneyt's efforts.

Regards,
- Graeme -

Where I live, "syntax diagrams" make much more sense. And online image search engines also seem to agree with that

Jürgen Hestermann wrote

The look of the diagrams is very good.

Call me old-fashioned, but what is wrong with the EBNF (Extended Backus-Naur Form) which is also an ISO standard.

https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_Form

I think the EBNF syntax is crystal clear to read and follow.

Here is an extract from my "Object Pascal Grammar" addendum to the official FPC Language Reference documentation:

===========================================================
Number -> [ "-" ], digit, { digit } ;
String ->  "'" , { all characters - "'" }, "'" ;

alphabetic character -> "A" | "B" | "C" | "D" | "E" | "F" | "G" | "H" | "I" | "J" | "K" | "L" | "M" | "N"
| "O" | "P" | "Q" | "R" | "S" | "T" | "U"

                      | "V" | "W" | "X" | "Y" | "Z" ;

digit ->  "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" ;
all characters ->  ? all visible characters ? ;

===========================================================

Regards,
- Graeme -

My public PGP key: http://tinyurl.com/graeme-pgp

[Badly-formatted stuff dropped]

Broadly speaking, I sympathise with your position. EBNF particularly makes more sense to people who for some reason need to know how to write a compiler for the target language.

However, Wirth used syntax diagrams in his description of Pascal, and didn't move to EBNF until later. As such it's part of the doctrine of the language, and discarding it would in effect be breaking backward compatibility.

http://www.standardpascal.org/The_Programming_Language_Pascal_1973.pdf

If you feel strongly about it I suggest doing the community a service by porting one of the diagrammers to Pascal, and then making sure that it can process your EBNF into diagrams suitable for the documentation. That would avoid having multiple language descriptions in different forms, which is obviously a hazard.

On Mon, Feb 1, 2016 at 4:32 AM, Graeme Geldenhuys <

Could you please categorize any page as "outdated"?

So later any page within the category could be considered for either update or removal.

thanks,
Dmitry

Dmitry Boyarintsev wrote

Could you please categorize any page as "outdated"?

Is there a "outdated" category? If not, I don't know how to add new categories.

So later any page within the category could be considered for either update or removal.

The problem is that some pages are completely out of date, others have out of date images (eg: screenshots of lazaras dialogs), and others might have a section of text out of date.

Do all those get categorised as out of date, or should only whole pages be marked as such? If the latter, that might help to some extent, but would be far from a real solution.

The bottom line is, Lazarus needs an actual documentation solution (per release, like FPC, fpGUI etc) - the wiki is not, and never will be that.

For example:

http://fpgui.sourceforge.net/docs-1.4/
http://fpgui.sourceforge.net/docs-1.0/
http://fpgui.sourceforge.net/docs-0.8/
http://fpgui.sourceforge.net/docs-0.4/

http://sourceforge.net/projects/freepascal/files/Documentation/3.0.0/ http://sourceforge.net/projects/freepascal/files/Documentation/2.6.4/ http://sourceforge.net/projects/freepascal/files/Documentation/2.6.0/

Though FPC doesn't have all previous releases available online (viewable via web browser), you can at least download PDF and HTML archives.

Regards,
- Graeme -

My public PGP key: http://tinyurl.com/graeme-pgp

On Mon, Feb 1, 2016 at 10:19 AM, Graeme Geldenhuys <

The category is there now.
In order to categorize a page just edit it and add

[[Category:Outdated]]

to the end of the page.

The problem is that some pages are completely out of date, others have out of date images (eg: screenshots of lazaras dialogs), and others might have a section of text out of date.

Potentially, images could be categorized as well. Once found, it won't be forgotten.

Do all those get categorised as out of date, or should only whole pages be marked as such? If the latter, that might help to some extent, but would be far from a real solution.

It's better to mark, if the whole page is outdated. Maybe create "Partial Outdated" category for the pages where only part of information is not up to date.

The bottom line is, Lazarus needs an actual documentation solution (per release, like FPC, fpGUI etc) - the wiki is not, and never will be that.

There're no releases in knowledge base. Live and learn :)

thanks,
Dmitry

Anthony Walter wrote

I've been working on writing Javascript to generate lexical language diagrams

Would this be of any interest? (Rather than reinvent the wheel.) http://www.bottlecaps.de/rr/ui

Peter

Peter wrote

Would this be of any interest? (Rather than reinvent the wheel.) http://www.bottlecaps.de/rr/ui

Very nice, thanks for sharing.

Regards,
- Graeme -

Michael Van Canneyt wrote

I can only concur with what Ewald writes.

Are the Latex sources for the documenation
available somewhere for download?

I would like to check why the syntax diagrams
created with the syntax.sty
are incorrect and how they can be improved.

They are in svn.

I would like to check why the syntax diagrams created with the syntax.sty are incorrect and how they can be improved.

Feel free to do so.
You can also simply submit bugs to the bugtracker, I usually fix documentation bugs ASAP.

Michael.