Hello,

I'm planning to write three parsers, and googling, I found some entries talking about lexical parsers.

After that, I did a 'find in files' in FPC sources, and I found many parsers (eg: jsonparser (jsonscanner), JSParser (JSScanner), fpsqlparser (fpsqlscanner), PParser (PScanner), fpexprpars etc.) that use lexical scanner.

Below, three possible string that I need to parse:

1)

${someVariable} -- 4 tokens

or

${a + b} -- 8 tokens - 1 expression

or

${fn:lenght('abc') * 3} -- 11 tokens - 1 function - 1 expression

2)

<c:forEach var="contact" items="${contactDao.list}"> ${contato.name}, ${contato.email}
</c:forEach>

3)

contacts[0].name=abc
contacts[0].email=abc at def
contacts[1].name=def
contacts[2].email=def at ghi

So my parser will allow to register dynamic variables, functions (to be called via script) and plugins (to extend the parser).

However, I have a question: is there any article about 'how to write lexical parsers' using Object Pascal?

I need any material about this subject, and I'm very grateful for any tip.

Thanks in advance!

This is a classic series of articles that show how to write a very simple compiler in Turbo Pascal. The fundamentals when it comes to scanning are the same:

http://compilers.iecc.com/crenshaw/

I've also written a BASIC implementation for Free Pascal and Lazarus. The scanner should be straightforward to understand:

http://runtimelegend.com/rep/rbasic/artifact/2350e85c36a77e4d2d76adde23fd7d45731b5b22

The compiler code shows how it can be used:
http://runtimelegend.com/rep/rbasic/artifact/93859f52fd424edfc1e0d5dfd16a92ed8ac04855

But you may also find the formatter code simpler. Although it is a bit too simple:
http://runtimelegend.com/rep/rbasic/artifact/f3e9fb2d1ed8e60d36b50754c2d9a7d7c109fc40

For general theory you can look on recursive descent parsers (they're the simplest to implement and AFAIK most compilers use them, either to build the token list or directly).

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20150306/f76a6548/attachment.html>

Hi Silvio,

About this parsing subject maybe looking at BeniBela Xidel and InternetTools help you or talking to Benito Aurthur of them. He is a very good developer and kind person like you.
http://videlibri.sourceforge.net/xidel.html
http://www.benibela.de/sources_en.html#internettools

Ara

Here is something I originally wrote in 2001. I had a product briefly for converting pascal code into documentation and it quite fast. Do with it as you will.

http://pastebin.com/qXJdHwGM
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20150306/12d71d9e/attachment.html>

On Fri, Mar 6, 2015 at 3:15 PM, Kostas Michalopoulos <

I downloaded the PDF with all articles.

Thank you very much!

I think that it uses regex as parser. I'll check when I have a free time. Thank you!

Nice parser. I had some ideas after seeing your code. Thank you!

However, I have a question: is there any article about 'how to write

lexical parsers' using Object Pascal?

First, you need to differentiate them correctly. Lexical scanner (or simply lexer) and parser, not lexical parser :)

I've written some simple languages, generating GraphViz, LLVM or C code, using both self learning reference from the almighty Prof. Niklaus Wirth's articles, Jack Crenshaw's, the legendary dragon book and many others plus my compiler course in college.

I really really suggest Prof. Niklaus Wirth's one
(http://www.inf.ethz.ch/personal/wirth/, click the Compiler Construction link) as it's concise, simple, doesn't use stupid generator (i.e.: easy to implement by hand) while covering most important parts of the subject. While he didn't use or stress or even discourage the usage of abstract syntax tree, he did explain it that's good enough to understand.

Here's what I've written in the past and is publicly available:
- https://bitbucket.org/leledumbo/erd-maker (generates GraphViz code for ERD visualization)
- https://bitbucket.org/leledumbo/linguc (generates LLVM code that can be compiled as a library for a mathematically provable language focusing in database driven application)

I don't use the direct code generation approach anymore as I found the abstract syntax tree to be usable for more than just code generation.

When there is NO license information in your repositories , this means that

"NO one can use them ."

with respect to copy right laws or conventions .

Thank you very much .

Mehmet Erol Sanliturk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20150311/ab39cfc1/attachment.html>

Mehmet Erol Sanliturk wrote

[...]

Pointer/Keyword/URL please? I presume for when it matters, one would ask the author first...?

el es

No idea with what's default by law or convention, but when I give no license, consider it public domain.

leledumbo wrote

No idea with what's default by law or convention, but when I give no license, consider it public domain.

By default author has copyright and sole right to redistribute. And that's what applies, when nothing else is declared.

I'd imagine that application of the law would vary depending on jurisdiction, and it might be unintentionally affected by the stated policy of wherever the files are posted.

However the /correct/ thing to do would be to look at the import list and to adopt whatever license is implied by the libraries and documentation sources that are referenced. If a body of code imports (only) libraries which conform to any of the recognised free (as in speech) licenses, then an unqualified "no one can use them" is not a reasonable conclusion.

Yes , the correct understanding is the above sentence . Without explicit declaration , nothing can be assumed "implicitly" that is permitted , i.e. , permission requires to be explicit .

Mehmet Erol Sanliturk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20150311/74fa8d10/attachment.html>

http://en.wikipedia.org/wiki/Copyright
http://en.wikipedia.org/wiki/Copyright_infringement http://en.wikipedia.org/wiki/Public_domain

http://en.wikipedia.org/wiki/Software_license
http://en.wikipedia.org/wiki/Public_domain_software http://en.wikipedia.org/wiki/Software_copyright
http://en.wikipedia.org/wiki/License-free_software

"Personal" use is a different concept , because no one may know that a copy is used whether usage is permitted or not for a copyrighted work .

Problem arises when "Redistribution" or ( "Derivative" works are "Published" ) .

One is the following :

http://www.acm.org/publications/policies/softwarecrnotice

Some of these collected software published before a year in US , is in "Public Domain" in the US with respect to a law ( I do not remember the year exactly but it is around 1970 ) .
ACM has denied this "Public Domain" concept by countering it with "Copyright Law" :

More restrictive rules are effective even they ( restrictive and permissive ) are in the laws for the copyright holder .

In summary , with respect to copyright laws ( there are minor differences in countries as related to Bern Convention agreement exceptions adopted ) anything is not explicitly mentioned is assumed to be not permitted .

Mehmet Erol Sanliturk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20150311/01bb1ea2/attachment-0001.html>

Buddy, thanks alot for that information. I'll take a look at this! =)