Sign in Join
Wiki copytemporary

Gold: what changed

Revision 0 to 1. Removed lines have a minus sign; added lines have a plus sign.

Title: Gold

OldNewLine
1+Gold is a free parsing system compatible with Free Pascal that you can use to develop your own programming languages, scripting languages, interpreters, and all kind of parsers and expression analyzers. It uses [LALR](https://en.wikipedia.org/wiki/LALR_parser) parsing, and a mix of [BNF](https://en.wikipedia.org/wiki/Backus–Naur_form) notation, character sets and regular expressions for terminals to define language grammars. Code and grammar are separated, so grammar is not tied to implementation language. This means that the same grammar can be loaded into engines made in different programming languages.
2+
3+There is a subjective [feature comparison table](http://goldparser.org/about/comparison-parsers.htm) of several parsers on Gold site, with special attention to [Gold vs Yacc](http://goldparser.org/about/comparison-yacc.htm) comparison.
4+
5+# Gold Parser Builder
6+
7+[GOLD Parser Builder](http://goldparser.org/) can be used to create, modify and test languages in Windows IDE (runs on Wine, too). Command line tools are also available.
8+
9+Features:
10+
11+- Grammar editor with syntax highlighting.
12+- Grammar generating wizard.
13+- Test window to step through parsing of a sample source.
14+- Templating system that can generate lexers/parsers or skeleton programs for various languages (including Delphi and FreePascal).
15+- Importing/exporting YACC/Bison format. Exporting XML and HTML format.
16+- Interactive inspection of the compiled DFA and LALR tables.
17+
18+Importing YACC/BISON can be very picky, so you will probably have to convert linux line endings and delete most of the not needed code leaving just BNF rules. Import from lex file does not exist, so you will have to examine lex file yourself to see if there are terminals that you should manually add to your GOLD grammar.
19+
20+# Gold Engines
21+
22+Gold Parser Builder has an option to build skeleton application for Delphi and FreePascal from existing grammar. However, there are also user made Gold Engines for FreePascal/Lazarus which can load compiled grammar (CGT v1.0 or EGT v5.0 file) and parse some file according to loaded grammar.
23+
24+Gold Parser Builder has changed CGT file format in version 5 to EGT. EGT v5.0 file format is used for all Gold Parser Builder versions newer then 5.0.0 (including 5.2.0 which is the actual version at the moment of writing this text), and CGT v1.0 for all older versions. This means that with older engines you will need to use older Gold Parser Builder. Although Gold parser Builder can still save CGT v1.0 file format, engines will sometime have trouble with it, so it's best to use older Gold Parser Builder if you need CGT v1.0. You can find old v3.4.4 version [here](http://www.semitwist.com/download/gold-parser-builder-3-4-4.zip).
25+
26+## Engines for FreePascal and Lazarus
27+
28+There are several Gold Engines for FreePascal and Lazarus, and one of them supports latest Gold EGT v5.0 file format.
29+
30+### Lazarus Gold Engine 2018
31+
32+This is a refactored cross platform Delphi engine used for JSON parsing example. It supports only CGT v1.0 file format so you need to use older Gold Parser Builder with it. You can find it [here](http://forum.lazarus.freepascal.org/index.php/topic,33830.msg295876.html#msg295876).
33+
34+### Gold Parser for Lazarus
35+
36+Another adaptation of the same Delphi engine for Lazarus. It's a little more commented then the 2018 version so it might be more appealing to the ones who want to understand the code. It also supports only CGT v1.0 file format so you need to use older Gold Parser Builder. You can find the engine [here](https://github.com/cgarciagl/goldparser-Lazarus).
37+
38+### Gold Parser for Free Pascal and Lazarus
39+
40+There are 2 versions of this engine. One derived from the same old Delphi engine compatible only with CGT v1.0, and the new one derived from Java which is compatible with both CGT v1.0 and EGT v5.0. This means that it will load grammars compiled with latest version of Gold Parser Builder (5.2.0 at the moment of writing). This engine was used for Pascal parsing example. You can find it [here](https://gitlab.com/teo-tsirpanis-718/gold-parser-lazarus).
41+
42+### FPC Gold Engine with VS Code parser plugin and code generator
43+
44+[FPC Gold Engine](https://github.com/Warfley/FPCGoldEngine) supports both CGT v1.0 and EGT v5.0, has very interesting [VS Code extension](https://github.com/Warfley/gold-parser-tools) to parse and debug grammars, and to generate Pascal and TypeScript code out of grammars. Some additional documentation is available [here](https://github.com/Warfley/GoldEngines/blob/master/docs/index.md).
45+
46+## Engines Usage Examples
47+
48+A lot of grammars found on the net are pretty old and many can not compile out of the box with recent versions of Gold Parser Builder. Therefore you can find here 2 examples which compile fine. These examples are recommended for your initial testing. They are just meant to show you how to parse sample text or code with a grammar, and will not go further then that.
49+
50+### JSON example
51+
52+You should copy/paste simple JSON grammar from below into 'Grammar Window' of Gold Parser Builder 3.4.4, save the grammar as 'json.grm" file, press 'Continue' button in the bottom right 3 times in a row, the press it 4th time and in file browser save compiled grammar as 'json.cgt' file. Now copy/paste JSON test code from below into 'Test Grammar' window of Gold Parser Builder 3.4.4, press 'Start' button, and then press 'Parse All' button. There should be no errors and you should see the populated 'Parse Tree' tab at the bottom of your 'Test Grammar' window. You will find nice parsing steps info in 'Parse Actions' tab, and you can even execute step by step if you use 'Step' button instead of the mentioned 'Parse All' button.
53+
54+After testing is finished, we load project GOLDParserEngineSource from *Lazarus Gold Engine 2018* into Lazarus, open Project Inspector to add 'sparta_generics' package as a new requirement, compile and start the executable, load file 'json.cgt' into 'Test Input' memo field, paste JSON Test Code into memo, and press 'Parse' button to start the parser. If everything went fine then you should see the memo populated with parsed tree represented in textual form. Of course, instead of showing parsed tree you can do whatever you want. You can implement semantic checks, emit code for template based code generator, interpret tree...
55+
56+JSON screenshots from below show what you should expect to see.
57+
58+#### JSON Grammar
59+
60+```bnf
61+"Name" = JSON Grammar
62+"Author" = Ars ne von Wyss
63+"Version" = 1.0
64+"About" = 'Grammar for JSON data, following http://www.json.org/'
65+! and compliant with http://www.ietf.org/rfc/rfc4627
66+
67+"Start Symbol" = <Json>
68+"Case Sensitive" = True
69+"Character Mapping" = 'Unicode'
70+
71+! ------------------------------------------------- Sets
72+
73+! Uncomment next line only on Gold 5.x.x and newer when running in new EGT v5.0 engine:
74+! {Unescaped} = {All Valid} - {&1 .. &19} - ["\]
75+! Older engines do not handle correctly {All Valid} with Unicode character mapping,
76+! so we will replace {All Valid} with {Printable}. Just have in mind that mentioned
77+! RFC4627 compliance is therefore lost.
78+
79+{Unescaped} = {Printable} - {&1 .. &19} - ["\] ! compatible with old CGT v1.0 engines
80+{Hex} = {Digit} + [ABCDEFabcdef]
81+{Digit9} = {Digit} - [0]
82+
83+! ------------------------------------------------- Terminals
84+
85+Number = '-'?('0'|{Digit9}{Digit}*)('.'{Digit}+)?([Ee][+-]?{Digit}+)?
86+String = ["]({Unescaped}|'\'(["\/bfnrt]|'u'{Hex}{Hex}{Hex}{Hex}))*["]
87+
88+! ------------------------------------------------- Rules
89+
90+<Json> ::= <Object>
91+ | <Array>
92+
93+<Object> ::= '{' '}'
94+ | '{' <Members> '}'
95+
96+<Members> ::= <Pair>
97+ | <Pair> ',' <Members>
98+
99+<Pair> ::= String ':' <Value>
100+
101+<Array> ::= '[' ']'
102+ | '[' <Elements> ']'
103+
104+<Elements> ::= <Value>
105+ | <Value> ',' <Elements>
106+
107+<Value> ::= String
108+ | Number
109+ | <Object>
110+ | <Array>
111+ | true
112+ | false
113+ | null
114+```
115+
116+#### JSON Test Code
117+
118+``` javascript
119+{"widget": {
120+ "debug": "on",
121+ "window": {
122+ "title": "Sample Konfabulator Widget",
123+ "name": "main_window",
124+ "width": 500,
125+ "height": 500
126+ },
127+ "image": {
128+ "src": "Images/Sun.png",
129+ "name": "sun1",
130+ "hOffset": 250,
131+ "vOffset": 250,
132+ "alignment": "center"
133+ },
134+ "text": {
135+ "data": "Click Here",
136+ "size": 36,
137+ "style": "bold",
138+ "name": "text1",
139+ "hOffset": 250,
140+ "vOffset": 100,
141+ "alignment": "center",
142+ "onMouseUp": "sun1.opacity = (sun1.opacity / 100) * 90;"
143+ }
144+}}
145+```
146+
147+#### JSON screenshots
148+
149+<img src="Gold_3.4.4_Json_Example.png" title="JSON example in Gold 3.4.4" width="640" alt="JSON example in Gold 3.4.4" />       <img src="GoldEngine2018_CGT_v1.0_parsed_Json_example.png" title="JSON example parsed with CGT v1.0 GoldEngine2018" width="240" alt="JSON example parsed with CGT v1.0 GoldEngine2018" />
150+
151+### Pascal example
152+
153+You should copy/paste Pascal grammar from below into 'Grammar' window of Gold Parser Builder 5.2.0, save the grammar as 'pascal.grm" file, press 'Next' button in the bottom right 3 times in a row, then press it 4th time and save compiled grammar as 'pascal.egt' file. Now copy/paste Pascal test code from below into 'Test Grammar' window of Gold Parser Builder 5.2.0, and press button with fast forward symbol icon (that's 'Parse All' button). There should be no errors and you should see the populated 'Actions' tab at the right side of your 'Test Grammar' window with all parsing steps info. You can see the whole parse tree in 'Tree' tab, and you can even execute step by step if you use button with play symbol icon. Now from the test menu choose 'Save The Test File As...' and save it as 'pascal.pas'.
154+
155+After testing is finished, we load package 'gold_parser_5.lpk' from 'Gold Parser for Free Pascal and Lazarus', compile it, then open 'GoldTreeParser.lpi' project and compile it too. Now from command prompt execute this command:
156+
157+``` bash
158+goldtrcc pascal.egt pascal.pas pascal.txt
159+```
160+
161+If everything went fine then you should see parse tree and parsed steps flying fast on your screen. Don't worry. Just open 'pascal.txt' file in your favorite editor and watch the result. Of course, instead of showing parsed tree and steps you can do whatever you want. You can implement semantic checks, emit code for template based code generator, interpret tree...
162+
163+#### Pascal Grammar
164+
165+Pascal grammar is relatively big, so it is hidden and to show it you need to expand it first with the linked button on the right.
166+
167+```bnf
168+! ---------------------------------------------------------------------------------
169+! Standard Pascal Grammar
170+!
171+! modified by Avra (Zeljko Avramovic) for compatibility with latest version of Gold
172+! ---------------------------------------------------------------------------------
173+
174+"Name" = 'Pascal'
175+"Version" = '1973'
176+"Author" = 'Niklaus Wirth'
177+"About" = 'PASCAL was developed by NIKLAUS WIRTH of the ETH Technical Institute of Zuerich in 1970-1971.(published in 1973)'
178+
179+"Case Sensitive" = False
180+"Start Symbol" = <Program>
181+
182+Comment Start = '{'
183+Comment End = '}'
184+Comment Line = '//'
185+
186+{Hex Digit} = {Digit} + [ABCDEF]
187+
188+{Id Head} = {Letter} + [_]
189+{Id Tail} = {Id Head} + {Digit}
190+
191+{String Ch} = {Printable} - ['']
192+{Char Ch} = {Printable} - ['']
193+
194+DecLiteral = [123456789]{digit}*
195+HexLiteral = '$'{Hex Digit}+
196+FloatLiteral = {Digit}*'.'{Digit}+
197+
198+StringLiteral = ''( {String Ch} | '\'{Printable} )* ''
199+CharLiteral = '' ( {Char Ch} | '\'{Printable} )''
200+
201+id = {Id Head}{Id Tail}*
202+
203+<constant> ::= DecLiteral
204+ | StringLiteral
205+ | FloatLiteral
206+ | HexLiteral
207+ | CharLiteral
208+
209+!=========================================== Program
210+
211+<Program> ::= <ProgramHeader> <Declarations> <CompoundStatement> '.'
212+
213+<ProgramHeader> ::= PROGRAM id ';'
214+ | PROGRAM id '(' <IdList> ')' ';'
215+
216+<Declarations> ::= <ConstantDefinitions> <TypeDefinitions> <VariableDeclarations> <ProcedureDeclarations>
217+
218+<ConstantDefinitions> ::= CONST <ConstantDefinitionList>
219+ |
220+
221+<ConstantDefinitionList> ::= <ConstantDef>
222+ | <ConstantDef> <ConstantDefinitionList>
223+
224+<ConstantDef> ::= id '=' <constant> ';'
225+
226+<TypeDefinitions> ::= TYPE <TypeDefinitionList>
227+ |
228+
229+<TypeDefinitionList> ::= <TypeDef>
230+ | <TypeDef> <TypeDefinitionList>
231+
232+<TypeDef> ::= id '=' <TypeSpecifier> ';'
233+
234+<VariableDeclarations> ::= VAR <VariableDeclarationList>
235+ |
236+
237+<VariableDeclarationList> ::= <VariableDec>
238+ | <VariableDec> <VariableDeclarationList>
239+
240+<VariableDec> ::= <IdList> ':' <TypeSpecifier> ';'
241+
242+<ProcedureDeclarations> ::= <ProcedureDec> <ProcedureDeclarations>
243+ |
244+
245+<ProcedureDec> ::= <ProcedureHeader> FORWARD ';'
246+ | <ProcedureHeader> <Declarations> <CompoundStatement> ';'
247+ | <FunctionHeader> FORWARD ';'
248+ | <FunctionHeader> <Declarations> <CompoundStatement> ';'
249+
250+<ProcedureHeader> ::= PROCEDURE id <Arguments> ';'
251+
252+<FunctionHeader> ::= FUNCTION id <Arguments> ':' <TypeSpecifier> ';'
253+
254+<Arguments> ::= '(' <ArgumentList> ')'
255+ |
256+
257+<ArgumentList> ::= <Arg>
258+ | <Arg> ';' <ArgumentList>
259+
260+<Arg> ::= <IdList> ':' <TypeSpecifier>
261+ | VAR <IdList> ':' <TypeSpecifier>
262+
263+<CompoundStatement> ::= BEGIN <StatementList> END
264+
265+<StatementList> ::= <Statement>
266+ | <Statement> ';' <StatementList>
267+
268+<Statement> ::= <CompoundStatement>
269+ | <AssignmentStatement>
270+ | <ProcedureCall>
271+ | <ForStatement>
272+ | <WhileStatement>
273+ | <IfStatement>
274+ | <CaseStatement>
275+ | <RepeatStatement>
276+ |
277+
278+<AssignmentStatement> ::= <Variable> ':=' <Expression>
279+! | <Variable> ':=' <FunctionCall>
280+
281+<ProcedureCall> ::= id <Actuals>
282+
283+<ForStatement> ::= FOR id ':=' <Expression> TO <Expression> DO <Statement>
284+ | FOR id ':=' <Expression> DOWNTO <Expression> DO <Statement>
285+
286+<WhileStatement> ::= WHILE <Expression> DO <Statement>
287+
288+<IfStatement> ::= IF <Expression> THEN <Statement> ELSE <Statement>
289+
290+<RepeatStatement> ::= REPEAT <StatementList> UNTIL <Expression>
291+
292+<CaseStatement> ::= CASE <Expression> OF <CaseList> END
293+
294+<CaseList> ::= <Case>
295+ | <Case> ';' <CaseList>
296+
297+<Case> ::= <ConstantList> ':' <Statement>
298+
299+<ConstantList> ::= <constant>
300+ | <constant> ',' <ConstantList>
301+
302+<Expression> ::= <SimpleExpression>
303+ | <SimpleExpression> '=' <SimpleExpression>
304+ | <SimpleExpression> '<>' <SimpleExpression>
305+ | <SimpleExpression> '<' <SimpleExpression>
306+ | <SimpleExpression> '<=' <SimpleExpression>
307+ | <SimpleExpression> '>' <SimpleExpression>
308+ | <SimpleExpression> '>=' <SimpleExpression>
309+ | <FunctionCall>
310+
311+<SimpleExpression> ::= <Term>
312+ | <SimpleExpression> '+' <Term>
313+ | <SimpleExpression> '-' <Term>
314+ | <SimpleExpression> OR <Term>
315+
316+<Term> ::= <Factor>
317+ | <Term> '*' <Factor>
318+ | <Term> '/' <Factor>
319+ | <Term> 'DIV' <Factor>
320+ | <Term> 'MOD' <Factor>
321+ | <Term> 'AND' <Factor>
322+
323+<Factor> ::= '(' <Expression> ')'
324+ | '+' <Factor>
325+ | '-' <Factor>
326+ | NOT <Factor>
327+ | <constant>
328+ | <Variable>
329+
330+<FunctionCall> ::= id <Actuals>
331+
332+<Actuals> ::= '(' <ExpressionList> ')'
333+
334+<ExpressionList> ::= <Expression>
335+ | <Expression> ',' <ExpressionList>
336+ |
337+
338+<Variable> ::= id
339+ | <Variable> '.' id
340+ | <Variable> '^'
341+ | <Variable> '[' <ExpressionList> ']'
342+
343+<TypeSpecifier> ::= id
344+ | '^' <TypeSpecifier>
345+ | '(' <IdList> ')'
346+ | <constant> '..' <constant>
347+ | ARRAY '[' <DimensionList> ']' OF <TypeSpecifier>
348+ | RECORD <FieldList> END
349+ | FILE OF <TypeSpecifier>
350+
351+<DimensionList> ::= <Dimension>
352+ | <Dimension> ',' <DimensionList>
353+
354+<Dimension> ::= <constant> '..' <constant>
355+ | id
356+
357+<FieldList> ::= <Field>
358+ | <Field> ';' <FieldList>
359+
360+<Field> ::= <IdList> ':' <TypeSpecifier>
361+
362+<IdList> ::= id
363+ | id ',' <IdList>
364+```
365+
366+#### Pascal Test Code
367+
368+``` pascal
369+program test; // simple pascal code for testing parser
370+
371+const
372+ PI = 3.1415;
373+
374+var
375+ a, b, c: real;
376+
377+procedure hello(s: string; b, c: real);
378+begin
379+ writeln(s);
380+ writeln(b);
381+ writeln(c);
382+end;
383+
384+function square(a: real): real;
385+begin
386+ result := a * a;
387+end;
388+
389+function half(a: real): real;
390+begin
391+ result := a / 2;
392+end;
393+
394+function twopi: real;
395+begin
396+ result := pi * 2;
397+end;
398+
399+begin
400+ a := PI;
401+ b := a * 10;
402+ c := square(half(twopi()));
403+ hello('Hello World!', b, c);
404+end.
405+```
406+
407+#### Pascal screenshots
408+
409+<img src="Gold_5.2.0_Pascal_Example.png" title="Pascal example in Gold 5.2.0" width="640" alt="Pascal example in Gold 5.2.0" />       <img src="Gold_Parser_for_Free_Pascal_and_Lazarus_EGT_v5.0_parsed_Pascal_example.png" title="Pascal example parsed with EGT v5.0 Gold Parser for Free Pascal and Lazarus" width="240" alt="Pascal example parsed with EGT v5.0 Gold Parser for Free Pascal and Lazarus" />
410+
411+# Goldie Tools
412+
413+Although [Goldie](https://www.semitwist.com/goldie/) is not originaly made for use with Pascal, it has many [command line tools](https://www.semitwist.com/goldie/Tools/) which can be useful to manipulate CGT files (Gold compiled grammar). You can show CGT file in human readable format, compile grammar to CGT v1.0 file without Gold Parser Builder executable, and parse source according to some CGT file to save the resulting tokens and parsed tree to JSON file.
414+
415+# Useful links
416+
417+- [[Make your own compiler, interpreter, parser, or expression analyzer]]
418+- [Gold Parser Builder home page](http://goldparser.org)
419+- [Gold Parser Builder 3.4.4 setup for older engines not compatible with latest Gold 5.x CGT files](http://www.semitwist.com/download/gold-parser-builder-3-4-4.zip)
420+- [Gold Parser Builder 5.2.0 setup](http://goldparser.org/builder/files/GOLD-Builder-5.2.0-Setup.zip)
421+- [Introduction to Gold Parser](https://www.codeproject.com/articles/10492/introduction-to-gold-parser)
422+- [Let Your Parser Go for the GOLD](https://www.developer.com/net/csharp/article.php/3689776/Let-Your-Parser-Go-for-the-iGOLDi.htm)
423+
424+---
425+*From the Free Pascal wiki, page "Gold", as it stood on 2023-12-15 after a change by Avra, out of the wiki's dump of 2024-05-26. Copyright stays with the author who wrote it, according to the old wiki's [[Licensing]] page. If you have concerns about this material, [message the admin](/wiki-contact).*
426+Category: Code
427+Category: Compiler design
428+Category: Utilities

Back to history