Sign in Join
Wiki copytemporary

Gold: what changed

Revision 1 to 2. Removed lines have a minus sign; added lines have a plus sign.

OldNewLine
11Gold is a free parsing system compatible with Free Pascal that you can use to develop your own programming languages, scripting languages, interpreters, and all kind of parsers and expression analyzers. It uses [LALR](https://en.wikipedia.org/wiki/LALR_parser) parsing, and a mix of [BNF](https://en.wikipedia.org/wiki/Backus–Naur_form) notation, character sets and regular expressions for terminals to define language grammars. Code and grammar are separated, so grammar is not tied to implementation language. This means that the same grammar can be loaded into engines made in different programming languages.
22
33There is a subjective [feature comparison table](http://goldparser.org/about/comparison-parsers.htm) of several parsers on Gold site, with special attention to [Gold vs Yacc](http://goldparser.org/about/comparison-yacc.htm) comparison.
44
55# Gold Parser Builder
66
77[GOLD Parser Builder](http://goldparser.org/) can be used to create, modify and test languages in Windows IDE (runs on Wine, too). Command line tools are also available.
88
99Features:
1010
1111- Grammar editor with syntax highlighting.
1212- Grammar generating wizard.
1313- Test window to step through parsing of a sample source.
1414- Templating system that can generate lexers/parsers or skeleton programs for various languages (including Delphi and FreePascal).
1515- Importing/exporting YACC/Bison format. Exporting XML and HTML format.
1616- Interactive inspection of the compiled DFA and LALR tables.
1717
1818Importing YACC/BISON can be very picky, so you will probably have to convert linux line endings and delete most of the not needed code leaving just BNF rules. Import from lex file does not exist, so you will have to examine lex file yourself to see if there are terminals that you should manually add to your GOLD grammar.
1919
2020# Gold Engines
2121
2222Gold Parser Builder has an option to build skeleton application for Delphi and FreePascal from existing grammar. However, there are also user made Gold Engines for FreePascal/Lazarus which can load compiled grammar (CGT v1.0 or EGT v5.0 file) and parse some file according to loaded grammar.
2323
2424Gold Parser Builder has changed CGT file format in version 5 to EGT. EGT v5.0 file format is used for all Gold Parser Builder versions newer then 5.0.0 (including 5.2.0 which is the actual version at the moment of writing this text), and CGT v1.0 for all older versions. This means that with older engines you will need to use older Gold Parser Builder. Although Gold parser Builder can still save CGT v1.0 file format, engines will sometime have trouble with it, so it's best to use older Gold Parser Builder if you need CGT v1.0. You can find old v3.4.4 version [here](http://www.semitwist.com/download/gold-parser-builder-3-4-4.zip).
2525
2626## Engines for FreePascal and Lazarus
2727
2828There are several Gold Engines for FreePascal and Lazarus, and one of them supports latest Gold EGT v5.0 file format.
2929
3030### Lazarus Gold Engine 2018
3131
3232This is a refactored cross platform Delphi engine used for JSON parsing example. It supports only CGT v1.0 file format so you need to use older Gold Parser Builder with it. You can find it [here](http://forum.lazarus.freepascal.org/index.php/topic,33830.msg295876.html#msg295876).
3333
3434### Gold Parser for Lazarus
3535
3636Another adaptation of the same Delphi engine for Lazarus. It's a little more commented then the 2018 version so it might be more appealing to the ones who want to understand the code. It also supports only CGT v1.0 file format so you need to use older Gold Parser Builder. You can find the engine [here](https://github.com/cgarciagl/goldparser-Lazarus).
3737
3838### Gold Parser for Free Pascal and Lazarus
3939
4040There are 2 versions of this engine. One derived from the same old Delphi engine compatible only with CGT v1.0, and the new one derived from Java which is compatible with both CGT v1.0 and EGT v5.0. This means that it will load grammars compiled with latest version of Gold Parser Builder (5.2.0 at the moment of writing). This engine was used for Pascal parsing example. You can find it [here](https://gitlab.com/teo-tsirpanis-718/gold-parser-lazarus).
4141
4242### FPC Gold Engine with VS Code parser plugin and code generator
4343
4444[FPC Gold Engine](https://github.com/Warfley/FPCGoldEngine) supports both CGT v1.0 and EGT v5.0, has very interesting [VS Code extension](https://github.com/Warfley/gold-parser-tools) to parse and debug grammars, and to generate Pascal and TypeScript code out of grammars. Some additional documentation is available [here](https://github.com/Warfley/GoldEngines/blob/master/docs/index.md).
4545
4646## Engines Usage Examples
4747
4848A lot of grammars found on the net are pretty old and many can not compile out of the box with recent versions of Gold Parser Builder. Therefore you can find here 2 examples which compile fine. These examples are recommended for your initial testing. They are just meant to show you how to parse sample text or code with a grammar, and will not go further then that.
4949
5050### JSON example
5151
5252You should copy/paste simple JSON grammar from below into 'Grammar Window' of Gold Parser Builder 3.4.4, save the grammar as 'json.grm" file, press 'Continue' button in the bottom right 3 times in a row, the press it 4th time and in file browser save compiled grammar as 'json.cgt' file. Now copy/paste JSON test code from below into 'Test Grammar' window of Gold Parser Builder 3.4.4, press 'Start' button, and then press 'Parse All' button. There should be no errors and you should see the populated 'Parse Tree' tab at the bottom of your 'Test Grammar' window. You will find nice parsing steps info in 'Parse Actions' tab, and you can even execute step by step if you use 'Step' button instead of the mentioned 'Parse All' button.
5353
5454After testing is finished, we load project GOLDParserEngineSource from *Lazarus Gold Engine 2018* into Lazarus, open Project Inspector to add 'sparta_generics' package as a new requirement, compile and start the executable, load file 'json.cgt' into 'Test Input' memo field, paste JSON Test Code into memo, and press 'Parse' button to start the parser. If everything went fine then you should see the memo populated with parsed tree represented in textual form. Of course, instead of showing parsed tree you can do whatever you want. You can implement semantic checks, emit code for template based code generator, interpret tree...
5555
5656JSON screenshots from below show what you should expect to see.
5757
5858#### JSON Grammar
5959
6060```bnf
6161"Name" = JSON Grammar
6262"Author" = Ars ne von Wyss
6363"Version" = 1.0
6464"About" = 'Grammar for JSON data, following http://www.json.org/'
6565! and compliant with http://www.ietf.org/rfc/rfc4627
6666
6767"Start Symbol" = <Json>
6868"Case Sensitive" = True
6969"Character Mapping" = 'Unicode'
7070
7171! ------------------------------------------------- Sets
7272
7373! Uncomment next line only on Gold 5.x.x and newer when running in new EGT v5.0 engine:
7474! {Unescaped} = {All Valid} - {&1 .. &19} - ["\]
7575! Older engines do not handle correctly {All Valid} with Unicode character mapping,
7676! so we will replace {All Valid} with {Printable}. Just have in mind that mentioned
7777! RFC4627 compliance is therefore lost.
7878
7979{Unescaped} = {Printable} - {&1 .. &19} - ["\] ! compatible with old CGT v1.0 engines
8080{Hex} = {Digit} + [ABCDEFabcdef]
8181{Digit9} = {Digit} - [0]
8282
8383! ------------------------------------------------- Terminals
8484
8585Number = '-'?('0'|{Digit9}{Digit}*)('.'{Digit}+)?([Ee][+-]?{Digit}+)?
8686String = ["]({Unescaped}|'\'(["\/bfnrt]|'u'{Hex}{Hex}{Hex}{Hex}))*["]
8787
8888! ------------------------------------------------- Rules
8989
9090<Json> ::= <Object>
9191 | <Array>
9292
9393<Object> ::= '{' '}'
9494 | '{' <Members> '}'
9595
9696<Members> ::= <Pair>
9797 | <Pair> ',' <Members>
9898
9999<Pair> ::= String ':' <Value>
100100
101101<Array> ::= '[' ']'
102102 | '[' <Elements> ']'
103103
104104<Elements> ::= <Value>
105105 | <Value> ',' <Elements>
106106
107107<Value> ::= String
108108 | Number
109109 | <Object>
110110 | <Array>
111111 | true
112112 | false
113113 | null
114114```
115115
116116#### JSON Test Code
117117
118118``` javascript
119119{"widget": {
120120 "debug": "on",
121121 "window": {
122122 "title": "Sample Konfabulator Widget",
123123 "name": "main_window",
124124 "width": 500,
125125 "height": 500
126126 },
127127 "image": {
128128 "src": "Images/Sun.png",
129129 "name": "sun1",
130130 "hOffset": 250,
131131 "vOffset": 250,
132132 "alignment": "center"
133133 },
134134 "text": {
135135 "data": "Click Here",
136136 "size": 36,
137137 "style": "bold",
138138 "name": "text1",
139139 "hOffset": 250,
140140 "vOffset": 100,
141141 "alignment": "center",
142142 "onMouseUp": "sun1.opacity = (sun1.opacity / 100) * 90;"
143143 }
144144}}
145145```
146146
147147#### JSON screenshots
148148
149-<img src="Gold_3.4.4_Json_Example.png" title="JSON example in Gold 3.4.4" width="640" alt="JSON example in Gold 3.4.4" />       <img src="GoldEngine2018_CGT_v1.0_parsed_Json_example.png" title="JSON example parsed with CGT v1.0 GoldEngine2018" width="240" alt="JSON example parsed with CGT v1.0 GoldEngine2018" />
149+![JSON example in Gold 3.4.4](/wiki-images/e/e0/Gold_3.4.4_Json_Example.png)       ![JSON example parsed with CGT v1.0 GoldEngine2018](/wiki-images/3/3e/GoldEngine2018_CGT_v1.0_parsed_Json_example.png)
150150
151151### Pascal example
152152
153153You should copy/paste Pascal grammar from below into 'Grammar' window of Gold Parser Builder 5.2.0, save the grammar as 'pascal.grm" file, press 'Next' button in the bottom right 3 times in a row, then press it 4th time and save compiled grammar as 'pascal.egt' file. Now copy/paste Pascal test code from below into 'Test Grammar' window of Gold Parser Builder 5.2.0, and press button with fast forward symbol icon (that's 'Parse All' button). There should be no errors and you should see the populated 'Actions' tab at the right side of your 'Test Grammar' window with all parsing steps info. You can see the whole parse tree in 'Tree' tab, and you can even execute step by step if you use button with play symbol icon. Now from the test menu choose 'Save The Test File As...' and save it as 'pascal.pas'.
154154
155155After testing is finished, we load package 'gold_parser_5.lpk' from 'Gold Parser for Free Pascal and Lazarus', compile it, then open 'GoldTreeParser.lpi' project and compile it too. Now from command prompt execute this command:
156156
157157``` bash
158158goldtrcc pascal.egt pascal.pas pascal.txt
159159```
160160
161161If everything went fine then you should see parse tree and parsed steps flying fast on your screen. Don't worry. Just open 'pascal.txt' file in your favorite editor and watch the result. Of course, instead of showing parsed tree and steps you can do whatever you want. You can implement semantic checks, emit code for template based code generator, interpret tree...
162162
163163#### Pascal Grammar
164164
165165Pascal grammar is relatively big, so it is hidden and to show it you need to expand it first with the linked button on the right.
166166
167167```bnf
168168! ---------------------------------------------------------------------------------
169169! Standard Pascal Grammar
170170!
171171! modified by Avra (Zeljko Avramovic) for compatibility with latest version of Gold
172172! ---------------------------------------------------------------------------------
173173
174174"Name" = 'Pascal'
175175"Version" = '1973'
176176"Author" = 'Niklaus Wirth'
177177"About" = 'PASCAL was developed by NIKLAUS WIRTH of the ETH Technical Institute of Zuerich in 1970-1971.(published in 1973)'
178178
179179"Case Sensitive" = False
180180"Start Symbol" = <Program>
181181
182182Comment Start = '{'
183183Comment End = '}'
184184Comment Line = '//'
185185
186186{Hex Digit} = {Digit} + [ABCDEF]
187187
188188{Id Head} = {Letter} + [_]
189189{Id Tail} = {Id Head} + {Digit}
190190
191191{String Ch} = {Printable} - ['']
192192{Char Ch} = {Printable} - ['']
193193
194194DecLiteral = [123456789]{digit}*
195195HexLiteral = '$'{Hex Digit}+
196196FloatLiteral = {Digit}*'.'{Digit}+
197197
198198StringLiteral = ''( {String Ch} | '\'{Printable} )* ''
199199CharLiteral = '' ( {Char Ch} | '\'{Printable} )''
200200
201201id = {Id Head}{Id Tail}*
202202
203203<constant> ::= DecLiteral
204204 | StringLiteral
205205 | FloatLiteral
206206 | HexLiteral
207207 | CharLiteral
208208
209209!=========================================== Program
210210
211211<Program> ::= <ProgramHeader> <Declarations> <CompoundStatement> '.'
212212
213213<ProgramHeader> ::= PROGRAM id ';'
214214 | PROGRAM id '(' <IdList> ')' ';'
215215
216216<Declarations> ::= <ConstantDefinitions> <TypeDefinitions> <VariableDeclarations> <ProcedureDeclarations>
217217
218218<ConstantDefinitions> ::= CONST <ConstantDefinitionList>
219219 |
220220
221221<ConstantDefinitionList> ::= <ConstantDef>
222222 | <ConstantDef> <ConstantDefinitionList>
223223
224224<ConstantDef> ::= id '=' <constant> ';'
225225
226226<TypeDefinitions> ::= TYPE <TypeDefinitionList>
227227 |
228228
229229<TypeDefinitionList> ::= <TypeDef>
230230 | <TypeDef> <TypeDefinitionList>
231231
232232<TypeDef> ::= id '=' <TypeSpecifier> ';'
233233
234234<VariableDeclarations> ::= VAR <VariableDeclarationList>
235235 |
236236
237237<VariableDeclarationList> ::= <VariableDec>
238238 | <VariableDec> <VariableDeclarationList>
239239
240240<VariableDec> ::= <IdList> ':' <TypeSpecifier> ';'
241241
242242<ProcedureDeclarations> ::= <ProcedureDec> <ProcedureDeclarations>
243243 |
244244
245245<ProcedureDec> ::= <ProcedureHeader> FORWARD ';'
246246 | <ProcedureHeader> <Declarations> <CompoundStatement> ';'
247247 | <FunctionHeader> FORWARD ';'
248248 | <FunctionHeader> <Declarations> <CompoundStatement> ';'
249249
250250<ProcedureHeader> ::= PROCEDURE id <Arguments> ';'
251251
252252<FunctionHeader> ::= FUNCTION id <Arguments> ':' <TypeSpecifier> ';'
253253
254254<Arguments> ::= '(' <ArgumentList> ')'
255255 |
256256
257257<ArgumentList> ::= <Arg>
258258 | <Arg> ';' <ArgumentList>
259259
260260<Arg> ::= <IdList> ':' <TypeSpecifier>
261261 | VAR <IdList> ':' <TypeSpecifier>
262262
263263<CompoundStatement> ::= BEGIN <StatementList> END
264264
265265<StatementList> ::= <Statement>
266266 | <Statement> ';' <StatementList>
267267
268268<Statement> ::= <CompoundStatement>
269269 | <AssignmentStatement>
270270 | <ProcedureCall>
271271 | <ForStatement>
272272 | <WhileStatement>
273273 | <IfStatement>
274274 | <CaseStatement>
275275 | <RepeatStatement>
276276 |
277277
278278<AssignmentStatement> ::= <Variable> ':=' <Expression>
279279! | <Variable> ':=' <FunctionCall>
280280
281281<ProcedureCall> ::= id <Actuals>
282282
283283<ForStatement> ::= FOR id ':=' <Expression> TO <Expression> DO <Statement>
284284 | FOR id ':=' <Expression> DOWNTO <Expression> DO <Statement>
285285
286286<WhileStatement> ::= WHILE <Expression> DO <Statement>
287287
288288<IfStatement> ::= IF <Expression> THEN <Statement> ELSE <Statement>
289289
290290<RepeatStatement> ::= REPEAT <StatementList> UNTIL <Expression>
291291
292292<CaseStatement> ::= CASE <Expression> OF <CaseList> END
293293
294294<CaseList> ::= <Case>
295295 | <Case> ';' <CaseList>
296296
297297<Case> ::= <ConstantList> ':' <Statement>
298298
299299<ConstantList> ::= <constant>
300300 | <constant> ',' <ConstantList>
301301
302302<Expression> ::= <SimpleExpression>
303303 | <SimpleExpression> '=' <SimpleExpression>
304304 | <SimpleExpression> '<>' <SimpleExpression>
305305 | <SimpleExpression> '<' <SimpleExpression>
306306 | <SimpleExpression> '<=' <SimpleExpression>
307307 | <SimpleExpression> '>' <SimpleExpression>
308308 | <SimpleExpression> '>=' <SimpleExpression>
309309 | <FunctionCall>
310310
311311<SimpleExpression> ::= <Term>
312312 | <SimpleExpression> '+' <Term>
313313 | <SimpleExpression> '-' <Term>
314314 | <SimpleExpression> OR <Term>
315315
316316<Term> ::= <Factor>
317317 | <Term> '*' <Factor>
318318 | <Term> '/' <Factor>
319319 | <Term> 'DIV' <Factor>
320320 | <Term> 'MOD' <Factor>
321321 | <Term> 'AND' <Factor>
322322
323323<Factor> ::= '(' <Expression> ')'
324324 | '+' <Factor>
325325 | '-' <Factor>
326326 | NOT <Factor>
327327 | <constant>
328328 | <Variable>
329329
330330<FunctionCall> ::= id <Actuals>
331331
332332<Actuals> ::= '(' <ExpressionList> ')'
333333
334334<ExpressionList> ::= <Expression>
335335 | <Expression> ',' <ExpressionList>
336336 |
337337
338338<Variable> ::= id
339339 | <Variable> '.' id
340340 | <Variable> '^'
341341 | <Variable> '[' <ExpressionList> ']'
342342
343343<TypeSpecifier> ::= id
344344 | '^' <TypeSpecifier>
345345 | '(' <IdList> ')'
346346 | <constant> '..' <constant>
347347 | ARRAY '[' <DimensionList> ']' OF <TypeSpecifier>
348348 | RECORD <FieldList> END
349349 | FILE OF <TypeSpecifier>
350350
351351<DimensionList> ::= <Dimension>
352352 | <Dimension> ',' <DimensionList>
353353
354354<Dimension> ::= <constant> '..' <constant>
355355 | id
356356
357357<FieldList> ::= <Field>
358358 | <Field> ';' <FieldList>
359359
360360<Field> ::= <IdList> ':' <TypeSpecifier>
361361
362362<IdList> ::= id
363363 | id ',' <IdList>
364364```
365365
366366#### Pascal Test Code
367367
368368``` pascal
369369program test; // simple pascal code for testing parser
370370
371371const
372372 PI = 3.1415;
373373
374374var
375375 a, b, c: real;
376376
377377procedure hello(s: string; b, c: real);
378378begin
379379 writeln(s);
380380 writeln(b);
381381 writeln(c);
382382end;
383383
384384function square(a: real): real;
385385begin
386386 result := a * a;
387387end;
388388
389389function half(a: real): real;
390390begin
391391 result := a / 2;
392392end;
393393
394394function twopi: real;
395395begin
396396 result := pi * 2;
397397end;
398398
399399begin
400400 a := PI;
401401 b := a * 10;
402402 c := square(half(twopi()));
403403 hello('Hello World!', b, c);
404404end.
405405```
406406
407407#### Pascal screenshots
408408
409-<img src="Gold_5.2.0_Pascal_Example.png" title="Pascal example in Gold 5.2.0" width="640" alt="Pascal example in Gold 5.2.0" />       <img src="Gold_Parser_for_Free_Pascal_and_Lazarus_EGT_v5.0_parsed_Pascal_example.png" title="Pascal example parsed with EGT v5.0 Gold Parser for Free Pascal and Lazarus" width="240" alt="Pascal example parsed with EGT v5.0 Gold Parser for Free Pascal and Lazarus" />
409+![Pascal example in Gold 5.2.0](/wiki-images/4/49/Gold_5.2.0_Pascal_Example.png)       ![Pascal example parsed with EGT v5.0 Gold Parser for Free Pascal and Lazarus](/wiki-images/9/90/Gold_Parser_for_Free_Pascal_and_Lazarus_EGT_v5.0_parsed_Pascal_example.png)
410410
411411# Goldie Tools
412412
413413Although [Goldie](https://www.semitwist.com/goldie/) is not originaly made for use with Pascal, it has many [command line tools](https://www.semitwist.com/goldie/Tools/) which can be useful to manipulate CGT files (Gold compiled grammar). You can show CGT file in human readable format, compile grammar to CGT v1.0 file without Gold Parser Builder executable, and parse source according to some CGT file to save the resulting tokens and parsed tree to JSON file.
414414
415415# Useful links
416416
417417- [[Make your own compiler, interpreter, parser, or expression analyzer]]
418418- [Gold Parser Builder home page](http://goldparser.org)
419419- [Gold Parser Builder 3.4.4 setup for older engines not compatible with latest Gold 5.x CGT files](http://www.semitwist.com/download/gold-parser-builder-3-4-4.zip)
420420- [Gold Parser Builder 5.2.0 setup](http://goldparser.org/builder/files/GOLD-Builder-5.2.0-Setup.zip)
421421- [Introduction to Gold Parser](https://www.codeproject.com/articles/10492/introduction-to-gold-parser)
422422- [Let Your Parser Go for the GOLD](https://www.developer.com/net/csharp/article.php/3689776/Let-Your-Parser-Go-for-the-iGOLDi.htm)
423423
424424---
425425*From the Free Pascal wiki, page "Gold", as it stood on 2023-12-15 after a change by Avra, out of the wiki's dump of 2024-05-26. Copyright stays with the author who wrote it, according to the old wiki's [[Licensing]] page. If you have concerns about this material, [message the admin](/wiki-contact).*
426426Category: Code
427427Category: Compiler design
428428Category: Utilities

Back to history