README

Rzemieślnik Kodu (Delphi Code Craftsman)

A modern developer toolkit for Object Pascal / Delphi built around DelphiAST. It features a production-grade, lossless hybrid code formatter (dfmt), an interactive VCL Abstract Syntax Tree visualizer (AstViewer), and diagnostic AST dumping utilities.

🌟 Key Highlights

• Hybrid Formatter Engine (dfmt): Combines high-level semantic AST understanding with lexical token stream preservation to format code without dropping comments, compiler directives, or formatting edge-cases.
• 100% Token Invariant Guaranteed: TokenFingerprint(Input) == TokenFingerprint(Output). Zero code loss—every non-whitespace token, comment, and preprocessor directive is preserved verbatim.
• Strict Idempotency: Format(Format(Code)) == Format(Code). Re-formatting code never causes secondary changes, ensuring clean Git diffs and CI/CD reliability.
• Interactive VCL AST Viewer: Live AST exploration with TTreeView, real-time parsing, and two-way source code location mapping.
• Comprehensive Golden Test Suite: 21 golden test cases covering language constructs from units and generics to Delphi 10.3+ inline variables (var X := 10;).

🏛 Architecture: The Hybrid Approach

Standard DelphiAST trees are semantic and lossy:

• Preprocessor directives ({$IFDEF}, {$I}, {$R}) and comments do not exist in the AST.
• String literals are dequoted (e.g., '#13' vs #13).
• Certain keywords and modifiers (such as packed, class operator) are omitted.

To overcome these limitations, dfmt implements a hybrid architecture:

1. TSourceTokens: Lexes the source using TmwPasLex with UseDefines := False. Captures every token, including whitespace, comments, compiler directives, and both branches of {$IFDEF}.
2. TPasSyntaxTreeBuilder: Parses the source into a semantic TSyntaxNode AST tree.
3. TLayoutPlanner: Traverses the AST and assigns layout marks (TLayoutMark) to code tokens: indentation levels, line breaks, blank lines, and expression joins (such as else if).
4. TTokenEmitter: Emits tokens sequentially from TSourceTokens, applying the computed layout while normalizing spacing around operators and symbols.

📦 Projects in this Repository

🚀 Building & Running

Prerequisites

• Embarcadero Delphi (Delphi 12 Athens or compatible, dcc32 available in your PATH), or
• Free Pascal (FPC 3.2.2+) in -Mdelphi mode for dfmt and dfmt_golden.

All binaries are compiled to the root .\bin folder.

1. Delphi Code Formatter CLI (dfmt)

Compile:

dcc32 -NSSystem -E".\bin" -NU".\bin\units" -U".\delphi-ast" -Q .\delphi-formatter\src\dfmt.dpr

Usage:

# Format to standard output:
.\bin\dfmt.exe input.pas

# Format and write to a new file:
.\bin\dfmt.exe input.pas output.pas

# Format files in-place:
.\bin\dfmt.exe -w file1.pas file2.pas

# Inspect AST node coverage:
.\bin\dfmt.exe --coverage input.pas

2. VCL AST Viewer (AstViewer)

Compile:

dcc32 -NSSystem -E".\bin" -NU".\bin" -U".\delphi-ast" -Q .\ast-viewer\AstViewer.dpr

Run:

.\bin\AstViewer.exe

3. Running Tests

A. Golden Runner (dfmt_golden)

Runs all 21 test cases in delphi-formatter/tests/golden/ without requiring external testing frameworks:

dcc32 -NSSystem -E".\bin" -NU".\bin" -U".\delphi-formatter\src;.\delphi-formatter\tests;.\delphi-ast" -Q .\delphi-formatter\tests\dfmt_golden.dpr
.\bin\dfmt_golden.exe --coverage

B. DUnitX Test Suite (dfmt_dunitx)

Runs 30 unit tests covering edge cases, AST invariants, and golden cases:

dcc32 -E".\bin" -NU".\bin" -NS"System;Data;Winapi;System.Win;Data.Win" -U".\delphi-formatter\src;.\delphi-ast" -Q .\delphi-formatter\tests\dfmt_dunitx.dpr
.\bin\dfmt_dunitx.exe

🧪 Golden Test Suite Coverage

The test suite in delphi-formatter/tests/golden/ validates format correctness, token preservation, and idempotence across 21 language constructs:

• 01_empty – Empty source handling
• 02_minimal_unit – Minimal Delphi unit
• 03_unit_uses – Formatted uses clauses
• 04_interface_type – Interface type declarations
• 05_class – Classes with visibility sections (private, public, strict)
• 06_record – Record types and fields
• 07_procedure / 08_function – Routine declarations and bodies
• 09_parameters – Parameter lists (const, var, out)
• 10_local_variables – Classic local var and const sections
• 11_nested_begin_end – Compound statements and blocks
• 12_if_else – Nested if..then..else and aligned else if
• 13_try_except / 14_try_finally – Exception handling and cleanup
• 15_for / 16_case – Loop structures and case..of statements
• 17_comments – Line (//), block ({}), and ANSI ((**)) comments
• 18_compiler_directives – {$IFDEF}, {$ELSE}, {$ENDIF}, {$I}, {$R}
• 19_generics – Generic classes, interfaces, and methods (<T>)
• 20_idempotency – Multi-pass stability checks
• 21_inline_vars – Modern Delphi 10.3+ inline variable declarations (var X := 10;, for var I := 0 to N do)

📂 Directory Structure

rzemieslnik-kodu/
├── ast-viewer/                 # VCL GUI application for AST visualization
│   ├── AstViewer.dpr
│   ├── MainForm.pas / .dfm
│   └── SemanticTools.DelphiAstParser.pas
├── delphi-ast/                 # Embedded DelphiAST parser and lexer library
│   ├── DelphiAST.pas
│   ├── DelphiAST.Classes.pas
│   └── SimpleParser.pas / SimpleParser.Lexer.pas
├── delphi-formatter/           # Delphi code formatter
│   ├── docs/                   # Architecture specifications and AST analysis
│   ├── src/                    # Formatter core engine and CLI (dfmt)
│   ├── tests/                  # DUnitX and golden test runners
│   └── tools/astdump/          # CLI AST dump utility
├── docs/                       # Presentation slides and project roadmap
│   ├── plan_ulepszen_gemini.md # Master improvement plan
│   └── demo.md                 # Demo script
├── bin/                        # Output directory for compiled binaries
└── README.md                   # This file

📚 Documentation & Further Reading

• delphi-formatter/docs/01-analiza-delphiast.md – In-depth technical analysis of DelphiAST and architecture decisions (Variants A through E).
• delphi-formatter/docs/02-architektura-formattera.md – Comprehensive architecture of the hybrid layout planner and token emitter.
• docs/plan_ulepszen_gemini.md – Project roadmap, prioritized backlog, and quality assurance matrix.

📄 License

This project is licensed under the terms of the open-source licenses used by DelphiAST (MPL / Mozilla Public License).