Catalog · How the catalog works
object-pascal-tree
object pascal AST parser
- ★ 5 on GitHub
Anyone may press Interesting, signed in or not. It helps others find this project.
README
PasTree
A simple, extensible, multithreaded Object Pascal parser and semantic analyzer - source in, AST + symbol table out - with a live editor host (SynEdit syntax highlighting, IDE-parity go-to-declaration and declaration↔implementation navigation) built on top of it.
PasTree parses Delphi source files (.pas, .dpr, .dpk, with full .inc
support) into an abstract syntax tree, resolves and type-checks it into a
project-wide symbol model, and exposes both through a small VCL demo that
behaves like a slice of the real RAD Studio IDE. It targets the language as
implemented by Delphi 13.x Florence and specified by
object-pascal-spec -
the companion specification this parser is built from.
Status: in progress. Lexer, preprocessor, parser, and the semantic layer (cross-unit resolution, overloads, generics,
.dproj-aware project loading) are all working end to end, and the demo already does real go-to-declaration and decl↔impl navigation over real projects (including the Delphi RTL/VCL) - with analysis now running fully asynchronously, so opening a project or editing a file never blocks the editor. See To do for what's still open.
Versioning
PasTreeVersion lives in source/PasTree.Version.pas. Two rules, and the same
two hold in the repository that builds on this one
(pastree-lsp - the LSP
server and its clients, which share one version between them), counting its
own commits:
- Every commit bumps the PATCH.
0.2.1->0.2.2->0.2.3, mechanically, no judgement call about whether a change "deserves" it. - A substantial change bumps the MINOR and resets the patch: a new capability, a reworked subsystem, anything a consumer might reasonably need to require.
The per-commit patch bump exists so that a version identifies a build,
unambiguously. This library is linked into tools that get deployed and then
debugged from the outside - the LSP server reports PasTreeVersion in its
serverInfo precisely so a client can ask "does the analysis in this server
have the fix I need" - and a number that only moved on release could not answer
that.
Compatibility is expressed by the consumers rather than by this number: each
declares the oldest version of its dependency it works with
(cMinPasTreeVersion in the server, cMinServerVersion in the plugin) and says
so when the requirement is not met. Those constants move only when a real
dependency appears. Usually that means naming a MINOR - but not always, and the
exception matters: a resolver fix is a patch here and can still be a hard
requirement for a consumer, so a cMin... naming a patch version is correct,
not a mistake.
What it does
- Parses a unit into a full-fidelity syntax tree - every token and every
piece of trivia (comments, whitespace, disabled
{$IFDEF}regions) is preserved, so tokens + trivia reconstruct the source byte-for-byte. Parsing never fails: malformed input produces error nodes with diagnostics instead of aborting (an editor/LSP has to parse broken code all day). - Resolves a whole project's
usesclosure into a symbol model: scopes, declarations, cross-unit references, overload resolution, generic instantiation, and compiler-intrinsic/builtin handling (Integer,TObject, the implicitSystemunit) - modeled closely enough ondcc's own rules to reproduce its precedence and its diagnostics. - Drives an editor (the VCL demo today, an LSP host later) with that model: real syntax highlighting, ctrl+click go-to-declaration, and a declaration↔implementation toggle, all reading the actual AST/symbol table instead of approximating with regex or heuristics.
Architecture
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
??? 1. SourceManager files, encodings/BOM, text cache, ???
??? .inc path resolution, .dproj parsing ???
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
??? 2. Lexer text -> flat token array ???
??? (incl. directives and trivia) ???
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
??? 3. Preprocessor $IFDEF/$IF branching, $I splicing ???
??? (include stack), option-state stack ???
??? -> visible token stream ???
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
??? 4. Parser recursive descent + Pratt expressions ???
??? -> homogeneous AST ???
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
??? 5. Sema per-unit scopes/symbols -> cross-unit ???
??? resolution -> type checking -> overloads/ ???
??? generics -> project-wide diagnostics ???
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
??? 6. Nav pure lookups over the analyzed model: ???
??? go-to-declaration, decl???impl toggle ???
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????Data model: index-based, allocation-friendly - tokens and nodes are records in contiguous per-unit arrays (an arena per unit); references are integer indices, not object pointers. Unit trees are immutable once built, which is also what makes the parallelism below safe.
Multithreading
Parsing and analyzing a unit is a pure function (text, defines) -> model,
with no shared mutable state on the hot path, so a whole project's uses
closure fans out across cores instead of being processed one file at a time:
- Parse + Phase 1 (lex -> preprocess -> parse -> per-unit scopes/symbols) run one worker per core over every file in the closure; results are registered back in input order afterwards, so model IDs stay deterministic regardless of which worker finishes first.
- One model per grab. Every parallel pass (
ForEachIndex) hands its workers one index at a time from a shared counter.TParallel.&For, which it used before, doubles a worker's share every core-count grabs up to 16 x the core count, so a pass's last grabs gave one worker 128-256 consecutive models while the others idled: 100 ms of tails on a 3750-unit project. - Cross-unit resolution is split into a parallel compute step (every
worker only reads other units' already-built reference maps) and a
sequential commit step, so the one pass that has a true ordering
dependency (dcc resolves inherited members before used units, before the
implicit
Systemunit) still runs safely without locking the hot read path. The few structures that genuinely need a lock (e.g. the shared generic-instance cache) are guarded explicitly; everything else relies on "workers only read, one thread commits." - Cold-start I/O is a separate phase from CPU-bound parsing. A first
analysis run pays for antivirus scan-on-first-touch and MFT lookups per
file, which is a latency problem, not a throughput one - so file prefetch
and the search-path index build run as their own phase that hands back raw
bytes only, while decoding to text and parsing come after. The gain is the
separation, not any queue depth: both phases run on the one shared default
TParallelpool (already warm) rather than a private thread pool - an earlier custom pool causedSetMaxWorkerThreadsto silently no-op below the machine's core count on higher-core machines. The staged driver (AnalyzeStaged) does not prefetch: its load engine's workers overlap their own I/O with each other's CPU. - A
SingleThreadedswitch runs every stage on the calling thread instead, for baseline timing comparisons and debugging - results are identical either way, since the parallel stages are pure per unit. - The thread pool's width is pinned (
TPasSemaProject.ConfigureThreadPool, logical-core width,CPUCount, by default; never below 2). Left to grow,TThreadPooladds workers when it believes they are blocked - and it cannot tell blocking from the memory manager spinning on allocation contention, which every one of these passes produces in quantity. Measured, it added threads until the same work cost an order of magnitude more CPU and more wall time (2643 ms vs 1853 ms, ~20.5 s of CPU vs ~5.8 s). Wider is not faster here. - So analysis must be called from a thread outside that pool. A pool
pinned at its width does not add a thread to rescue a task that waits on
another queued task, and every pass forks onto the pool and joins on its
caller. Call
Analyze*from the main thread, aTThread, or throughTPasAsyncSession(which owns aTThread); from aTTaskor aTParallel.Forbody it raisesEInvalidOperationinstead of hanging once every pool thread is such a caller. The cancel predicate anAnalyze*takes is called by the pool workers, concurrently: it must be thread-safe.
The one line every host must set
System.NeverSleepOnMMThreadContention := True; // before analyzingThis is a requirement of hosting this library, not a tuning knob, and it cannot be set from inside the library: it is a global the application owns, and a library flipping a process-wide memory-manager mode behind its host's back is exactly the kind of thing that has no business being implicit. So every host has to do it, and the honest cost of that design is that a host can forget.
With the default (False), Delphi's memory manager sleeps when it cannot
take its lock. Every pass above is allocation-heavy by nature - a token stream,
a tree and a model per unit - so the workers spend their time in Sleep instead
of allocating, and the fan-out is not merely wasted but actively harmful.
Measured 2026-08-20 on a 3757-unit project, same closure and inputs, only this flag differing:
| wall | CPU | intf |
full |
cross |
|
|---|---|---|---|---|---|
| flag set | 15.4 s | 135 s | 2 928 | 4 825 | 7 598 |
| flag NOT set | 70.3 s | 50 s | 18 586 | 29 836 | 21 847 |
SingleThreaded, for scale |
44.7 s | 47 s | 6 421 | 10 982 | 27 314 |
Read the middle row carefully: without the flag the parallel run is slower
than a deliberately single-threaded one, while burning less CPU - the
signature of threads waiting rather than working. Sampling confirmed it: one
thread running, thirty-odd asleep. And note which stages move. cross allocates
comparatively little and stays fine; intf and full are 3x worse than serial.
A slowdown concentrated in the allocation-heavy stages, with CPU time going
down, means this flag.
The LSP server (pastree-lsp) shipped without it and lost 4.5x. It looked like
the cost of running analysis out of process, which it was not - the demo and
every CLI driver in tools/ set it, and the server was simply the one host that
did not. Building the server with full release flags, for comparison, was worth
about 2% - inside the noise. One line was the entire difference.
The stack every host should reserve
{$MAXSTACKSIZE $01000000} // 16 MB per thread - in the program's .dprThe pool's workers are created with the stack size in the executable's
header, so this directive in the host's program file is the only way to give
them more than the default 1 MB; a library cannot. The walks over a tree are
recursive, one level per nesting level of the source, and the depth that is
not the source's nesting - an operator chain 'a' + 'b' + ... and an
else if chain, both as long as generated code makes them - is walked in
loops (audit A1-01, A1-07). The reserve is for what is left: a recursion
nobody has met yet at a depth nobody has written yet. It costs address space
only; a page is committed when the stack reaches it. Every program in
tools\ and the demo sets it.
An overflow that does happen is caught where the analysis absorbs failures -
a unit's load, a pass body - and reported there as the unit's or the pass's
failure. Those handlers call PasRecoverStackOverflow (PasTree.Types),
because Windows does not restore the guard page it spent raising the
overflow: the next one on that pool thread would end the process with no
exception at all. A host with an except of its own that catches an
analysis exception and lets its thread go on should call it too. (Win64. A
Win32 process may not survive even the first: when the RTL's handler has no
stack left to run on, the process ends there.)
Considered and parked: the Windows heap as memory manager
The RTL memory manager returns a 1.25 MB block pool to the OS only when every
block in it is free, and an analysis interleaves what it keeps with what it
drops, so the pools stay mapped around a few live blocks each. A host can swap
in a private Windows heap instead (an opt-in unit listed first in the
program's uses). Measured 2026-09-23 on a 3767-unit closure: analysis wall
unchanged, CPU -7%, private bytes 3177 -> 2785 MB with the full state kept and
2447 -> ~1500 MB after DemoteClosedUnits - but freeing a whole project
takes ~2.1 s instead of ~0.5 s. Not adopted for now: no host links it, and
the implementation (the unit, demo wiring, a background free of the replaced
project) is parked on the branch perf/mm-retention.
Start from the spec
object-pascal-spec is the reference this parser is written against, not documentation written after the fact. Before fixing or adding anything, read the section that governs it and work from there. In order:
- Find the rule. If the spec already states it, the fix follows its wording.
A fix that needs more cases than the spec has rules is a signal you are
patching symptoms - sec. 5.7 says a
withtarget is a structured-typed designator, one rule, while the implementation grew a case per shape. - If the spec is silent, ask dcc, then write it down. A probe compiled
against
dcc32settles it; the answer belongs in the spec in the same session, with the probe's shape as evidence. Otherwise the next person re-derives it. - If the spec and the code disagree, do not assume the spec is right. It is
evidence, not scripture: sec. 1.2.2 once cited
Generics.Collectionsas the example for a rule that excluded dotted names - the example was right and the rule was wrong, and only dcc could say which. Correct whichever loses. - Check before you write. sec. 13.1.4 already described default array
properties, including the
default;versusdefault 0;distinction, and was nearly duplicated by someone who fixed it first and read it after.
Reading first is usually faster, not slower: several fixes here were
re-derivations of something the spec already said in one line, and one of them
(&Foo and Foo being the same identifier, sec. B.3) would have gone straight to
the both-ends conclusion instead of half of it.
Performance discipline
Every change ships with a timing run, not just a correctness run. This is not advice; it is a rule the project learned twice the hard way, both times reported by the user rather than caught by a measurement:
- a lock + string keys memoizing a member lookup cost +16% while the scan it memoized cost 4 ms;
- a defensive
PasNameKeycall added insideFindLocal- the hottest function in the analyzer - cost 3.3x total analysis time.
Both passed every test. Diagnostic counts do not detect a slowdown.
Before/after protocol
tools\out\PasTreeSemaProject.exe <corpus-dir>over a flattened corpus, and read thestages:line - a total alone cannot tell you which phase moved.- Take the best of 5 runs, never a single one. Machine noise on the 665-unit corpus is roughly ±150 ms.
- Compare against the preceding commit, rebuilt and run in the same session. A number from yesterday is not a baseline: it may itself contain a regression (one such stale figure sent an entire investigation at the wrong phase).
- If a corpus count moves, confirm the analysis did not otherwise change:
the tool's full dump should be byte-identical unless the change was meant to
alter resolutions (
-stversus parallel is the cheapest such check). - Say the honest size in the commit message, including when a change is inside the noise - a stage-level win that does not move the total should be described that way, so the next person does not over-credit it.
Build the suites with -$R+ -$Q+ when anything looks flaky: the tool builds ship
without range checks, so an out-of-range index reads adjacent memory instead of
failing, which once presented as non-determinism (three different outcomes in
eight runs) rather than as the one-line bug it was.
Asynchronous analysis
Opening a project, or editing a file, never blocks the editor: analysis runs on a background worker while the previous (still-valid) model stays live and usable.
- Staged pipeline (
TPasSemaProject.AnalyzeStaged): one load wave parses every unit of theusesclosure in full, breadth-first, with the open file's own direct dependencies front-loaded so they're ready first - the closure of the interfaceusescommitted first (progress phaseintf), then the units only an implementationusesreaches (full); a finalizer then runs the existing cross-unit passes over the whole closure. Each module tracks its own progress -msQueued -> msFullReady -> msCrossReady,msIntfReadyfor a library unit read interface-only. - One wave, the ids of two. The load used to run two waves: every unit
interface-only, then each one upgraded to a full parse. Nothing could use
the interface wave - no host reads a project during its run - and the
upgrade re-parsed every unit and re-ran its Phase 1, so a single wave of
full parses is 0.26 s and 11% CPU cheaper on a 3750-unit project (perf
audit 2026-10). Model ids stay the ones the two waves gave, because the
first file loaded under a unit name owns it (
RegisterUnitName): a unit discovers its interfaceusesat its commit, its implementationusesonce the interface closure is in, in id order - while the units only an implementation reaches already parse ahead of their turn. - Snapshot swap, not mutation. A module's tree/model is an immutable
snapshot; a re-parse (a re-decided
$IF, an edit) builds a new model and replaces the old one with it, freeing the old one. The replacement is not synchronized: nothing may read a project while a thread is analyzing it (TryGetSnapshotandModuleStatusincluded). A host gets a complete project fromTPasAsyncSessionbelow, which hands one over only when its build is done. TPasAsyncSessionruns the whole pipeline on a background thread with double-buffered ownership: the host polls a lock-guarded progress snapshot (phase done/total, with total growing as the closure is discovered) and, once finished, swaps the built project in on its own thread - no shared mutable state between the worker and the UI to race. Cancellation is cooperative, checked between chunks/passes, so switching projects or closing the app never waits on a stale in-flight analysis.- Background analysis and the synchronous batch driver (
AnalyzeProject) now load the same way - one wave of full parses - and differ only in the order they commit, so in model ids. - Text-only demotion for incremental hosts (
DemoteText, 0.47.0): after a build, a host can free the token layer of every unit it does not keep (typically: keep open files and the project's own units, drop the library) while the project stays open to the single-module fast path and to donor rebuilds - they rehydrate the few units they touch, ~1 ms each. Measured on a 3767-unit closure: 2848 -> 2008 MB allocated, 3352 -> 2765 MB private, every differential step still identical to a full build. The trade is first-touch latency per demoted unit; details indocs/incremental-analysis.md(the memory dial). - Light library units (
TPasSemaProject.LightPaths, 0.113.0, opt-in): a host names the directory trees of somebody else's code (RTL, VCL, third-party) and their units are analyzed interface-only for good - no bodies, the implementationusesclause alone (0.116.0) - while the project's own units are analyzed exactly as before. Measured on a 3,782-unit closure: 3.1 -> 1.8 s, 2.3 -> 1.2 GB held, 1.5 GB -> 0.86 GB afterDemoteText(its text read only that far, 0.117.0), every binding of the own units identical.PromoteUnit(0.114.0) turns one light unit into a fully analyzed one in place - System.pas included - in tens of milliseconds, nobody else redone. The uses lint answers on the safe side, and exactly afterLintPromotions(0.115.0). Since 0.118.0 a query asks first which library bodies its answer needs (Demand*, from the uses graph and an implementation word index),EnsureFullpromotes them in one batch - or says a full rebuild is cheaper - and the answer is the full analysis's, withNotSearchedcounting what a query run without it left out;tools/PasTreeLightSmokechecks that query by query. Since 0.119.0 every public query has its demand - the hierarchy, the conditional symbols (a preprocess-only completion that leaves the light unit light), the built-ins, positions and renames included. Since 0.120.0 a host asks through aTPasWorkspace(PasTree.Sema.Workspace), which owns the project and its navigator and meets each query's demand inside the query- promotion, rebuild, escalation, with progress and cancel - so a host
never runs that loop itself; the demo asks every Find All query and
Rename through it (0.121.0). What a host stops
seeing, the exempt rules, promotion, the lint, the demand and the
workspace:
docs/library-light-mode.md.
- promotion, rebuild, escalation, with progress and cancel - so a host
never runs that loop itself; the demo asks every Find All query and
Rename through it (0.121.0). What a host stops
seeing, the exempt rules, promotion, the lint, the demand and the
workspace:
Current features
- Non-blocking background analysis - opening a project or editing a file
kicks off analysis on a worker thread (see Asynchronous
analysis); the demo shows live
phase done/totalprogress and swaps in the finished model without ever freezing the editor. - Light library units - the demo's
Light librariescheckbox analyzes the IDE's library and Studio sources interface-only (LightPaths): the analysis takes about half the time and the memory, the project's own units bind exactly as before, a library file opened in a tab is promoted to a full analysis in the background, and the uses lint promotes what it needs first (docs/library-light-mode.md). - Syntax highlighting for SynEdit (
TPasTreeSynHighlighter) - a realTSynCustomHighlighterdriven by PasTree's own lexer/preprocessor/parser, not a regex approximation: keyword/identifier/number/string/comment/ directive/BASM coloring, context-sensitive "weak keyword" coloring (read,deprecated, visibility words, routine directives, a package'spackage/requires/contains...) that only lights up where the AST actually proves them to be directives,$IFDEF'd-out code greyed out like the real IDE, and ctrl+hover link rendering. - Go-to-declaration (ctrl+click) - parity target is "any identifier
navigates exactly like the real IDE does," and the current matrix covers:
local and cross-unit declarations, member access through inheritance/
generics/aliases, the implicit
Systemunit, compiler intrinsics with no source declaration (routed toSystem.pas's header, like the IDE),Result,usesclause names and qualifiers, the whole project closure (not just the main file), units only reachable via the IDE's own registry- read library/browsing search paths, unit namespaces (-NS) and aliases (-A). - Include files open like anything else - ctrl+click (and ctrl+hover's link)
on the file name in an
$I/$INCLUDEdirective jumps into it, and Open File at Cursor (Ctrl+Enter, Delphi's own command, first in the editor's context menu) opens whatever the caret names: ausesitem, an$Iargument, a quoted path. Both ask the ANALYSIS before the filesystem, which is what picks the rightcommon.incwhen three copies sit on different search paths and what makesuses FormsopenVcl.Forms.pasthrough the namespace and alias rules; the filesystem fallback (the current file's directory, then the last analysis's search paths, then the name as written) still opens a file the analysis never reached. An$Iargument is TRIVIA with no identifier and no AST node, so neither goes through the resolver - the span is a line scan (PasTreeDemo.Includes, 18 shapes pinned inDemoSettingsSmoke, including the three that must NOT match:$I+,$I-and$I%), and the link range is the directive's own raw token. - View Unit (Ctrl+F12, the top toolbar, or the file tree's context menu) -
a modal picker over the project's units: type to filter (substring, case-
insensitive), Up/Down while the caret stays in the filter box, Enter or a
double-click opens. Each row is two lines, the name over its directory,
because the name alone stops being an identifier the moment the list widens:
its Implicit Units box adds everything the finished analysis reached, and
a real closure holds both
System.Types.pasandVcl.Types.pas. Names are deduplicated (a project copy wins over a library one - the same precedence the analysis itself applied), and the box is enabled from a LIVE query rather than a snapshot, since a background analysis can finish while the dialog is open. A status bar carries the counts and the project's name. The list and filter rules live inPasTreeDemo.UnitList, apart from the form and tested there (UnitListSmoke). - Declaration ↔ implementation toggle (Ctrl+Shift+Down / Ctrl+Shift+Up) -
jumps from a method or routine's forward declaration to its body's first
statement and back to its name, overload-precise (matched by full
parameter-type signature, not just arity), correctly handling comment-only
bodies and refusing to cross from inactive
$IFDEF'd-out code into active code. - Semantic diagnostics - project-wide
E20xx-style errors from a.dproj-aware loader (search paths, namespaces, unit aliases, platforms), validated continuously against the real Delphi RTL/VCL/FMX source tree. Opens a program or a package:.dpr,.dpkor.dproj, with a bare main source redirected to its sibling.dprojwhen there is one, so naming either file opens the same project. The packages this analyzer is measured against (BuildWinRTL/VCL/FMX.dpk) are the reason the.dpkhalf is not decoration. A.dprojis evaluated as MSBuild would: every environment variable and the reservedMSBuildProject*/MSBuildThisFile*properties are defined before its first line, property names are case-insensitive, and a package's.dcpreferences are itsRequires, not its files.
Layout
| Path | Contents |
|---|---|
source/ |
the library: PasTree.Types, PasTree.SourceManager, PasTree.Lexer, PasTree.Preprocessor, PasTree.Ast, PasTree.Parser, PasTree.DProj, PasTree.Platforms, PasTree.Ast.Json, PasTree.Ast.Check (the tree checker: span, link and Aux invariants over any tree, and the own-token table - which tokens each node kind may own, and which of them hold a fact the tree does not), PasTree.Printer (the structural printer: the source regenerated from the tree - keywords and punctuation from the node kinds, text only from leaves and the documented contract reads - and its two checks, the print against the parsed tokens and the print parsed back; docs/tree-contract.md), PasTree.Project, and the semantic layer PasTree.Sema.* (Model, Resolver, Types, Project, Builtins, Nav, Async, Diagnostics, Dump) |
source/PasTree.Version.pas |
this library's semver (PasTreeVersion), CompareVersions, and BinaryBuiltOn. Deliberately a standalone unit that pulls in nothing else, so a consumer can report which PasTree it is built against without linking the analysis machinery - which is how the LSP server can put it in its serverInfo |
demo/ |
PasTreeDemo - a VCL host (SynEdit + VirtualTreeView) exercising the highlighter and navigation features interactively over real projects |
tests/ |
16 DUnitX-style smoke suites (ParserSmoke, ResilienceSmoke, StagedParseSmoke, DProjSmoke, SemaSmoke, SemaTypeSmoke, SemaXTypeSmoke, SemaOverloadSmoke, SemaProjectSmoke, SemaNavSmoke, SemaCompleteSmoke, AsyncSmoke, UnitListSmoke, NavHistorySmoke, DemoSettingsSmoke, DcuSmoke) plus golden JSON trees and full-corpus runs. tests\build.bat builds and runs all of them - use it rather than hand-rolling dcc32 lines: the last three link demo units through relative in paths and compile only with the current directory set to tests\ |
out/ |
every build's .dcu, under out\dcu\win32 and out\dcu\win64. Intermediate output that nothing reads between runs (every build passes -B), kept in one place so it is trivial to delete and to leave out of a backup. Split by platform because the same units compile both ways and PasTree.Types.dcu would otherwise exist twice under one name |
tools/ |
CLI drivers per pipeline stage (PasTreeLex, PasTreePP, PasTreeParse, PasTreeJson, PasTreeSema, PasTreeSemaProject), PasTreeTreeCheck (the tree checker over a directory or the golden rows: every invariant violation, a per-kind histogram of the tokens each node kind owns with the own-token table's class for each, and the loss list), PasTreePrint (the structural printer over a directory or the golden rows: every token the print misses or misplaces, the filed losses per finding, and every file whose print does not parse back to the same tree), PasTreeDcu (a .dcu -> interface source, a raw dump, or the read-and-parse sweep over a library directory - the .dcu reader's regression gate), the node-kinds generator, and the compile-compare harness: PasTreeXform rewrites a unit along its tree - parenthesizes every operator (t1), wraps every statement in begin/end (t2), reprints it from its tree in place of its own tokens (t3, and t3x with both), writes the preprocessor's decisions into the text (t0f), or writes every name the analysis bound as the declaration it chose - Unit.Name, Self.Name (tq, the resolver's rung, with its selftest tqs), and every member through a cast to the type declaring it - Owner(Base).Name (tm, selftest tms; tqm both) - fidelity.ps1 compiles original and rewrite under dcc and compares the .dcu bytes (identical exactly when the tree, or the preprocessor, agrees with dcc), naming the differing routine from the dump and, through its localizer (the rewrite's sites bisected by recompiling), the node - and fidelity-gate.ps1 runs all of it as one command before a parser, preprocessor or printer change is committed: the tools built with range checks, the comparator's selftest, the tree checker and the printer over the golden rows, the repository and the Studio source, every rewrite over the self-host, the Studio RTL and any extra corpus, each outcome judged against the known exceptions in fidelity-gate.txt |
docs/ |
parser-fidelity.md - how the tree is proven dcc's: the three claims (coverage, structure, completeness), the checks without a compiler and the compile-compare harness through it, the rules its rewrites follow, what none of it can see, where it stands, the findings still open, and the gate. tree-contract.md - what a consumer may read from the tree and from where: the tokens every kind regenerates, the leaf texts, the few documented reads of a token, the facts the tree still does not hold, and how each claim is checked. incremental-analysis.md - how a single edit is re-analyzed without rebuilding the closure: the parse donor, AnalyzeModuleOnly, the guards, every reason a fast path is refused, and what is still open. editor-features.md - the living IDE-parity spec for the demo's editor features. coverage.md - every place PasTree knowingly implements LESS than object-pascal-spec describes, keyed by spec section; the spec itself stays a description of the LANGUAGE, so a gap in this parser is recorded here, beside the code that closes it. dcu-reader.md - the .dcu format findings (Delphi 11 -> 12/13 deltas, all additive), the decision to write our own reader with the patched public parser as the oracle, and what was built: the reader, the interface printer, the source-manager fallback, what the generated text does and does not carry, the gates |
Line endings: CRLF for everything Delphi and cmd.exe read
A .pas, .dpr, .dpk, .inc, .dproj, .dfm or .bat in a working copy
is CRLF. Docs (.md, LICENSE) are LF. Declared in .gitattributes, here and
identically in the pastree-lsp repository - keep the two files in step.
The rule is aimed at tools and scripts as much as people: a sed -i, a
heredoc, or any editor writing "just a newline" produces LF and nothing
complains at the time. RAD Studio then re-saves the file its own way on the first
edit, and the next diff is the whole file instead of the lines that actually
changed - which is how a real change gets lost in review. cmd.exe is worse than
cosmetic: it is the one interpreter here that can genuinely misparse an LF-only
.bat, and every build goes through one.
Renormalized on 2026-08-20. The drift was not confined to recently-touched
files: PasTree.Parser.pas, PasTree.Sema.Resolver.pas, the whole demo/
directory and most of tests/ had been LF for a while. To check:
git ls-files --eolEvery line's w/ must match its attr/. i/ is LF for everything, which is
correct - normalization happens in the repository, eol= decides the working
copy.
To do
Still open, roughly in the order we're tackling it:
-
Incremental reanalysis - BOTH HALVES NOW SHIPPED (0.9.0); this entry is kept for the reasoning, and the sketch below differs from what was built. A is
TPasSemaProject.AdoptParseDonor- reshaped from the standalone cache sketched under A into a donor project (the host's still-alive last-good project), which needs no new lifetime and pins no memory: 192/200 parses reused on the demo closure, interface+full waves 1894 -> 1326 ms. B isTPasSemaProject.AnalyzeModuleOnlyplusTPasAsyncSession.CreateForModule: a body edit costs 19-33 ms against a 3400 ms rebuild on the demo closure, an interface edit is refused and the host rebuilds. The guards are the ones described below, with one correction the harness forced: interface membership is decided by walking a scope's PARENT chain, not by comparing scope indices - the implementation scope is created early (index 2), while an interface class's member scope gets a much higher one. The differential harness istools\PasTreeDiffHarness.dpr(-modulefor B,-selftestas the negative control). -
Incremental reanalysis. Every analysis today rebuilds the whole closure, and the LSP server (
c:\Repos\pastree-lsp) has made the cost concrete: on the demo's own 197-unit closure a rebuild is 5.3 s, and the server pays it for every real edit. The host side is already as good as it can get without library support - versioned overlay buffers, a 300 ms debounce, a background session, mid-pass cancellation, and a gate that skips rebuilding when a document's text does not actually differ from what was analyzed - so what remains is here. Measured split (the server logsStageTimings):stage demo closure intf- interface-only closure parse1088 ms full- upgrade to full parse1418 ms cross- every cross-unit pass2757 ms So parsing is only about half of it, which rules out "cache the parse" as a complete answer and splits the work in two.
A. Reuse the parse artifacts across analyses. A cache keyed by (full path, content hash, define set, platform) handing back the parsed
TPasTree; an unchanged unit is neither re-lexed nor re-parsed and can be registered straight asmsFullReady, skipping BOTH waves. Worth ~48% (5.3 s -> ~2.8 s on the demo), and it is sound by construction rather than by argument: a tree is immutable once parsed - the onlyTree :=anywhere outside the parser isPasTree.Project.pas's assembly of the parse result - so a shared tree cannot be corrupted by the model that borrows it. Two things it must respect: a unit whose preprocessing consumed the$IF Declared(...)/symbol oracle is NOT a pure function of its own text (that is exactly whatRunDeclaredPassre-parses), so anything with a non-emptyTree.Source.UnresolvedDeclaredor a recorded symbol question stays uncached - a dozen units in the whole RTL; and the cache owns the trees, so its lifetime must outlive any one project while the models only ever reference them (the ownershipTPasSemaModelalready documents).B. Reanalyze ONE module. This is the actual editor target - a keystroke inside a routine body should cost ~100 ms, not a closure rebuild - and it is structurally within reach, because the cross passes are already written as "one body per model" (
ForEachIndexoverCrossResolve,CheckCalls,BindTypesX, ...). What blocks it is that symbol identity is an index:- every other model's
ExtRefMapholds(UnitId, Sym)pairs pointing into the edited unit, andFInstances/ the helper index hold the same shape; - interface symbol indices are stable across a re-parse only while the interface TEXT is unchanged (the collect walk is deterministic), which is the common typing case but must be verified, not assumed;
- implementation symbol indices shift on the most ordinary edit there is - adding a local variable - so any instance or helper entry that references an implementation-local symbol of that unit goes stale, and a stale instance index is silent corruption, not an error.
Hence the shape:
AnalyzeModuleOnly(AId)re-parses the file, then GUARDS - the interface symbol list (name + kind + order) must be unchanged, and no instance/helper entry may reference an implementation-local symbol of that model - and falls back to a full rebuild whenever a guard fails. Being unclever there is the whole design: a wrong fast path shows up as navigation that lands somewhere plausible but wrong, which is the hardest class of bug this project has. Also needed: per-model entry points for the passes that currently loop rounds over everything (RunDeclaredPass,CrossResolveDecl's round loop,RunInheritedPass's pending lists,RunCrossTypePass), andPasTree.Sema.Asyncneeds a "reanalyze this module" mode for the host to drive it.The deliverable of B that matters is not the speed-up but the differential harness: run the full pipeline and the incremental one over the same edit sequence and compare RefMap, ExtRefMap and diagnostics across the entire closure. Without that, "it seems right on the demo" is not evidence, and the corpus suites only prove the FULL path still works.
- every other model's
-
**A source with no BOM is decoded as ANSI, and every editor disagrees.**Decided 2026-08-20: UTF-8 first, ANSI only when the bytes are not valid UTF-8.DecodeBytesused to default a preamble-less file toTEncoding.Default- the system ANSI codepage - because that is what dcc does, and matching the compiler is the right instinct. It was the wrong answer here, and what settled it was seeing what the old rule actually cost:- Positions shifted, silently. Every modern editor decodes a
preamble-less
.pasas UTF-8, and THIS repository's own sources are UTF-8 with no BOM and full of em-dashes in comments - so the analyzer and the editor genuinely read different text out of the same bytes, a 3-byte dash arriving as three ANSI characters. Any column on a line with a non-ASCII character before the identifier was off by that inflation. Nothing reported it: navigation just landed beside the name, or inside the preceding string literal. - It defeated the host's rebuild gate, unfixably from outside. The LSP server compares an editor buffer against the file's BYTES, because a decode difference is not an edit (that comparison replaced a string one which cost ~14 s of rebuilds per peeked declaration). Sound for deciding whether to rebuild - but it left the analysis holding its own ANSI reading of a file the editor had open, with no rebuild ever due to correct it. A host cannot work around this: it would have to re-decode every file the analysis reads.
- "Matching dcc" bought less than it looked like. Identifiers are almost always ASCII, so the semantic model rarely differs (a non-ASCII name is one identifier in any case by dcc's fold, spec B.3); the difference is confined to the CONTENTS of string literals and comments - where dcc, reading UTF-8 bytes as ANSI, produces mojibake. Reproducing that faithfully is worth nothing to a caller, while the position skew costs correctness everywhere.
Pure-ASCII files decode identically under both rules, so for the overwhelming majority this is a superset rather than a change. A genuinely ANSI source - high bytes that are not valid UTF-8 - still reads as ANSI, and says nothing: landing on ANSI without a preamble is the rule, not a recovery, so it must not reach
PPENC, whose entire meaning is "the recovered text may not be what the author wrote". Reporting it anyway was the first attempt, and it put 10 warnings on a 197-unit closure for files that had been read perfectly- nine SynEdit units and
System.DateUtils.pas, all of them tripping on a Latin-1 letter in a copyright header (Maël Hörz,Flávio Etrusco). ANSI decoding is total and loses nothing there, so the warning named a correct reading as a problem, with a message that was also false on its face ("text after the bad byte may differ"). That is how a diagnostics list stops being read.PPENCstays for the case it was written for: a file that DECLARED an encoding its bytes then failed.
Pinned in
SemaProjectSmoke(four cases: no-BOM UTF-8, no-BOM 1251, BOM, ASCII), which spells its fixtures with explicit character codes and byte values because this file has no BOM either - a literal in the test source would otherwise be read under the very rule under test.Corpus re-run, 113-unit
.dprojclosure, before vs after: identical in every number - 113 units, 9 unresolveduseswith 5 units gated, 9 diagnostics all in project files and none in library units, stage times inside noise. A wider corpus has not been re-run. - Positions shifted, silently. Every modern editor decodes a
preamble-less
-
**Overlay buffers and cancellation in the library facade**- Done 2026-08-16, the two preconditions for hosting PasTree out-of-process (the LSP server lives inc:\Repos\pastree-lsp, spec there). Most of it turned out to already exist:SetBufferoverlays were consulted before the Prefetch cache and the disk everywhere (LoadText AND IncludeStream), andAnalyzeStagedalready polled its cancel predicate every 64-file chunk and between cross passes. What was actually missing and got added: (1) overlay buffers carry the HOST's version stamp (SetBuffertakesAVersion,BufferVersionreads it back through session -> project -> source manager), so an async host can recognize a result computed from older text; (2) cancellation now lands MID-PASS -ForEachIndexreads the running staged analysis's predicate and turns the rest of a pass into no-ops (safe because every commit loop already tolerated a nil/unset slot, the same shape as a worker that threw), plus checks in the sequential finalizer loops (ResolveUses, BindTypesX, after RunDeclaredPass and RunCrossTypePass) - a cancel no longer waits out a multi-second cross pass on a big project. The demo grew a Stop button that cancels the in-flight background analysis and logs it; the previous project stays live (double-buffering). -
**An interface's implicit `IInterface` ancestor is not walked.**Done 2026-08-06 -FindMemberXmakes the same hop for a heritage-less interface that it already made for a heritage-less class (14 sec. 14.1.1: the default ancestor isIInterface, orIDispatchfor a dispinterface), sowith I do QueryInterface(G, O)resolves. Worth recording that the audit predicted this hop would also clear the WinRT_AddRef/_Releaseflood and was WRONG about that - see the constraint entry below for what actually did. -
**A third-party collections/DI library is a corpus now.**Clean since 2026-08-05. Three package projects of 73, 121 and 71 units, run like any.dproj. Its value was density - compiling projects of ordinary size that reported more than the 3747-unit client project does: 27 false diagnostics in the base package, now ZERO, and 34 in the core package, now 2 - both of those the same honestF1027, and an instructive one. The core package setsDCC_UnitSearchPath=..\..\Source(not recursive) and reaches a unit one directory deeper through itsrequiresof the base package; the source DOES exist in that tree. So a host that followed therequiresclosure to the required PACKAGE's own search paths would resolve it - the one concrete lead the source-less-units item below has.Four causes, all four fixed, and three of the four were "a name that is also something else" again:
The 19 `E2010 Incompatible types: 'Pointer' and '_nil'`- fixed 2026-08-05, all 19 with one rule. They were one cause, traced to its declaration site: its base unit declares a GENERIC record namedPointer<T>, and a bare reference toPointerbound to it instead of to the builtin, so everyPointer(X) := nilandResult := nilin aPointer-returning routine became a type error. Arity is part of a type's identity (sec. 16.1.2), and the project pass already knew it for CROSS-unit references (PreferNonGeneric/FindTypeInUsesArity) - the resolver's own binding did not, soRefMapwas wrong before any of that ran, which also sent ctrl+click to the generic. Now a name that lands on the WRONG arity keeps looking (TPasSemaModel.ResolveByArityAt): the same-scope chain, then joined scopes, then outward - the joined step being what reaches the SEEDEDPointer, a case sec. 16.1.2 did not mention and now does. The guards are tested inline in that order, for the reasonPreferNonGenericdocuments: the common case pays one set membership and no call.The 2 `E2015` on `not HasValue`- the same rule the other way round. That base unit also declaresNullable = record class var HasValue: string; endbesideNullable<T>, whoseHasValueis a Boolean property - so a parameter typedNullable<T>was resolving to the arity-0 record andnot other.HasValuebecamenot <string>. sec. 16.1.2 states both directions and the project pass had both (FindTypeInUsesAritytakes an arity); the resolver now does too, which is why one function serves both.Two deliberate limits: with only the other arity in scope the binding is KEPT (dcc errors there, but dropping the reference would cost navigation for nothing), and an identifier inside
<...>counts as bare, sinceTDict<Pointer, TList>means the arity-0Pointer.The 5 `E2004 Identifier redeclared`- a LEXER gap.class function &&op_Equality(...)is how that library'sTValuedeclares its comparison operators, and only the FIRST&escapes (B.3): the rest belong to the name, so&&op_Equalitynames&op_Equality, a different member fromop_Equality. dcc-verified in both directions - it accepts the two in one record, and rejects&op_Equalitybesideop_Equalityas a redeclaration, and it accepts&&&optoo, so the escape is one&and not a pair. We emitted a straytkUnknownfor the second&, which derailed the whole class body: its parameters, result type andstaticwere declared into the enclosing scope, and the next such declaration collided with them.PasNameKeyalready strips exactly one leading&, so the name key came out right the moment the token did. Across3rdlib13this removed 34 unknown tokens.The 1 `E2003 MemoryBarrier`- an unseeded compiler intrinsic, dcc-probed the way that seed list demands (a unit with an EMPTY uses clause resolves it), and a sibling of theAtomic*family already there. -
**Unresolved MEMBERS after a dot are reported only behind a FLAG.**Its work list is EMPTY as of 2026-08-07 - every RTL/VCL/FMX corpus reports zero, from 8 348 when the flag was new. The flag itself stays (see below: the LIBRARY default is off on purpose), and this entry stays with it because what the floods turned out to be is the useful part.O.Readwhere the class has noReadisE2003for dcc and silence for us; only unresolved BARE identifiers are diagnosed by the LIBRARY by default - and that default is deliberate: error-tolerant is the analyzer's primary mode, because that is what an editor embedding it needs, while a compiler FRONT END needs every unresolved name visible. The switch is what makes both modes available:TPasSemaProject.ReportUnresolvedMembers(TPasAsyncSession.SetReportUnresolvedMemberson the background path, CLIPasTreeSemaProject -members).The demo turns it on unconditionally, and there is no checkbox for it: this host's job is to show what the analyzer still gets wrong, and a switch that hides a real gap also hides progress towards closing it. Show Errors then means all of the semantic ones, and its sibling Syntax Errors (on by default) the lexer's and the parser's - dcc's "expected but found" family, classified by code in
IsSyntaxDiagCode; the two are read differently, a syntax row being a fact dcc agrees with and a semantic row possibly our own gap. Both stay pure DISPLAY filters - toggling re-filters the list instead of costing a re-analysis, which is what a control that changed the analysis would have cost (15 s on the client project). Until member binding is overload- and generic-aware, expect that list to be long and mostly binding gaps; the counts and their three causes are below.It reports at the single place a member reference is finally given up on -
CrossType'snkMemberbranch, afterRefMap,ExtRefMapandFindMemberXhave all failed, and only when the container type is KNOWN, the unit'susesall resolved, and the site is not in an unopenedwith. Off, the analysis is byte-identical.On, 2026-08-05, it was a flood, and the shape of the flood was the finding - three mechanisms, not hundreds of cases. All three are closed now, and what is left is a short tail of unrelated shapes:
corpus 2026-08-05 frames + builtin helpers + body constraints + Create/Free+ seed shadowing + alias identity rtlflat (403) 8 348 104 51 23 6 6 0 BuildWinRTL (317) - - 51 23 5 5 0 BuildWinVCL (271) - - 0 bigflat (726) 8 638 387 194 162 136 110 0 BuildWinFMX (362) - - 89 0 client project (3747) 3 609 824 7 server project (2121) 56 0 library, base (73) 70 60 60 60 60 19 0 library, core (121) 27 3 library, data (71) 20 1 Every RTL/VCL/FMX corpus reports NOTHING under the member flag as of 2026-08-07 -
rtlflat,bigflat, and all three real packages. That is the same bar definition-of-done item 4 sets forE2003, now met by the opt-in check written to find what those corpora could not see.The VCL's last two were one rule: writing the name of a parameterless function reference CALLS it, so
ValueFunc.GetValueon aTFunc<IValue>asks IValue for the member, not the closure (System.Bindings.Outputs, 6 sec. 6.6.1 - which stated the syntax and not this). The hop carries the instantiation frame, sinceTFunc<TResult>declares its result as a PARAMETER; a proc type with parameters and aproceduretype both end the walk instead.The server project, 2026-08-08: 56 -> ZERO. Nine causes over two passes. The first four took it to 9, and the first of THOSE was not in the analyzer at all:
- the demo saw four IDE search paths instead of 141 -
StudioRootreturned the registry'sRootDirverbatim, trailing\and all, whileExtraSearchPathsmatched it against the same value passed throughExcludeTrailingPathDelimiter. The two never compared equal, so every BDS version was skipped and the wholeLibrary/Browsingset was lost -Vcl.Formsand every third-party unit came back asF1027on a project that compiles. A missing unit GATES its importers' diagnostics, so this one bug both invented reports and hid them. - explicit generic-method type arguments (~30 of the 56):
Unsafe.Cast<TSomeEdit>(Edit).ActiveProperties.InferMethodFramereads the frame off the ARGUMENTS, and a component suite'sCast<T: class>(AObject: TObject): Tdeclares every parameter concretely - nothing to infer from, so the call stayed typed as the openT.ExplicitMethodFrameuses the written list, which 16.5.1 says wins outright; inference supplies only what was not written. Its other half: a call writtenName<...>must skip a NON-generic of that name, orTJSONObject.GetValue(Name)swallowsTJSONValue.GetValue<T>(APath)and the chain after it loses everything. - a helper indexed under only one of its target's identities (10):
An editor component's helper unit writes
record helper for TRectwithWinapi.Windowslast in its uses, so it registered Winapi's ALIAS, while another of its units declaresR: TRectwithSystem.Typesin its implementation uses. Same type, different key, helper invisible.BuildHelperMap.Publishnow indexes the whole plain-alias chain - the non-builtin twin of the per-model seed rule above. inherited Namewhere the class REDECLARES that name (7):inherited Alignment.Horzin two of a suite's editor units, where the derivedAlignmentis an enum and the ancestor's is an object. The inherited pass only retries names that bound NOWHERE, so a redeclared one never reached it; the head of an inherited chain is now resolved from the ancestor inCrossTyperegardless of what it already bound to (12.1.2).
The remaining 9 were five more, each a singleton or a pair, and four of the five are one theme - a rule stated for the common case and never widened:
- a
withtarget indexed through a DEFAULT array property (2, and a 2447-report near-miss).with Values[I - 1] doover a TcxValuesViewInfo means itsproperty Values[Index]: TcxValueInfo ... default, so the body opens the ELEMENT - the intra-unit resolver opened the COLLECTION, which also has aValuesmember, soValues.Separatorbound to the element and lostSeparatorwith no diagnostic anywhere near the cause. The pass-through it replaced is still right when the base designator IS an array property (its declared type is per-element already); getting that distinction wrong first cost 2447 reports from a single.inc, which is the amplification pattern in its purest form. T.Createunder a bareconstructorconstraint (1). Only a class can satisfy it - a record has no constructor to require - so it is a class constraint that additionally promisesCreate. Onlyclasswas recognised (Atomic, a threading library).- several constraints on one parameter (2).
TKey: IComparable<TKey>, IEquatable<TKey>, IHashableguarantees the members of ALL of them (16 sec. 16.4.1); the hop took the FIRST, and a utility library callsAKey .GetHashCode(the third) andA.Equals(B)(the second) in adjacent methods. - a parameterless-except-for-defaults function reference (1).
TVTStyleServicesFunc = function(AControl: TControl = nil): TCustomStyleServicesis called by writing its name, so the rule is "no parameter the caller MUST supply", not "no parameters". - a bare name used as a QUALIFIER (1) means the PARAMETERLESS overload:
an RPC library declares
function Add(anEntity): Integerahead offunction Add: TRODLEntity, andAdd.Assign(...)typed as Integer.
Plus two last binding gaps: a bare reference to a same-named GENERIC keeps the intra-unit binding even when the arity says otherwise (
FCollection Enumerator: TCollectionEnumeratordeclared insideTCollectionEnumerator<T>- the non-generic is System.Classes', and the resolver's own arity rule can
only see one unit), and an alias to another class's NESTED type
(
TNotify = TRESTComponentAdapter.TNotify) had no definition at all, because nothing binds the last segment of a dotted nested name -ResolveTypeExpr Nestedis the answer there, for the fourth time in a different function.
The client project, 2026-08-08: 824 -> 8, and only ONE of those eight is unexplained. Five are the charting component's
F1027s (it ships no.pasanywhere) and two areInnerGetParentByType, a confirmed true positive dcc reports too. Two causes covered 732 of the 816:- the instantiation frame was dropped on the OVERRIDE path (~700). The
inherited pass carries a member's substituted type with it, because
nothing downstream can recover which hop it came from - but only on the
branch for names that arrived UNBOUND. A name that arrives BOUND is there
to be overridden (a unit reference or a compiler seed), and that branch
parked
XNil. Invisible until a name was BOTH:uses UITest.Paramsregisters the unit underParams, andTUITest<TParams>.Params: TParamsis the member that outranks it - so everyParams.fieldin the client project's UI tests typed as the OPEN parameter, fell back to its CONSTRAINT, and resolved the base params class's fields while failing on the actual one's. The symptom was only ever which field name was undeclared. - a SAME-UNIT helper now hides the extended type's own member (~60), the
long-standing precedence gap this To-do listed. 15.3.3 says the helper
wins;
JoinHelperScopesjoined it as an ordinary fallback, soFindLocalDeepfound the type's own first. Scopes grew aShadowinglist searched BEFORE their own names, and helper injection is its only user - everything else joined there (uses, with, ancestors, enums) is genuinely a fallback. A suite's rich-edit units are what need it: aclass helper for TTagBasethere redeclaresImporterat the DERIVED importer type, and the whole HTML importer reads members off that.
The last one closed the same day and was not the helper it looked like. A component suite's layout-unit converter declares
LayoutUnitsToPixelsfor TSize, TPoint and TRect at one arity;XSameTypecompares SYMBOLS, and the argument'sTRectand the parameter'sTRectare read in different units, so all three candidates scored 0 and the FIRST - TSize - won the tie. The call then typed as TSize, and nothing said so until.ToRectFwas read off it two units away. Two changes: argument and parameter types are canonicalized through their plain alias links before comparison (CanonTypeX, the non-builtin twin of the seed canonicalization the helper index does), and - the first crack in "ScoreCandidate never REJECTS" - two DIFFERENT record types now reject the candidate outright. That corner is decidable where the general rule is not: distinct records are not assignable, and the one thing that could make them so, aclass operator ImplicitorExplicit, is a MEMBER and can be looked for.The third-party library, 2026-08-08: base 19 -> 0, core 27 -> 2, data 20 -> 1 - and all three left are the honest
F1027above. Three causes, every one a rule that was right for the single shape it was written against and had never met a second:- 16.1.2 by COUNT, not just generic-vs-not (16 of the data package's 20).
The intra-unit arity rule asked one question - is the name
written with type arguments or not - and
TNodes<T>andTNodes<TKey, TValue>are BOTH generic. Only a name's head is registered, soTNodes<TKey, TValue>.PNestedRefbound the arity-1 declaration's nested type, and both arities declare that name. The wrong node record differs from the right one in exactly one field (fKeywhere the other hasfPair), which is the whole reason it went unnoticed for so long: nothing fails until a field is read. Same NextOverload walk the qualified-implementation-name case already did. - a helper's ANCESTOR helper (2 in the core package).
class helper (X) for T(15.3) inherits X's members, and since at most ONE helper is active per type, the derived one is the only route to them. One of its units declaresTRttiMethodHelper = class helper(<its base unit's own helper>) for TRttiMethod; a unit using only the base resolvedReturnTypeHandleand one using both did not - adding a used unit REMOVED members.HelperMemberHitnow walks that chain, still reading helper member scopes only, so a malformed helper graph cannot cycle. - an ARRAY parameter meeting a record/class/interface argument now rejects
the candidate - the third narrow bite out of "
ScoreCandidatenever REJECTS". A binding stops at the FIRST declaration of a name, and there a derived interface declares only theTArray<T>overload of a method while INHERITING the single-item one: the array overload was the only candidate, fitted on arity, and typed the call asTArray<...>. Rejecting it is what lets the existing "nothing in the type's own chain fits, so this means an INHERITED routine" fallback do its job.
FMX's 89 were four shapes, and all four are about what a QUALIFIER means:
- a NESTED type as a type ARGUMENT -
TDispatchMessageWithValue< TCustomMemoModel.TLineInfo>, where nothing binds that last segment cross-unit this early (the same gap a HERITAGE reference has, and the same helper answers it). Losing one argument loses the whole instantiation FRAME, and with it every member of a field typed by the parameter: 82 of the 89 wereMessage.Value.<anything>through this one miss. - a NESTED type as a member QUALIFIER -
TObjectAppearance.TDataMembers .Create(...), where that nested name is an alias ofTArray<TDataMember>. A dotted name binds on its LAST SEGMENT, so the "is this a type?" test read nothing off thenkMembernode and the dynamic-array pseudo-constructor did not apply. Third time that particular lesson has been learnt this week. - the re-export idiom,
X = X(2 sec. 2.5.1, written up from this): the right side means the OUTER X, since a declaration cannot alias itself. Bound to itself instead, the alias is a type whose definition is itself - no diagnostic, and every member reached through it simply absent. FMX does this twice, and the resolver now resolves such a right side with the symbol being declared excluded. - a member NAME that collides with a compiler SEED -
Model.Text, whereTextis the predefined FILE type. A seed is never anyone's member, so a binding that says otherwise is corrected rather than trusted. This is the member-side twin of the seed-shadowing fix above, and it needed its own rule because a member name is resolved through its qualifier and so never reaches the pass that fixed the bare-name side.
The seed-shadowing column is the one to read carefully: it changed 14 of rtlflat's dump lines and none of its six reports. A wrong binding costs no diagnostic - that is the whole reason this table cannot be the only measurement.
The last two causes, both found by running the RTL package once the tail was short enough to read:
- a builtin type has several NAMES, and a helper targets the type.
TUInt32Helper = record helper for UInt32(System.Classes) must answer for a value declaredCardinal; dcc-probed, a Cardinal helper applies toCardinal,LongWordandUInt32alike and isE2671on anInteger. Two halves: follow a plain alias (UInt32 = Cardinal) to what it names, and group the separately-seeded names by hand (PasBuiltinAliasGroup).
The measurement earned its keep immediately: the first version registered
TEditMask = type string's helper understringtoo and added 17 false reports while removing 14, because= typedeclares a DISTINCT type and at most one helper is active per type (15 sec. 15.3.3) - so it HID TStringHelper across FMX.MaskEdit. The parser marks that form withAux = 1on thenkTypeDecl, which is the whole difference between the two.- a generic METHOD's constraints are on its own declaration (16 sec. 16.2.1),
where the body repeats a bare
<T>exactly as a generic type's does.System.Rtti.GetNamedObject<T: TRttiNamedObject>callsObj.HasNamein its body - the RTL's last two member reports. The parsing detail that decides it: an implementation name is a FLAT run of segments, each able to carry its own parameter list, so the owner of a list is the segment immediately before it. Taking the first segment answers with the CLASS's name, which is what the first attempt did.
The three mechanisms named when the flag was new, each one gap rather than hundreds:
**a generic ANCESTOR's member typed by its parameter.**Fixed 2026-08-06 - and it took the two flat corpora from 8 300 / 8 590 reports to 104 / 387. 535CreateInstance+ 134CreateInstanceWithOwner+ hundreds ofget_XxxPropertyin the WinRT units are allStatics.XwhereStaticscomes fromTWinRTGenericImportS<S>and its type ISS.
Both mechanisms the diagnosis had suspected really were present, as the previous session's narrowing said:
AncestorOfXcomposes frames,ResolveTypeExpr'snkTypeArgsbranch creates one for a heritage reference, and the two-unit fixture proved it end to end - the inherited pass typed bareStaticsas the ARGUMENT (IController), frame and all. The loss was one node later.CrossType'snkIdentbranch types an identifier fromRefMap/ExtRefMap- the member's DECLARED type, which for this shape is the openS- and never looked atExprTypeX, where the inherited pass had already parked the frame-substituted answer. So the next dot searched an open parameter and gave up. The branch now prefers that answer, gated on what it computed being an open parameter (or nothing), so the common identifier still costs no dictionary lookup on that hot path.The general lesson is the one
TPasInhPending.Xalready states and this walk did not honour: a frame cannot be recovered downstream - nothing at a node says which hop its member came from - so wherever one pass computed a type WITH a frame, a later pass must read that type rather than re-derive it from the declaration.Measured: no honest diagnostic moves (rtlflat 0, bigflat its 9 known
F1027s, that library's base package 0, suites 859), the only dump lines that change are therefs:/typed exprs:summaries, unresolved references drop by 8 203 on rtlflat and 8 236 on bigflat, and the clock is unchanged (966-1 000 ms rtlflat, ~1 960 ms bigflat, both before and after).**helpers on a builtin-typed value.**Fixed 2026-08-06 - rtlflat 104 -> 51, bigflat 387 -> 194.Trim,StartsWith,ToLower,IsEmpty,Length:TStringHelper/TCharHelpermembers reached through a chain.
A builtin type has no single identity - every model SEEDS its own
string, so which symbol a value carries depends on where its type was read.S.Trimon a local worked (this model's seed, which is exactly whatBuildHelperMap'sPublishindexes), whileUpperCase(S).Trimcarried System.SysUtils' seed andGive(S).Trima third unit's, and both missed a helper that was active, correctly registered and looked up by the right name. Four probe lines in one program separated those cases in a minute, which is worth more than the fix: the corpus reports namedTrimandToLowerand said nothing about WHICHstring.HelperMemberHitnow canonicalizes to the referring model's own seed and probes once more. Both guards keep it off the hot path - the retry runs only for askBuiltinTypethat came from ANOTHER model, and only after the ordinary integer-keyed probe missed. Registering every model's seed up front was the alternative and is quadratic; this pays a name lookup on a miss instead. Clock unchanged (1 962-1 971 ms on bigflat over three runs).What the leftovers turned out to be is worth the next session's attention:
Text.IsEmptyin the FMX canvas units is not a helper gap at all -Textis a seeded builtin (the FILE type, 10.3) and it beat the enclosing class's ownTextproperty, so the base is typed as a file. That is the "a name that is also something else" family again, and this time it is a WRONG BINDING rather than a missing one.**`_AddRef` / `_Release` / `QueryInterface`**- fixed 2026-08-06, and NOT by theIInterfacehop the audit predicted (that hop was already there). The real cause is one line up:System.Win.WinRTdeclaresclass var FFactory: FwithF: IInspectable, so the walk arrived at a generic PARAMETER and stopped, having no rule for one. 16 sec. 16.4.1 says what it should do - a value typed by an unbound parameter has the members its CONSTRAINT guarantees - and fromIInspectablethe existing interface hop reachesIInterfaceand those three names. 48 reports gone (all_AddRefand_Release, 39 of 40QueryInterface), the frame is deliberately dropped on the hop because a constraint is written in the DECLARING scope.
The surviving `QueryInterface` is a different shape still to look at.It was the same rule, one scope out - fixed 2026-08-06, rtlflat 51 -> 23 and the wholeSystem.Net.Mimecluster with it. The constraint hop read the parameter's OWNnkGenericParamgroup, and a method BODY's<T>has no constraints in it: 16 sec. 16.4.1 puts them on the declaration and dcc REFUSES them on the implementation header (procedure TListBase<T>.Add;). So the hop worked exactly where constrained parameters are rarely used and failed where they always are -TAcceptValueListBase<T: TAcceptValueItem, constructor>accounted for 26 of the RTL package's 51 reports on its own, every one of themLItem.FWeight/LItem.Parse/T.Createinside a method body.ConstraintOfParamXnow falls back to the DECLARING type's same-named parameter. Two details are load-bearing: the link is the parameter SYMBOL's scope chain and not the node's (a header's generic-parameter idents are declared into the routine scope but never stamped intoNodeScope, soStructSymOfNodehad nothing to walk), and parameters are matched by NAME, so a body that renames them finds nothing rather than the wrong constraint.**`Create` and `Free`, the tail after those three.**Fixed 2026-08-06 - rtlflat 23 -> 6, the RTL package 23 -> 5. Two unrelated rules wearing one pair of names, and the spec had neither, so both were settled by dcc probes and both are now written down (sec. 8.2.3 is new; sec. 16.4.1 grew the table):
T: classguarantees TObject's members. sec. 16.4.1 listed the constraint KINDS and never said which members each makes reachable, which is the only question a member lookup asks.V.Freecompiles under a bareT: classand isE2003underT: recordor a loneT: constructor- so exactly one kind keyword answers a type, andTObjectList<T: class>.Notify'sValue.Freewas the RTL's wholeFreebucket.- A dynamic array type has a
Createpseudo-constructor.TBytes .Create($EF, $BB, $BF)builds the array and noCreateis declared anywhere - there is no member to bind, so this is typed (as the array, like any constructor) and deliberately left unbound. The two negatives are what make it safe, and both are dcc's: a STATIC array and a VARIABLE qualifier areE2671. Aliases hide the shape (TBytesisTArray<Byte>isarray of T), and the seededTBytes/TArrayhave no declaration at all, so the test follows the alias chain and answers from the seed's category when it runs out.
One refinement the flag earned immediately: a member on a
Variantis never reported, because late binding makes any name compile and dcc checks nothing- 312
ActiveWorkbookand 94Worksheetson one real project, 740 reports in total, all from OLE automation. That rule is in the check now.
So the flag did its job: it converted "the corpora say nothing about member-lookup completeness" into three named mechanisms and a number to beat, and two days later all three are closed and the number is 51 / 194. The demo runs with it on for exactly that reason - the number is the work list, and a host that hides it cannot show the list shrinking. The LIBRARY default stays off until those numbers are near zero, because an editor embedding PasTree should not inherit a work list as diagnostics.
What is left on rtlflat is six reports:
HasNamex2,FromBytesx2,ToString,Wrap. bigflat's 136 are a longer tail of the same kind, led byIndex(20),Length(15) and theTextbinding bug below. There is no bucket left to name - from here it is one site at a time. - the demo saw four IDE search paths instead of 141 -
-
**Inline `var`/`const` visibility is not POSITIONAL.**Done 2026-07-31 (TPasSemaModel.ResolveAt/DeclaredAfter) - the entry outlived the fix and is corrected here on 2026-08-07, the second one in this list to do that. 3 sec. 3.1.3 scopes an inline variable "from its declaration to the end of the enclosing block", and only a REFERENCE lookup consults the position, forsckBlockscopes alone, so everything the language keeps order-independent stays that way. The comparison is on the visible-stream token index, monotonic in source order across include boundaries.Re-probed against dcc32 37.0 on 2026-08-07, five shapes, and we match on all five: a reference above the declaration binds OUTWARD (
E2010), an inline var's own INITIALIZER still sees the outer name (var GX: Integer := GX + 1compiles), a nested block's inline name does not leak out, inlineconstfollows the same rule, and with no outer name at all the earlier reference isE2003.One dcc diagnostic we do NOT emit, and it is a missing check rather than a wrong binding: dcc adds
E2004 Identifier redeclaredat the inline declaration when a reference above it already resolved the name in that block. Belongs with the other missing checks below if it is ever worth having. -
**Member visibility is recorded but not ENFORCED.****Both halves are done- recording 2026-07-31, enforcement 2026-08-05, and its tail worked down to
TWO reports on 2026-08-07** (from 546 / 998). What is still deliberately
absent is named at the end of this entry:
protected(E2362, needs an ancestry walk) and the BARE-name half, which dcc reports asE2003and which therefore belongs in the member walk rather than in a check after it. Recording landed on 2026-07-31 -CollectStructtracks the section as it walks and stamps every symbol a child declares,automatedincluded (kept distinct frompublished). Members before any section marker staysvDefaultrather than taking a guess: the real default ispublishedunder{$M+}andpublicotherwise, which depends on a directive AND on the ancestry.
Enforcement landed 2026-08-05 as an opt-in -
TPasSemaProject.ReportVisibility, CLI-visibility, default OFF, with the analysis byte-identical when off. It covers a QUALIFIED access to aprivateorstrict privatemember, reported as dcc spells it:E2361 Cannot access private symbol TType.Member. Every rule was probed, and the two that keep code LEGAL matter most:privateis visible to the whole declaring UNIT (the "friend" rule), soA.FPrivfrom a sibling class one line below is fine and enforcing per-type would reject correct code everywhere;strict privateis refused even there. Both directions are pinned inSemaProjectSmoke.**Its first measurement found a BINDING bug rather than visibility errors - 546 reports on rtlflat, 998 on bigflat - and fixing that took it to 170 and
-
-
The top names gave it away:
Exception.Create(475 on bigflat),TMonitor.Enter/Exit,TThread.Synchronize- each of those types declares a PRIVATE member of that name beside the PUBLIC one people call (Exceptionhas a privateclass constructor Createat the top of its private section and the publicconstructor Create(const Msg: string)fifteen lines below). The argument-matched overload was already being SELECTED for every call and recorded inCallTargetXfor navigation - butRefMap/ExtRefMapstill held the chain HEAD, so anything reading a binding saw the first candidate. Two changes, both 2026-08-05:- the callee's binding is re-pointed to the selected overload
(
RepointCallee, same discipline asCheckCalls' own re-point: own model'sRefMapdirectly, another model's through the deferred dictionary). The maps now agree withCallTargetXinstead of contradicting it, which fixes every binding consumer at once and not just this check. - the visibility check moved out of the cross pass into its own pass after
it (
RunVisibilityPass). A member's binding is not final while that pass runs - the nkMember node is visited before the enclosing nkCall selects - so a check running inline necessarily judged the head. Any future check that inspects a binding rather than producing one belongs there too.
Measured effect on the corpora: the only dump lines that move are the
refs:summaries (bindings shifting between local and cross-unit), the total UNRESOLVED count is identical, and no symbol line changes - 139 572 unresolved before and after on rtlflat. client project still the known 7, server zero.Then class-vs-instance took it to 52 and 111. A callee qualified by a TYPE (
TMonitor.Enter(X)) cannot mean an INSTANCE method, and that alone separated the pair arity and argument scores could not: System'sTMonitordeclares a private instancefunction Enter(Timeout: Cardinal): Booleanbeside the publicclass procedure Enter(const AObject: TObject)- same arity, so whenever the argument's own cross-type is unknown both score zero and the first candidate, the private one, won.TMonitor.Enterwent from 150 reports to 2. The test is cheap and needs no new bookkeeping: the parser already marks aclassmethod withAux = 1on itsnkRoutine(6.1), and a CONSTRUCTOR counts as callable on the type becauseTFoo.Create(...)is how one is called.Then the tail, worked one site at a time on 2026-08-07: 546 -> 2 on rtlflat and 998 -> 2 on bigflat, in two rounds. Eight causes between them, and not one was a visibility rule - the flag has now said the same thing about every flood it has produced. The first four:
- an anonymous method LITERAL rejects a candidate outright (41 of
bigflat's 111).
TThread.Synchronize(nil, procedure ... end)fits the private(ASyncRec: PSynchronizeRecord; QueueEvent: Boolean = False; ForceQueue: Boolean = False)on arity,nilscores the same against either first parameter, and the first candidate won the tie. This is the "ScoreCandidatenever REJECTS" gap named below, deliberately taken in its narrowest SYNTACTIC form: a written-outprocedure ... endis a value of procedural type and nothing else. The general rule cannot be had so cheaply- it would have to answer for record
Implicitoperators,Variant, untyped parameters, and the rule that a parameterless function reference in a value position is CALLED (which is exactly why atcProcARGUMENT may legitimately meet aBooleanparameter, seeE2012). A literal cannot be called that way, so it is provable at the node.
- it would have to answer for record
strict privatereaches a NESTED type (theFJSONWriter/FParentTypesfamily, and the FIELD accesses that had no selection to read). dcc-probed both ways and the relation is asymmetric: a nested class may read the outer class's strict private member, and the outer class may NOT read a nested one's - that direction is dcc's ownE2361. Now in 11 sec. 11.2.1.- a
class constructoris never what a name means (15 sec. 15.1.5 said "not callable" and stopped there).TRegistrydeclares a private one fourteen lines above the public parameterlessconstructor Create, so the member walk stopped at it. Two traps on the way: theclasskeyword is NOT in the routine node's token span (the struct-body parser consumes it and recordsAux = 1), soIsConstructorSymanswers True for these and only the PAIR separates them; and the name is not fixed -class constructor Init;compiles. - when nothing in a type's own chain fits, the call means an INHERITED
routine (
TButton.Create(Self)is TComponent's). A binding stops at the first declaration of the name, so the ancestor's overloads were never candidates. OneFindMemberXfrom the owner's ancestor, run only when nothing fit, which keeps it off the common path.
The last twelve, read end to end the same day: 6 -> 2 on rtlflat and 12 -> 2 on bigflat. Four more causes, four more binding gaps:
- a UNIT-QUALIFIED type is still a type qualifier.
System.TMonitor .Enter(X)writes the type asUNIT.TYPE, so the callee's qualifier is itself annkMemberand neither map holds anything for that node - the class-vs-instance test read nothing and let the private INSTANCEEnterback in. Its last segment is the type name. - a
class constructorthat is a class's ONLY ownCreatemust not end the member walk. The overload-chain skip added an hour earlier only helped where there WAS a chain;FEngineClass.Create(Vcl.Themes, through a class reference) has one strict private class constructor and nothing else, so the walk has to carry on to the ancestor - dcc-probed, that call is TObject's. - a declaration site is inside its own type.
FGlow: TSystemTitlebarButton .TGlowWindowdeclared INTSystemTitlebarButton, whoseTGlowWindowisstrict private type-StructSymOfNodedeliberately answers NIL for a node in a type declaration, so the check thought the class was an outsider to itself.DeclStructsOfNodeanswers for declaration sites. - a PROCEDURE designator rejects a non-procedural parameter, the same rule
as the anonymous-method literal and for the same reason one step on:
TThread.Synchronize(nil, DoProvide)passes a method VALUE, and the implicit call that lets a procedural argument meet an ordinary parameter needs a FUNCTION and its result. A procedure has none.
What still reports is two sites, the same one twice:
AURIStr.IndexOfAny(['@', '/', '\', '?', '#'], Pos, Limit + 1 - Pos), whereTStringHelperdeclaresIndexOfAny(const Values: array of string; var Index: Integer; StartIndex: Integer)beside the public(const AnyOf: array of Char; StartIndex, Count: Integer). Same arity, and both parameters accept what is written - a one-character literal is assignment-compatible withstringtoo, so this is dcc PREFERRING the exact element type, not rejecting the other. It therefore needs the bracket constructor's ELEMENTS typed and the open-array parameter's element type scored, which is the general typing work the entry below names and not another special case.Still not covered, and each named rather than approximated:
protected/strict protected(E2362, needs an ancestry walk to answer "is the accessing type a descendant"), and a BARE name in a descendant, where dcc reportsE2003 Undeclared identifierinstead - an inaccessible member is not in scope rather than in scope and refused, so that half belongs in the member WALK, not in a check after it.Two rules also come out RIGHT today because the walk ignores visibility, and both must survive: a nested enum's VALUES leak out of a
privatetype (2 sec. 2.2.4) and astrict privatenested helper still activates (15 sec. 15.4). Both are pinned by tests, and neither is a member ACCESS, so neither reaches the new check. - the callee's binding is re-pointed to the selected overload
(
-
- recording 2026-07-31, enforcement 2026-08-05, and its tail worked down to
TWO reports on 2026-08-07** (from 546 / 998). What is still deliberately
absent is named at the end of this entry:
-
Missing semantic CHECKS the spec names, none of which can produce a false positive - collected here rather than as separate items because they share a shape: we accept code dcc rejects. In rough order of how often the shape occurs:
no "ordinal type required" check anywhere- done 2026-08-05 (E2001for an array index type, aset ofbase and acaseselector,E2032for aforcounter). Half of the old entry here was a SPEC BUG, not a missing check: dcc accepts a sparse (explicit-valued) enum in every one of those positions, so the code was right to accept it, and sec. 2.1.1/sec. 2.2.4 have been corrected. dcc also splits the rule per POSITION -Variantis an error as an index/base/counter and legal as acaseselector or anifcondition - which is now a table in sec. 2.1.1. Thecaseselector followed the same day, once the guard check below gave it the typer's result to work from, and so did the 64-bitset ofbase - see the entry below;`set of` base-type limit of 256 values / ordinals in `0..255`- done 2026-08-05 (E2028, one code for both halves, including a negative lower bound). Reported only for a cardinality it can compute exactly: a builtin whose range is fixed on every target (so notNativeInt, the 64-bit types or the*Boolones), a subrange with two LITERAL bounds, or an enum whose values are literals or implicit - tracked in order, so(cA = 250, cB, cC, cD, cE, cF, cG)is caught and its six-element sibling is not, both dcc-verified. A named constant as a bound is left alone rather than folded.set of Char(andset of 'a'..'z') is honoured by staying silent, since dcc makes itW1050, a warning. A 64-bit ordinal base is dcc's other code,E2001- andNativeInt/NativeUIntare 64-bit only where the target says so, which is why the typer now takes a platform: the sameset of NativeIntisE2028underdcc32andE2001underdcc64, both verified.ByteBoolisE2028despite being one byte, whileBooleanis fine. Not reported:FixedIntandUCS4Char, which have real declarations inSystem.pasrather than seeds, so their category is cross-unit and unknown intra-unit;conditions are not required to be Boolean- done 2026-08-05 (E2012, plusE2001for thecaseselector, which needs the same expression types). Three exemptions, each dcc-verified and each necessary:Variant(legal as a guard AND a selector), records as guards only (anImplicitoperator toBooleanmakes one legal, toIntegerillegal - the answer is in its operators, so guards exempt records and selectors do not), and procedural types in both, because a parameterless function reference in a value position is CALLED and the guard's type is its RESULT. That last one was found by the corpora and not by reasoning: sixif AShouldStop thenconditions in a component suite, allreference to function: Boolean, after the suites and both flat corpora were green. Still not reported: a record guard with no Boolean conversion, which is the exemption above;assigning to a `for` counter inside the body- done 2026-07-31 (E2081: a direct assignment,Inc/Decbare orSystem.-qualified, a nestedfororfor ... inon the counter,Read/ReadLn,Val's value - the last four since 0.144.0 - against a named or an inlinefor varcounter; a var-parameter pass is only dcc's W1015 and is not reported). Zero new diagnostics across ~12 000 corpus units, which is the bar every remaining entry here has to clear;bare `raise` outside an exception handler- done 2026-08-05 (E2145). The context turned out to be purely lexical and decided by the NEAREST enclosing part of atrystatement, so afinallyor atryblock resets what an outer handler established, while an anonymous method body does not - all dcc-verified, and now written into sec. 18.3.1, which had said only "the analyzer must track handler context". Zero new diagnostics across all four corpora;`Slice` outside an open-array argument position (4 sec. 4.11)- done 2026-08-05 (E2193), and dcc turned out to be stricter than sec. 4.11 said: an argument of an INTRINSIC is never a valid position either, not evenInsert's, whose parameter really is an open array. Reported for every position - assignment RHS, statement, index base, array-constructor element, any intrinsic's argument,Sliceof aSlice, and a non-open-array PARAMETER of an ordinary call. That last one needs the parameter at an argument index, and it is taken only from a SINGLE unambiguous candidate: a plain callee name bound to a routine with no further overload and with its params visible in this unit. An overload set is deliberately not ranked - picking the wrong candidate would invent a diagnostic, andSliceis far too rare for that trade. SoOver(Slice(A, 3))against an overloadedOverstays silent, as does any cross-unit callee. Still not implemented and now the only gap:Slice's first argument may not be a DYNAMIC array (E2016 Array type required), which is a different check on a different argument;multiline-string indentation (B sec. B.6.3)- done 2026-08-05 (dcInconsistentIndentChars, dcc'sE2657 Inconsistent indent characters). The lexer already found the terminator; it now compares the closing run's indentation against every content line. dcc compares CHARACTERS, not widths, and stops at the end of a short line - one rule that explains all six probed shapes: a tab where the closer has spaces is an error, a whitespace-only line shorter than the closer is not, and anything past the matched prefix is free. sec. B.6.3 said "less indented", which predicts neither of those; it now has the table. One report per offending line, dcc-verified.
Two things about the measurement, since they matter more here than usual: the diagnostic is a LEXER one, so it does not reach the sema report and the bar had to be met with
PasTreeLexinstead (zero across ~16 000 files: both flat corpora, both real project trees,3rdlib13and the whole Studio source tree) - and that corpus is thin for this feature specifically: the whole ~16 000 files hold exactly ONE multiline literal (an embedded JS function in one client-project unit, which is in that project's analysis closure and lexes clean), the feature being Delphi 12 and most of this code older. So the corpora prove no regression (full sema dumps byte-identical, lexer throughput 94.6 vs 93.5 MB/s) plus that one accepted literal, and the dcc-matched probes carry the rest of the weight. Still not implemented, and now the only known gap in B.6.3: the token's VALUE is raw source text, not de-indented - harmless today because nothing folds string constants. -
**Three rules the CODE implements that the SPEC never states.**Written up on 2026-07-31: helper inheritance and its class-vs-record asymmetry (sec. 15.3.1), the dynamicarray of array of Tindexing sugar (sec. 8.2.1), and aTypeRefposition resolving to a TYPE over a nearer same-named non-type (sec. B.11). Behaviour unchanged - the code already did all three. -
Where the audit stands (2026-08-05). ALL FIFTEEN of its items are closed - both false-positive/wrong-binding defects (
IInterface, inline-var position), visibility recording, the three spec write-ups,E2081,E2145,E2193, the whole ordinal/Boolean/set-limit family (E2001,E2012,E2028,E2032) - where one of the items turned out to be a wrong spec rule rather than a missing check - and multiline-string indentation. That is every "missing check" the audit named, bar the two one-line residues noted in this list (Slice's first argument may not be a dynamic array;FixedInt/UCS4Charset bases are invisible intra-unit). The last two - member-lookup reporting and visibility enforcement - shipped as OPT-IN switches, since both can only ADD diagnostics. They shared one acceptance bar, met by every check added so far and worth restating rather than re-deriving - zero new diagnostics across rtlflat, bigflat, both real projects and a third-party library's base and core packages (~12 000 units), plus a probe whose output matchesdccline for line. It has earned its keep once already:E2012passed every suite and both flat corpora while wrongly rejecting sixif AFunctionReference thenconditions, and only the 3747-unit third-party closure showed it.What replaces this list is one finding the last three checks produced independently, and it is worth carrying forward as the next objective rather than as three anecdotes: where this analyzer is imprecise, it is imprecise in BINDING, and every check that trusts a binding inherits that. The member flag's 8 348 reports are generic-ancestor frames and builtin-type helpers;
E2012's six false rejections were parameterless function references;-visibility's 998 are constructor overloads picked first out of a member scope. Three checks, three binding gaps, no rule wrong in any of them.So the work the switches are waiting on, in the order that unlocks the most. The first item is done: a call's callee is re-pointed to the argument-matched overload, and a type-qualified callee no longer considers instance methods - 998 visibility reports on bigflat became 111 in two steps. Argument TYPES were already weighed by
ScoreCandidate; what it still does not do is REJECT a candidate whose argument types definitely do not fit (a mismatch scores 0 rather than -1), which is the next refinement there and wants its own measurement, since it can change a selection rather than only break a tie. The second item, generic-ancestor frames in the member walk, is done too (2026-08-06: 8 300 -> 104 on rtlflat, and the gap was a later pass re-deriving a type instead of reading the frame-substituted one an earlier pass had already computed). So is the third, helpers on builtin-typed values (rtlflat 104 -> 51: a builtin type is seeded PER MODEL, so the helper index was keyed on onestringand asked about another).All three named buckets are therefore closed, and a fourth that only became visible once they were: a constrained parameter used in a method BODY (rtlflat 51 -> 23). The list is out of named items, so the next objective comes from reading the tail rather than from here. What the tail says today:
`Create` (12 on rtlflat) and `Free` (5)- done the same day, and they were two unrelated rules sharing a pair of names (rtlflat 23 -> 6): aclassconstraint guarantees TObject's members, and a dynamic array type has aCreatepseudo-constructor. Both were spec gaps, now filled - this is the first tail item where the spec had to be WRITTEN before the code could be, and both probes are in it;a seeded builtin name beating a class's own member- done 2026-08-06, bigflat 136 -> 110, and the point of it is the reports it did NOT change.Textis a predefined FILE type (10 sec. 10.3) and also an inherited property on FMX'sTTextLayout; dcc gives the member precedence throughout the method body (Text.IsEmptycompiles, andvar F: Text;there isE2007- it wins in a TYPE position too, both probed and now in 11 sec. 11.4). We bound the seed, so the name RESOLVED and only the member after the dot failed: a wrong binding produces no diagnostic of its own, which is why the fix shows up in the dumps (14 unit summaries move on rtlflat, whose report count does not budge) rather than in a count.
The inherited pass already re-decided nodes that arrive bound - that is how a unit reference loses to a member - so this is one more kind admitted to that queue. The reason it was not there already is written in the old comment: queueing every seed-bound identifier measured +3.6%, since
LengthandIntegerare seed-bound in every method body. Two attempts to filter by NAME were both net losses (RTL member names overlap the intrinsics heavily, and the name test itself allocated); what worked is memoizing the MISSES per worker on an integer key -(struct, the symbol it is bound to), no string built, no lock - which is +1.2%, and the difference from the +16% memo this file's history records is exactly that key.Known and deliberate imprecision: the walk ignores visibility, so a
privateancestor member in another unit - invisible to dcc - can win here. That is the same imprecision-visibilityexists to measure, not a new one, and its numbers did not move (52 / 111).ScoreCandidatenever REJECTING a type-mismatched overload - taken in two narrow forms so far. 2026-08-07: an anonymous method LITERAL rejects a non-procedural parameter, provable at the node, 41 of the 111 falseE2361. 2026-08-08: two DIFFERENT RECORD types reject each other, which is decidable because distinct records are not assignable and the one thing that could make them so is aclass operator Implicit/Explicit- a MEMBER, so it can be looked for. That one also needed argument and parameter types canonicalized through their alias links first (CanonTypeX), since the two sides are read in different units andXSameTypecompares symbols. The GENERAL rule is still open and still wants its own measurement, because "these types do not fit" has to answer forVariant, untyped parameters and implicit calls of parameterless function references;- on rtlflat there are exactly six member reports left, so that corpus can no
longer drive this -
bigflat's 110 and the client project are the ones with anything to say.
-
Audit coverage, so the next pass knows where to start. The 2026-07-31 sweep ran pass 1 (spec -> code) over every chapter and pass 2 (code -> spec) over the member-lookup, property, interface, helper, array and generic machinery, with
dccprobes for anything either side left unstated. What pass 2 has still NOT covered: the parser's own branch set (appendix B and the statement/declaration grammar), the preprocessor's, and the typer's -PasTree.Sema.Typesin particular has never been read against ch.02 sec. 2.6.1, which is where the assignment-compatibility contract lives. -
**One cross-model expression typer, not two.**Tried on 2026-07-29 and rejected on measurement; done on 2026-09-19 (v0.39.0) the other way round, and this time it paid. The record of the first attempt stays below because its measurements are what shaped the second.What changed: instead of teaching
Walkthe with-typer's kinds AND keeping both,WithTargetTypeXnow isWalk-CrossTypegained a probe mode (TPasXProbe): the same walk over one subtree, every table write redirected into the probe's own dictionaries, nothing committed, the answer read off the probe. The with pass keeps one probe per unit per round, so a target is typed once per body instead of once per body NAME, and the with-specific rules the old dispatch carried became value-position rules in the walk (IsWithTargetfor the parameterless overload,CtorResultX, the node-basedElementXfallback undernkIndex,nkInheritedas a value).BindTypesXmoved ahead of the with pass - it needs only declaration-site bindings, and the "4 symbols the with pass unlocks" below are now read on demand byDeclTypeX. Measured on the frozen client (3767 units,-members): with 373 -> 342 ms, xtype 618 -> 647 ms, wall 5.5 -> 5.6 s, diagnostics 0 -> 0 with an identical list; the other corpora unchanged; the FMX grid library that prompted it 60 -> 0. The trigger was the fourth corpus in a row to find a designator formWalktyped and the with-typer did not (a generic class instantiation as a qualifier, a generic method with written type arguments) - each costing whole with bodies of false E2003. See 5.7 in the spec for the rule this implements: the target's type is the expression typer's answer, and a second typer is where the gaps live.There were two:
CrossType'sWalk, which types every expression node, andWithTargetTypeX, which types awithtarget. They are not subset-and-superset - each lacks kinds the other has (WalknonkIndex/nkDeref/nkBinaryOp,WithTargetTypeXnonkTypeArgs/nkInlineIf) - so the duplication is real and a shape fixed in one can stay broken in the other. Unifying them still lost on the only test that matters here, simple and fast:step total baseline 1893 ms Walkgains the designator kinds1952 ms + split BindTypesX, shareDeclTypeX1999 ms +5.6% wall time, zero diagnostic change, and MORE code - an
AInBodiesparameter, a two-phase ordering protocol, four extra call sites, a fallback chain. The 35k extra expression types produced nothing observable. Reverted.What the attempt did establish, and what makes it cheap to retry only if a real need appears:
- The two cannot share the ident primitive without
BindTypesXrunning before the body passes, and that reorder is not free: of 53,643 symbols untypeable right afterCrossResolve, 43,470 are declared inside a method body, and the inherited pass unlocks 42,039 of them (the with pass a further- Moving all of
BindTypesXearly loses 181k of 874k cross-model types with diagnostic counts completely unmoved. Splitting it by declaration site works to within 17 symbols, so the dependency is precisely method locals - not "the body passes" in general.
- Moving all of
- Three of the four
withdefects fixed that day had one root cause, and it was never the shape list: code askingRefMap/ExtRefMapfor a binding that a LATER pass produces. At with-pass time those maps are legitimately incomplete; the rule is "derive from types, never read the ref maps". That rule lives in one place (DesignatorSymX's fallback), and new special cases should be routed through it rather than grown. That is the cheap generalization; merging the typers is not.
- The two cannot share the ident primitive without
-
**Zero-diagnostic parity on the real RTL/VCL/FMX.**Reached on 2026-07-30. The flattened RTL+VCL+FMX corpus (726 files) reports ZEROE2003/E2034/E2035; the only diagnostics left are honestF1027s for units genuinely absent from the flat directory.BuildWinVCL.dpk(271 units) andBuildWinFMX.dpk(362 units) are both clean end to end. Stable run to run and identical single-threaded. This is a floor to HOLD, not a finished job - and the two precedence gaps below are known places where the binding is right-ish for the wrong reason.Held, and raised on 2026-08-07: the same five corpora also report zero under
-members, which is the stricter bar - it asks about every member after a dot, not only bare names. The gaps that bar found were all bindings, and the "right-ish for the wrong reason" worry above is exactly what it was measuring.Two real applications are clean too as of 2026-07-31. The 3747-unit /
- M-line client reports 7 diagnostics, not one of them a defect we can fix
here: five
F1027for a charting library that ships.dcu+.hppand no.pas(the source-less-units item below), and two a CONFIRMED true positive
- M-line client reports 7 diagnostics, not one of them a defect we can fix
here: five
-
a method-resolution clause whose right-hand side is declared nowhere in the shipped sources, which
dccreports identically on a reduced probe. Down from 899 on 2026-07-28. Its 2121-unit Win64 server reports zero, down from 94 the same day. 2026-09-17: the client is at zero too - the two true positives went with v0.36.1 (dcc ignores that clause), and the fiveF1027s with the.dcureader (v0.37.0): the charting library's units are now analyzed from their compiled form, 3767 units in 5.4 s. -
Two
with-target diagnostics we accept where dcc refuses, both found by auditing 5.7 against the implementation rather than by the corpus, and both missing-diagnostic rather than false-positive:with P doover a pointer. The implicit dereference that makesP.Fieldlegal does NOT extend to a with target - dcc saysE2018 Record, object or class type requiredand thenE2003on every member.FindMemberX's pointer hop is shared with member access, so we open the scope happily.with (R) do X := 1. Parentheses demote the target to a VALUE: dcc opens the scope but reportsE2064 Left side cannot be assigned to. We have no assignability model for with members, so the write is accepted.
-
OLE named arguments are exempted syntactically, not by callee type. 4.11.3:
Name := Valueis an argument only when the callee isVariant/OleVariant- on a class method, an interface, a statically typed
dispinterface, or a plain routine (even one whose parameter really is namedName) dcc reportsE2003on the name. Nothing knows the callee's type at parse time, so the parser accepts the form anywhere and the resolver never looks the name up. A lost diagnostic, never a false one. Closing it means deferring the decision to the typer, which knows whether the callee isVariant. We also do not implementE2166(named arguments must follow positional ones).
- on a class method, an interface, a statically typed
-
Two over-permissive bindings, both found on 2026-08-08 and both left in deliberately - they can only ADD a binding, never invent a diagnostic, and each would cost precision elsewhere to fix properly:
- A lone
constructorconstraint unlocks all of TObject, where dcc unlocks onlyT.Create. 16 sec. 16.4.1 is explicit and probed: underT: constructoralone,V.Freeon a VALUE of that type isE2003, whileT.Createon the type is legal.ConstraintOfParamXmaps the kind constraint to TObject wholesale, so both resolve. Doing it exactly means splitting the type-designator use from the value use at the constraint hop. - An inherited overload set is merged whether or not the declarations say
overload. dcc-verified this day, in both directions: withIMore = interface(IBase)redeclaring a name, the ancestor's signature is callable when both carryoverloadandE2010 Incompatible typeswhen neither does - the derived declaration hides. Our "nothing in the type's own chain fits, so this means an INHERITED routine" fallback does not read the directive, so it accepts the hidden call too. Now in the spec (6 sec. 6.3.1). Since 0.98.5 a derived set sayingoverloadoroverrideclimbs by the- .1 rule (SelectCallTarget's ConsiderAncestors); only the "nothing fits" fallback, one ancestor level, still ignores the directive.
- A lone
-
Invert the overload chain:
PrevOverload, nearest-declaration-wins. TodayDeclareSymkeeps the FIRST declaration registered under the name and chains later ones forward throughNextOverload. Object Pascal's rule is the opposite - the NEAREST declaration wins and shadows what came before - so every place that needs "what does this name mean here?" has to work against the grain, and every place that needs "all the overloads" has to know where the chain starts. Suggested from a previous parser where the inverted link handled every case; the evidence here agrees:- We already hand-rolled nearest-wins once, as a special case: a declaration
that HIDES a used unit's leaf name re-binds the name to the newer symbol
(
DeclareSym'sskUnitRefbranch). WithPrevOverloadthat is just the normal path. - An implementation-section overload joining the interface set needed the
same fix written three times, in
CheckCalls,SelectOverloadandSelectCallTarget(UnitSetHeads, 0.98.5), because the two declarations live in different SCOPES and nothing linked them. A prev link established at declaration time - looking UP the scope chain, not only in the current scope - carries that relation once, and every consumer that walks "what I shadow" gets it for free. - Walking outward is also the order resolution actually wants: nearest scope,
then enclosing, then interface, then imports.
Surface is small: 19
NextOverloadreferences across four units. ⚠️ What it does NOT solve, so the work is not mistaken for more than it is: the CROSS-UNIT cases. Whether one unit'sTObjectListshadows another's depends on the REFERRING unit'susesorder, so the relation differs per importer and cannot be a single per-symbol link -FindTypeInUsesArityand friends stay either way. Of the three overload/shadowing defects fixed on 2026-07-30 and -31, this design would have prevented one. ⚠️ Migration risk is the symbol IDENTITY change: whateverFindLocalreturns becomes the newest rather than the oldest declaration, and navigation, dumps and several tests assert on which symbol that is. Hold it to the same test as any other refactor here (README's own rule, and the merged-typers episode above): fewer special cases AND no slower, measured - not "cleaner".
- We already hand-rolled nearest-wins once, as a special case: a declaration
that HIDES a used unit's leaf name re-binds the name to the newer symbol
(
-
**Arity check in the intra-unit typer ignores inherited members.**Fixed on 2026-08-23, and the fix is NOT the ancestor walk this item assumed. Reproduced first: auses-less unit with a globalBarand an inheritedTBase.Bar(A: Integer)reported a falseE2034onBar(1)inside a method - dcc compiles that unit. Walking the same-unit ancestry would not have been enough to clear it, because EVERY class also inheritsTObject, which such a unit has no model of (noSystemis loaded andTObjectis deliberately not seeded), soToString(X)in a method there really can meanTObject.ToString. The candidate set inside a method is therefore never provably complete, andSelectOverload's gate now also requires the CALL SITE to be outside any struct (TPasSemaModel.EnclosingStructSym, new - the scope-chain climb the completion engine already had per-scope). The diagnostic keeps firing for unit-level code and plain global routines, where the globals really are the only candidates: the same probe unit reportsE2034on exactly the line dcc does, and nothing on the method call. One enabling change: the resolver publishesNodeScopeto the model BEFORERunrather than after, because the typer is a phase of that same run (the array is shared, not copied). -
**Helper precedence inside the same-unit join.**Fixed on 2026-08-08- the ordering exception this item asked for is a
Shadowinglist on a scope, searched byFindLocalDeepBEFORE the scope's own names, andJoinHelperScopesis its only user. Everything else joined onto a scope (uses, with, ancestors, enums) is a fallback and stays inAdditional. It was worth ~60 reports on the client project, all in a suite's HTML importer, where a helper redeclaresImporterat a derived type.
- the ordering exception this item asked for is a
-
**Declaration-site precedence: inherited member vs used-unit global.**Done in 0.93.11 (F53 of the fidelity plan), and half of it was never a gap: in a class's own declaration its ancestors' members are not visible at all (dcc64 37.0:E2003), so the used unit's global IS what the name means. The other half was real - in a NESTED type's declaration the outer types' ancestries come before the used units - and costs no deferral of every declaration-site name: only a name in a nested type that CrossResolve bound to a used unit or System is looked at again byCrossResolveDecl(QueueOuterDeclRecheck), which overrides it when an outer type's ancestry declares it. -
**`{$IF Declared(X)}` cannot be answered, and the wrong branch is taken.**Done on 2026-07-31.Declared()asks whether an identifier is in scope, and the symbol table that knows sits behind the token stream this very decision produces - so it is answered in two stages instead of guessed:- the first pass answers the compiler-provided names, which need no models at all, and RECORDS every other name it was asked about;
RunDeclaredPassthen re-preprocesses only the units whose recorded names now answer differently, once their imports have models and before any cross pass - the one window where the imports exist and nothing yet holds a reference into the models being replaced.
The two-stage split is what makes it affordable: the commonest guards (
DECLARED(AnsiChar),declared(UInt64)) sit in the largest RTL units, and answering them in stage one took the second pass from 7 units to 2 and a +4% regression to nothing. ONE round across units, deliberately - see the code for what that gives up. The unit's OWN declarations answer by position, as dcc does (2026-09-27): a name declared above the directive and in scope there - a local in its routine's body included - is declared. Counting the own scope whole would make the idiom it exists for ({$IF not declared(X)} X = ...) oscillate; a guarded declaration always lies below its guard, so the positional rule cannot. An earlier guard the pass itself flips can still move a declaration a later guard reads, so a unit is re-decided until its own-name answers agree with the stream they produced. -
An AST printer, and the two very different tests it enables. Item 2 of the definition of done is a token-level roundtrip (concatenated tokens + trivia == source, byte-for-byte), which proves the LEXER lossless and says nothing about the tree. A printer that regenerates source from the AST is the next rung, and it is worth being precise about what each variant actually tests:
- Verbatim printing -> parser/AST fidelity. Print the tree and diff against the original. Every mismatch is a node that DROPPED information - a directive the parser swallowed, an operand order it normalized, a construct it folded. This is a strong, cheap, fully automatable check over the whole RTL, and it is the one to build first. It does NOT test the semantic layer: the tree is the same tree whether or not a single name resolved.
- Qualified printing -> the semantic layer, end to end. Print every
unqualified reference in its FULLY QUALIFIED form (
Unit.Type.Member, eachwithbody member rewritten totarget.member- literally what sec. 5.7's parser guidance suggests keeping the tree shaped for). Now the output encodes every binding decision, so a diff against a known-good rendering is a total test of resolution, not of parsing. Better still, the result must still COMPILE under dcc32 and must produce byte-identical.dcus - which makes dcc the oracle and removes the need for a hand-maintained expectation file. This is the real payoff: a wrong binding stops being invisible and becomes a compile error or a binary difference. Started 2026-09-28 (v0.61.0):PasTree.Printerprints expressions and statements from the tree alone, and since v0.63.0 declarations too - the whole unit, with the tokens the tree does not hold yet (a parameter'svar/const, ...) read in place and counted as losses. Over the Studio source, both flat RTL corpora and the third-party trees (~90M tokens) the print misses or misplaces no token, and parses back to the same tree in both layouts but where one recovery heuristic reads a line break in valid code (casealone on its line in a variant record). The contract it follows isdocs/tree-contract.md. Since v0.64.0 the reprint goes through dcc too - the whole unit printed from its tree compiles to the original's.dcu- and the verbatim rung is closed as definition of done item 5 (docs/parser-fidelity.md). Since v0.90.0 the qualified rung runs too, as that one more rewrite (tq,docs/parser-fidelity.mdsec.- unit-level names and a method's own members are judged by dcc over the
self-host and the Studio RTL, and since v0.91.0 the members reached after
a dot, through a default array property or a
withtarget as well (tm); since v0.92.0 the rung is a gate stage (tqm, its selfteststqsandtms, with floors for the names it cannot judge) and has run over every Studio rtl/vcl/fmx unit and four third-party libraries. What it found is the resolver's work list (docs/parser-fidelity.mdsec. 6): a member's visibility from another unit ignored, an inherited member missed for a unit-level name, two standard procedures System also declares, awithbody falling past Self's members, a nested ancestor of the same name, a string helper'sLength.
- unit-level names and a method's own members are judged by dcc over the
self-host and the Studio RTL, and since v0.91.0 the members reached after
a dot, through a default array property or a
-
A real formatter for Delphi code, built on the structural printer. The printer regenerates a unit from its tree, but lays it out only for checking - one token per line, all on one line, or on the original's lines. A formatter is the same regeneration with a LAYOUT of its own: indentation by nesting, line breaks and wrapping by construct, spacing around operators and punctuation, alignment where a style asks for it, keyword and identifier case,
beginplacement - as a configurable style, with the options a user of the IDE's own formatter expects. What it needs beyond what exists:- trivia: comments re-attached to the nodes they belong to (a doc comment to its declaration, a trailing comment to its line, a comment between statements to the one after it), and conditional directives with the inactive code between them kept as written - a formatter must not lose a branch the current defines skip, so regions it cannot parse are copied, and a unit whose directives cut across the tree is formatted around them or left alone;
- include files formatted as themselves, not flattened into the includer;
- idempotence (formatting formatted code changes nothing) and a range mode
(format a selection, an edit) for the editor and the LSP server.
The safety net comes for free: a formatter changes only layout, so its
output must compile to a byte-identical
.dcu- the compile-compare harness (tools\fidelity.ps1) proves every formatted unit, and T3r proves it parses back to the same tree.
-
Follow a package's
requiresclosure to the required package's search paths. Small, and it is the whole of the last honestF1027we still report: a.dpkwhose ownDCC_UnitSearchPathis one non-recursive directory reaches units that live a level deeper only because itrequiresanother package whose paths do cover them. dcc resolves it through the required.dcp; a host reading.dproj/.dpkfiles can resolve it by reading the required project's paths. Worth doing before the source-less-unit work below, because it costs a.dpkparse and removes a whole class of reports that look like ours and are not. -
**Units with no source (`.dcu`-only third-party libraries).**Done on 2026-09-17 (v0.37.0), stage 3 straight away: seedocs/dcu-reader.md. A unit with no.pason any path is read from its.dcu(source/PasTree.Dcu.pas, Delphi 11 to 13, Win32 and Win64), printed as an interface-only unit (source/PasTree.Dcu.Source.pas) and analyzed like any other; navigation lands in that generated text, read-only. All 10,199 units of the three installed Studios'libdirectories read and parse; the client project went 7 -> 0 diagnostics. The original item follows for the record. The blocker for real projects: the client project pulls in several large third-party component suites, and where only.dcuships, every importer gets anF1027and - far worse - its diagnostics are then gated off entirely by theAllUsesResolvedrule, so the unit reads as clean when it was never analyzed. Target design: produce a virtual, in-memory interface-only.pasfor such a unit and feed it to the ordinary pipeline, so nothing downstream needs to know the declarations did not come from a file. Navigation then opens that generated buffer as a read-only tab - the mechanism already exists for$Iincludes, which likewise resolve to a path that is not the model's own file. Staged, cheapest-first, because the last stage is the expensive one:- Stub search path. Accept a directory of hand- or tool-written
interface-only
.pasfiles that stand in for missing units. No new format work at all, unblocks a real project immediately, and it is the same ingestion path every later stage feeds. Do this first regardless. - Whatever the vendor already ships.
.hppheaders (C++Builder export) and.intfiles where they exist are already declaration-only text. Not hypothetical: the first real project run hit exactly this - the client project's last 5F1027are one charting component's, which ships its.dcuunderlib\win32\releaseplus a C++Builder.hppwith no.pasanywhere in the Studio tree. A.hppreader would close that cluster without touching the.dcuformat at all - and those 5 are now the only project-file diagnostics that corpus has. - A
.dcureader. Probed 2026-09-17, decided: seedocs/dcu-reader.md- the gap to 13.0 turned out to be a handful of additive fields, and the public parser, patched, decodes the whole RTL of three Studios; it stays as the oracle for a reader of our own. The original budget warning below is kept for the record. ⚠️ Do not budget this as a small job. The format is undocumented, proprietary, and changes with essentially every compiler release - a version magic at the head and per-version tag tables. The public knowledge is reverse-engineered (dcu32int, IDR), lags current releases, and nothing published covers 37.0. It cannot be written from documentation; it has to be reverse-engineered against samples. The one big lever: the RTL ships source and.dcuside by side for ~700 units, which is a labelled training corpus almost nobody else has - take a unit whose interface we already parse correctly, and the.dcuis the answer key for what the encoding must mean. Scope the reader to what the analyzer actually needs (unit name, uses list, exported type/const/var/ routine declarations with signatures) and stop there - code, debug info and line tables are out of scope. Resist one shortcut: scraping identifier NAMES out of the.dcustring table and treating "name is present" as resolution. It suppressesE2003without types, so every member access through such a name silently degrades, and real errors get hidden along with the false ones. A stub with honest types (stage 1) beats a name list.
- Stub search path. Accept a directory of hand- or tool-written
interface-only
-
**Navigation history (Back / Forward).**Done on 2026-07-31. Alt+Left / Alt+Right, the mouse's back/forward buttons, and two items on the editor's context menu. Entries are file path + line/col, never node or symbol indices: every re-analysis (ReanalyzeForNav, armed by any edit) rebuildsFSemaProjectfrom scratch and invalidates every index into it.Two things the plan above got wrong, both found by building it:
EditorMouseDownwas not "the single place a jump happens" - the Goto Declaration/Implementation actions and the message window's double-click jump too, and a history fed from one of four sites is worse than none. All four now go throughNavigateTo, which is what the plan wanted to be true.TSynEdit.OnMouseDowndoes NOT receive the X buttons. VCL'sTMouseButtonis(mbLeft, mbRight, mbMiddle)and has no XButton member at all, so the handler is never called for them.WM_XBUTTONDOWNgoes to the focused control, soApplication.OnMessageis the one place that sees it without subclassing every editor as it is created.
One decision taken in advance held: a jump may open a closed unit and Back never closes anything. Positions also die with the project - opening one closes every tab, so entries pointing into them would point at nothing.
The other, "recorded positions drift after an edit and are jumped to anyway", turned out not to be necessary. The plan's own suggestion was right: SynEdit already does this arithmetic, for its gutter marks, its indicators and its selections, and offers the same two notifications to plugins - so
TNavHistoryPlugin(one per editor,phLinesInserted+phLinesDeleted) shifts the recorded lines and there is no drift to accept. Two things worth knowing before reusing it:- the convention is asymmetric and stated only in passing:
FirstLineis 0-based while a mark/indicatorLineis 1-based (TSynIndicators.LinesInserted's comment is where SynEdit says so); - SynEdit's two implementations disagree by one. Indicators shift on
Line > FirstLine, gutter marks onMark.Line >= FirstLine- which drags the line ABOVE an insertion point down with the text below it. The indicator rule is the correct one;NavHistorySmokepins that boundary, and swapping in the mark rule fails exactly that test.
The list rules and the arithmetic live in
PasTreeDemo.NavHistory, apart from the form, because none of it is observable in the running program: a wrong rule shows up as Back landing somewhere slightly unexpected several clicks later, by which time the history that would explain it is gone. -
**Recent projects on `Open Project...` (split button).**Done on 2026-07-31 (PasTreeDemo.Settings).Open Project...is a realbsSplitButton, so its primary half still browses and Windows handles the arrow; the drop-down lists the last 20 (2026-08-07; ten before), most-recent-first, deduped by full path and numbered 1..N in that order. The numbers used to be(index + 1) mod 10, which was reaching for a single-digit accelerator and, past ten entries, printed 1..9, 0, 1..9 again - numbers that repeat and appear to run backwards at the tenth row. The accelerator is kept only where it is unambiguous, the first nine:&10binds the key1, which row 1 already owns, and Windows resolves that by cycling rather than choosing. The.inisits beside the executable and carries the sticky settings too (highlighter, threading, highlight colour) - but NOT the platform, whichOpenProjectsets from the.dprojand would overwrite before it was ever seen. A missing file stays REMEMBERED and is only left out of the menu, so a disconnected share does not silently drop off the list.Tested in
DemoSettingsSmoke, which is worth the suite for one reason: none of this shows itself until a drop-down opens, and the rules that make the list useful (order, one entry per project however it was spelled, a cap that shrinks the FILE and not just the list) are exactly the ones that break quietly. -
.groupproj(project groups), and a project TREE in the demo. Today the demo opens one.dprojand shows its units as a FLAT list, which stops being readable at the client project's 1542 files and cannot represent a group at all. Two halves, and the second is the one with design in it:- Reading
.groupproj. MSBuild XML like.dproj, soPasTree.DProjis the place - anItemGroupof<Projects Include="...">plus per-project<Dependencies>. Straightforward, but note the analyzer already has the hard part: several projects in a group share most of their closure, andTPasSemaProjectcaches by full path (FByPath), so ONE project instance across the whole group parses each shared unit once. Building a project instance per group member would multiply the 7.4M-line closure by the member count for no gain. - The tree. Group -> project -> the .dproj's own grouping (Contains /
Requires for a package, unit vs form vs resource), and separately the
closure a project actually pulls in, which is where the units NOT listed in
any
.dprojlive (3747 analyzed vs 1542 listed for the client project - the other 2205 are library units nothing in the UI currently surfaces). Worth keeping those two axes distinct in the model rather than merging them into one list: "files the project declares" and "units the analysis reached" answer different questions, and the second is what the diagnostics are keyed to.
- Reading
-
A 64-bit build of
PasTreeSemaProject.and of the demo- the demo is Win64-only as of 2026-08-13 (demo\build.batcallsdcc64; seedemo\README.md, which records the requirement and why it is not a preference). Nothing in the library is 32-bit-specific - it is purely address space, and the figure is now measured rather than assumed: the client project's closure (3750 units) holds 3.5 GB, printed on the demo'sDone:line and in the tool's report viaPasTree.Types.AllocatedBytes. That also rules outLARGEADDRESSAWAREas a workaround (3.5 GB against a 4 GB ceiling).A Win32 OOM is worth recognising by its shape, because it does not look like what it is: every unit that fails to load gates its importers, so the run reports tens of thousands of unresolved
uses, suppresses every unit'sE2003, prints0 diagnostics, and files the OOM underINTERNAL: N unit(s) failed to parse - analyzer defect. One cause, many costumes.Still open for the TOOL: build it with
mkdir ..\out\dcu\win64thendcc64 -B -Q "-U%BDS%\lib\win64\release;..\source" "-NSSystem;System.Win;Winapi;Data;Xml" "-N0..\out\dcu\win64" -Eout64 PasTreeSemaProject.dpr- quote
-N0, and create the directory first:dccneither creates it nor says clearly that it is missing (it reportsF2039 Could not create output fileagainst the first unit, which reads like a locked file). - what is missing is making that the default rather than a manual step.
- quote
-
LSP/LSIF server. The demo (VCL-hosted) is the only editor integration today; a Language Server Protocol server (live highlighting/navigation/ diagnostics/rename/etc. over the same Sema/Nav layer) plus LSIF dump generation (precomputed navigation for code browsing without a live server) are the planned next consumers.
-
ToolTip Insight - hover any identifier and get a popup hint saying where it is DECLARED:
<module>: <line>, plus enough of the declaration to be worth reading (kind and name at minimum -property TTextLayout.Text: string). Cheapest of the wish-list items and the one with the best ratio, because the answer is already computed and already reachable: ctrl+click resolves exactly this, so the hint is the same(unitId, sym)lookup rendered as text instead of as a jump. Build it on the SAME entry point the click uses, never a second resolution path - two of them would drift, and a hint that disagrees with where the click lands is worse than no hint.Three things to decide, and each has a right answer already visible elsewhere in this codebase:
- what to show when the click has nowhere to go - and the answer is always a
WORD, never a blank hint or a made-up line. An intrinsic says so:
Length - compiler intrinsic, because it has no source declaration at all (go-to-declaration routes those toSystem.pas's header, which is a navigation convenience and not where they are declared). An identifier nothing resolves says<unknown id>, spelled exactly that way - the user's own wording, and the honest one: this analyzer is error-tolerant by design, so "we could not bind this" is a normal state and worth showing plainly rather than hiding.Result, ausesname (show the resolved FILE) and the dynamic-arrayCreatethis session added (bound to nothing on purpose) each need their own word for the same reason. - when a hint is WRONG rather than missing. The seed-shadowing bug fixed on
2026-08-06 is the cautionary one:
Textresolved, to the predefined file type instead of the class's property, and nothing reported it. A hint makes every such binding visible to the user for the first time - which is a feature (it is a free audit of the binding tables) and a risk (the tail in the To-do above is exactly what it will surface). - hover timing and cancellation. The analysis is asynchronous and a hover is
not a click: the hint must never block the UI thread and must be dropped if
the caret moves or a re-analysis lands.
TPasAsyncSessionalready owns that discipline for the message window; reuse it rather than re-deriving it.
- what to show when the click has nowhere to go - and the answer is always a
WORD, never a blank hint or a made-up line. An intrinsic says so:
-
Find References - for any identifier, list every place it's actually USED (by resolved symbol identity via
RefMap/ExtRefMap/CallTargetX, not a text search - two same-named locals in different scopes must not cross-pollute, and an overload-precise call site must land on the actual overload, not just any same-named routine), in its own results window (module / line / a trimmed source snippet around the identifier), double- click to navigate - same shape as the existing message window, but with REAL columns (the message window today is a single flat text column; this needsHeader.MainColumn/actualTVirtualStringTree.Columns, a first for this codebase's VST usage). Walks every loaded model's RefMap/ExtRefMap/CallTargetX for a match against the target (unitId, sym) pair - cross-unit by construction, no new resolution needed, just a new consumer of data already computed. This is also the direct PREREQUISITE for Rename Symbol below (rename = find references + apply edits) - build this first. -
**Find References / Go to Definition for conditional symbols**- DONE (TPasNavigator.DefineAt/FindDefineReferences/GotoDefine, 0.27.0): the name in$DEFINE/$UNDEF/$IFDEF/$IFNDEF/Defined()is a fourth identity next to symbol, unit and builtin. It reads the preprocessor's own record of every directive it walked (TPasPreprocessed.DefineRefs, dead branches included and flagged), never the text; ctrl+click goes to the nearest preceding$DEFINE, and a project/platform define is reported as such rather than as "not found". Seedocs/editor-features.md§9. -
**Rename symbol** (Ctrl+Shift+E)- DONE (TPasNavigator.PlanRename/PlanUnitRename, demo ctrl+shift+E / context menu): renames the declaration and every reference to it project-wide off the same resolved identity Find References walks, so it only touches the actual declaration's references and never a textually matching name from an unrelated scope. Also renames a UNIT (its header + everyusesitem, dotted names whole; the required file name is handed back to the host). A compiler builtin is refused outright - the name is the compiler's - and so is a rename that would touch a LIBRARY source (RTL/VCL/third-party: the host declares those trees throughTPasNavigator.LibraryPaths) or a read-only file, whole rather than in part. Seedocs/editor-features.md§3 for the edit/preview contract and what it deliberately does not check. -
**Find Overrides**- DONE (TPasNavigator.MethodAt/FindOverrides, demo editor context menu, next to Find References): for a class METHOD, every declaration sharing its VMT slot - thevirtual/dynamicroot the chain starts at plus everyoverridebelow it, in any unit of the closure - and the two shapes that belong to such a report without sayingoverride: amessagehandler (implicitly virtual, dispatched through the message table, and dcc rejectsoverrideon one) and areintroduce(deliberately leaving the chain, which a reader must not mistake for staying in it). Not a reference search: no call site is a row, and neither is a same-named method in an unrelated hierarchy. Climbs to the root first, so any link of a chain answers for the whole chain; then descends over a reverse-heritage index built from every class'sclass(TBase)reference as the RESOLVER bound it. Seedocs/editor-features.md§4 for the per-shape table, including the two honest gaps (an ancestor named through a type alias). -
**Find Implementations**- DONE (TPasNavigator.InterfaceMethodAt/FindImplementations, demo editor context menu): the interface-side twin of Find Overrides - for an INTERFACE method, every class method that implements it, project-wide. Harder than an override chain because there is no keyword to key on (an interface method is implicitly virtual and an implementor writes no directive at all), so the tie is the class's ownclass(TBase, IFoo)list: the interface plus every interface descending from it, then every class listing one of those, and for a class that declares no such method itself, the ANCESTOR method that satisfies it (positioned on the code that actually runs, credited to the class that took the interface on). A method resolution clause and a delegatedimplementsproperty are honest gaps - seedocs/editor-features.md§5. -
**Unused Units**- DONE (PasTree.Sema.Lint.FindUnusedUses, 0.93.0): theusesentries none of whose names a unit uses, for the unit in the editor or every project unit - pastree-mcp'slint unused-usesrule over one analysis, with what may make a removal wrong (an initialization the program would lose, a dead branch naming the unit) said on the row. A unit an inline routine the unit calls needs counts as used: without it dcc does not expand the call (H2443). AndFindUnreferencedUnits: the project units nobody uses - what the program would no longer take in once every unused entry is gone, the.dpr's own list included. Seedocs/editor-features.md§12. -
Show Defines - for the identifier/position under the cursor, list every preprocessor define ACTIVE there (module where
$DEFINEd / project- or platform-level, line, the define name), in its own results window, click to navigate to the$DEFINEsite. Needs NEW preprocessor infrastructure, not just a UI:TPasDefinesis a live flag set (Define/Undefine/IsDefined) with no positional history - nothing today records WHERE in the token stream a define turned on/off, so "what's active at offset N" can't be answered from the current model. Requires logging (position, name, on/off) events during preprocessing and replaying them up to a query position; also needs to decide how project/platform-level defines (MSWINDOWS,WIN32, ...) are attributed in the module column (no real$DEFINEsite - "project settings" or similar sentinel). -
Show Units Dependency - a dependency tree (TreeView, root = the open project's main source, children = each unit it
uses, recursively) in its own window, offTPasSemaModel.UsesList/UnitId(the same graphAnalyzeProject/AnalyzeStaged's BFS closure walk already has, just not currently exposed anywhere) - real dcc disallows genuineusescycles so this is a DAG, though a diamond dependency (two units sharing a common base) will naturally repeat in a plain tree view, same as a real IDE's own uses-browser. The SAME window should also offer a flat INITIALIZATION ORDER list (dependencies-first) - this is NOT just "the order units were discovered" (TPasSemaProject's model list is BFS-discovery order, which does not guarantee a unit's own dependencies were already fully discovered first); it needs an actual topological sort over the uses graph to match how dcc really sequences unitinitializationsections. -
Generate method body from declaration (Ctrl+Shift+C) - given a method or routine declaration with no implementation yet, emit an empty matching body (right unit, right place, right signature), the inverse of the existing decl->impl navigation.
-
Formatter - reformat source to a configurable style, off the same AST the highlighter and navigation already use rather than a separate regex-based pass.
-
PasTree's own diagnostics, beside dcc's. Today every diagnostic is one dcc would give (its code, its wording), so what dcc accepts in silence is not reported at all - and some of it is a mistake waiting to bite. The case that prompted this (2026-10-03): a real application's server project names a unit
ina file that does not exist, in its.dprand as aDCCReferencein its.dproj. dcc does not complain - it compiles the unit of that name the search path finds (probed on dcc32 37.0; F1026 on the in-path only when there is none) - so the build is green while the project file points at a file that is gone, and which copy is compiled depends on the search path order. PasTree now resolves it the same way (a pin to a missing file is dropped, 0.91.1), and with that it, too, says nothing. What is wanted: a code range of our own that cannot be mistaken for dcc's (e.g.PT1xxx), each a warning or a hint with the severity carried on the diagnostic, never an error - dcc's verdict on whether the code compiles stays the only error source. Candidates, each something the resolver already knows at the point it decides:- an
in 'path'(or a host-pinned project file) that does not exist, with the file actually used: "X: 'gone\X.pas' does not exist, the search path's lib\X.pas is compiled"; - an
in 'path'whose file declares a different unit name than the item; - a unit the project file list (
.dproj) and the program'susesdisagree on - listed in one, absent from the other, or at two different paths; - a unit found in two places on the search path, the later copy shadowed
(which copy wins is decided by path order alone, and a reordered search
path silently swaps the code compiled).
Hosts then show them as they show the rest (the LSP server's
publishDiagnosticscarries a severity already; pastree-mcp'sstatusanddiagnostics), and the definition of done's "zero false positives" holds for them as for dcc's codes - each needs its probe that dcc really does what the message says.
- an
-
Error Insight (live diagnostics, à la Embarcadero's own IDE feature) - today's diagnostics are error-only (
E20xx) and shown as a flat list after a full analysis run; this extends it to a live, severity-classified feed (errors/warnings/hints) in its own panel, plus squiggly underlines in the editor itself, color-coded by severity, on the offending expression's own span as analysis re-runs on edit. -
Longer-term / exploratory: a compiler front-end built on the same model.
Definition of done (v1)
- Lexes and parses the entire Delphi 13 source tree (RTL/VCL/FMX/...) with zero errors.
- Roundtrip: concatenated tokens + trivia == original source, byte-for-byte.
- Golden AST tests keyed to the spec's feature numbering (e.g. test
5.4.1covers the inline-ifexpression). - Semantic analysis over the same tree produces zero false-positive
diagnostics. ✅ Reached 2026-07-30 on the flattened RTL+VCL+FMX corpus and
on the
BuildWinVCL/BuildWinFMXpackages, and extended on 2026-08-08 to every corpus we measure: two real applications (3747 and 2121 units) and a third-party library's three packages report no falseE2003under the opt-in member flag either. What is left anywhere is a unit whose source genuinely is not on the search path, plus one confirmed true positive. See the To-do for what holding that now depends on. - The tree is the compiler's. Over the self-host and the Studio source:
the tree checker reports no invariant violation and no token outside the
own-token table; every rewrite computed from the tree - the
preprocessor's decisions written out, every operator parenthesized, every
statement wrapped in
begin/end, the unit reprinted from its tree alone, and all of them at once - compiles under dcc to a byte-identical.dcu; and the structural printer reproduces every unit's tokens modulo a written normalization list, and parses back to the same tree. ✅ Reached 2026-09-28 (v0.64.0) on the Studio 37 source, Win64 and Win32, and the self-host, with three exceptions filed as findings: one recovery heuristic reads a line break in valid code (the reprint does not parse back in the token-per-line layout, 10 files), and two spellings the tree takes as one that dcc writes apart in bytes the code does not depend on (3 units). The method, the blind spots and the open findings:docs/parser-fidelity.md; held bytools\fidelity-gate.ps1.
Requirements
- Delphi 13.0 Florence or later (the parser is written in the language it parses).
License
MIT © 2026 Oleksandr Skliar.
PasTree is an independent open-source project, not affiliated with or endorsed by Embarcadero Technologies. "Delphi" is a trademark of Embarcadero Technologies, Inc.

