parser: decompose into atomic testable steps (docs/parser/) #3
crowmaki
Loading…
Reference in a new issue
No description provided.
Delete branch "docs/parser-decomposition"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What
Decomposes the parser — the second pipeline phase, after
lexer/— into clear, atomic, testable steps, and locks the first round of design decisions the decomposition surfaced.Files
docs/parser/README.mddocs/parser/00-scaffold.mdparser/crate under the workspace;parse() -> Result<Program, ParseError>; golden-dump harnessdocs/parser/10-ast.mddump()inspector so tests assert a printed formdocs/parser/20-type-refs.mddocs/parser/30-expressions.mddocs/parser/40-statements.mdlet/const)docs/parser/50-functions.mdfunction/ lambda declarations,::/this/ overloadsdocs/parser/60-user-types.md#[SOA]docs/parser/70-top-level.mdmain, attribute attachment, hello-worlddocs/prd/language-grammar.mdlexer/src/token.rsimpl/operatorasKeywordKindlexer/src/lexer.rsEach parser step follows the same skeleton: Deliverable → Grammar → AST → parser-function names → Acceptance (inline Rust unit tests and the fixture(s) it completes) → RED→GREEN note. Steps are bottom-up so each is independently testable, and each names exactly which of the 32 fixtures in
tests/fixtures/parse/it completes.Design decisions now locked
These came out of review; the PRD EBNF and the lexer token changes implement them:
docs/prd/language-grammar.mdas an EBNF appendix — the language's first formal grammar. Parser steps defer to it.implandoperatorare reserved keywords. Both were plain identifiers; the lexer now emitsKeywordImpl/KeywordOperator(enum member + scanner map + a unit test). No parser-sideIdentifier("impl")contextual hack.Two further points (the golden-
dump()test contract and every AST node carrying a sourceSpan) were confirmed as accepted defaults rather than open questions and are recorded as such in the README.Notes
cargo fmt-clean and passescargo test(9 passed, 0 failed).main; the docs and lexer change depend on nothing not already onmain.Adversarial grammar review (latest)
The EBNF (
docs/prd/language-grammar.md) went through an adversarial reviewagainst the spec, the lexer token vocabulary, and all 32
parse/fixtures.The reviewer confirmed 5 CRITICAL + 3 MAJOR + 2 MINOR findings; two additional
gaps (functional static/instance
Type::m = (...) ->, and parameter'd functiontypes
(f32) -> float) were caught on independent re-derivation. All fixesare applied so that every fixture in
tests/fixtures/parse/is nowderivable from the grammar. Notable:
-> { ... }) — the return type is inferred[N]T,[]T { ... },[..]T { ... }let/const (x, y) = ...with_;optional&&=/||=restored to compound-assignment (lexer emits them)ismoved to its own stricter level; unbound spana..spelled out?disambiguation rules pinned (no silent guesses)Response to the adversarial grammar review of docs/prd/language-grammar.md. Confirms the reviewer's findings and fixes both its and inherited gaps: - array-type literals in expression position: [N]T / []T {...} / [..]T (static-arrays, array-initialization, array-iteration, c-style-loop) - bare arrow with no return,type: 'function f(this) -> { ... }' (inferred) - destructuring declarations: const (x, y) = point; with '_' target - interface-method trailing ';' now optional (fixture omits it) - &&= and ||= restored to compound-assignment (lexer emits them) - functional static/instance spelling: Type::m = (...) -> (no 'const') - is at its own level 12, looser than relational, in both table and chain - unbounded span a.. as additive (..|...)? [additive], requires a start - paren-first disambiguation (function-type/lambda vs tuple vs parenthesized) stated for and type position - LL(2) lookahead note for '?' postfix-vs-ternary conflict - multiplier vs additive precedence preserved (postfix > unary > cast)