Restructure front end into a Cargo workspace #1

Merged

crowmaki merged 1 commit from workspace-lexer-crate into main

2026-08-21 16:47:02 +00:00

crowbot commented

2026-08-21 16:43:57 +00:00

Collaborator

Catlang — Cargo workspace (multi-crate)

Catlang is a statically-typed, C-like systems language. There is no compiler
yet; this is the start of the front end, organized as a Cargo workspace so
each concern is an isolated, fully-tested library crate and the eventual
executable is a thin CLI binary on top of them.

Workspace layout

Cargo.toml            workspace manifest: declares the member crates (this file)
lexer/
  Cargo.toml          the `lexer` library crate's own manifest
  src/
    lib.rs            the crate root: re-exports the three modules below
    token.rs          token kinds + literal payload
    source.rs         SourceBuffer, Span, and 1-based line:col positions
    lexer.rs          the single-pass scanner + its inline test suite

Building and testing

From the workspace root, cargo build builds every member and cargo test
runs every member's test suite (the lexer suite is inline in scanner.rs).

The compiler is intended to bootstrap — be rewritten in Catlang itself —
once the language is stable. Until then, Rust is the host for every pipeline
phase, and each phase is a thin, independently-testable crate in this
workspace. The future executable will be a cli crate depending on these
libraries.

Layout notes

The scanner/token/source modules were extracted into the lexer crate from a
single-crate layout; this move preserves git history (git mv).
Standard library only — no external dependencies, kept deliberately small
(see the rationale at the top of lexer/src/lexer.rs).

# Catlang — Cargo workspace (multi-crate) Catlang is a statically-typed, C-like systems language. There is no compiler yet; this is the start of the front end, organized as a **Cargo workspace** so each concern is an isolated, fully-tested library crate and the eventual executable is a thin CLI binary on top of them. ## Workspace layout ``` Cargo.toml workspace manifest: declares the member crates (this file) lexer/ Cargo.toml the `lexer` library crate's own manifest src/ lib.rs the crate root: re-exports the three modules below token.rs token kinds + literal payload source.rs SourceBuffer, Span, and 1-based line:col positions lexer.rs the single-pass scanner + its inline test suite ``` ## Building and testing From the workspace root, `cargo build` builds every member and `cargo test` runs every member's test suite (the lexer suite is inline in `scanner.rs`). The compiler is intended to **bootstrap** — be rewritten in Catlang itself — once the language is stable. Until then, Rust is the host for every pipeline phase, and each phase is a thin, independently-testable crate in this workspace. The future executable will be a `cli` crate depending on these libraries. ## Layout notes - The scanner/token/source modules were extracted into the `lexer` crate from a single-crate layout; this move preserves git history (`git mv`). - Standard library only — no external dependencies, kept deliberately small (see the rationale at the top of `lexer/src/lexer.rs`).

crowbot added 1 commit

2026-08-21 16:43:57 +00:00

Restructure the front end into a Cargo workspace a51423d31f

Extract the lexer into its own library crate so each pipeline phase can be
isolated and fully tested, with a thin CLI binary to be added later on top of
the libraries.

- Root Cargo.toml is now a [workspace] manifest (members = lexer, resolver 3).
- lexer/ is a dependency-free library crate: token kinds, SourceBuffer/Span,
  and the scanner, each in its own testable module.
- Root
running 8 tests
test lexer::lexes_char_literal ... ok
test lexer::lexes_comments ... ok
test lexer::lexes_float_literals ... ok
test lexer::lexes_let_declaration ... ok
test lexer::lexes_operators_longest_match ... ok
test lexer::rejects_unterminated_block_comment ... ok
test lexer::lexes_string_and_interpolation ... ok
test lexer::lexes_integer_and_hex_literals ... ok

test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s runs the lexer crate suite: 8 passing / 0 failing.
- CRATE.md documents the layout; AGENTS.md notes the workspace model.
- git mv preserves history for the moved files.