Duolith docs

0023. The TextMate grammar is generated from the language definition

  • Status: Accepted (provisional)
  • Date: 2026-11-14
  • Decided by: integrator (internal only)
  • Planning item: none; WP2 grammars and WP3 editors v0

Context

Every editor needs highlighting for page files before the language server exists, and the element list will keep growing. editors/AGENTS.md already says keyword lists are generated from spec/language/, never typed by hand. VS Code, JetBrains (through its TextMate bundle support), Sublime, Zed's fallback and Shiki all read TextMate grammars.

Decision

  • tools/generate/grammar.ts writes editors/grammars/duolith.tmLanguage.json from spec/language/elements.yaml; pnpm generate runs it and tools/check-generated.ts fails CI when the committed grammar is stale.
  • Scopes, so every theme colours page files sensibly:
    • elements from the definition: entity.name.tag.duolith keyword.other.element.duolith (themes show them as keywords);
    • capitalised tags: support.class.component.duolith (as JSX components are in VS Code);
    • any other tag: entity.name.tag.duolith;
    • prefix:name attributes such as on:press and bind:value: the prefix is keyword.control.directive.duolith; events from the definition are entity.other.attribute-name.event.duolith;
    • the script block and {expressions} embed source.ts; <style> embeds source.css; their content names (meta.embedded.block.script.duolith, meta.embedded.expression.duolith, meta.embedded.block.style.duolith) let editors switch comment and bracket behaviour by language.
  • Snapshot tests tokenise fixtures with vscode-textmate and Oniguruma, the same engine VS Code uses, and compare with committed snapshots.
  • Editor shells copy the grammar at build time; nothing commits a second copy.

Alternatives considered

OptionWhy not
A hand-written grammarDrifts from the definition the first time an element is added
tree-sitter firstNeeds a C toolchain and per-editor packaging; planned later for Neovim, Helix and Zed, and does not replace TextMate for VS Code or JetBrains
Semantic tokens from the language server onlyNothing to show until WP6, and no highlighting on GitHub or in docs

Consequences

Adding an element to spec/language/elements.yaml needs pnpm generate and, when a fixture uses the element, UPDATE_SNAPSHOTS=1 pnpm grammars:test with the snapshot diff reviewed. Directive prefixes are matched generically, so a new prefix is highlighted without a grammar change.