Duolith docs

0019. Write the data core ourselves, behind the adapter meant for TanStack Query

  • Status: Accepted (Mik, 17 Nov 2026)
  • Date: 2026-11-14
  • Decided by: Mik
  • Planning item: D23

Context

D23 recommends wrapping TanStack Query's core behind our own API, for its proven cache, retry and deduplication. Decision 0011 adopted that provisionally. Since then the walking skeleton (0012) fixed how the runtime reaches the browser: crates/emit-web carries every file of packages/client/src inside the compiler and serves it, types removed, as plain ES modules under /_duolith/runtime/. There is no bundler and no node_modules at runtime, so a bare import "@tanstack/query-core" cannot load.

Using TanStack would mean vendoring its compiled core (about 40 files, its own observer and notification scheduler) into packages/client, keeping that copy up to date by hand, and bridging two reactive systems: TanStack's observers and our state cells. What WP8 needs from it is small: a map of entries by key, one request in flight per key, stale time, retry with backoff, invalidation by key prefix, and reload on focus and reconnect.

Decision

Write the data core in packages/client/src/data/, behind the seam D23 asked for:

  • core.ts defines the interface DataCore (watch, load, read, write, invalidate, clear) and the key rules. Resources and actions talk only to it.
  • cache.ts is the one implementation, MemoryCache, about 150 lines, on top of state cells.
  • host.ts gathers everything taken from the browser (fetch, timers, storage, focus and online events, EventSource, WebSocket) into one object, so tests run against a fake network and clock.

If a later need (garbage collection of unused entries, persistence of the cache, structural sharing of results) outgrows it, a TanStack-backed DataCore can replace MemoryCache without changing the public API or any page, once the runtime can load vendored modules.

Alternatives considered

OptionWhy not
Vendor @tanstack/query-core into packages/clientA large hand-maintained copy; two reactive systems to bridge; the import paths would need rewriting to relative .js paths for the browser
Add a bundling step to duolith buildChanges the emitter and CLI, outside WP8, to serve one dependency
Wait for D23 to be revisitedBlocks WP8

Consequences

The data layer has no third-party code and loads as plain modules. We own the correctness of the cache: tests in packages/client/test cover deduplication, stale time, retry and backoff, focus and reconnect reloads, invalidation, optimistic changes stacking over loads and failures, offline replay, shared and cross-tab queues, typed keys, and cancelling the retries of keys nobody watches. Each entry keeps the server's value apart from the optimistic changes waiting on top of it, so a failed write takes out only its own change. The gaps this record listed for later are closed (see below).

What Mik decided (17 Nov 2026)

Mik compared the core with TanStack Query and chose to keep our own and add everything TanStack has built in that ours lacked. Each is in packages/client/src/data/, named in the glossary's words rather than TanStack's, and tested in packages/client/test/data-core-features.test.ts:

TanStackOursWhat it does
gcTimeforgetAfter (default five minutes)A value nobody watches is forgotten, so memory does not grow. Screens dropped from the stack stop watching what they loaded.
persisterssaveOnDevice: true | { keepFor } (saved.ts)The last value is kept in the host's storage and shown at once on the next start, counted as loaded when it was saved, so staleTime still holds.
structural sharingalways on (share.ts)A reload equal to what is shown changes nothing; a partly changed one keeps every unchanged list item and object, so bindings and keyed lists update only where something changed.
refetchIntervalrefreshEveryReloads while followed, skipping turns while offline.
selectselectPicks or reshapes the value per caller; runs again only when the loaded value changes.
placeholderData: keepPreviousDatakeepPrevious, with showingPreviousThe last key's value stays shown while a new key loads.
enabledwhen, with waitingNothing loads until a condition, read as state, is true.
prefetchQueryprefetch(key, loader, options)Loads ahead of a screen, without watching; forgotten after forgetAfter unless asked for.
devtoolsinspectData(), watchData(), __duolith_dataEvery key with its value, freshness, watchers, waiting changes and when it is forgotten.

DataCore gained prefetch, inspect and onChange, so a TanStack-backed core could still replace MemoryCache behind the same seam. The offline write queue and typed API clients, which TanStack leaves to plugins, stay as they were.