This page as markdown

Architecture Plan

The parts, the process model, events, coverage, and the store.

This plan follows the goals in 00-initial-prompt.md, the ideas in 01-agent-ideation.md, and the decisions in 02-decisions.md. Each design choice below points at a number in 03-measurements.md (written as "M 2.3" for section 2.3 of that file).

Status: the user accepted the plan on 2026-11-11, with the answers in 02-decisions.md (D9 to D14). No product code exists yet.

Fixed points:

• The package is named ubqtest. The executable is named ubq (D13).
• The minimum Node version is 24, because the plan needs type stripping, node:sqlite, module.registerHooks, using, and the compile cache with no flags. Raise it when a feature needs a newer version (D12).
• The end state is one npm package with no dependencies at install time. The source has separate parts, and the build makes one small bundle (D13).

1. Parts#

test file ──imports──> core library
                          │ publishes typed events
                          ▼
                 diagnostics_channel
                          │
        ┌─────────────────┼──────────────────┐
        ▼                 ▼                  ▼
   text sink          db sink          pipe sink (fd 3)
   (stdout)          (SQLite)               │
                                            ▼
                                         runner ──> text sink
                                                ──> db sink

One test process has two possible setups:

• Plain program. node test/foo.ts. The core library attaches the text sink and the database sink in the same process.
• Child of the runner. The runner sets an environment variable. The core library attaches only the pipe sink. The runner owns stdout and the database.

The sinks are the same code in both setups. The runner feeds them events that it read from pipes. A plain program feeds them events from diagnostics_channel.

2. Process model#

One process for each test file. Tests in a file run one at a time. Files run in parallel.

Reasons:

• Coverage for each test needs serial tests in a process (M 2.4).
• A process for each file removes shared state between files. The failures that shared state causes depend on run order, and those are the hardest failures for a weaker model to debug (D3).
• The cost is known: about 41 ms of CPU for each file under load (M 3). The prototype ran 200 files in 747 ms with coverage on (M 5). That ties or beats each Node-based tool that isolates files (M 4).

Worker threads are not in the first version. They save 25% of the time (M 5), but process.chdir does not work in a worker, and process.env, signal handlers, and native addons are shared with other files. Shared processes ran the suite in 191 ms (M 5). Both stay as later options for projects that accept less isolation (D9).

Fast path for one file. When the selection contains one file, the CLI does not start a child. It turns on the compile cache and coverage, then imports the test file into its own process. The prototype measured 37.5 ms for this case (M 5). This is the common case in an edit loop.

Scheduling. The runner starts files in this order: files that failed in the last run, files that cover changed code, then all others with the slowest first (from stored durations). The first failure prints at once.

Child flags. The runner starts each child with the compile cache on (saves 19 ms for each file, M 3) and with a preload module that starts coverage before any user module loads. A plain program has no preload, so it loses coverage of module top-level code that ran before the core library loaded (M 2.4). The tool marks such a run in the store as "partial coverage".

3. Events and transport#

The event types, as a sketch. The real definitions get their own document before any code.

Each event has a version field. A passing assertion carries only the test ID and ok, so the common case stays small.

Transport: JSON lines on file descriptor 3. The child collects the events of a test and writes them with one sync write at the end of that test.

• Speed is not a factor. All methods cost about 1 microsecond for each event (M 6).
• A sync write cannot be lost if the process exits or dies after the write returns.
• JSON lines are plain text. A person can read them, and any runtime can write them.
• Stdout stays free for the output of the code under test.

Children do not write to the database. Writers to one SQLite file wait for each other (M 6).

On Deno, writeSync(3) fails (M 9): Deno has its own numbers for open files. The process still has the descriptor, so the pipe sink opens /dev/fd/3 there and writes to that.

4. Coverage#

Method: V8 precise coverage through node:inspector, with callCount: true, detailed: true. This is the only mode that gives correct data for each test (M 2.1).

Flow:

1. Start coverage as early as possible in the process.
2. Take one snapshot after the test file loaded. It lists each module that loaded, which is the dependency list of the file.
3. Take one snapshot at each test end. Keep ranges for project files only. Drop node_modules, the tool itself, and runtime internals. Remove the ?mock=N suffix from URLs (M 10).
4. Publish a coverage event with byte offsets.
5. The store converts offsets to lines when a query needs lines. Offsets point into the original TypeScript source (M 2.4).

Cost and how the plan pays for it:

• A snapshot costs 0.13 to 3.5 ms, by count of loaded functions (M 2.3). A process for each file keeps that count low.
• Coverage counters slow hot loops by up to 13 times and typical code by about 6% (M 2.2). The plan accepts this (D1). Other tools that use V8 coverage pay the same cost.
• Subtests: a snapshot at the end of each leaf test. A parent test gets the union of its children plus its own code.

Source counters, for Bun#

Bun has no usable API for per-test coverage (M 11), so the Bun collector adds counters to the source (D10).

• A load plugin (Bun.plugin) parses each project file and inserts a counter at each function start and each branch. It inserts text on the same line, so line numbers in stack traces stay correct (M 11). Bun then compiles the TypeScript as usual.
• Each counter maps to a source range. The collector publishes the same coverage event as the V8 collector, so the store and the queries do not change.
• A snapshot reads the counter array and sets it to zero. This costs microseconds, and the counters do not stop optimization.
• The runner starts each Bun child with --preload. A plain bun test/foo.ts has no preload, so the plugin cannot change static imports (M 11). The tool states that such a run has no coverage, and names the bunfig.toml preload line that fixes it.
• The plugin needs a TypeScript parser. It goes into the bundle, so the package still has no dependencies at install time.
• The transformed source is cached by content hash.

Source counters could also replace V8 coverage on Node and Deno later, if snapshot cost or hot-loop cost becomes a problem. They need a preload there too.

A separate project can propose a per-test coverage API to Bun. When Bun has one, the Bun collector can drop the counters.

5. Store#

One SQLite file at .ubq/runs.db in the project root. The tool writes .ubq/.gitignore with * in it, so the folder never goes into git. The folder is in the project, so two git worktrees do not share state.

Tables, as a sketch:

Rules:

• The writer commits a batch every 50 ms while a run is in progress, so other processes can query partial results.
• Cost is low: 1 ms to open, 0.74 microseconds for each row (M 7).
• Retention: status and duration for the last 50 runs. Coverage, output, and assertion detail for the last run of each test only. Both limits need a check against real project sizes.

Queries the CLI must answer (from 01, sections 3 to 6): failures, show <test>, diff <run> <run>, history <test>, slow, uncovered <file>, covers <file>:<line>, and a raw SQL mode.

6. Test IDs and selection#

• A test ID is the file path plus the names from the root test to the leaf: test/parse.ts > parse > empty input.
• Two tests with the same name under the same parent are an error. This keeps IDs unique and stable.
• The CLI also accepts file:line. It selects the test whose source range contains the line.
• Selection happens in the test process. The file still loads, and tests that do not match do not run. The summary states how many tests the selection left out.
• A selection that matches zero tests is an error (D2).

Selection modes:

--changed works from the store:

1. Compare the content hash of each file with the hash in the last run.
2. Select each test that ran code in a changed file, and each changed or new test file.
3. If a changed file has no coverage data (a config file, a fixture, a lock file), run all tests and state the reason.

The first version works at file level. A later version can use ranges, so that a change in one function selects only the tests that ran that function.

--repeat <n> goes with each selection. It runs the selected tests n times, one repeat after the other, each in new processes. The run has one result for each test: fail if the test failed one time or more, with the events of its first failure and the two counts (test-end.repeat). The sinks get these results after the last repeat, so the store has one run, and the text has one block for each failed test.

--baseline <commit> goes with files and test IDs, or with no selection (D26):

1. Make a git worktree of the commit in .ubq/baseline/.
2. Run ubq there, on the test files of the selection that the commit has. The run is at file level: a test ID or a line selects its file. That run has its own store in the worktree.
3. Read the result of each test from that store, and remove the worktree.
4. Run the selection in the working tree. The text leaves out the block of a failure that the baseline has too, and ends with the comparison.

No result cache in the first version. A cache that skips a test by mistake reports a pass that did not happen. File reads and environment variables are inputs that the dependency list does not contain. --changed gives most of the speed, and the user asks for it by name, so the output can state plainly what did not run.

7. Failure checks#

These checks make a wrong pass hard to produce (D2, D3).

Console capture: the core library replaces the write functions of stdout and stderr in a child. AsyncLocalStorage gives the current test. The text becomes an output event.

8. Timeouts and hangs#

Async tracking is off in a normal run, because an init hook makes await-heavy code 5 times slower (M 8).

The watchdog for a child process is a timer in the runner. It costs the child nothing. A process with no runner above it (a plain program, or one file in the CLI process) has a watchdog thread, because its own timers cannot run while it is in the loop. A thread in each child would add about 300 ms to a 200-file run.

When a child ends, the runner kills the process group of the child, so processes that its tests started and did not stop die too.

For creation stacks ("which line opened this server?"), the runner runs a timed-out test one more time, alone, with the init hook on. The result is a hang event. The limit is 3 such reruns for each run. This is on by default, and the hangRerun key of the configuration turns it off (D14). Built in src/hang.ts; the status document has the limits.

9. Output#

Two text sinks, as in D4:

• Compact (default): nothing for a pass, one block for each failure, one summary line.
• CI (when CI is set): the same, plus console output, full diffs, hang reports, seed, and runtime for each failure.

Each failure block ends with the command that runs that test again. The summary line has a fixed format and includes the run ID.

Exit codes, as a proposal:

10. Module mocks#

module.registerHooks supports this with no flags (M 10):

const subject = await t.mockImport<
  typeof import('../src/subject.ts'),
  { '../src/dep.ts': typeof import('../src/dep.ts') }
>('../src/subject.ts', {
  '../src/dep.ts': { fetchUser: id => ({ id, name: 'x' }) },
})

• The type checker rejects a mock that does not match the real export (M 10).
• Each call gives a new module instance. Nothing leaks into the next test.
• No hoisting and no source transform.

The exact signature belongs in the API document. The mechanism is proven.

11. Runtimes#

The order is Node, then Bun, then Deno. Bun comes before Deno because of its growth in agent workflows (D10).

The V8 collector is for Node only (D33).

12. Speed budget#

Targets on the machine in 03, for the suite in M 4. A benchmark in the repository must check these in each change.

The one-file targets were met in step 3 (45.5 ms and 64.6 ms) and are not met after step 4. A process with no runner above it now starts a watchdog thread, so that a loop that does not return cannot wedge it (section 8). The thread costs 3 to 4 ms in a file of 10 tests. The machine was also about 2.5 ms slower for each process start on the day of the step 4 numbers (31.3 ms for an empty Node program, 28.7 ms before).

The 200-file target is not met. A child process takes 44 ms: 29 ms for Node, 1.9 ms to start coverage, about 5.5 ms for 11 snapshots, and about 8 ms to load and run the library and the test. A snapshot costs 0.43 ms in a real test process, because the cost follows all functions that the process loaded, with the ones in Node itself. The prototype child took 40 ms. The run store adds about 50 ms to the run. The known ways to get faster are source counters in place of V8 snapshots (section 4), and the background server (below).

The first two targets were 40 and 65 ms before step 2. The prototype did not have the store, and its 10 snapshots were cheaper. In the real library, 10 tests take 4.8 ms (0.48 ms for each, most of it the snapshot), coverage start takes 1.8 ms, and the store takes about 1.5 ms.

Where the plan stands against other tools (M 4):

• Faster than node --test (1462 ms) and Vitest default (1529 ms) by 1.7 times, and faster than Jest with coverage (1898 ms) by 2 times, with per-test coverage on.
• Slower than Jest with no coverage (778 ms) and deno test --parallel (671 ms).
• Slower than Vitest with isolation off (343 ms). Shared processes would beat it (191 ms).
• Slower than bun test in all cold cases. Bun starts in 5 ms and runs all files in one process. A tool on Node cannot match 38 ms for 200 files, and the user accepts that (D11).

The speed goal is: CI runs never get near a timeout, and the edit loop feels immediate (D11). --changed runs the few tests that matter, and stored results answer questions with no run at all.

Later speed work, in order of expected gain:

1. A background server that keeps warm child processes ready. It removes most of the 27 ms start from the edit loop. Built: 11-server.md.
2. Range-level --changed.

Worker threads and shared processes are not planned (D34).

13. Build order#

Each step ends with a check that must pass before the next step.

Until step 3, the tool's own tests run on node:test.

14. Open points#

The user answered the first set of questions (D9 to D14). These points are still open, and each one has a step that settles it:

• Source counters on Node in place of V8 snapshots, to get nearer to the 200-file target. The instrumenter exists (D22). The user has not decided this.
• Hang reports on Bun (step 5).
• Retention limits for the store, against real project sizes (step 2).