This page as markdown

Test API

The test file API, and what is absent on purpose.

The API for test files. Status: the user accepted it on 2026-11-11 (D15 to D18 in 02-decisions.md).

The type sketch is in experiments/api/ubq.ts. good.ts there compiles, and each line of bad.ts gives the compile error that this document claims. mock-rt.ts proves the mock call at run time on Node and Deno.

Not in this document: the CLI, the config, the fake clock, and snapshots. Each one gets its own document.

1. A full example#

import { test, describe, assert, spy, testdir, mockImport } from 'ubqtest'
import { parse } from '../src/parse.ts'

describe('parse', () => {
  test('circle', () => {
    const shape = parse('c')
    assert(shape)                       // narrows away undefined
    assert.equal(shape.kind, 'circle')  // narrows the union
    assert.equal(shape.r, 1)
  })

  test('unknown input', () => {
    assert.throws(() => parse('?'), /unknown shape/)
  })

  for (const [input, kind] of [['c', 'circle'], ['s', 'square']] as const) {
    test(`kind of ${input}`, () => {
      assert.match(parse(input), { kind })
    })
  }
})

test('greet reads the user from the db', async () => {
  const fetchUser = spy(async (id: number) => ({ id, name: 'ada' }))
  const { greet } = await mockImport(() => import('../src/greet.ts'), [
    [() => import('../src/db.ts'), { fetchUser }],
  ])
  assert.equal(await greet(7), 'hello ada')
  assert.deepEqual(fetchUser.calls[0]?.args, [7])
})

test('writes a report', { timeout: 500 }, async ({ signal }) => {
  const dir = testdir({ 'in.json': '{"n":1}' })
  await writeReport(dir, { signal })
  assert.match(readFileSync(`${dir}/report.txt`, 'utf8'), /n = 1/)
})

node test/parse.ts runs this file. So does ubq test/parse.ts.

2. Design rules#

1. The first guess must work, or the error must name the fix. Names come from node:assert/strict, Vitest, and tap, in that order of preference, because those names are the most common in existing code. A common wrong guess (test.skip, assert.ok, assert.strictEqual) is a compile error whose text is the correct call. See section 9.
2. One way to do each thing (D2). No aliases.
3. Few tokens. No chains, no wrappers, no setup code that repeats in each file.
4. Types carry the rules. An assertion narrows the type of its argument. A mock must match the module that it replaces. A comparison of a string with a number does not compile.
5. A test contains all that it needs. No hooks, and no state that a reader must look for in a different part of the file.
6. Plain code. test and describe are function calls. A loop makes many tests. An if makes a test conditional.

3. test#

test(name: string, fn: (ctx: { signal: AbortSignal }) => void | Promise<void>): void
test(name: string, options: { skip?: string | false; timeout?: number }, fn): void

• A test passes if fn returns (or its promise resolves) and one or more assertions ran.
• A test with zero assertions fails.
• Tests in a file run one at a time (D5).
• Tests must not depend on each other or on their order. The CLI runs single tests by ID, and a later step adds random order with a seed.
• name must be unique in its describe group. A duplicate is an error, because the name is part of the test ID (test/parse.ts > parse > circle).
• timeout is in milliseconds. The default is 5000, the same as Vitest and Jest.
• signal aborts when the test times out. Pass it to fetch, child processes, and timers, so that they stop with the test.
• skip takes the reason as a string. The summary prints each skipped test with its reason. false means "do not skip", which permits a condition: { skip: process.platform === 'win32' && 'needs fork' }.
• A call to test or describe from inside a running test is an error.

4. describe#

describe(name: string, fn: () => void): void
describe(name: string, options: { skip?: string | false; timeout?: number }, fn): void

A group gives its tests a name prefix and can give them options. It does nothing else.

• fn runs at once, and it must be synchronous. A promise from fn is an error.
• Groups can nest.
• There are no hooks. For shared setup, write a function and call it in each test (section 7).

5. assert#

A failed assertion throws. The test stops at its first failure.

assert(value, message?)                        // asserts value
assert.equal(actual, expected, message?)       // asserts actual is typeof expected
assert.notEqual(actual, expected, message?)
assert.deepEqual(actual, expected, message?)
assert.match(actual, pattern, message?)
assert.throws(fn, expected, message?)          // returns the Error
await assert.rejects(promise, expected, message?)  // resolves to the Error
assert.snapshot(value, message?)               // compares with saved text
assert.fail(message)                           // returns never

match patterns. One function covers "looks like this":

• A RegExp matches a string that it tests true for.
• An object matches if each key in the pattern matches the same key in the value. Extra keys in the value are fine.
• An array matches an array of the same length, element by element.
• Any other value matches by Object.is.

assert.match(res, { status: 200, body: { user: { name: /^a/ } } })

This keeps expected values short when the full object is large.

expected in throws and rejects is required. A test that accepts any error also accepts a typing mistake in the code under test.

• A RegExp tests the message.
• A class tests instanceof.
• An object is a match pattern for the error.

Both return the error, so more assertions can follow:

const err = assert.throws(() => parse('?'), ParseError)
assert.equal(err.message, 'unknown shape: ?')

rejects must be awaited. If the test ends and nothing awaited the result, the test fails with a message that says so.

snapshot compares the text of a value with the text that an older run saved (D31).

• The text of a string is the string. The text of any other value is its util.inspect form, with sorted keys and no size limit.
• Each test file has one snapshot file: ubq-snapshots/<test file path>.snap. Commit it. The key of an entry is the names of the test and the number of the call in the test: parse > circle 1.
• A value with no saved text fails. A plain run never writes the file. ubq --update <file> or ubq --update '<test id>' runs the named tests and writes their snapshots. --update with no test is an error.
• An update of one test keeps the entries of the other tests. An entry of a test that the file no longer has is removed.
• The failure shows the lines that differ, and the command that writes the new text.
• The text can differ between runtimes: Node, Deno, and Bun do not print each value the same way.

Types.

• assert(value) and assert.equal narrow, as the example shows. This works because assert is an import with a declared type. It would not work for a method of a context argument (TypeScript requires an explicit type on the receiver of an assertion function). That is the reason assertions are not on a t object.
• expected must be assignable to the type of actual. assert.equal(count, '1') does not compile when count is a number.

Assertions find the current test through AsyncLocalStorage, which has no measurable cost (03-measurements.md, section 8). An assertion that runs after its test ended fails that test.

6. Mocks and spies#

spy, spyOn#

spy<F>(impl?: F): Spy<F>
spyOn(object, key, impl?): Spy<typeof object[key]>

• A spy is a function that records each call in spy.calls, as { args, result, error }.
• spyOn replaces a method and puts the original back when the test ends.
• There are no special assertions for spies. Use assert.equal and assert.deepEqual on calls.
• With no impl, spy() returns undefined, and spyOn calls the original method. result of an async function is its promise. error is present only if the call threw.
• The key of spyOn must name a function of the object. A different key is a compile error.

mockImport#

const subject = await mockImport(
  () => import('../src/greet.ts'),
  [
    [() => import('../src/db.ts'), { fetchUser }],
    [() => import('node:fs'), { readFileSync: () => '{}' }],
  ],
)

• The first argument loads the module under test. The result is a new instance of that module, with its own state.
• Each pair names a dependency and the exports that replace it.
• All types come from the import() expressions. A mock with a wrong signature, or a key that the real module does not export, is a compile error.
• The mock exists only in the module instance that the call returns. Nothing needs a reset.
• Nothing moves to the top of the file, and the source is not rewritten. The mock functions are normal values in normal scope.

The loaders must be arrow functions of the form () => import('...'). The library reads the module path from the import() call through a resolve hook. A mock loader does not load the real dependency.

• A mock of a function with overloads (readFileSync) can be a function that fits the last overload.
• The module under test gets a new instance, and so does each project module that it imports. Packages in node_modules and builtins are not loaded again.
• If the code under test imports a name that the mock does not have, the call fails with an error that names the export to add.
• A mock with no default export gets one: the mock object. So import fs from 'node:fs' works with a mock of named exports.

Built in src/mock.ts. Node and Deno use the hooks of module.registerHooks. Bun has no such hooks, so the library uses Bun.plugin there. Differences on Bun:

• The library writes the mock number into the import paths of each module instance, when Bun loads the instance. It finds the imports by their text: from '...', import '...', import('...'). An import() with a path that is not a string literal gets the real module.
• A mock loader of a package or a builtin loads the real module one time. Bun has no resolve hook for them. The library reads the name from the text of the loader, and finds the package from the project root.
• The error for a mock that lacks an export names the file by its path from the project root (src/db.ts), not as the code under test wrote it (./db.ts).

7. Fixtures and cleanup#

A fixture is a function that a test calls. It sets something up and removes it when the test ends. The test does not have to ask for the removal.

testdir(files?: Files): string     // a new folder, removed at test end
env(vars: Record<string, string | undefined>): void   // set or delete, put back at test end
cleanup(fn: () => void | Promise<void>): void         // run fn at test end

• testdir({ 'a.txt': 'hi', sub: { 'b.json': '{}' } }) makes the files and returns the path.

• The path comes from the test ID: .ubq/testdir/<test file path>/<test name>, with characters that are not safe in a path replaced by -. A second call in the same test adds -2. An agent can find the folder with no need to read the output.

• When the test passes, the folder is removed. When the test fails, the folder stays, and the failure block prints its path. The next run of that test removes the old folder first.

• All of .ubq/ is ignored by git: the tool writes .ubq/.gitignore, and one .ubq/ line in a project .gitignore also covers it. The folder is in the project, so git worktrees do not collide.

• cleanup functions run in reverse order. An error in one fails the test.

• A fixture call outside a test (at the top level of the file) lasts until the file ends. This is the way to share a costly resource: const server = await startServer(); cleanup(() => server.close()).

• A fixture at the top level must come before the first test call, and the test calls must have no await between them. The cleanup functions of the file run after the last test in the queue. A test call or a fixture call after that is an error.

• A fixture call after its test ended is a failure of that test (late). Each cleanup function has the time limit of its test.

fakeClock#

const clock = fakeClock({ now?: number | Date })
clock.now                 // milliseconds since 1970
clock.timers              // count of timers that wait
await clock.tick(ms)      // move forward, run each timer that is due
await clock.runAll()      // move from timer to timer until none waits
clock.restore()           // the real clock comes back

• The clock belongs to the test that made it. The real clock comes back when the test ends, or at clock.restore(), or at the end of a using clock = fakeClock() block. One clock at a time.
• It replaces all that it can (D30): setTimeout, setInterval, setImmediate and their clear functions, Date, performance.now, process.hrtime, AbortSignal.timeout, and the same functions in node:timers and node:timers/promises.
• Time moves only in tick and runAll. Timers run in the order of their times. Promise callbacks run after each timer, so a chain of await sleep(10) calls ends in one tick.
• tick and runAll return promises. There is no sync form.
• runAll fails after 10000 timers: a setInterval never ends.
• The time limit of the test, and the times in reports, use the real clock.
• queueMicrotask and process.nextTick are not replaced.

To share setup between tests, write a function:

const setup = () => {
  const dir = testdir({ 'config.json': '{}' })
  env({ APP_HOME: dir })
  return { dir, app: createApp() }
}

test('starts', () => {
  const { app } = setup()
  assert.equal(app.state, 'ready')
})

Each test gets new values. Nothing is shared by accident.

8. What is absent, and what to use#

9. Errors for wrong guesses#

Names that an agent is likely to try exist in the types as properties that cannot be called. The compile error contains the fix. At run time, the same call throws the same text.

test.skip('x', () => {})
  error TS2349: This expression is not callable.
  Type 'Hint<"Use test(name, { skip: \"reason\" }, fn)">' has no call signatures.

it, expect, and the hook names are not exports. An import of one fails with the normal "no exported member" error. The agent guide in the package lists the replacement for each.

10. Where each part comes from#

11. Open points#

• match patterns have no type matchers (tap permits { id: Number }). Add them only if tests need them often.
• Snapshots across runtimes. The text comes from util.inspect, which differs between runtimes. A formatter of our own would fix this.