Level 3: Build Output Verification

Testing built files, SSG output, templates, and bundler configuration.

What Level 3 Tests

Level 3 tests verify build output -- the actual files produced by your build tool. Instead of testing source code directly, you run the build and inspect the results.

Typical targets:

• SSG (Static Site Generation) output HTML

• Template rendering results

• Bundler output (correct chunks, code splitting)

• Generated configuration files

• Build-time data transforms (MDX compilation, content collections)

Tools

Run the Build First

Level 3 tests read files under dist/ -- they never trigger a build themselves. Run the build before running the suite, or the tests read stale (or missing) output:

pnpm build && pnpm vitest run --project build

--project build scopes the run to a named build project if your Vitest config defines one (see Vitest Patterns); without a named project, a plain vitest run after the build works the same way.

To avoid relying on the developer remembering the build step, wire it into Vitest itself with globalSetup so a bare vitest run always exercises a fresh build:

// vitest.config.ts
import { defineConfig } from "vitest/config";

export default defineConfig({
  test: {
    globalSetup: ["./tests/build-setup.ts"],
  },
});

// tests/build-setup.ts
import { execSync } from "node:child_process";

export default function setup() {
  execSync("pnpm build", { stdio: "inherit" });
}

A Built Artifact With More Than One Writer Is Shared Mutable State

The Warning above assumes a single writer: you build, then you read. That holds for a solo pnpm build && vitest run and stops holding the moment anything else can write dist/ while the suite reads it. Package lifecycle hooks are the writers people miss, because nobody types them -- they fire as a side effect of commands that look read-only.

Treat a built artifact the way you would treat a shared mutable variable: enumerate every writer before you trust it.

Enumerate the Writers

Before you rely on any build-once/read-many setup, list every process that can write the artifact:

• Lifecycle hooks -- prepare, prepack, prepublishOnly, postinstall. Nobody invokes these; the package manager does.

• Watchers -- tsup --watch, tsc --watch. These write for as long as the dev server is alive.

• Force-rebuild scripts -- test lanes, workspace builds, local heavy lanes that rebuild unconditionally on every invocation.

• The publish path -- npm pack, npm publish, and whatever they run on your behalf.

From a real pnpm monorepo, packages/<pkg>/dist/ had four independent writers -- one from each category:

Nothing mechanically prevented these from overlapping, so the project fell back on a human-remembered rule in its CLAUDE.md -- don't run the test lane while pnpm dev is live. That is the shape of the problem: a race that only discipline prevents. What it produced, over time:

• A --watch build wiped dist/ and regenerated no .d.ts at all, until clean was made conditional on watch mode (clean: !options.watch).

• A dead .d.ts watcher left dist/*.d.ts frozen while the JS kept updating, so the suite -- in the package's own words -- typechecks cleanly against stale types instead of failing loudly. A green suite against an artifact that no longer matches its source, caused by concurrency, not by forgetting to build.

• A workspace build guard that, by its own documentation, checks existence rather than freshness: it accepts a stale artifact by design and hands the freshness question back to a human.

--ignore-scripts Is a Flag, Not a Property

Here is the incident that makes the abstraction concrete. A fast-tier unit test asserted the npm tarball's file list, and passed --ignore-scripts precisely so the call would be read-only:

execFileSync("npm", ["pack", "--dry-run", "--json", "--ignore-scripts"], { cwd: PKG_ROOT });

Whether that flag actually suppresses prepare turns out to depend on the npm version:

Local dev ran Node 24 and saw a genuinely read-only pack. CI pinned Node 22, so the same command fired prepare, which ran tsup with clean: true, which deleted dist/ outright -- while a sibling spec spawned the package's bin/*.mjs, whose static import of ../dist/eject/index.js then failed with ERR_MODULE_NOT_FOUND. Mid-publish. The release stopped after one of three packages.

The Stray Log Line Was the Tell

Months before the release broke, CI had already seen a build's own log line -- [copy-theme-packs] dist/theme-packs/ OK -- printed ahead of the pack's JSON, breaking JSON.parse. That string is emitted only from the build's onSuccess chain, so it was direct proof that the lifecycle was already running inside a supposedly read-only command. The fix applied at the time was a tolerant JSON scanner that skipped past the stray line. The symptom was hidden, the rebuild kept running, and it eventually took down a release.

The Rule: Own an Immutable Snapshot

A test that reads a built artifact must own an immutable snapshot of it, and any build-freshness key must hash the artifact's own bytes, not just its sources.

Two concrete shapes.

Shape 1 -- snapshot the artifact. Pack it (npm pack, then extract the tarball) or copy {bin,dist} into a scratch directory, and point the test at the copy. Whatever the lifecycle does to the live dist/ afterwards, the test is reading bytes no one else can reach.

// tests/setup/snapshot-package.ts
import { cpSync, mkdtempSync } from "node:fs";
import { join } from "node:path";

const PKG_ROOT = join(__dirname, "../..");

export function snapshotPackage(): string {
  // Inside the package root on purpose -- os.tmpdir() breaks bare-specifier
  // resolution. Prefix is gitignored and removed in teardown.
  const dir = mkdtempSync(join(PKG_ROOT, ".artifact-snapshot-"));
  for (const entry of ["bin", "dist", "package.json"]) {
    cpSync(join(PKG_ROOT, entry), join(dir, entry), { recursive: true });
  }
  return dir;
}

Shape 2 -- hash the output, not only the input. If you cache builds behind a freshness marker, include the built output in the key. A key derived from sources alone still matches when a concurrent writer truncated, half-wrote, or deleted the very artifact the marker vouches for -- the sources did not change, so the marker happily accepts a dist/ that no longer exists in the state it describes. See Mechanism 2: skip the rebuild when nothing changed for the freshness-hash recipe this extends.

What a Guard Against This Can Honestly Claim

A mechanical guard helps: scan the default lane's specs for launched build or package-lifecycle commands -- npm|pnpm|yarn followed by pack|publish|install|ci|rebuild|run build, a bare tsup, and so on -- and fail any spec that launches one without an explicit annotation.

Document its limit in the same breath. A string scanner catches launched commands only. It does not catch a shell wrapper, a script alias, or a direct rmSync/writeFileSync under dist/. The mechanically enforced rule is therefore narrower than the invariant it protects, and the docs have to say which is which: the guard enforces "no spec launches a build command"; the invariant is "no spec writes the artifact".

Example: SSG Output Verification

// tests/build-output.test.ts
import { describe, it, expect } from "vitest";
import { readFileSync } from "fs";
import { join } from "path";

const DIST = join(__dirname, "../dist");

describe("build output", () => {
  it("generates index.html with correct title", () => {
    const html = readFileSync(join(DIST, "index.html"), "utf-8");
    expect(html).toContain("<title>My Site</title>");
  });

  it("generates sitemap.xml", () => {
    const sitemap = readFileSync(join(DIST, "sitemap.xml"), "utf-8");
    expect(sitemap).toContain("<urlset");
  });

  it("includes all expected pages", () => {
    const pages = ["index.html", "about/index.html", "docs/index.html"];
    for (const page of pages) {
      const content = readFileSync(join(DIST, page), "utf-8");
      // Case-insensitive: toolchains emit the doctype as lowercase `<!doctype html>`.
      expect(content.toLowerCase()).toContain("<!doctype html>");
    }
  });
});

Example: MDX Formatter Contract Testing

A real-world pattern from mdx-formatter: the Vitest suite tests the Rust formatting engine by running it on fixture files and comparing output. This is Level 3, not Level 1, because format() is a thin JS wrapper that drives a compiled native engine produced by the build -- the object under test is that build artifact, not hand-written TypeScript logic. (If your formatter were pure JS/TS with no compiled backend, this same idempotency check would belong at Level 1 instead.)

// tests/format.test.ts
import { describe, it, expect } from "vitest";
import { readFileSync } from "fs";
import { join } from "path";
import { format } from "../src/format";

describe("mdx formatting", () => {
  it("is idempotent", () => {
    const input = readFileSync(join(__dirname, "fixtures/sample.mdx"), "utf-8");
    const first = format(input);
    const second = format(first);
    expect(first).toBe(second);
  });
});

Executing Emitted Code

The examples above compare emitted output as static text -- read the file, assert on its contents. But some build tools generate JavaScript that runs: adapters that wrap a user handler, bundler wrappers, scaffolders. For these, a string-equality check proves the template was written -- it does not prove the generated code behaves correctly when executed. To verify that, you have to write the artifact, import it, and run it.

The reusable recipe is temp dir + dynamic import + env stub:

1. Write a synthetic inner bundle into an mkdtemp scratch directory.

2. Run the emit step (emitWorker()) to produce a real _worker.js.

3. import(pathToFileURL(...)) the emitted file, then call default.fetch(new Request(...), envStub).

4. Assert that (env, ctx) are threaded through correctly -- the wrapper your build emits must pass env to the user handler.

5. Clean the scratch directories in afterEach.

// tests/emit-worker.test.ts
import { mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { pathToFileURL } from "node:url";
import { afterEach, describe, expect, it } from "vitest";
import { emitWorker } from "../src/build.js";

let scratchDirs: string[] = [];

afterEach(async () => {
  for (const d of scratchDirs) {
    await rm(d, { recursive: true, force: true });
  }
  scratchDirs = [];
});

async function scratch(): Promise<string> {
  const d = await mkdtemp(join(tmpdir(), "emit-worker-"));
  scratchDirs.push(d);
  return d;
}

describe("emitWorker threads env/ctx into the inner bundle", () => {
  it("the generated wrapper passes env through to the user handler", async () => {
    const dir = await scratch();

    // A synthetic inner bundle that reads env from the request scope
    // the wrapper sets up (here via AsyncLocalStorage), then echoes it.
    const inputPath = join(dir, "inner.mjs");
    await writeFile(
      inputPath,
      `import { AsyncLocalStorage } from "node:async_hooks";
const STORAGE_KEY = "__adapter_als__";
function getStore() {
  const g = globalThis;
  let als = g[STORAGE_KEY];
  if (!als) { als = new AsyncLocalStorage(); g[STORAGE_KEY] = als; }
  return als.getStore();
}
export default {
  async fetch(request) {
    const store = getStore();
    const env = store?.env ?? {};
    return new Response(
      JSON.stringify({
        token: env.API_KEY ?? null,
        ctxKind: typeof store?.ctx?.waitUntil,
      }),
      { status: 200, headers: { "content-type": "application/json" } },
    );
  },
};
`,
      "utf8",
    );

    // Run the real emit step to produce a real _worker.js.
    const out = await emitWorker({
      inputBundlePath: inputPath,
      outdir: join(dir, "dist"),
    });

    // Import the emitted artifact as ESM and drive it with a
    // synthetic Request + env + ctx -- no wrangler / miniflare needed.
    const worker = (await import(pathToFileURL(out.workerPath).href)) as {
      default: {
        fetch: (
          request: Request,
          env: Record<string, unknown>,
          ctx: { waitUntil: (p: Promise<unknown>) => void },
        ) => Promise<Response>;
      };
    };

    const envStub = { API_KEY: "sk-test-1234" };
    const ctx = { waitUntil: () => {} };
    const request = new Request("https://worker.test/api/foo", { method: "POST" });

    const response = await worker.default.fetch(request, envStub, ctx);
    const body = (await response.json()) as { token: string | null; ctxKind: string };

    // The headline check: env actually reached the inner handler.
    expect(body.token).toBe("sk-test-1234");
    // ctx was threaded too -- its surface matches the runtime shape.
    expect(body.ctxKind).toBe("function");
  });
});

Blind Spots

When to Use Level 3