Publishing-Pipeline Verification

The publish pipeline is a test lane no execution tier exercises -- lifecycle hooks that run only at publish time rot until a release fails on them, and a local heavy lane on a different toolchain than CI is blind by construction. Platform-binary packages add three transport failure modes on top.

A publish is the one pipeline stage that cannot be undone. A red check, a failed deploy, a bad build are all retryable; a version pushed to npm is permanent, and a multi-package release that dies halfway leaves the registry in a state no re-run fully repairs. That makes the publish pipeline worth verifying as a lane in its own right.

This page covers it in two parts. The first is package-shape-agnostic and applies to any repo that publishes anything: gates that run only at publish time, and the toolchain gap that hides them from your local heavy lane. The second is the special case of packages that ship platform binaries, where three transport-level failure modes live between pack and a working install.

Publish-Only Gates Are a Lane No Tier Runs

The execution tier model covers five places a test can run: the local fast pass (T0), the PR gate (T1), a split e2e suite (T2), a scheduled re-exam (T3), and the local heavy lane (T4). A lifecycle hook that runs only on npm publish sits outside all five. prepack, prepublishOnly, and their siblings fire on the release path and nowhere else, so nothing exercises them between releases. They rot silently, and the rot surfaces at the moment it costs the most.

The shape is easy to miss, because the hook looks like a gate. In one repo, a package's prepack chained fifteen check-*.mjs contract guards plus a codegen drift check -- a serious verification suite by any measure. Grepping every workflow file and the local heavy lane for those fifteen script names returned zero hits. Their first-ever execution was inside a three-package lockstep publish driven by an autonomous release script: v5.6.1 died halfway through, one package already on npm and the other two stranded. A rotted check in that position reds a release, not a PR.

One definition, two callers

The rule: any lifecycle hook that runs only on publish must have a CI job invoking the same script under a different name. Do not copy the hook's commands into a workflow -- that is a second definition, free to drift from the first. Extract the hook body into one named script, have the hook delegate to it, and call that same script from a PR job.

{
  "scripts": {
    "check:prepack-contract": "node scripts/check-prepack-contract.mjs",
    "prepack": "pnpm run check:prepack-contract"
  }
}

The release path and the PR gate are now two callers of one definition. A guard added to check-prepack-contract.mjs is exercised on the next PR instead of the next release, and the hook cannot quietly gain a step that CI does not run.

Make the real lifecycle path run on every PR

Calling the named script proves the script works. It does not prove the hook works -- a typo in the prepack key, a package manager that fires a different hook set, or a files field that omits the script all break the release path while the direct call stays green. Run a pack in CI so the actual lifecycle path executes:

# .github/workflows/pr.yml (excerpt) -- the publish-only gates, pulled into T1.
jobs:
  publish-gates:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          # One declaration of the toolchain major, read by CI and local dev alike.
          node-version-file: .nvmrc
      - run: pnpm install --frozen-lockfile

      # 1. The named script the prepack hook delegates to -- the direct call.
      - run: pnpm run check:prepack-contract

      # 2. The real lifecycle path: `pack` fires `prepack`, so a green here
      #    proves the hook itself, not a re-implementation of it.
      - run: pnpm pack --dry-run

      # 3. `prepublishOnly` is NOT fired by pack -- its commands are mapped to
      #    CI steps explicitly. Keep these in sync with the hook.
      - run: pnpm build
      - run: pnpm test

pnpm pack --dry-run (or npm pack --dry-run) assembles the tarball the registry would receive and discards it. It costs seconds, needs no registry credentials, and fires prepack for real.

prepublishOnly is not triggered by a pack

The two hooks are not symmetric, and the asymmetry is the part that gets missed:

A pack-based PR job therefore covers prepack and nothing else. If prepublishOnly is pnpm build && pnpm test, the PR gate very likely runs both already -- and "very likely" is exactly the failure. Write the mapping into the workflow as a comment beside the steps standing in for the hook (step 3 above), so the next person adding a command to prepublishOnly can see where its CI counterpart belongs.

A Local Heavy Lane Must Run CI's Runtime

The second half of the same incident explains why nobody saw it coming. The affected repo pinned node-version: 22 at 21 separate workflow sites and declared no local Node version anywhere -- no .nvmrc, no .node-version, no root engines. Local development ran Node 24 with npm 11. The behavior that killed the release was an npm 10 bug, fixed in npm 11.

So the T4 local heavy lane ran the exact suite that later failed on the release path, and passed -- not by luck, but by construction. It was executing a toolchain in which the bug did not exist. No number of local re-runs could have surfaced it, and the same blindness covers anything else runtime-dependent.

The structural fix is cheap -- declare the version once and make both sides read it:

# Before: every workflow pins its own number, and nothing declares the local one.
- uses: actions/setup-node@v4
  with:
    node-version: 22

# After: .nvmrc at the repo root holds the major; nvm and setup-node both read
# it, so the two sides cannot silently diverge.
- uses: actions/setup-node@v4
  with:
    node-version-file: .nvmrc

The count is what makes this structural rather than cosmetic. At 21 pinned sites, keeping the numbers aligned by hand is a losing game and a new workflow silently opts out of the convention; with node-version-file, a new workflow cannot introduce a divergence, and bumping the major is one edit that moves CI and local dev together.

The Platform-Binary Special Case

A CLI shipped the esbuild way -- one meta package plus a set of per-platform binary packages resolved through optionalDependencies -- has a verification surface that neither the PR gate nor the nightly registry drift net can see: the publish pipeline itself. The PR gate installs from a lockfile, so it never resolves against the live registry. The nightly drift net does resolve against the registry, but it runs once a day, on one platform, and it only ever exercises install-and-build. Between them sits an unverified gap -- pack, publish, and CDN propagation -- where three distinct failure modes live, each silent, each a property of the npm registry and client rather than of any one project.

The rest of this page documents those three failure modes and the publish-time gate that closes each. The patterns are the standard ones (esbuild, swc, Biome, and Turborepo all ship binaries this way), so the gates generalize to any project distributing platform binaries through npm.

The Package Shape

The layout is a single meta package that users install (my-cli) plus one package per target triple (my-cli-linux-x64-gnu, my-cli-darwin-arm64, my-cli-win32-x64-msvc, and so on). Each platform package carries only the prebuilt binary for its triple. The meta package lists every platform package under optionalDependencies, and at install time npm resolves only the entries whose os/cpu fields match the current machine -- so a Linux user downloads the Linux binary and nothing else.

The word optional is the whole problem. npm's semantics for an optionalDependency are "install it if you can; if anything goes wrong, skip it and exit successfully." That is exactly the behavior you want when a package genuinely does not apply to this platform -- and exactly the behavior that turns every distribution bug into a silent one. A tarball that fails to download, a binary that ships with the wrong mode, a version that half-published -- none of these make npm install fail. They all make it exit 0 with the binary quietly absent or unusable, and the failure surfaces later, at the user's first invocation, far from the pipeline that caused it.

Failure Mode 1: Silent Skip on Unpropagated Tarballs

Registry metadata propagates through the CDN faster than large binary tarballs. On a project whose per-platform tarballs run ~78 MB, the version metadata becomes resolvable within seconds of publish while the tarball itself is still replicating across CDN edges. In that window, a client that resolves the optionalDependency and then fails to fetch the tarball hits npm's "optional = skippable" path: npm install exits 0, the binary is simply absent, and the user's first invocation fails at runtime -- a broken release that looks green from the pipeline's side.

A post-publish smoke test that merely installs the meta package proves nothing here until the tarball is actually fetchable, and even then it only ever proves the runner's own platform. The gate is to force a real tarball fetch for every platform package before declaring the release good:

npm pack --dry-run <platform-pkg>@<dist-tag> forces a genuine tarball download rather than mere metadata resolution, so it fails until the tarball has propagated. Poll it with backoff, for every platform package -- not just the smoke runner's arch -- placed between the publish job and the clean-room smoke install:

# .github/workflows/release.yml (excerpt) -- runs AFTER the publish job and
# BEFORE the clean-room smoke install, gating on real tarball fetchability.
on:
  workflow_dispatch:
    inputs:
      dist_tag:
        description: "npm dist-tag to verify and smoke-test (e.g. latest, next)"
        required: true
        type: string
  workflow_call:
    inputs:
      dist_tag:
        required: true
        type: string

jobs:
  publish:
    # ...publishes every platform package + the meta package under dist-tag
    # ${{ inputs.dist_tag }} (see the workflow_dispatch/workflow_call inputs above)

  wait-for-propagation:
    needs: publish
    runs-on: ubuntu-latest
    steps:
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      # Probe EVERY platform package, not just the smoke runner's arch.
      # `npm pack --dry-run` forces a real tarball fetch -- metadata resolving
      # is not enough; the ~78 MB tarball itself must be downloadable.
      - name: Wait for every platform tarball to propagate
        run: |
          # ${{ inputs.dist_tag }} is this workflow's own workflow_dispatch/
          # workflow_call input (declared in the `on:` block above) -- the
          # same tag the publish job just pushed to the registry.
          TAG="${{ inputs.dist_tag }}"
          for pkg in \
            my-cli-linux-x64-gnu \
            my-cli-darwin-x64 \
            my-cli-darwin-arm64 \
            my-cli-win32-x64-msvc; do
            for attempt in $(seq 1 30); do
              if npm pack --dry-run "${pkg}@${TAG}" >/dev/null 2>&1; then
                echo "ok: ${pkg} tarball is fetchable"
                break
              fi
              if [[ "$attempt" == 30 ]]; then
                echo "::error::${pkg}@${TAG} tarball never propagated after 30 attempts"
                exit 1
              fi
              sleep $(( attempt < 10 ? attempt * 5 : 60 ))
            done
          done

  smoke-clean-room:
    needs: wait-for-propagation
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
    # ...npx create-my-cli@<tag> in a temp dir, build, assert output

Failure Mode 2: Pack Strips the Exec Bit

Observed on pnpm 9.x -- re-verify against whatever version you pin, since this is pnpm's own packing behavior rather than a documented, timeless guarantee of the npm ecosystem: pnpm pack normalizes file modes to 0644, silently stripping the 0755 executable bit off a shipped binary. The install succeeds, the file lands on disk, and spawning it fails with EACCES -- another green pipeline, another runtime-only failure. There are two gates here, and they work together.

First, publish platform packages with npm publish, which preserves the source file modes. Reserve pnpm for the packages that actually need its workspace:* rewriting; the platform packages carry a single prebuilt binary and no workspace protocol dependencies, so they gain nothing from pnpm publish and lose the exec bit to it.

Second, assert the mode inside the packed tarball before publishing, so the constraint is enforced by CI rather than remembered by whoever last touched the release script:

# CI gate: binary must be 0755 (493 decimal) inside the tarball
MODE=$(cd "$pkg_dir" && npm pack --dry-run --json 2>/dev/null \
  | jq -r --arg b "$BIN_NAME" '.[] | .files[] | select(.path == $b) | .mode')
if [[ "$MODE" != "493" ]]; then
  echo "::error::${pkg_dir}: binary mode is '${MODE}', expected 493 (0755). Was pnpm publish used instead of npm publish?"
  exit 1
fi

The 493 is 0755 in decimal -- the mode as npm pack --json reports it. Run this gate over every platform package except the Windows ones: execute bits are not meaningful on Windows and tar may report a different value there, so asserting 493 on a win32 package would fail the gate for a non-problem.

Failure Mode 3: The Double-PUT Lost-ACK Race

npm's publish is an HTTP PUT, and a network blip can lose the success ACK after the registry has already stored the version. The client, seeing no acknowledgement, retries the PUT -- and the retry gets 403 ... cannot publish over the previously published versions (EPUBLISHCONFLICT). The publish succeeded; it only looks failed. Under set -e, a multi-package release then dies mid-list -- publish 7 of 9 packages, hit the phantom conflict on the 8th, and the job aborts. Worse, it cannot simply be re-run: every already-published package now 403s on the second attempt, so a naive retry of the whole list fails immediately.

Two gates in the publish script fix this, and both belong per package:

• Idempotent. Before publishing a package, check whether npm view <name>@<version> already resolves; if it does, skip it. This makes a partially-completed release re-runnable to completion -- the second run walks past the packages that already landed and finishes the rest.

• Conflict-tolerant. When a publish fails, match the specific conflict error text (plus a post-failure registry recheck with backoff to confirm the version really is up there), and tolerate that one case as a skip. Fail hard on everything else -- auth errors, validation errors, a genuine hard network failure. Matching a bare 403/E403 is too broad; it would swallow real permission failures.

is_publish_conflict() {
  printf '%s' "$1" | grep -qiE 'cannot publish over the previously published versions|EPUBLISHCONFLICT'
}
# per package: precheck registry → publish → on failure, is_publish_conflict OR
# registry recheck (N attempts, backoff) → tolerate as skip; anything else → fail

Publish one package per invocation so a mid-list conflict can never abort the rest of the release. Combined with the idempotent precheck, this makes the whole publish a set of independent, re-runnable steps rather than one fragile sequence that a single phantom 403 can wedge.

Where These Sit in the Tier Model

The two halves of this page get different answers, and the difference is worth stating plainly.

The publish-only lifecycle gates have no tier because they are in the wrong place, and the remedy is to move them rather than to invent a tier for them. Once check:prepack-contract runs on every PR, it is an ordinary T1 check that happens to also fire on release -- it stops being publish-only, and the untiered lane disappears.

The three transport failure modes above cannot be moved that way. Nothing about tarball propagation, file modes inside a packed archive, or a lost publish ACK exists before there is a real release to observe, so these stay where they are. They are release-pipeline gates: they run once per publish, and they alert or block right at the moment the bug would otherwise ship. That makes them a different animal from the nightly registry drift net, which runs on a schedule, on one platform, and exercises install-and-build. The drift net is the scheduled cousin of these gates -- it catches a registry-side regression eventually and on one arch; these gates catch a publish-side regression immediately and across every platform. Failure mode 3 makes the boundary concrete: the drift net structurally cannot see a lost-ACK race at all, because that failure happens inside the publish step, before there is anything to install.

The post-publish clean-room smoke -- npx create-my-cli@<tag> in a fresh temp directory, build a scaffold, assert the output -- sits in a third position again. It is an alerting guard, not a gate: by the time it runs, publish has already happened, so it cannot prevent a bad release, only detect one that already shipped. Wire its failures into the same one-tracking-issue-per-workflow pattern the drift net uses, so a broken release opens exactly one tracking issue and closes it when the next release goes green.