Verify vendored pgstencil provenance and defer its code audit to pgstencil

[nedtwigg]

Stacked on #742 (pgstencil-peers). Pairs with diffplug/pgstencil#9 (and its follow-up, the pgstencil security-audit workflow).

Decision

pgstencil audits its own packed code, in its own repository, against its own SECURITY.md. Dormouse stops auditing the tarballs' contents and verifies the link instead. The Hosted audit keeps everything about how hosted/ configures the adapter (origin gate, cookies, authPolicy, providerBindings, preview and production paths).

What

• scripts/sync-pgstencil.mjs reads each tarball's own package/dist/provenance.json and refuses one that is missing, marked dirty by pgstencil's --allow-dirty, or naming a commit other than the one packed. --working-tree keeps Vendor clean pgstencil with kysely as a Hosted peer, and fail on unmet peers #742's semantics (dirty: true, refused by production preflight). It warns when the vendored commit is not on pgstencil origin/main.
• docs/specs/security-hosted.md -> Deployment boundary: Must vendor pgstencil from a commit on its main (a Dormouse branch may vendor a pgstencil branch while a cross-repo change is in flight); the archive FAIL IF now covers the packed provenance, and a new FAIL IF requires the vendored commit to be on pgstencil main with a success security-audit check run. Budget ratcheted 450 → 550.
• .github/audit/application-security.md no longer reads the packed modules; it runs the provenance checks (two gh api reads of a public repo, no PAT) and audits the consumer configuration.
• hosted/server/tests/artifacts.test.ts and verifyPackages in hosted/scripts/production.mjs check the tarball provenance and build.json's dirty: false.
• Re-vendored from pgstencil#9's tip (ddbf96c), the first pack that carries provenance.

Before merging

ddbf96c is a pgstencil branch commit. After diffplug/pgstencil#8, #9 and the audit-workflow PR merge, re-run pnpm pgstencil:sync ../pgstencil <main sha> on this branch so main vendors a pgstencil main commit with a passing security-audit check run; until then the nightly audit's new FAIL IF fails by design.

Verified

pnpm lint:specs, pnpm --filter dormouse-hosted test (16 deploy + 19 vitest), verifyPackages() against the new archives.

🤖 Generated with Claude Code

[cloudflare-workers-and-pages]

Deploying mouseterm with    Cloudflare Pages

Latest commit:	67c0fb7
Status:	✅  Deploy successful!
Preview URL:	https://d8dba830.mouseterm.pages.dev
Branch Preview URL:	https://pgstencil-provenance.mouseterm.pages.dev

View logs

[dormouse-bot]

Feedback on work in progress, not a merge verdict — mark ready when you want the full pass.

Deferring the packed code to pgstencil's own audit and verifying the link instead is the right split. One thing in that split looks unresolved, plus two smaller items inline.

The Origin and Account boundary FAIL IF lines still send this same agent into the tarballs. .github/audit/application-security.md now tells it "do not audit the tarballs' contents here", but three FAIL IF bullets in its own scope direct it to do exactly that: in docs/specs/security-hosted.md, the request-URL/CORS/CSRF bullet ends inspect hosted/server/worker-app.ts and the packed adapter, the cookie bullet inspect the adapter and hosted/src/api.ts, and the authPolicy bullet inspect hosted/server/policy.ts and the packed adapter. The prompt and the spec disagree, and the spec is the normative one — so the auditor either breaks its prompt or reports three checks as unrun. These three are also the weakest candidates for deferral: cookie attributes, credentialed CORS, and explicit-linking policy are properties of how Dormouse configures the adapter, and pgstencil auditing its packed code against its own SECURITY.md is not evidence that this consumer's configuration produces them. Either scope the prohibition to the qualitative sweep and leave the adapter readable for those three bullets, or rewrite the bullets to name what Dormouse can inspect without opening an archive.

Smaller, and separable: the claim.dirty === true refusal the sync gains can't fire from that entry point. pgstencil's build-packages.ts refuses pnpm packages:pack on a modified packages/ or scripts/ tree unless --allow-dirty is passed, and sync-pgstencil.mjs neither passes that flag nor accepts it — an unknown -- option hits the usage throw. Worth keeping as a defense against a pgstencil that later packs dirty by default, but the comment above it reads as though it guards a reachable path. The same mechanism means hosted/README.md's "To try uncommitted pgstencil changes, pass --working-tree" no longer describes what the mode does: it packs a clean checkout against the local install, and uncommitted pgstencil changes fail at the pack step with pgstencil's own error.

[dormouse-bot]

--working-tree records dirty: true in build.json by design (const dirty = workingTree; in scripts/sync-pgstencil.mjs), even when the pgstencil checkout is clean — so this assertion makes pnpm --filter dormouse-hosted test fail for every --working-tree sync. hosted/README.md says to re-run the integration tests after every refresh, which that mode can no longer pass.

verifyPackages already refuses a dirty build.json, and it is the gate the README and docs/specs/hosted.md -> "Production releases" both name as owning this. Dropping the assertion keeps the three in agreement; if you'd rather the hosted suite refuse it too, the README's --working-tree paragraph is the thing that needs to change.

Suggested change

	// A --working-tree sync records dirty: true; production preflight refuses it.
	expect(build.dirty).toBe(false);
	// `dirty: true` from a --working-tree sync is refused by production
	// preflight (verifyPackages), not here.

[dormouse-bot]

The archive is written into vendor/ before its provenance is read, so a rejected pack still lands: a stale or mismatched tarball overwrites the good one while build.json keeps naming the old commit, and with two archives the first can be replaced before the second throws. The resulting tree fails the hash check everywhere, which is the right direction, but recovering needs a git checkout vendor/ the error message doesn't hint at.

Reading the claim from dist/packages and copying only once both pass leaves vendor/ untouched on failure — and makes the comment's "stops the sync before build.json can record a commit the bytes do not carry" true of vendor/ as well:

  for (const filename of archives) {
    // Both archives are held to the commit packed here, so a stale pack —
    // or two archives disagreeing with each other — stops the sync before
    // build.json or vendor/ can hold a commit the bytes do not carry.
    const claim = provenanceOf(resolve(checkout, "dist/packages", filename));
    if (claim.dirty === true)
      throw new Error(
        `${filename} was packed --allow-dirty; a dirty pack cannot be vendored`,
      );
    if (claim.commit !== commit)
      throw new Error(
        `${filename} was packed from ${claim.commit}, not the ${commit} packed here`,
      );
  }
  for (const filename of archives)
    copyFileSync(
      resolve(checkout, "dist/packages", filename),
      resolve(root, "vendor", filename),
    );

Left as prose rather than a suggestion block because the edit spans the whole loop.