feat(sandbox-kit): version and publish the Docker Sandboxes kit (PFM-7373) - #3455
Conversation
Give the chainloop-trace-claude kit the same release lifecycle the Helm chart already has: it declares a `version:` that tracks the Chainloop release whose CLI it ships, the release bump script rewrites it alongside the chart's appVersion, and merging that bump PR publishes the kit to docker.io/chainloop/sbx-kit-claude:<version> with a signature and SLSA provenance attached. Docker Hub rather than ghcr because sbx's default kit.allowedSources is already ["docker.io/"], so consumers need no host settings change to pull the kit. The spec moves under devel/sandbox-kit/claude/ to make room for further kits; the packaging workflow and the bump script both glob the directory, so additional kits are picked up without further edits. sbxenv.yaml keeps pointing at the in-repo copy so the local environment always runs the kit as it exists on the current branch. Also corrects the README's sbx run invocations: the kit is `kind: sandbox`, so it is the agent and belongs in the positional slot, not behind --kit, which takes mixins only. Assisted-by: Claude Code Signed-off-by: Miguel Martinez <miguel@chainloop.dev> Chainloop-Trace-Sessions: 7da50f6b-428c-4749-82b5-562ae1ba890e
AI Session Checks — 🟡 75% ·
|
| Avg score | Sessions | Failing policies | Attribution | Files | Lines | Total Duration |
|---|---|---|---|---|---|---|
| 🟡 75% | 1 | 100% AI / 0% Human | 6 | +857 / -842 | 50h56m22s |
🟡 75% — 100% AI — ⚠️ 2 policies failing
-
Sep 18, 2026 14:08 UTC · 50h56m22s · $67.52 · 1.0k in / 346.2k out · claude-code 2.1.276 (claude-opus-5)
Change Summary
-
- Moves the sandbox kit spec under
devel/sandbox-kit/claude/and trims kit comments/docs. - Adds kit
version:handling and updates the release bump script to rewrite sandbox kit specs. - Adds and then hardens
package_sandbox_kit.yamlfor discovery, validation, publishing, attestation, andlatesttagging. - Updates
sbxenv.yamland the kit wrapper for persistent-only tracing and stricter repo detection.
- Moves the sandbox kit spec under
AI Session Overall Score
-
🟡 75% — Useful session, but claim drift and unexercised workflow changes leave reviewer work.
AI Session Analysis Breakdown
-
🟢 92% · user-trust-signal
-
🟢 The user kept moving to the next task without sharp corrections. · High Impact
🟢 90% · scope-discipline
-
🟢 Code changes stayed inside the sandbox-kit area the user kept steering. · High Impact
🟢 84% · solution-quality
-
🟢 Replaced the coarse skip guard with version-change discovery. · High Impact
🟡 72% · alignment
-
🟠 The AI twice described shipped behavior more strongly than the branch supported at the time. · Medium Severity
💡 Verify capability claims against the current branch before stating them in summaries or PR text.
🟡 71% · context-and-planning
-
🟠 A broad workflow, script, spec, and config change ran mostly without a visible plan. · Medium Severity
💡 For multi-file release changes, write a short visible plan before editing so later fixes stay anchored.
🟡 58% · verification
-
🔴 Workflow edits were never rerun end-to-end after the final changes. · High Severity
💡 For CI or workflow changes, rerun the workflow or an equivalent end-to-end publish path before calling it done.
-
File Attribution
████████████████████100% AI / 0% HumanStatus Attribution File Lines modified ai devel/sandbox-kit/claude/spec.yaml+554 / -353 deleted ai devel/sandbox-kit/spec.yaml+0 / -396 modified ai .github/workflows/package_sandbox_kit.yaml+243 / -59 modified ai devel/sandbox-kit/README.md+25 / -11 modified ai sbxenv.yaml+16 / -16 modified ai .github/workflows/utils/bump-chart-and-dagger-version.sh+19 / -7
Policies (4, 2 failing)
Status Policy Material Messages ✅ Passed ai-config-ai-agents-allowedai-coding-session-7da50f- ⚠️ Failedai-config-no-dangerous-commandsai-coding-session-7da50f- Forbidden bash pattern /curl[^|\n]|\s(?:bash|sh)\b/ matched command: cat > .github/workflows/package_sandbox_kit.yaml <<'YAML' name: Package Sandbox Kit on: # Only push the kit if the kit sources have changed. The release bump...
- Forbidden bash pattern /curl[^|\n]|\s(?:bash|sh)\b/ matched command: git add .github/workflows/package_sandbox_kit.yaml && git commit -S -s -F - <<'MSG' && git push 2>&1 | tail -2 fix(sandbox-kit): harden the packaging workflow (...
- Forbidden bash pattern /curl[^|\n]|\s(?:bash|sh)\b/ matched command: python3 - <<'PY' import sys, pathlib p = pathlib.Path(".github/workflows/package_sandbox_kit.yaml") s = p.read_text() old = """ - name: Install Chainloop ...
⚠️ Failedai-config-no-secretsai-coding-session-7da50f- Secret (generic-[REDACTED:generic-password) detected in session content [turn=831, source=tool_result, line=62]: |
⚠️ Failed |ai-config-no-secrets| [ai-coding-session-7da50f](https://app.chainloop.dev/u/chainloop/projects/chainloop/versi... - Secret (generic-credential-uri) detected in session content [turn=886, source=assistant-text, line=27]: test-token https://gitlab-ci-token:[REDACTED:generic-credential-uri]@github.com/chainloop-dev/chainloop.git
- Secret (generic-password) detected in session content [turn=15, source=tool_result, line=33]: [REDACTED:generic-password]: ${{ secrets.GITHUB_TOKEN }}
- Secret (generic-password) detected in session content [turn=29, source=tool_result, line=33]: 33 [REDACTED:generic-password]: ${{ secrets.GITHUB_TOKEN }}
- Secret (generic-password) detected in session content [turn=34, source=tool_result, line=93]: 93 [REDACTED:generic-password]: ${{ secrets.GITHUB_TOKEN }}
- Secret (generic-password) detected in session content [turn=35, source=tool_result, line=115]: 115 [REDACTED:generic-password]: ${{ secrets.GITHUB_TOKEN }}
- Secret (generic-password) detected in session content [turn=493, source=tool_result, line=33]: 33 [REDACTED:generic-password]: ${{ secrets.GITHUB_TOKEN }}
- Secret (generic-password) detected in session content [turn=563, source=assistant-tool_use:Bash, line=46]: [REDACTED:generic-password]: ${{ secrets.DOCKERHUB_TOKEN }}
- Secret (generic-password) detected in session content [turn=624, source=assistant-text, line=12]: |
DOCKERHUB_TOKEN| Docker Hub access token (Account Settings → Personal access tokens), scoped Read & Write, not your [REDACTED:generic-password] | - Secret (generic-password) detected in session content [turn=632, source=assistant-tool_use:Bash, line=1]: grep -rn "login-action" -A 6 .github/workflows/.yaml .github/workflows/.yml 2>/dev/null | grep -iE "registry:|username:|[REDACTED:generic-password]:|login-action" | head -20
- Secret (generic-password) detected in session content [turn=633, source=tool_result, line=11]: .github/workflows/package_chart.yaml-33- [REDACTED:generic-password]: ${{ secrets.GITHUB_TOKEN }}
- Secret (generic-password) detected in session content [turn=633, source=tool_result, line=15]: .github/workflows/release.yaml-93- [REDACTED:generic-password]: ${{ secrets.GITHUB_TOKEN }}
- Secret (generic-password) detected in session content [turn=633, source=tool_result, line=4]: .github/workflows/build_external_container_images.yaml-115- [REDACTED:generic-password]: ${{ secrets.GITHUB_TOKEN }}
- Secret (generic-password) detected in session content [turn=633, source=tool_result, line=7]: .github/workflows/package_sandbox_kit.yaml-45- [REDACTED:generic-password]: ${{ secrets.DOCKERHUB_TOKEN }}
- Secret (generic-password) detected in session content [turn=650, source=tool_result, line=17]: The workflow must grant the id-token: write permission, pass the Docker Hub organization name as username, omit [REDACTED:generic-password], and set the OIDC connection ID in DOCKERHUB_OIDC_CONNECTION...
- Secret (generic-password) detected in session content [turn=650, source=tool_result, line=1]: Web search results for query: "Docker Hub OIDC GitHub Actions push without [REDACTED:generic-password] trusted publishing"
- Secret (generic-password) detected in session content [turn=655, source=tool_result, line=32]: -
username: Docker Hub organization name (no [REDACTED:generic-password] field) - Secret (generic-password) detected in session content [turn=658, source=assistant-text, line=7]: The change to my workflow is small — drop the [REDACTED:generic-password], and the two remaining values are variables, not secrets:
- Secret (generic-password) detected in session content [turn=703, source=assistant-tool_use:Bash, line=16]: [REDACTED:generic-password]: ${{ secrets.DOCKERHUB_TOKEN }}"""
- Secret (generic-password) detected in session content [turn=831, source=tool_result, line=62]: |
⚠️ Failed |ai-config-no-secrets| [ai-coding-session-7da50f](https://app.chainloop.dev/u/chainloop/projects/chainloop/versi... - … and 6 more — view all ↗
✅ Passed ai-config-mcp-servers-allowedai-coding-session-7da50f- -
Security Checks — ✅ 10 passing
✅ secret-scan
| Status | Policy | Messages |
|---|---|---|
| ✅ Passed | secrets-detection |
- |
✅ github-actions-scan
| Status | Policy | Messages |
|---|---|---|
| ✅ Passed | ci-pipeline-security |
- |
✅ sast-scan
| Status | Policy | Messages |
|---|---|---|
| ✅ Passed | owasp-top10-2025 |
- |
| ✅ Passed | sast |
- |
| ✅ Passed | cwe-top25 |
- |
| ✅ Passed | cwe-top26-40-cusp |
- |
✅ iac-scan
| Status | Policy | Messages |
|---|---|---|
| ✅ Passed | iac-misconfiguration |
- |
PR info
| Status | Policy | Messages |
|---|---|---|
| ✅ Passed | pr-min-approvals |
- |
| ✅ Passed | pr-description-required |
- |
| ✅ Passed | pr-user-story-linked |
- |
⏭️ 1 scan not applied
| Scan | Reason |
|---|---|
vulnerability-scan |
no manifest/lockfile changed |
View attestation ↗
View attestation ↗
Powered by Chainloop and Chainloop Trace
Replace the DOCKERHUB_USERNAME/DOCKERHUB_TOKEN secrets with an OIDC connection, so publishing needs no long-lived Docker Hub credential in the repository. GitHub mints a short-lived identity token per run and Docker exchanges it for a registry token that expires with the job; access is governed by the connection's ruleset on the subject claim. Assisted-by: Claude Code Signed-off-by: Miguel Martinez <miguel@chainloop.dev> Chainloop-Trace-Sessions: 7da50f6b-428c-4749-82b5-562ae1ba890e
Kusari Analysis Results:
Dependency analysis found no pinned version dependency changes and raised no concerns. Code analysis flagged a Docker Hub secret detection at line 63, but determined it is a DOCKERHUB_OIDC_CONNECTIONID identifier, not an actual credential - the workflow uses OIDC federation with short-lived tokens rather than storing long-lived secrets, which is a security best practice. No codeIssues or workflow_issues were reported. Both analyses independently recommend proceeding, and the combined findings do not change the overall risk profile. No action items required before merging. Note View full detailed analysis result for more information on the output and the checks that were run.
Found this helpful? Give it a 👍 or 👎 reaction! |
![kusari-inspector[bot]](https://avatars.githubusercontent.com/in/1220830?s=60&v=4){kind=small}
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed
Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.
Re-trigger cubic
Each release moves docker.io/chainloop/sbx-kit-claude:latest to the version just published, so consumers can track the current kit without pinning. The attestation keeps naming the immutable tag. Assisted-by: Claude Code Signed-off-by: Miguel Martinez <miguel@chainloop.dev> Chainloop-Trace-Sessions: 7da50f6b-428c-4749-82b5-562ae1ba890e
|
Kusari PR Analysis rerun based on - 7ffa222 performed at: 2026-09-18T17:05:50Z - link to updated analysis |
There was a problem hiding this comment.
Review completed against the latest diff
Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.
Re-trigger cubic
There was a problem hiding this comment.
All reported issues were addressed across 1 file (changes from recent commits).
Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.
Re-trigger cubic
Address the SAST and Kusari findings on the new workflow: - Replace the `curl | bash` installer with a fetch, sha256 digest check and then execute. The job holds an OIDC token that mints Docker Hub credentials and signs artifacts, so an unverified install script is a privilege escalation path. - Pass the kit version into the publish step through env rather than interpolating the step output into the shell source, removing the template injection surface. Assisted-by: Claude Code Signed-off-by: Miguel Martinez <miguel@chainloop.dev> Chainloop-Trace-Sessions: 7da50f6b-428c-4749-82b5-562ae1ba890e
migmartri
changed the title
|
Kusari PR Analysis rerun based on - 7c9bee1 performed at: 2026-09-18T17:11:02Z - link to updated analysis |
The kit now requires a repository that has already been initialized with `chainloop trace init` and refuses to start otherwise, telling the user to run it in the repository first. Identity always comes from the committed .chainloop.yml, so the traceMode, chainloopOrg, chainloopProject and chainloopWorkflow arguments are gone. `chainloop trace run` is dropped deliberately: it ignores .chainloop.yml by design, and its teardown wipes .git/chainloop-trace/ and strips the committed hooks, which is destructive on exactly the repositories this kit accepts. The trade-off is that a session whose work is never pushed attests nothing. Also adds a usage header pointing at the guide, drops the edition wording from the CLI install and egress comments, and condenses the commentary. Assisted-by: Claude Code Signed-off-by: Miguel Martinez <miguel@chainloop.dev> Chainloop-Trace-Sessions: 7da50f6b-428c-4749-82b5-562ae1ba890e
There was a problem hiding this comment.
All reported issues were addressed across 3 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
Move the long-form rationale out of spec.yaml and leave short notes with pointers to the README, which already carries the same material under "Why nightly" and "Why the token is passed in". Removes the build-requirement banner, the gRPC/ALPN analysis, the commented-out proxy-managed credential block, the v1-to-v2 migration asides and the per-host debugging notes. No functional change: the egress allowlist, environment variables and wrapper behaviour are unchanged. Assisted-by: Claude Code Signed-off-by: Miguel Martinez <miguel@chainloop.dev> Chainloop-Trace-Sessions: 7da50f6b-428c-4749-82b5-562ae1ba890e
There was a problem hiding this comment.
All reported issues were addressed across 1 file (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
…(PFM-7373) Review follow-ups. Publish every kit under devel/sandbox-kit rather than the hardcoded claude one: a discover job derives the matrix from the directories that hold a spec.yaml, so a new kit needs no workflow edit. Narrow the trigger to devel/sandbox-kit/*/spec.yaml. The previous devel/sandbox-kit/** also matched the README, so a docs edit republished the released tag with fresh content under the same immutable version. Guard the same hazard from the other side by skipping a version already present in the registry, since the trigger still fires on spec edits that leave the version alone. Make a zero-match glob fatal in the bump script. It previously ran from the invocation directory and swallowed a miss, so a release run from the wrong place would leave every spec at the old version and publish nothing, with no error. Move checkout and the version read ahead of the tooling installs so a missing version fails in seconds rather than after a 95MB download. Drop the wrapper's run_plain indirection and the detected flag left over from two-mode dispatch, along with the last stale references to the removed trace mode. Assisted-by: Claude Code Signed-off-by: Miguel Martinez <miguel@chainloop.dev> Chainloop-Trace-Sessions: 7da50f6b-428c-4749-82b5-562ae1ba890e
There was a problem hiding this comment.
All reported issues were addressed across 5 files (changes from recent commits).
Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
…-7373) Review follow-ups on PR 3455. Select kits in the discover job by diffing each spec's `version:` across the pushed range, and drop the registry-existence guard that replaced. The guard skipped the whole publish block, so a run whose version push succeeded but whose `latest` push or attestation failed could never be reconciled by a re-run; selecting on the version change instead leaves a re-run of that same push selecting the kit and completing the work, while a comment-only spec edit selects nothing. Spell out the two config filenames the wrapper accepts. The .chainloop.y*ml glob also matched files Chainloop never reads, such as .chainloop.yolo.ml, which would let the sandbox start without the tracing it promises. Restore the proxy re-test probe next to the credentials note, which pointed at a README section carrying the reasoning but not the procedure. Assisted-by: Claude Code Signed-off-by: Miguel Martinez <miguel@chainloop.dev> Chainloop-Trace-Sessions: 7da50f6b-428c-4749-82b5-562ae1ba890e
There was a problem hiding this comment.
All reported issues were addressed across 2 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
3 participants
Gives the
chainloop-trace-claudeDocker Sandboxes kit a release lifecycle, and narrows it to a single supported mode.Publishing
The kit declares a
version:in its spec that tracks the Chainloop release whose CLI it ships.bump-chart-and-dagger-version.shrewrites it on release alongside the chart'sappVersion, and merging the resulting bump PR triggers a newPackage Sandbox Kitworkflow that publishes todocker.io/chainloop/sbx-kit-claudeunder both the version tag andlatest, with a signature and SLSA provenance attached — mirroring howpackage_chart.yamlreacts to changes underdeployment/chainloop/.Docker Hub rather than ghcr: sbx's default
kit.allowedSourcesis already["docker.io/"], so consumers pull the kit without changing their host settings. Authentication is via a Docker Hub OIDC connection, so no long-lived registry credential is stored in the repository; access is governed by a ruleset matching the subject claimrepo:chainloop-dev/chainloop:ref:refs/heads/main.v1.109.0andlatesthave been published manually to bootstrap the repository.Persistent tracing only
The kit now requires a repository already initialized with
chainloop trace initand refuses to start otherwise, pointing the user at that command. Identity always comes from the committed.chainloop.yml, so thetraceMode,chainloopOrg,chainloopProjectandchainloopWorkflowarguments are gone.chainloop trace runis dropped deliberately: it ignores.chainloop.ymlby design, and its teardown wipes.git/chainloop-trace/and strips the committed hooks — destructive on exactly the repositories this kit accepts. The trade-off is that a session whose work is never pushed attests nothing.Layout and docs
The spec moves to
devel/sandbox-kit/claude/to make room for further kits. Adiscoverjob derives the publish matrix from the directories holding aspec.yamland the bump script globs the same tree, so a new kit is published with no workflow edit. The trigger is scoped todevel/sandbox-kit/*/spec.yamlso a README edit cannot republish a released tag, and a version already present in the registry is skipped rather than overwritten.sbxenv.yamlkeeps pointing at the in-repo copy so the local environment runs the kit as it exists on the current branch.The spec gains a usage header pointing at the guide, and sheds its long-form rationale in favour of short notes referring to
devel/sandbox-kit/README.md, which already carries the same material. The README'ssbx runinvocations are corrected: the kit iskind: sandbox, so it is the agent and belongs in the positional slot rather than behind--kit, which accepts mixins only.AI assistance: this change was produced with Claude Code.