feat(sandbox): grant a GCP sandbox read on the registry it pulls from - #663
ItamarZand88 wants to merge 6 commits into
Conversation
A GCP sandbox runs a container it pulls from the registry Alien hosts, but the grant machinery counted a sandbox only on AWS, so the image it needs was never covered. The constant that excluded it said why, and that reason no longer holds. Each compute service pulls as its own Google-managed service agent, so a service type is the unit of the grant. Cloud Run's robot account does not serve a sandbox, and three adjacent Vertex agents would be refused, so the account is pinned by a test rather than described in a comment. The extra service-account emails a deployment carries were the cheaper place to put this and the wrong one. They are revoked per deployment, so the first deployment deleted in a project would strip a binding its siblings still need. The service-type list is refcounted and survives until the last consumer goes, which is also why the revoke names every type rather than the ones it granted. Both grant and revoke now read one list, since a member the revoke misses stays on the repository policy with nothing left to remove it.
…pin proves The agent domain decides whether every GCP sandbox pull 403s, and its only source was a status doc. The test that looks like it guards the domain compares two copies of the same constant, so it says what it does catch instead.
…leanup Counting a GCP sandbox widened a prefix-blind guard, so a deployment whose only image Alien never hosted stopped short-circuiting and demanded a registry binding a manager that hosts no images does not have. It was never granted, so there is nothing for the cleanup to strand. The service types the grant and the revoke name are one definition rather than two literals four hundred lines apart, since a type the revoke omits stays on the repository policy with nothing left to remove it.
The sandbox agent does not exist until a project's first session, so the grant names a principal that may not exist yet. The binding is accepted; whether it becomes effective is the live pull test's answer, not a property of this code.
Google grants the sandbox agent a project-scoped role that already carries artifact download, so a same-project pull needs nothing from us. This binding exists for the cross-project case, which is the one Alien's registry is in.
|
| fn gcp_service_types() -> Vec<ComputeServiceType> { | ||
| vec![ComputeServiceType::Worker, ComputeServiceType::Sandbox] |
There was a problem hiding this comment.
Existing grants stay incomplete
Existing GCP deployments with registry_access_granted already set return before applying this new service-type list. Their old Worker-only policies are therefore never updated with the sandbox service agent. If such a deployment already has or later adds a sandbox, its private image pull fails with a 403. The grant state needs versioning or a backfill so existing policies are reconciled.
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/alien-manager/src/registry_access.rs
Line: 417-418
Comment:
**Existing grants stay incomplete**
Existing GCP deployments with `registry_access_granted` already set return before applying this new service-type list. Their old Worker-only policies are therefore never updated with the sandbox service agent. If such a deployment already has or later adds a sandbox, its private image pull fails with a 403. The grant state needs versioning or a backfill so existing policies are reconciled.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.The marker is written after the cloud grant lands, so a false one still covers a live grant, and returning early on it would leave registry read alive after the deployment is gone. The comment above the guard already said so. The read path now derives its service agents from the table the write path uses, so a sandbox member decodes as the type that produced it rather than as an ordinary account, with a round-trip test to keep the two from drifting.
|
Two fixed in 49a4fb2, one is real and tracked but out of scope here. Cleanup Skips Live Grants — fixed, and it was my regression. You are right, and the comment six lines above the guard already said so: the marker is written after the cloud grant lands, so a false one still covers a live grant. My early return would have left registry read alive after the deployment was deleted. Reverted to failing loudly. For context on what I was trying to fix: counting a GCP sandbox widened a prefix-blind guard, so a GCP sandbox whose image Alien never hosted now reaches the binding lookup and errors on a manager that hosts no images. That behaviour is pre-existing on AWS, where sandboxes were always counted, so GCP now simply matches it. Erroring loudly beats silently dropping a revoke. Sandbox Permissions Decode Incorrectly — fixed. The read path now derives its service agents from the same table the write path uses, so a member decodes as the type that produced it. Added Existing Grants Stay Incomplete — real, tracked, not fixed here. Worth knowing alongside it: this binding only matters cross-project. Google grants the sandbox agent a project-scoped role that already carries |
|
@greptile review |
| // Not `Ok` on an unset marker: the grant lands before the marker is persisted, so a false | ||
| // one still covers a live grant, and the only revoke path is through this binding. | ||
| return Err(AlienError::new(ErrorData::RegistryAccessCleanupFailed { |
There was a problem hiding this comment.
Cleanup Blocks Ungranted Deletions
If a manager has no artifact-registry binding, removing the false-marker escape makes cleanup fail for a never-granted GCP sandbox that uses a public image. The prefix-blind guard admits this deployment even though no repository matched and no grant was created. The unavailable binding then causes every deletion reconcile to return RegistryAccessCleanupFailed and retry instead of completing. This path must distinguish a possible unpersisted partial grant from a deployment for which no grant could have existed.
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/alien-manager/src/registry_access.rs
Line: 335-337
Comment:
**Cleanup Blocks Ungranted Deletions**
If a manager has no artifact-registry binding, removing the false-marker escape makes cleanup fail for a never-granted GCP sandbox that uses a public image. The prefix-blind guard admits this deployment even though no repository matched and no grant was created. The unavailable binding then causes every deletion reconcile to return `RegistryAccessCleanupFailed` and retry instead of completing. This path must distinguish a possible unpersisted partial grant from a deployment for which no grant could have existed.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.|
Both valid. Neither is fixed here, and here is why. Issue 1 — the distinction you are asking for cannot be made at this call site. You are right that the path should tell an unpersisted partial grant apart from a deployment that could never have been granted. It cannot, because That leaves two wrong answers, and I have now written both. Returning I am keeping the loud one. A cleanup that retries is recoverable; a grant that outlives its deployment is not. The real fix is to stop discarding the reason one function up, which changes that signature for every caller, so it is filed separately rather than attempted as a third version of this guard inside a feature PR. Issue 2 — real, filed, and broader than this diff. One thing worth knowing about the blast radius of both: this binding only matters cross-project. Google grants the sandbox agent a project-scoped role that already carries |
Summary
A GCP sandbox runs a container it pulls from the registry Alien hosts, but the
grant machinery counted a sandbox only on AWS, so the image it needs was never
covered. This grants it, and makes the service types the grant and the revoke
name one definition instead of two.
When a deployment reconciles its registry access:
registry at all. AWS builds a MicroVM image from a published bundle and GCP
pulls a container, so both earn a grant; Azure names an image from its own
catalog and earns nothing.
A sandbox pulls as a different principal from a worker, so a grant made for
one reaches neither the other.
the revoke does not name stays on the repository policy with nothing left to
remove it.
This changes a hardcoded "only AWS sandboxes pull from us" to a question asked
per platform.
What I did
Replaced the
INCLUDE_SANDBOX/EXCLUDE_SANDBOXconstant pair with a functionthat answers per platform, and added a sandbox compute-service type mapped to the
Vertex sandbox service agent.
Moved the grant off the deployment's extra service-account emails, which was the
wrong place: those are revoked per deployment, so the first deployment deleted in
a project would strip a binding its siblings still need. The service-type list is
refcounted and survives until the last consumer goes.
Fixed a regression the widened guard introduced. That guard is deliberately
prefix-blind, so it now admits a GCP sandbox whose image Alien never hosted. Such
a deployment was never granted, and on a manager that hosts no images there is no
registry binding to load, so its delete reconcile started returning an error every
tick. It now answers that there is nothing to strand.
How I tested
cargo test -p alien-manager -p alien-bindings --lib— 239 and 338 pass, andcargo check --workspace --all-targetsis clean, on a branch rebased onto currentmain.
Pinned the new behaviour rather than assuming it: a GCP sandbox-only stack is
covered by the grant, the shared project grant survives while any sibling
deployment is alive, and a never-granted GCP sandbox on a public image reaches
the binding lookup with nothing recorded to clean up.
Checked the attacks this change makes possible:
diff: GCP has one registry repository per environment, so the worker agent
already holds this. Tracked separately; this adds a second principal inside the
same customer project, which holds no capability the customer lacks today.
removal is a
retain, so naming one never granted is a no-op while omitting onewould orphan a member (
gar.rs:207).members are revoked only in the last-consumer branch, and the per-deployment
identity stays per-deployment (
registry_access.rs:92-152).created at a project's first session, so the binding necessarily precedes it.
The binding is accepted; whether it becomes effective when the agent appears is
measured by the cross-project pull test, not claimed here.
Nothing turned up.