Unified clients for the Python SDK (phase 1)

[eunomie]

Draft, for review of the approach as much as the code.

One generated client artifact per module, used by a module and by a plain Python program alike. It replaces the per-dependency bindings and [[dependencies]].

The design is committed with the code: hack/designs/2026-09-15-unified-clients.md.

What a scope looks like

Any directory that uses clients — a module's own directory, or a plain project:

<scope>/pyproject.toml          the uv workspace root, written by the SDK
<scope>/sdk/                    the SDK files, plus dagger_global/ while migrating
<scope>/clients/core/           the generated core client
<scope>/clients/<name>/         one member per declared client

Each client is a PEP 420 namespace package under dagger_clients, a uv workspace member with no path of its own, so the same artifact is byte-identical in two different scopes.

from dagger import function, object_type
from dagger_clients.core import Container, core
from dagger_clients.lib import lib


@object_type
class Demo:
    @function
    async def hello(self, name: str) -> str:
        return await lib().greeting(name=name)

    @function
    def base(self) -> Container:
        return core().container().from_("alpine:3.21")

What works

Verified by hand with the released CLI, outside the check harness. hack/try-unified-clients.sh builds the whole walkthrough from nothing in a temporary workspace.

• A module calling another module through a client, on released v1.0.0-beta.14, through Query.serveModule and under both entrypoint forms.
• A plain program with no connection handling: it provisions an engine on the first query, prints its answer, exits 0, and leaves no session process.
• A plain scope with no [project] table at all, calling a client through dagger.connection().
• An existing module migrating: dag.container() and dagger.Container keep working behind [tool.dagger] global-client = true, and clearing the flag gives back exactly the tree of a module that never had one.

What is open, and not fixable here

A changed client target does not invalidate its caller's cached result. [[dependencies]] used to put the target into the caller's identity; a unified client records a path and loads the module at run time, so nothing about the target reaches the caller's cache key. Change the target, call the caller with the same arguments, and the old answer comes back; call it with a different argument and the change appears.

This reproduces on a released engine and on 284cd849 alike, so it is a property of loading a module at run time rather than of the new field. Every SDK that drops [[dependencies]] inherits it. Written up language-neutrally, with the measured behaviour kept separate from the inferred mechanism: hack/designs/2026-09-18-serve-module-cache.md.

A module driven by a Dang entrypoint cannot reach a second hop. In a chain demo → lib → leaf, lib's entrypoint is handed the caller's container-derived workspace rescoped to /.dagger/modules/lib, so lib cannot load its own local client. One hop works; two do not. core/sdk/entrypoint/entrypoint.go, Workspace() at beta.14.

The SDK that generates a module must be the one that runs it. A generated manifest names the published shared entrypoint, which predates this layout and refuses the module. An entrypoint source may name only a git ref or a path inside the module, so a checkout under development cannot point at its own. Written up in hack/designs/2026-09-18-workspace-sdk-runtime.md. Before release, tag entrypoint/v1.x from this branch.

Smaller, also engine-side: dagger module client add ../lib records ./.dagger/modules/lib, but dagger module client rm ../lib refuses the same argument, because withoutClient does not normalise the address the way withClient does.

To try this branch today: hack/try-unified-clients.sh builds the walkthrough from nothing and copies this checkout's entrypoint into each module, which is what makes a development checkout runnable.

Modules run on a Dang entrypoint, and are handed only their clients

Generated manifests carry no [runtime]: a module runs through the shared Dang entrypoint, or the one --static-entrypoint generates inside it. The floor is released v1.0.0-beta.14, which drives entrypoints and has serveModule; the load path for engines without it is gone.

That exposed a defect worth recording. A Dang entrypoint runs the module's Python in an ordinary nested exec, and the engine attaches module context only to execs it starts itself — so the process reports its own container as its workspace and no currentModule at all. Whatever module context the code needs, the entrypoint must hand over.

What it hands over is the least thing that works: one module source per declared client, built from the files the engine already loaded, detached from the workspace they came from, re-read from the caller's config at every call. Not the workspace, and not a handle that leads back to one — both were tried and both leaked, the second because a ModuleSource from Workspace.moduleSource keeps its origin and withIncludes("../…") climbs back out.

A probe check runs module code that walks every ID the loader holds and every ModuleSource route that could rebuild a directory — at every climb depth, distinguishing an absent field from a refused capability — and reads nothing: 4,305 routes, reads=[], absent=0, under both entrypoint forms. Against the leaking shape the same probe reads the caller's file, so the check has a negative control.

The rule most of the review time went into

Generation writes into a directory the user also owns, so it must touch only what it made. Making that true took one critical defect and eleven more:

• The [tool.dagger] generated marker is read as TOML, never as text, and only client, core and runtime count. A regex over raw text once deleted a user's directory because a multi-line string in it happened to quote a marker.
• When the marker cannot be read, nothing is deleted and nothing is overwritten — a corrupted clients/core stops generation rather than being replaced.
• The scope file is edited byte-preservingly. A user's comments, table order, member and dependency entries survive; regeneration over an untouched scope reports no changes to apply.
• The parser boundary is where the user's content begins. [tool.uv.workspace] members and [project] dependencies are parsed with tomllib in the SDK's pinned image. A hand-written reader failed this twice in opposite directions: first too generous, then refusing "tomli; python_version < \"3.11\"" as "not an array of strings".

Checks

• sdk: 546 passed.
• helpers/pyproject: 39 Go tests.
• 33 end-to-end checks, including a module calling a client through each entrypoint form, and the capability probe above. hack/e2e-local.sh runs them against your engine; hack/e2e-floor.sh pins the floor release and asserts it before any check runs.
• Each fix carries an inversion: break the behaviour and a named check fails.

Phases

This is phase 1. Phase 2 publishes the SDK files and ends the migration: the sdk/ copy leaves the scope, and the global client is removed after a deprecation period.