LightconeResearch · EiffL · Sep 28, 2026
diff --git a/README.md b/README.md
@@ -1,38 +1,59 @@
 # Lightcone Research Stack documentation
 
-[![Docs](https://img.shields.io/github/actions/workflow/status/LightconeResearch/docs/docs.yml?branch=main&style=flat&label=docs&color=darkgreen)](https://docs.lightconeresearch.org)
-[![License](https://img.shields.io/badge/License-BSD_3--Clause-426b78.svg?style=flat)](LICENSE)
+User guides and reference material for the [Lightcone Research Stack](https://lightconeresearch.org/): the `lc` execution tools, research agent skills, and their integration with the external [ASTRA specification](https://astra-spec.org/latest/).
 
-The **Lightcone Research Stack** is [Lightcone Research](https://lightconeresearch.org/)'s
-tooling for research analyses described with [ASTRA](https://astra-spec.org/latest/)
-(Agentic Schema for Transparent Research Analysis). You describe an analysis in an
-`astra.yaml` specification; the stack validates it and takes care of the rest —
-execution, environments, and provenance.
+**[Read the documentation →](https://docs.lightconeresearch.org/)**
 
-**→ Read the documentation at <https://docs.lightconeresearch.org>**
+## Start here
 
-## Where to start
+- [Install](https://docs.lightconeresearch.org/user/install/) — one CLI setup, with an optional agent plugin.
+- [Your first analysis](https://docs.lightconeresearch.org/user/getting-started/) — run a complete local example and inspect its provenance.
+- [Work with an agent](https://docs.lightconeresearch.org/user/agents/) — scope, implement, and resume a research project.
+- [Meet the stack](https://docs.lightconeresearch.org/user/) — understand how the tools fit together.
 
-- [Install](https://docs.lightconeresearch.org/user/install/) — uv, git, and the `lc` command
-- [Getting started](https://docs.lightconeresearch.org/user/getting-started/) — your first analysis, from `lc init` to a published result
-- [Core concepts](https://docs.lightconeresearch.org/user/concepts/) — projects, output identity, and how provenance is recorded
-- [Running on a cluster](https://docs.lightconeresearch.org/user/cluster/) — SLURM, containers on HPC, and parallel filesystems
-- [Troubleshooting](https://docs.lightconeresearch.org/user/troubleshooting/) — common errors and how to fix them
+These docs currently target the upcoming explicit compute workflow. The installation guide pins a source revision that includes `lc compute`; the published `0.5.0rc4` release uses an earlier workflow. Keep installation instructions, tutorials, and command reference aligned when moving to a new release.
 
-## Components
+## Build locally
 
-| Component | What it does | Repository |
-| --- | --- | --- |
-| **lightcone-cli** | The `lc` CLI: project scaffolding, locked environments, sandboxed execution, and the provenance layer | [LightconeResearch/lightcone-cli](https://github.com/LightconeResearch/lightcone-cli) |
-| **astra-tools** | The SDK and `astra` CLI for ASTRA specifications: schema, validation, and evidence verification helpers | [LightconeResearch/astra-tools](https://github.com/LightconeResearch/astra-tools) |
+With [uv](https://docs.astral.sh/uv/getting-started/installation/) installed:
 
-## Feedback
+```bash
+uv sync --locked
+uv run zensical serve
+```
 
-The stack is in early alpha, and bug reports, design challenges, and use cases it
-doesn't cover yet are welcome. Report a problem with a tool on that tool's
-repository; report a problem with the documentation itself — a page that is wrong,
-unclear, or out of date — [here](https://github.com/LightconeResearch/docs/issues).
+Before submitting a change:
 
-## License
+```bash
+uv run zensical build --clean --strict
+```
+
+Pull requests run the same strict build. Updates to `main` are deployed through GitHub Pages by [the docs workflow](.github/workflows/docs.yml).
+
+## Where to edit
+
+| Location | Purpose |
+| --- | --- |
+| `docs/index.md` | Stack landing page |
+| `docs/user/` | Installation, tutorials, and task-oriented guides |
+| `docs/cli/` | CLI command reference |
+| `docs/api/`, `docs/architecture.md` | CLI implementation reference |
+| `docs/contributing/`, `docs/maintainer.md` | Contributor guidance |
+| `zensical.toml` | Navigation and site configuration |
+| `docs/stylesheets/extra.css`, `overrides/` | Website-aligned typography, colors, and layout |
+
+The design follows [lightcone-website](https://github.com/LightconeResearch/lightcone-website): Quattrocento headings, Newsreader prose, Alegreya navigation, JetBrains Mono code, parchment surfaces, and antique-gold accents. The landing-page engraving is the same *Uranometria* (Bayer, 1603) asset used by the website.
+
+Check technical claims against the relevant source:
+
+- [lightcone-cli](https://github.com/LightconeResearch/lightcone-cli) — execution and provenance.
+- [agent-skills](https://github.com/LightconeResearch/agent-skills) — plugin installation, skills, and hooks.
+- [ASTRA documentation](https://astra-spec.org/latest/) and [astra-tools](https://github.com/LightconeResearch/astra-tools) — the external specification and validation tools.
+
+Preserve existing page URLs when reorganizing navigation. Keep advanced implementation details in the reference and contributor sections, and put a runnable path before optional setup.
+
+## Feedback and license
+
+Report documentation problems in [this repository's issues](https://github.com/LightconeResearch/docs/issues). Report tool behavior in the relevant tool's repository. The stack is in early alpha and feedback from real analyses is welcome.
 
 BSD 3-Clause — see [LICENSE](LICENSE).
diff --git a/docs/api/assets.md b/docs/api/assets.md
@@ -15,7 +15,7 @@ Source: `src/lightcone/engine/assets.py`.
 | `classify(...)` | The one rule: `current` / `behind` / `stale`, with the why. Two callers — the worker and the read-only walk. |
 | `Verdict.calls_for_a_remake(refresh=)` | The one place a state becomes an action: `stale` always, `behind` only when asked. |
 | `data_version(path)` | Content hash of a directory or file — computed in the worker, before anything is annexed. |
-| `Versions` | Per-run memo so a shared declared input hashes once, not once per dependent. |
+| `Versions` | Memoizes content hashes within a classification or execution context. Separate worker processes do not share a mutable cache. |
 | `read(sidecar)` / `write(...)` | The manifest, `.<output_id>.manifest.json`. Both take the sidecar's own path, so a caller holding an output path has to say `manifest_path` out loud. |
 | `output_path(root, u, id, fmt)` | The output's file, guarded: any part that is not a single path component is refused, and so is a format that could not be an extension. |
 | `manifest_path(output)` | The sidecar beside it, named from the id alone — so it keeps its path, and its history, across a re-declared format. |

diff --git a/docs/api/compute.md b/docs/api/compute.md
@@ -0,0 +1,57 @@
+# lightcone.engine.compute
+
+The allocation boundary shared by CLI lifecycle operations and execution.
+`Compute` loads resource policy and obtains fresh native observations.
+It owns no service, registry, or saved current-cluster selection.
+
+| Symbol | Contract |
+|---|---|
+| `Request.parse(...)` | Common exact/minimum CPU and memory requests, node count, walltime, startup class. |
+| `Catalog.load(path)` | Ordered fixed shapes and stable connection namespaces; use the built-in local catalog only when the implicit default file is absent. |
+| `Compute.plan(request)` | Select an eligible offer and freeze its native launch settings without allocation. |
+| `Compute.launch(plan)` | Submit once and return a self-contained `Identity`. |
+| `Compute.discover()` | Snapshots and per-connection errors, querying each authority once. |
+| `Compute.status(id, wait=False, timeout=300)` | Native allocation state plus authenticated Dask readiness. |
+| `Compute.down(id)` | Native termination independent of scheduler health. |
+| `connect(id, timeout=10, config_path=None)` | Context manager borrowing a standard Dask client; closes the client, never the allocation. |
+| `Provider` | `plan`, `launch`, `discover`, `inspect`, `connect`, `terminate`. |
+
+The built-in catalog exposes one `local` offer: one CPU, 1 GiB, one node,
+fast startup, 30-minute default and two-hour maximum lifetime. It creates no
+configuration file or allocation. Configured catalogs replace it completely.
+Missing paths selected through an argument or `LC_COMPUTE_CONFIG`, unreadable
+files, and invalid catalogs remain errors. Stable connection namespaces let
+separate invocations discover and attach to the same local allocations.
+
+`local.py` and `slurm.py` implement the provider protocol. Adding an adapter means
+adding one provider factory and its native mapping; `run` and `materialize` only
+borrow clients through the common API. Provider settings stay behind that seam.
+`runtime.py` owns private files, standard TLS material, and authenticated scheduler
+identity checks. `local_runtime.py` and `slurm_bootstrap.py` compose stock Dask
+components; they do not define custom workers or membership protocols.
+
+`Snapshot` distinguishes native allocation evidence from scheduler observations.
+No live allocation size is filled from today's catalog. Connection namespaces
+persist independently of offers, and IDs encode native incarnation evidence
+without a UUID-to-job lookup database. Exceptions retain known cluster IDs and
+submission tokens for partial/ambiguous acceptance.
+
+Execution submits ordinary tasks through the borrowed client's `submit` method.
+Dask chooses the workers and handles dependencies; invocation-specific keys prevent
+unintended reuse across commands. There is no worker-selection layer, per-worker
+preflight orchestration, source fingerprinting, or login-node guard. Driver-side
+preparation and the existing task runtime/sandbox checks remain in their owners.
+`output.py` transports byte chunks through standard Dask events so detached
+workers' output reaches the invoking CLI. Probes preserve both streams;
+materialization sends recipe output to stderr to leave stdout for its report.
+
+Local teardown drains the allocation's validated process group rather than
+assuming the owner's exit proves every child stopped. Failed unpublished launches
+are cleaned up, and incomplete locator directories do not hide healthy allocations.
+Cancellation and concurrent project writers are not made safe by allocation
+management; callers must respect the documented execution limits.
+
+Tests cover deterministic selection, malformed identities and catalogs, partial
+native failures, acceptance ambiguity, PID reuse, detached local lifetime, standard
+Dask bootstrap, and explicit execution through borrowed clients. Slurm command
+contracts are simulated; a real NERSC submission remains a deployment check.
diff --git a/docs/api/container.md b/docs/api/container.md
@@ -57,7 +57,7 @@ Sources: `src/lightcone/engine/image.py`,
   `OCIBackend`, data-parameterized; the podman family is stated once
   (`_PODMAN_FAMILY`) and asked positively, so a new runtime falls
   outside it by default. podman-hpc adds exactly one step (`migrate`,
-  outside the load branch) and joins `_SHARED_STORE_RUNTIMES`.
+  outside the load branch). Execution verifies the prepared image on each worker.
   Detection order podman-hpc → podman → docker; docker's daemon is
   probed at detection.
 - **The architecture gate refuses before the load** — a wrong-arch

diff --git a/docs/api/crate.md b/docs/api/crate.md
@@ -1,9 +1,10 @@
 # lightcone.engine.crate
 
 The publication view: the repository described as a Workflow Run
-RO-Crate. The project *is* the crate — `ro-crate-metadata.json` sits at
-the root, describes what the repository already holds, and a deposit is
-`git archive`, not an export step. lc's manifests stay the canonical
+RO-Crate. `ro-crate-metadata.json` sits at the root and describes the
+repository's research objects. A complete deposit must include the annexed data
+and results; `git archive` alone carries their Git representations, not their
+content. lc's manifests stay the canonical
 record; the crate is the same facts in schema.org vocabulary for
 archives and viewers that will never run `lc`.
 

diff --git a/docs/api/index.md b/docs/api/index.md
@@ -17,7 +17,7 @@ is responsibility and contract, not every signature.
 | [`assets`](assets.md) | One output: its directory, manifest, and state | pure |
 | [`worker`](worker.md) | Making one output; the rerun entry point | impure |
 | [`materialize`](materialize.md) | The driver: gates, scheduling, the save/restore loop, status | impure |
-| [`venue`](venue.md) | Where a run executes: SLURM detection, the login guard | impure |
+| [`compute`](compute.md) | Resource requests, native allocation lifecycle, borrowed Dask clients | impure |
 | [`sandbox`](sandbox.md) | The exec boundary: policy, backends, attestation, denials | mixed |
 | [`image` & `container`](container.md) | The container hatch: declaration → image → archive → runtime | pure / impure |
 | [`crate`](crate.md) | The publication view: the repo as an RO-Crate | pure |

diff --git a/docs/api/materialize.md b/docs/api/materialize.md
@@ -7,22 +7,25 @@ its classification walk.
 
 Source: `src/lightcone/engine/materialize.py`.
 
+Recipes are ordinary Dask tasks. Their stdout/stderr is forwarded as bytes to the
+driver's stderr, independently of success or failure, leaving stdout for the report.
+
 ## Key symbols
 
 | Symbol | Role |
 |---|---|
-| `materialize(root, targets, *, refresh)` | The run: guards → converge → plan → fetch → schedule → save/restore loop → crate converge. |
+| `materialize(root, targets, *, cluster_id, refresh)` | The run: guards → converge → plan → fetch → schedule → save/restore loop → crate converge. |
 | `check(root, targets, *, refresh)` | The same classification without executing, committing, or fetching. Exempt from the dirty refusal. |
 | `status(root)` | The report: every output's state and provenance commit, plus the mode/image/sandbox header facts. |
 | `MaterializeReport` / `StatusReport` | The JSON surfaces; `ok` and `up_to_date` first. |
-| `cluster_for_run()` | The venue ladder, and the two-method scheduler seam (`submit`, `completed`). |
+| `cluster_for_run(cluster_id)` | Borrow the cluster; the submit/completed scheduler seam (`submit`, `completed`). |
 | `run_record(...)` / `datalad_run_subject(...)` | The commit message `datalad rerun` replays, and the one spelling of its subject line — shared with the foreign-write comparator, because two strings here would drift. |
 | `_engine_requirement()` | How a record pins its engine: by version for a release, by source commit (hatch-vcs) for a dev build. |
 
 ## The run's order, and why
 
-1. **Login guard first** — the allocation is the remedy with queue
-   latency, so the user submits it before fixing anything else.
+1. **Explicit cluster first** — validate native allocation identity and connect
+   to its scheduler before preparing the project.
 2. **Dirty refusal before the environment converge** — in
    containerized mode the converge can commit an image archive, and
    `dataset.save` commits the whole index; on a dirty tree the user's
@@ -37,9 +40,9 @@ Source: `src/lightcone/engine/materialize.py`.
    — the driver commits as results arrive, so any per-task read could
    answer differently mid-run. Nondeterminism in a provenance field is
    worse than either answer.
-6. **Save on `ok`, restore otherwise, `try/finally` around the loop**
-   — an interrupt restores whatever is still outstanding; the tree
-   ends as clean as it started.
+6. **Save on `ok`, restore reported failures** — unreported outputs are retained
+   after interruption because their tasks may still be writing. Allocation
+   management does not provide concurrent-writer or cancellation guarantees.
 
 ## What must stay true
 
@@ -51,9 +54,9 @@ Source: `src/lightcone/engine/materialize.py`.
 - **`up_to_date` is `ok and not made and not planned`** — a run where
   every recipe failed must not report "nothing to do", and `behind`
   never counts against it.
-- **A read-only verb never tracebacks.** Anything `check`/`status`
-  cannot read classifies as "will be remade" and the real error
-  belongs to the recipe that follows.
+- **Unreadable output state is reportable.** An unreadable manifest or input
+  can classify as "will be remade". An invalid spec, universe, or lock instead
+  raises `ProjectError`, which the CLI presents as a command error.
 - **The run record is genuinely re-runnable**: engine pinned by
   requirement, project environment rebuilt by the worker from the
   rerun commit's own lock, format tested *through datalad's parser*
@@ -69,4 +72,4 @@ Source: `src/lightcone/engine/materialize.py`.
 `tests/test_materialize.py` — real repositories, real recipes, a real
 `LocalCluster` through the seam exactly once, real `datalad rerun` for
 the record's whole claim. `cluster_for_run` is the one monkeypatch
-point for venue-free tests.
+point for allocation-free tests.
diff --git a/docs/api/sandbox.md b/docs/api/sandbox.md
@@ -18,10 +18,14 @@ plus `lightcone/_sandbox_exec.py`, the Landlock shim.
 | `Capability` | What this host can do — `detect()`'s answer, the only `sys.platform` branch. |
 | `Attestation` | What was actually enforced, derived from the flags applied — never from what the matrix says should have happened. |
 | `Backend.wrap(policy, argv)` | The pure rewrite. `contains_prefix` declares whether the uv hop rides inside (a container is a world; a host mechanism trusts host plumbing). |
-| `exec_policy(...)` | The one policy: probe and recipe get the same thing. Building it is where the impurity lives (the per-run private `$HOME`); `scope()` owns its cleanup. |
+| `exec_policy(...)` | Shared policy builder with a recipe output directory or probe `results/` write scope. Creates the private `$HOME`; `scope()` owns its cleanup. |
 | `Unavailable` | A real backend that wraps to the same argv and attests `fs: open`. Saying so is the caller's job; pretending is nobody's. |
 | `denial.explain()` / `denial.trailer()` | Best-guess remedies (allowed to return nothing) and the unconditional trailer on every nonzero sandboxed exit. |
 
+An optional output receiver gets stdout/stderr byte chunks. Capturing output never
+decodes or normalizes stdout; only the retained stderr tail is decoded for denial
+classification. Without a receiver, stdout remains inherited.
+
 ## What must stay true
 
 - **`wrap` stays pure** — no temp files, no FDs, no global state