santos/dev: crew family, Pareto crewing and the Model Pool, custom connections

[santoshkumarradha]

Integration branch santos/dev into dev. Everything below through #1192 landed on dev as #1194 on 2026-09-18 (merged by Abir with his own fixes on top: a zombie-aware process-group liveness check and a CI-mode guard in the pool test), and dev was merged back into santos/dev at 631237c, again at 2ac2360 (the Model Pool mirror workflow, #1116) and at c8d0046 (the /connect custom row fix, #1195), and at ae07e93 (the v0.3.0 changelog roll-up, #1196), so the branch is level with dev. #1203 makes two planner tests count their fake client's calls atomically, removing a test-only data race that failed one full check under load. #1109 brings the worker harness: one loop under /task and codeaf do, with a plan store per run, a review round after each piece of work, and the task tree on the rail. It is off unless CODEAF_TASK_BELT=bash is set, so the default road does not move; today a /task the person types and codeaf do take the new road, and a task the chat starts by itself joined it in #1205. #1207 seats a review check on the crew's checking seat instead of the thinking seat, so the model a person picks to check is the one that checks; the door resolves that seat from --check-model, then CODEAF_CHECK_MODEL, then a pinned plan seat, then the profile's careful row. The two harness waves landed at 164548f (full check on Spark green, 114 packages): #1204 draws the plan as a tree on the chat rail with a dot row, a line saying what is happening now, a bounded read-only question about a run, and a work tab that lists runs; #1205 gives the chat one way to put work out, a landing that wakes no model turn, and a reply only when an answer is owed, and a task the chat starts now takes the new road too. #1210 reads a run's result from the store when the root's worker returns after the run ended, which removes a race that lost the answer under load. #1209 shapes answers to lead with the action, number their steps and end on one next thing. #1211 joins a test's background writer before its state directory is torn down (test-only, landed at 7cda67c, full check green). Three fixes from a first real session on the new road landed together at b94df33 (full check green, 114 packages): #1212 counts a landing's report as evidence that the ask is finished, so a correct answer is not followed by an empty turn, and a check refused for a leading directory change is told the form that passes; #1213 keeps a reopened conversation's tasks, by reading back a task born from a run's plan, never overwriting a checkpoint it could not read, never reusing an id, and saving a run's ending with its row; #1214 tells a refused proposal every problem at once and settles where a task works from the folders its brief names before asking the person. #1215 reads a declared check as one command the way a shell reads its quotes, so a search for one of several words is admitted, run and judged by one shared reader, and a refusal names the character and the form that passes; #1221 judges the router scenario on scripted first tokens so its verdict no longer depends on how busy the machine is (both landed at 39ffbc5, full check green). Seven small PRs landed together at fe9eb62 (full check on Spark green, 114 packages): #1222 lets the conversation's tasks tool read a run's tasks from the run's own record, so a finished hand-off is asked about and not redone; #1223 says once, at load, which top-level profile config keys nothing reads, so a nested object written where a dotted key was meant is no longer silent; #1225 stops a dispatch pass from reclaiming the leaf it just released, which ran the same piece of work twice and spent its budget twice; #1227 seals the place sweep's note at close, so nothing is written under state after the process closes and quitting never waits on the folder walk; #1226, #1228 and #1229 make three tests independent of machine load (a wait helper that checks its condition before the clock, a usage writer joined before cleanup, a first token ordered by a signal). Three more landed together at 4e176a5 (full check on Spark green, 114 packages, first run): #1230 lets a conversation whose engine runs in another process, which is the default, read and steer its run: the rail's rows, a task's page and the steering verbs now cross the wire, a compile-time line fails the build if a later method is not carried, and the plan is re-read on a beat only while a row is queued or running, so a conversation at rest reads nothing; #1231 opens a task's page from its row, gives a run a row of its own with its parts under it, and answers a click from the same drawn view the frame used; #1233 seats the review of a codeaf do run whose root finished itself before its worker returned, which used to end the run with no check and failed the full check under load. Five more landed together at 608c459 (full check on Spark green, 114 packages): #1220 makes a worker's declared checks the review contract: a holds verdict rests on a recorded, zero-exit run of every declared command, the worker records each command's own exit, a record opens with a line marking it as one that records exits, so a declared check that was never observed is refused, and the basis of every verdict is kept with the task (replaces #1206); #1232 gives every run a copy of its own, so a hand-off no longer works in the person's folder, and brings the work home as one commit when the run ends; #1235 paints a wait whose stream has stopped arriving at the spinner's own pace, measured at about 0.6 points of a core less on the wait path, with its method and harness under bench/idle-surface; #1240 stops a letter typed into a note on a running task's page from raising the stop card, and a finished task's page no longer offers verbs the store would refuse or shows a store id; #1237 takes the connect tests off the fixed callback ports, so nothing else on a machine can fail them. Two more landed at 1cbcb4b (full check on Spark green, 114 packages, first run): #1239 tells the person once, in the conversation, which config.json keys nothing reads, in hosted and unhosted launches alike, again only when that set of keys changes, and never for a key the product itself writes; #1238 identifies a surviving process in a test by its identity and start time, since a bare process number can be reused by an unrelated process. Four more landed at 12840ee (full check on Spark green, 114 packages, first run): #1234 keeps every run a conversation has made readable, and closes a lock the plan reader could leave held when a conversation had no run yet, which stopped the engine; a new guard law now fails the build if a reader of that shape can return while still holding its lock; #1242 makes two profile writes in one process compose, so a second write no longer drops the first one's key; #1243 adds the regression test for seating a check after a worker whose root finished itself returns; #1241 adds a repeatable drive of a run on the real binary in hosted mode under scripts/, outside the full check. Five more landed at 443c661 (full check on Spark green, 114 packages, first run): #1244 gives what a person types after pressing a run's row to that task's page, so a note meant for a task is never sent to the model as a message; #1245 keeps the unread-key notice quiet about keys the product itself retired, with a ledger law so a key cannot be retired silently; #1236 lets a reconnect reuse its registered identity, reports a finished sign-in only after its listener is released, and hands the product a held listener in tests; #1247 and #1246 make two tests independent of machine load. Two more landed at 8ed67b7 (full check on Spark green, 114 packages, on the second run; the first run failed one temp directory cleanup in a package the stack does not touch): #1248 keeps both keys when two codeaf processes write the profile at once, with a lock that holds on every platform the repo builds for, a wait bounded at two seconds, and no stale lock to reclaim after a crash; #1249 makes the job shell tmux tests read one marker line, so a large environment can no longer cut the line they assert on or let an absence pass unproven. Still to come as small PRs: the task's own page and its reader over the wire, a copy of its own for every run, and the review contract where a worker declares its own proof command and the engine judges it by what ran (replaces #1206). This PR now carries what came after #1194: the lazy lexer registry (#1193) and whatever lands next. This PR is the running index of what the branch carries at the product level. It stays a draft until every feature below has landed on santos/dev, the independent end-to-end pass on Spark is green, and Santosh has e2e-tested it.

What a person gets

Feature	What it does for you	How you use it	Where it is
Crew model family: open or all	The three crew words (frugal, balanced, max) answer from open-weight models only, or from every model including frontier ones.	models.crew.source = open|all (default all), or the open/all switch in /crew.	On santos/dev (#1073)
Pareto crewing: crews computed from the live catalog	Crew presets are computed from OpenRouter's prices and quality indexes on the cost-quality front, refreshed daily, instead of a hand-picked table. Any seat can be auto.	/crew catalog, or models.crew.pick = catalog.	On santos/dev (#1093, squash 2fb296b)
Pick mode: table, catalog, learn	Choose the fixed table, today's catalog, or a crew learned from judged runs, per role (worker, checker, planner).	/crew table|catalog|learn, models.crew.pick.	On santos/dev (#1093, squash 2fb296b)
Learn: judged runs move the picks	After each chat /task run an independent judge scores every seat (billed to a judge seat). Scores go to your own sheet and feed the picker next time, blended with the pool by observation count, so your own runs improve your own picks first. Starts from a seed built from 145 real runs, so learning works on day one. A seat you type by hand is judged too.	Automatic under pick = learn. codeaf pool show lists what the picker knows.	On santos/dev (#1093, squash 2fb296b)
Model Pool: collective, text-free evidence	Every install can share its score rows through a relay and pull a signed index back, so everyone picks better because the others ran. Rows are computed numbers and ids under a per-install nonce: never code, prompts, paths or identity.	model_pool on|read|off (default on, notice shown once). codeaf pool show|status|verify [--json].	On santos/dev (#1093, squash 2fb296b)
Public relay and GitHub mirror	The relay Worker ships in-repo under relay/ with a runbook; it is live at https://codeaf.agentfield.ai/pool. The signed index is mirrored to the model-pool branch of this repo so anyone can watch it update. A team can run its own private relay with its own key (CODEAF_MODEL_POOL_RELAY_URL, _MIRROR_URL, _PUBLIC_KEY).	Nothing to do; works out of the box.	On santos/dev (#1093, squash 2fb296b)
The paper	docs/design/model-pool/pareto-crewing.pdf ("Pareto Crewing"): the objective, estimator and picker, with figures from real runs.	Read it.	On santos/dev (#1093, squash 2fb296b)
Named, multiple, switchable custom connections	Connect several OpenAI-compatible endpoints (local models, proxies), each with its own name, key and model list; switch between them, including mid-session; rename one and every stored pick follows. /model groups models under each connection's name. Switching onto a keyless default service is refused with a reason.	/connect → Custom OpenAI-compatible API; /settings → Providers → add / edit / active connection.	On santos/dev (#1092, squash 825ef2c; Santosh e2e-tested locally against a local OpenAI-compatible proxy)
Worker harness: one loop under /task and do	The internal hands (patch, doc, web, image) become one worker loop with a plan store, shared by the chat /task door and codeaf do.	No new surface; fewer steps and lower cost per task (smoke: 4/4 parts in 28 steps vs 3/4 in 42).	#1109, draft, wave 1 landing

Bug fixes on the branch (question answer-box keys, landing answer fate, retry avoiding the failed upstream, recovery evidence, taxonomy lane read, and, with #1092, mid-session /connect now reaching the live conversation on the engine road, a pre-existing hole) are not listed above; each has its own change note under docs/changes/unreleased/.

Known limits, stated on purpose

• The learned picker is greedy on its estimate; there is no exploration yet, so a model that is never picked is never scored. The next algorithm pass (Thompson sampling, hierarchical cap, drift) is a follow-up PR.
• The judge runs on the chat /task door only; codeaf do, ask and remote sessions do not feed the pool yet.
• The pool index is read at start-up (24 h cache); codeaf pool verify pulls a fresh copy, read at the next start.
• A pool cell becomes readable at 3 rows, from one install or three (the paper says three installs); follow-up.

Relay status

Live at https://codeaf.agentfield.ai/pool (healthz 200, index signed, codeaf pool verify good). The model-pool branch exists and carries the current signed index (refreshed hourly by a stand-in job until #1116 puts the workflow on dev). The pool is primed with the same measured runs the embedded seed encodes, so the published index carries real cells from day one.

End-to-end pass on Spark (independent session, done)

All 12 steps pass on #1093's head (cf4ea35; final fd29d79 differs by a runbook sentence). Report: spark:~/src/e2e-learn-2026-09-17.md with KNOWN-LIMITS.md and evidence/INDEX.md beside it; 18 capped runs, about $26.

Step	Result
1 fresh install, learn, judged tasks, own sheet + outbox flushed	pass (first pass failed: judge picked a rate-limited free model; fixed)
2 rows reach the relay, fold, publish	pass (index 1789681435, 19 cells)
3 fresh install D reads the pool and picks differently from the seed	pass (pool: worker gpt-5.6-sol; seed only: glm-5.3-flash)
4 privacy: rows are numbers/ids/nonce; read and off send nothing	pass
5 resilience: bogus relay → mirror → seed; task still runs; verify good	pass
6 manual /model override under learn is still judged	pass
7 crew words × open/all, table vs catalog, real tasks on frugal-open and max-all	pass
8 auto/catalog/learn seats resolve on headless doors	pass (first pass fell to the table; fixed)
9 /crew chooser and /settings pool row with notice	pass (note: pick mode is a settings row, not a chooser switch)
10 `pool show	status
11 codeaf models, --refresh, daily cache	pass
12 headless do/exec under a learn crew	pass (fixed with 8; judge on headless doors stays a follow-up)

Follow-ups from the e2e, now on santos/dev: a task's thread stops its parts before closing and a machinery cut records its reason (#1118); a decision card under --yolo takes its default (#1117); the auto crew row shows the rung it actually resolved to (#1119); a bare auto row under picked from = catalog reads the catalog's figures alone (#1120). codeaf do seats from the settings the run will use, so a fresh profile with the key in the environment reads the catalog, not the table (#1122); the relay accepts the exec and run doors (#1121); a launch's pool errands stop when the process closes instead of writing into a closed profile (#1124); a launch compiles internal/verify's patterns on first use and the engine host runs on a capped scheduler (#1125); the client's min_installs floor counts installs, as the relay's does (#1127); the relay folds a retried batch once by its nonce (#1128), carries an acceptable metric beside the judge score (#1129), publishes only cells that meet the floor (#1131) and writes once per cell per batch (#1136); a refused row leaves the outbox and the file is compacted (#1130, #1133); the pool errand tracker's critical sections unlock from a defer (#1134); a run a process death left unjudged, and any headless door's landing, is judged on the next chat start (#1132). the finished-tree check reads a project cloned into a subdirectory (#1135); the exec, run and do doors leave a pending record for the sweep (#1137, #1138, #1139); the wall admits a first round it cannot cost and refuses an overrun it cannot hold, so a long codeaf do lands instead of re-planning to the wall (#1144), with the delivery gate over a grown job root pinned by a test (#1147); pool show, pool verify and pool status print both metrics, the pending judge records and the last sweep (#1141, #1142); a test's crash fixture stays in a profile the test owns, and the package's test floor clears any profile the environment named, and the opening-hint test reads a profile it made and pins the greeting contract both ways (#1145, #1150, #1153), and tui3's package law makes every surface-building helper name a profile of its own (#1173); the manual's status row says whose ledger it is (#1143); the seed index is regenerated from the live relay by one in-repo command, so a fresh install's fallback is a verified copy of the pool (#1148); pool show --cells lists the held index's cells with their installs and dims (#1149); a landing that leaves gitignored build products behind stays complete and the reconciled row names them (#1152); codeaf version on an unstamped binary names the condition, no .git directory for the toolchain to read, instead of blaming a Makefile target (#1151). pool status says when the relay dropped a row and why, instead of the row vanishing silently (#1154); a job teardown records the group leader's identity at launch and never signals a process group the kernel has since handed to someone else (#1155); a codeaf do continuation after an overrun continues the plan it has from the recorded turns instead of re-planning from scratch (#1156); the process sets a soft memory limit of half the machine's memory, nothing below a 512 MiB floor, an explicit GOMEMLIMIT untouched, carried into engine --daemon (#1158), and the bound is the smallest finite one of physical memory and the cgroup v2/v1 limits walked from the process's own level to the root (#1162); pool status says when this run cached the index, so the seed-to-cache flip is visible in the run that caused it (#1161). The pool's copy of a seat's id is the bare vendor/id, so a fresh install on the default model (whose seat carries codeaf's alias marker) no longer has its first rows refused by the relay, and the judge picker's same-vendor exclusion works on it (#1163); no cmd/codeaf test can push rows to the default relay, the package floor pins a dead address (#1164); the relay bounds each judge's fitted severity to ±10, clamps every adjusted score to the rubric, publishes mean and sd on one scale and falls back to the median when spread is zero, which is what had published a cell at 103.5 (#1165); the relay can purge stored rows from a fixture vendor and refuse vendors not in ALLOWED_VENDORS (#1167), with ALLOWED_VENDORS set to the catalog's vendors plus codeaf (#1177, Worker version 47102a26; 624 fixture keys purged from the live store); the built-in seed is regenerated from the relay's first clean publish after the purge, 18 cells, no fixture rows (#1180); a job's shell no longer inherits the host's TMUX handle and gets a profile-owned TMUX_TMPDIR, so a command the model runs cannot kill the terminal hosting codeaf (#1166); an unattended run (--yolo, nobody to answer) is refused a git push, merge or rebase that would move a branch it did not create, including the branch the checkout began on, while its own task branch stays pushable and attended sessions are untouched (#1168, the road it took is in #1160's note); pool status says the install's identity, the judged total across all doors, and "none yet" when no landing has been judged (#1169); an exec run's worker spend is minted into the usage ledger at the door, one row per run, so the status row, the cap and the pool row count it (#1170); the task door honours a stored models.crew word (every arm had resolved to balanced) and a pinned tier row survives pick=learn instead of being recomputed (#1171); a landing card raised on the check road under --yolo takes its default and records that the dial answered and that the check ran out of time, instead of parking a headless run forever, while conflict, shift and ground landings still park and now say so in the record (#1174); the model catalog warm is seated on the profile's start-up errand tracker so a process close joins it, instead of a warm outliving the process and writing its cache into whatever state root was current (#1179); the auditor's calls tag themselves auditor with the node they check, and its usage rows carry the auditor role and the task, so a ledger tells an audit call from a session turn (#1181); a headless do --json whose node ends failed or unverified names the kept branch and the verdict in its envelope, so nobody digs through tasks.json for kept work (#1182); the settle turn a landing note wakes under task.settle auto runs under the checker's own bound, a call ceiling and a share of the run's cap, one call for a clean tree with no declared check, and records "not settled within its bound" instead of running to the wall, which had been a one-to-two-dollar tax on most headless runs (#1183); the chat-side tasks text and a resumed session's record line name a failed or unverified node's kept task/<slug> branch and its verdict word, the same two words the headless envelope carries (#1184); the identical-effects test is deterministic, its forty saves carrying distinct call signatures so the turn's repetition guard cannot end the turn and the leash is the only thing the assertion is about, which closes a margin-of-one race that had read as a load flake since #284 (#1185); a non-verified codeaf exec or codeaf run names its kept branch and verdict in its envelope through the one result-envelope seam the three headless doors share, so do, exec and run answer alike (#1186); a task worker parked on a command it started (a bash call the 30-second background clock promoted to a job) waited the node's whole allowance when the job never ended, which is what held a DOE run silent for 53 minutes under a live cap; the park now has a bound of its own, a third of the allowance, posts "the park was not settled within its bound" and hands the turn back while the command keeps running, with the diagnosis at docs/notes/yolo-task-worker-parked-on-promoted-bash.md (#1187); the model pool runbook's plan line says metered, not capped (#1188, #1189, docs only); the status row says working while the door's task subtree turns, read from the same signal the tab strip draws from so the two cannot disagree, with the manual's screen and task-page sections moved to match (#1191); a job subtree is cut when it passes a CPU or process bound, 60% of the machine sustained for 30 s or 128 processes when nothing else bounds it, the process backstop alone under a cgroup or affinity quota, and the run learns it as "the job subtree was not settled within its bound", which is what a model that spawns infinite loops under a dollar cap had escaped (#1190); a note records what a task's audit step can execute and where that boundary is set, the declared checks plus read-only git and coreutils, which is why a brief that declares no check leaves a cautious auditor nothing it may run (#1192, docs only); the prose lexer registry loads a curated set of forty languages on first use instead of parsing all 279 of chroma's lexers at init, plain text for the rest, which takes about 3 ms off cold start and 2 MB off the binary, with chroma's files and licence carried beside the copies (#1193). Still open, PRs in progress: the footprint benchmark method and table under docs/benchmarks (#1126); Pareto crewing v2 with a model-free grader (#1123, Santosh reviews). The "task-branch merge fails 1 in 3" observation was the test driver killing runs mid-check, not a product defect.

Related, not on this branch

• telemetry: audit fixes — hosted chat counters, no unversioned sends, silent smoke tests #1111 telemetry audit fixes → dev directly (must ride staging and the next main release; the website relay fix is already live). Also merged into santos/dev so the e2e covers it.
• ci: Model Pool mirror workflow (signed index onto the model-pool branch) #1116 Model Pool mirror workflow → dev directly (workflow file + verifier only), so the model-pool branch is refreshed by Actions; until then a stand-in job on Spark refreshes it hourly. Also on santos/dev.

Before this leaves draft

• connect, settings: named, multiple, switchable custom connections (#1089) #1092 (825ef2c) and Pareto crewing: the crew picker, the Model Pool with its judge and relay, and the paper #1093 (2fb296b) merged into santos/dev; dev, telemetry: audit fixes — hosted chat counters, no unversioned sends, silent smoke tests #1111's and ci: Model Pool mirror workflow (signed index onto the model-pool branch) #1116's branches merged on top (6092393).
• The e2e report above green (done).
• Santosh's e2e pass on santos/dev.
• Full make check on the final santos/dev head, on Spark.