rushd: WS3/WS4 integration with Reporter prerequisites - #6018
Sean Larkin (TheLarkInn) wants to merge 254 commits into
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Consume the repository experiment before Rush version selection, keep agent detection out of pre-major defaults, strip frontend-only controls before engine handoff, and preserve legacy verbosity compatibility. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Make the demo independent of inherited reporter controls, verify exclusive output destinations, and correct log-level, alias-position, and latest-log guidance. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Validate non-public diagnostic codes against the registry, bucket unknown categories, and bound retained telemetry dimensions deterministically. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Stop reporter control scans at the pass-through separator and add an exactly-once frontend close contract across success, failure, and termination paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Keep protocol metadata root-owned, gate mixed-privacy diagnostics, and bound producer attribution with trusted deterministic retention. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Keep reporter controls out of ts-command-line globals, gate incompatible engines before initialization, and enforce bounded signal and close-error behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Preserve unsupported custom reporter values until frontend ownership is unambiguous, and narrow emergency legacy stripping to the reporter selection flag. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Probe reporter ownership without requiring a value, then enforce strict reporter parsing only after frontend ownership is established. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Derive bounded producer retention priority from parent-session provenance instead of child-controlled package namespaces. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d6318e80-5da9-4858-a147-817e8692f10e
Document the existing R5 full-detail requirement separately from configurable primary presentation and additional file destinations. The automatic invocation artifact remains debug-complete in file mode; no runtime behavior changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Preserve the standalone flag strip list alongside consumed value controls in both real bootstrap compatibility fallback paths. Keep this combined-boundary fix on the corrected review baseline, outside the scoped 6019 follow-up delta. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Reconcile the three daemon-adjacent configuration/export/template conflicts after the R2A squash landing, and remove one automatically duplicated reportingConfiguration declaration. Preserve every reviewed frontend, client, Rush-lib, daemon, reporter source/test and public API blob exactly while inheriting the landed main build-tooling and publication changes. Verified dependency/consumer build, full Rush-lib tests and focused frontend lifecycle/control regressions. The independently discovered Windows watch-fixture failure remains a separate pending correction. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Resolve physical cwd at parser entry so native Windows short names and directory aliases match Git repository paths. Keep real watch cancellation coverage and add symlink/junction regressions without mocking input analysis or watcher behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Native Node 24 and 26 validation showed that generic realpathSync and FileSystem.getRealPath retain 8.3 names. Use the existing native-realpath pattern to resolve the physical directory before configuration discovery. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Explicit pipes prevent Git line-ending notices from being mirrored onto the parent test stderr. Real setup failures still throw with the original captured error text. Reproduced the actual Rush production gate with process-local core.autocrlf=true and core.safecrlf=warn: unchanged tests exited with warnings before the fix and passed cleanly after it, without changing CI warning policy or watch assertions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Resolve both expected and actual link locations using native-backed realpath before comparing them. Add a real directory-alias regression that still rejects wrong and missing targets, and run it alongside the unchanged npm and Yarn integration workflows. Reproduced the previous lexical mismatch before the fix; the regression and complete suite pass under an invocation-owned aliased temporary root. No production code, dependency versions, or CI gates changed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Port the existing R8 owned-worker regression mechanism to the integration tests. Isolate missing-log, unrelated-remediation, and raw-byte mutations; terminate and join workers before cleanup or the following test. Preserve all existing case deadlines, production code, corpus fixtures, raw-byte thresholds, privacy and actionability gates. Cover delayed, rejected, cancelled, and timed-out mutation lifetimes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Reconcile pinned main b6c53f0 with the accepted AI isolation head. Verify all 64 incoming paths against integration semantics before resolving the 25 textual conflicts. Preserve the complete integration source, API and test trees, including native-engine lifetime, physical cwd, Git stdio, link identity and owned AI workers with the unchanged 5000ms negative-test deadline. Remove only the auto-reintroduced early graph registration, already superseded by joined execution-time registration. Retain main's existing JSON-control separator change note. No rebase, history flattening, future-main changes, new production changes or relaxed gates. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Replay the accepted 7c4e9f3 shared watch-test ownership correction after the pinned-main integration merge. Register the complete asynchronous test before setup awaits, cancel and join setup/execution/finalization before releasing locks or deleting fixtures, and retain all existing integration assertions and 5000ms deadlines. No production changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Reconcile only pinned main 130daa8 after proving all 100 incoming paths against published 29c1350. Resolve 35 textual conflicts without changing integration production or API bytes. Retain native engine, protocol, output, privacy and lifetime contracts, including owned AI workers and joined watch cleanup with unchanged deadlines. Accept the landed native-help matrix expansion and its previously absent unscoped-output change note. Preserve both parents and existing history; no rebase, future-main alignment, runtime changes or gate relaxation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
A controlled working-set trim of the verified fixture-owned Windows24 daemon reproduced two legitimately retained children below budget after the original allocation was sized. Keep the public IPC request admitted while the child reports actual allocated RSS, perform one measured adjustment, and verify both real pressure and single-runner headroom before releasing it. Preserve all four measured-score versus opposite-recency cases, native builds/processes, watcher/generation/eviction assertions, and existing deadlines. Add captured-sample allocation regressions. The same real trim fails before and passes after; normal Windows24, Windows26 and Linux selections pass. This proves the fixture precondition vulnerability, not the precise cause of the uninstrumented hosted failure. No production changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
|
Published A real working-set trim of the verified fixture-owned Windows 24 daemon reduced RSS between allocation sizing and admission release. Combined RSS remained below the 512 MiB budget, so maintenance correctly retained both projects while the fixture incorrectly expected one. This proves the fixture defect, not the precise cause of the uninstrumented hosted failure. The same public IPC request now remains admitted while the fixture measures actual RSS, performs at most one bounded adjustment, and verifies real pressure plus single-runner headroom before releasing admission. All four real policy cases, exact retained/watched/generation assertions, memory budgets and existing deadlines remain unchanged. Production and API code are unchanged. The same controlled native counterexample failed before and passed after. Normal Windows 24, Windows 26 and Linux each passed 11 focused cases, including all four real retention cases; build/types/lint passed. Fresh full CI run |
Publish allocation, adjustment, release, cancellation and fixture inputs through an exclusive owned temporary sibling, closing the payload before atomic rename. Read only newline-complete IPC event frames and bind pressure samples to a unique request gate. Preserve immediate errors for malformed published JSON and keep request joining even when cancellation publication fails. Cover held partial publication with real files and a separate reader process, staging cleanup failures, complete malformed frames, input replacement and stale phases. Keep all four measured-pressure policies, one RSS adjustment, real process/build behavior and existing deadlines unchanged. No production or API changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
Keep the initialization promise and assignment continuation owned through teardown. Cancel timed-out setup and join its real host before disposing a late fixture, preserving the original five-second hook deadline. Retain setup and cleanup failures together and add deterministic late-host, partial-failure and cancellation regressions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10
|
Published both fixture corrections together at The Linux 24 failure was a created-but-not-completely-written release JSON file. Allocation, adjustment, release, cancellation and fixture inputs now publish a closed owned temporary sibling atomically; event logs use complete NDJSON frames and current request identity. Malformed completed data still fails explicitly. The Windows 26 failure exposed unowned initialization after a setup-hook timeout. Teardown now cancels and joins late creation and disposal before using the fixture, preserving both original setup and cleanup errors. A forced 5,000 ms timeout still fails its case, without the secondary undefined-disposal error or leaked host/folder. The exact nine-path union is test/helpers and two type-none change notes only. Production, APIs, configuration, defaults, memory budgets, original retention assertions and all existing deadlines are unchanged. On the actual combined head, Linux 24 passed app 36/36 and daemon 29/29; one full uninstrumented serial Windows 26 qualification passed app 36/36 and daemon 14/14. This closes the earlier local full-run qualification gap; it does not claim the prior intermittent startup/timing cause was eliminated. Previous Windows 24 evidence is reused only by exact source/runtime identity, not represented as a new combined run. Hosted CI verified September 11 at 11:29 UTC: |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10 Copilot-Session: 719d36bf-0fcf-419e-b464-844ddf140881
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10 Copilot-Session: 719d36bf-0fcf-419e-b464-844ddf140881
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10 Copilot-Session: 719d36bf-0fcf-419e-b464-844ddf140881
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10 Copilot-Session: 719d36bf-0fcf-419e-b464-844ddf140881
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10 Copilot-Session: 719d36bf-0fcf-419e-b464-844ddf140881
Reject oversized diagnostic codes before registry lookup or syntax validation without changing the diagnostic API. Limit non-public projection to the registered local-sensitive allowlist, and cover code boundaries, floods, counters, and secret exclusion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10 Copilot-Session: 719d36bf-0fcf-419e-b464-844ddf140881
Preserve machine reporter writes during legacy capture, recognize owned additional stdout outputs during bootstrap without claiming custom parameters, and classify version installation paths as local-sensitive. Keep legacy defaults and canonical full-detail logging unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10 Copilot-Session: 719d36bf-0fcf-419e-b464-844ddf140881
Require child diagnostic categories and supplied template keys to match the registered definition, preserving optional detail omission and severity overrides. Enforce positive safe integer source coordinates when present, with contextual coverage for the older child-process suite. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10 Copilot-Session: 719d36bf-0fcf-419e-b464-844ddf140881
Reuse the qualified nonempty secret-value alias classifier for machine JSON source fields and private producer identity, while preserving owner-only FileReporter context and unrelated source data. Apply producer privacy checks to combined machine output without changing the 13-case corpus, thresholds, defaults, or deferred AI flush behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10 Copilot-Session: 719d36bf-0fcf-419e-b464-844ddf140881
Adapt the inherited R6 additional-output matrix to the integration branch's existing file-reporter stdout reservation while keeping plaintext output live. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10 Copilot-Session: 719d36bf-0fcf-419e-b464-844ddf140881
Apply the qualified R8 matcher to the integration branch's human-readable diagnostic surface without importing the rejected flush slice. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4747f826-8c83-495d-80df-3d1168662e10 Copilot-Session: 719d36bf-0fcf-419e-b464-844ddf140881
Preserve daemon integration, control ownership, stdout privacy and lifecycle fixes while incorporating the landed R6/R7 prerequisite changes and regression coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Repair Windows lock acquisition and release recovery; centralize lock backing paths; isolate request selection options; normalize shell and identity handling; complete publication metadata and environment documentation; add a file-level review map; and apply the pinned formatter without weakening strict lint rules. Retain intentional frontend dependency, same-user request shell selection, and explicit permission failures with documented rationale and regression coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
What this adds
An opt-in Rush build daemon that keeps configuration and the build graph in memory between commands. Repeated builds can reuse that setup instead of starting the engine from scratch.
Implements WS3 (#5898) and WS4 (#5899) of #5894, with Reporter prerequisites from #5858. Existing
rushandrushxbehavior stays unchanged.Developer outcomes
rush-clientand supported package scripts throughrushx-client.start,stop,restart,status, andlogscommands.The PR also fixes shared Windows lock cleanup, shell/environment handling, and request-selection isolation. Those fixes affect ordinary Rush commands too, not just daemon execution.
Trying it
Install a matching daemon-capable package set and select its Rush version in
rush.json, then opt in:The client starts the daemon if needed. Use
--no-daemonfor in-process execution. CI stays in-process unless explicitly enabled withRUSH_DAEMON=1.Merging this PR does not publish those packages. See the client guide and environment-variable reference for setup and controls.
Current limits
.envinitialization—fall back before execution. A successful build alone does not prove it used the daemon.daemonIpcconfiguration and a compatible worker implementation; this mode is unsharded and non-cacheable.daemon.watchobserves files; it does not start builds automatically. Memory limits are best-effort, not hard process-tree limits.Making ordinary
rush/rushxuse the daemon by default is the later WS5/Rush 6 change (#5900). This PR does not make that switch.Review scope
This branch includes Reporter work as well as daemon integration. The file-level review map separates exact inherited files from integration and mixed changes. Review fixes and repository-wide formatting are in
1b4f89ae37.The environment reference is included here; publishing it to the separate website repository remains a release follow-up.