Skip to content

fix(bazel): Cargo feature parity and Windows builds - #3133

Merged
ScriptedAlchemy merged 28 commits into
masterfrom
claude/bazel-windows-fixes
Oct 7, 2026
Merged

ScriptedAlchemy merged 28 commits into
masterfrom
claude/bazel-windows-fixes

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Bazel builds were resolving different Rust features and test runtime inputs from the existing workspace. That changed canonical JSON identity digests, omitted grammars and test-transport behavior, and left test subprocesses and fixtures inaccessible.

This change corrects target-side serde_json features, shares the workspace test feature resolution, and generates test binaries with the features their suites require. Test launchers stage runtime resources, provide absolute executable and toolchain paths, retain canonical test crate names, and isolate each test's home and profile. Installed host configuration prefers the stable PATH executable over a Bazel build output.

Windows fixes shorten external build-script paths, patch the aws-lc header include, and apply platform configuration after generic defaults so the Windows workspace-status command and UTF-8 compiler flags actually take effect. CI queues are scoped to each PR instead of blocking unrelated validation globally. The macOS filename probe proves ordinary writes succeed before recognizing an unsupported filename.

Validation through Bazel:

  • Original domain suite: 169 tests passed, including strict JSON identity assertions and runtime fixture reads.
  • Focused macOS churn tests: 2 passed under the sandbox.
  • Runtime environment probe, private-fs, and framing targets passed.
  • Generated BUILD validation, repository gates, dashboard, and feature gates passed on earlier shared-fix commits.

Full CI is running on the current head. Windows launcher and dependency-resolution failures still need confirmation from the diagnostic run; this draft is not yet green across all platforms. Temporary Windows diagnostic output will be removed after the failures are resolved.

claude added 2 commits October 6, 2026 08:23
rules_rust matches per_crate_rustc_flag on a prefix of the crate root's
exec path, which carries the canonical repository name
rules_rust++crate+crates__<name>-<version>; the generated +crate+crates__
filters matched nothing, so the perf opt-levels never applied.

Windows cannot execute workspace_status.sh, which left the product SHA
genrule without its stable key, and its runners have python but no
python3 for the dashboard digest genrule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo
rules_rust stages each external build script's sources under
<repo>/_bs.cargo_runfiles/<repo>/, spelling the canonical repository
name twice, so the deepest vendored headers (tree-sitter grammars,
aws-lc's jitterentropy) reached 260-270 characters and MSVC could not
open them. A one-letter crate hub and a D:/b output base on Windows
shorten every such path by 15 characters.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo
@changeset-bot

changeset-bot Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 58fa8e2

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Bazel hands genrule actions a stripped PATH on Windows, so neither
python3 nor python resolved for the dashboard digest; pass the runner's
PATH and take the first interpreter that starts. The workspace status
wrapper now reads git directly, since a bare bash can resolve to WSL.
CI also prints failing commands and keeps large build-script output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo

Copy link
Copy Markdown
Owner Author

Windows lane status after 558b65f (run 37438541376):

  • Fixed by the MAX_PATH change: tokensave-large-treesitters' build script now compiles.
  • Still failing, addressed in f46beb2:
    • //:product_git_sha: the workspace status wrapper now reads git directly, because a bare bash can resolve to WSL.
    • //dashboard:bundle_sha256: Windows genrules get a stripped PATH with no Python. They now get the runner's PATH, and the genrule uses the first interpreter that starts.
  • Still failing, cause not yet known:
    • aws-lc-sys build script: its 6 MB of stderr was cut off by --experimental_ui_max_stdouterr_bytes. CI now keeps up to 16 MB and passes --verbose_failures, so the next run will show the real error.
    • tracedecay-code-index and tracedecay-lsp: every variant fails with E0463 can't find crate for all of its dependencies, members and externals alike. Other crates compile. Both crates depend directly on tracedecay-code-extraction. I'm waiting on the verbose rustc command from the next run.
  • Test failures still open:
    • serde_json preserve_order leaks from tree-sitter's build-dependency into the Windows target build. This changes JSON key order and identity digests.
    • Timing-sensitive tests under load.

A new workflow_dispatch run with run_os=true is queued on f46beb2.


Generated by Claude Code

crate_universe folds a host triple's build-dependency features into its
normal build, so tree-sitter's build script widened serde_json with
preserve_order: JSON objects kept insertion order and every digest over
them drifted from Cargo builds. A generated crate annotation rebuilds
serde_json from Cargo's target resolution.

rust_test(crate=) takes cfgs from its own crate_features, so unit tests
compiled without their features; they now carry the variant's set.
Rustc runs unsandboxed so env!("CARGO_MANIFEST_DIR") stays readable at
test time, test binaries get 30 minutes, and the cold-cache Linux and
Clippy jobs get room to finish.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo
@ScriptedAlchemy ScriptedAlchemy changed the title fix(bazel): build and test on Windows fix(bazel): Cargo feature parity and Windows builds Oct 6, 2026
Cargo starts a test binary in its crate directory, and 53 suites read
../../tests/fixtures from there; the launchers now cd into the crate's
runfiles directory and every test carries //tests:fixtures. aws-lc's
four-level relative jitterentropy include still overran MAX_PATH once
MSVC joined it to the staged build-script tree, so a crate patch names
the header the builder already puts on the include path. Bazel runs the
Windows status command through cmd.exe, which needs backslashes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo

Copy link
Copy Markdown
Owner Author

Test Windows failed on 14b8ffd4 (run 37511829834): 53 tests passed, 12 failed and 95 failed to build. I addressed it in db0abfdb.

  • //:product_git_sha. Bazel runs the workspace status command through cmd.exe, which reads scripts/bazel/... as the command scripts followed by the switch /bazel. That's why neither the .sh nor the .bat ever ran. The Windows path now uses backslashes.
  • aws-lc-sys build script. cl.exe failed with C1083 opening crypto\fipsmodule\rand/entropy/../../../../third_party/jitterentropy/jitterentropy-library/jitterentropy.h. MSVC joins relative includes without normalizing them, which made that path about 275 characters.
    • A crate.annotation patch changes the include to "jitterentropy.h". The aws-lc builder already puts that directory on the include path whenever jitter entropy is built, and the include sits inside #if !defined(DISABLE_CPU_JITTER_ENTROPY).
    • No other aws-lc relative include goes past 249 characters.
  • Fixture paths. code-extraction's main and unit tests, along with 53 suites across 14 crates, read ../../tests/fixtures relative to the crate directory, which is where Cargo starts tests.
    • The launchers now cd into the crate's runfiles directory before running the test.
    • Every test now carries //tests:fixtures (160 files, 1.1 MB). Linux would have hit the same failures, but its run timed out before reaching those tests.
  • Exit 9009 on five early tests. The probe, framing and private-fs tests failed because their .exe wasn't in the runfiles tree. The same launcher worked for later tests, and the cause isn't known yet. If it happens again, the Windows launcher now reports the missing binary and lists the runfiles tree.
  • tracedecay-code-index and tracedecay-lsp: E0463 can't find crate for every dependency, with no extern-location note. The process_wrapper command line is 27,908 characters, so this isn't command-line truncation. A failure-only bazel aquery --include_param_files step now prints lsp's exact rustc arguments.

The Linux and macOS lanes of run 37511829834 are still running. They're the first check of the serde_json and unit-test feature fixes, so I'll dispatch db0abfdb once they finish rather than cancel them.


Generated by Claude Code

claude and others added 14 commits October 6, 2026 16:06
The golden Pi fixture gained a contentless user message, which capture
refuses, so the malformed-file test's fixture has nine lines and
session_info spans the last one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo
Each member's tests resolved features on their own, as `cargo test -p`
does, so every test build carried its own featured copy of its
dependency closure: 534 libraries for 50 crates, more than a CI lane
compiles inside its timeout on a cold cache. Tests now share two
workspace-wide resolutions, as one `cargo test --workspace` does: the
plain one, and one enabling every required feature plus the
test-transport surfaces. That leaves 181 libraries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo
… HEAD

# Conflicts:
#	crates/tracedecay-agent-hosts/BUILD.bazel
#	crates/tracedecay-application/BUILD.bazel
#	crates/tracedecay-automation-runtime/BUILD.bazel
#	crates/tracedecay-capture/BUILD.bazel
#	crates/tracedecay-code-index-retention/BUILD.bazel
#	crates/tracedecay-code-index-runtime/BUILD.bazel
#	crates/tracedecay-code-index/BUILD.bazel
#	crates/tracedecay-configuration/BUILD.bazel
#	crates/tracedecay-daemon-control/BUILD.bazel
#	crates/tracedecay-daemon-identity/BUILD.bazel
#	crates/tracedecay-daemon-protocol/BUILD.bazel
#	crates/tracedecay-daemon-service/BUILD.bazel
#	crates/tracedecay-dashboard-api/BUILD.bazel
#	crates/tracedecay-global-db/BUILD.bazel
#	crates/tracedecay-graph-db/BUILD.bazel
#	crates/tracedecay-graph-query/BUILD.bazel
#	crates/tracedecay-host-admission/BUILD.bazel
#	crates/tracedecay-lcm/BUILD.bazel
#	crates/tracedecay-lsp/BUILD.bazel
#	crates/tracedecay-maintenance/BUILD.bazel
#	crates/tracedecay-mcp-catalog/BUILD.bazel
#	crates/tracedecay-mcp/BUILD.bazel
#	crates/tracedecay-privacy/BUILD.bazel
#	crates/tracedecay-private-fs/BUILD.bazel
#	crates/tracedecay-project/BUILD.bazel
#	crates/tracedecay-query/BUILD.bazel
#	crates/tracedecay-runtime-core/BUILD.bazel
#	crates/tracedecay-rusqlite-runtime/BUILD.bazel
#	crates/tracedecay-session-memory/BUILD.bazel
#	crates/tracedecay-session-runtime/BUILD.bazel
#	crates/tracedecay-sessions/BUILD.bazel
#	crates/tracedecay-source-edit/BUILD.bazel
#	crates/tracedecay-store-runtime/BUILD.bazel
#	crates/tracedecay-store/BUILD.bazel
#	crates/tracedecay-temporal-query/BUILD.bazel
#	crates/tracedecay/BUILD.bazel
#	scripts/bazel/gen_builds.py
5118be5 added a contentless user message to the Pi golden fixture, which
capture refuses, but the malformed-file test still split the fixture into
eight lines and expected session_info at the eighth line's byte range.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo
All test contexts now share a single workspace resolution with every
required feature and test-transport surface enabled, as the hosted test
partitions ran them: one test build per crate, 162 libraries in all.

Cargo builds the binaries a test run spawns with that run's features, so
each binary gains a `__test` build the suites' runtime overrides and
CARGO_BIN_EXE_* name; the plain target stays the shipped one. Binaries'
own #[cfg(test)] modules run as `<bin>_unit_test`, which covers the CLI's
inline tests, and the host-CLI fixture example reaches its installer
through TRACEDECAY_HOST_CLI_FIXTURE.

CI exports the toolchain the rustup proxy hands `cargo test` (CARGO,
RUSTUP_TOOLCHAIN, and both homes) and passes it to tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo
The test-run builds of every binary (`<bin>__test`, from the shared test
resolution) replace the tracedecay-cli `__test_transport` builds, keeping
their testonly flag and shipped file name; the launcher's own CARGO,
RUSTC, and RUSTUP_TOOLCHAIN supersede exporting them from CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo
Binary unit tests stage the same runtime resource directories as library
unit tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo

ScriptedAlchemy commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner Author

Status at bcc51b4b (dispatched with run_os):

Linux at b7d27ef (job 112555776231). The whole suite ran in 3h15m: 151/160 pass, against 14 failures plus a timeout at 14b8ffd. The 9 failures, sorted:

Already fixed on head:

  • search-eval:unit_test: 912022e8 re-pins the receipt.
  • core_cli_suite: missing host-CLI example, fixed by TRACEDECAY_HOST_CLI_FIXTURE.
  • code-extraction:unit_test: crate fixtures/, fixed by debb3f47.

Rustup reached through a redirected HOME ("missing manifest in toolchain 1.97.1", "syncing channel updates"):

  • mcp:unit_test
  • mcp_suite (affected-tests cases)
  • runtime_acceptance_suite (compiler lookup, LSP negotiation)

The runner leaves CARGO_HOME/RUSTUP_HOME unset, so the launcher's $HOME fallback points into the test scratch dir. bcc51b4b exports both homes.

SDK journeys:

  • transport_acceptance_suite: pnpm run build in sdks/typescript fails to spawn. bcc51b4b adds setup-pnpm to the test tools, as the Cargo-era test jobs had.

In-process vs nextest isolation:

  • cli_non_interactive_test (core_cli_suite): "core publication was not held".
  • session_store_read_cost: per-message writer work 1.3× against a 1.2× bound.
  • session-temporal-store:unit_test: "full root candidate window: Unavailable" (to confirm).
  • runtime_surface_acceptance: one MCP git-preview problem, not yet classified.

Build parity fixes, both host-wide:

  • 1baabab6: rules_rust replaces a build script's CFLAGS/CXXFLAGS with the C toolchain's, so --action_env never reached cc-rs. Vendored C ran without -DNDEBUG everywhere and without /utf-8 on Windows, which is where the 347 Windows extraction Utf8Error/poisoned-LazyLock failures came from. The flags now travel as HOST_/TARGET_CFLAGS, which cc-rs appends.
  • 3f51fa9e: .cargo/config.toml's LIBSQLITE3_FLAGS never reached Bazel; a crate annotation adds it.

Windows E0463. The params file does name every --extern. Locally, a missing, wrongly named, empty or directory extern each produces its own rustc message, so a bare E0463 comes from a rejection the locator doesn't record. f208a4e7 reruns one lsp compile with RUSTC_LOG=rustc_metadata::locator=info on failure.


Generated by Claude Code

claude and others added 5 commits October 7, 2026 00:00
#3073 (lexical roster carry) and #3067 (ambiguous candidate gaps) merged
on 2026-10-05 seconds apart and moved validation ranking; Linux tests
moved to Bazel that evening and no master run has reached search-eval
since, so the drift went unseen. The train receipt still matches its pin
under Bazel, so the fixture inputs are unchanged and only the validation
rows moved. The workload identity pin excludes receipts and stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo
tracedecay-lsp and tracedecay-code-index fail on Windows with a bare
E0463 for every --extern their params file names, which is the locator
dropping each candidate without recording a reason. On failure, rerun
one lsp compile from its expanded params with the locator's info log.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo
rules_rust sets a cargo_build_script's CFLAGS and CXXFLAGS from the C
toolchain's own arguments, overwriting --action_env, so vendored C never
got Cargo's -DNDEBUG on any host and MSVC never got /utf-8: tree-sitter
grammars spelled symbol names in the ANSI code page, and every Windows
extraction test panicked on invalid UTF-8. cc-rs appends HOST_ (native)
or TARGET_ (cross) flags to CFLAGS, and rules_rust leaves those alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo
claude and others added 4 commits October 7, 2026 03:06
.cargo/config.toml hands libsqlite3-sys's build script the worker-thread
cap and disabled memory-status counters through [env]; Bazel never set
them, so the Bazel lanes tested a differently configured SQLite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo
Tests reached rustup through a HOME Bazel had redirected, so cargo,
rustc, and rust-analyzer found an empty toolchain home ("missing
manifest in toolchain"); the runner leaves CARGO_HOME and RUSTUP_HOME
unset, so export them. The SDK journeys build sdks/typescript with pnpm
from the checkout, which the Cargo-era test jobs installed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8Xg2pQQcTiyVB8ndJc5oo
… HEAD

# Conflicts:
#	.github/workflows/ci.yml
@ScriptedAlchemy
ScriptedAlchemy merged commit cda29a8 into master Oct 7, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants