Conversation
zzylol
changed the base branch from
main
to
refactor/query-plan-dag-execution
September 22, 2026 20:20
This was referenced Sep 22, 2026
This was referenced Sep 22, 2026
zzylol
changed the base branch from
refactor/query-plan-dag-execution
to
perf/issue-758
September 22, 2026 20:45
This was referenced Sep 23, 2026
This was referenced Sep 23, 2026
zzylol
force-pushed
the
perf/issue-758
branch
from
September 24, 2026 12:55
4530670 to
999cafd
Compare
zzylol
changed the base branch from
perf/issue-758
to
refactor/query-plan-dag-execution
September 24, 2026 12:56
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dependency stack: main → #768 → #737 → #749 → #771 → #728 → #770 → #763 → #765 → #761 → #742 → #759
Independent follow-ups to #765: #756 (diagnostics), #766 (runtime controls and overhead inspection).
Stacked directly on #765. Scoped evidence and workload costing depend on installed query execution, not diagnostic logging, the differential runner or overhead inspection. Closes #752.
Before this PR
The backend can select costed Planner candidates, but query/data/snapshot-scoped external evidence and ERP resource measurements are not integrated into complete backend-owned workload costing. Callers must provide complete physical-candidate quotes.
After this PR
Scoped evidence supplies quantile-operand domains, Hydra bounds, TopK intervals and distribution/cardinality facts, with scope/freshness validation. Applicable ERP benchmark resources take priority over explicitly identified analytical estimates; resource measurements do not certify accuracy. Both startup and compile/publish combine unit resources with query recurrence, data scale, shared state, physical layouts and transport, calculate complete workload costs and compare feasible physical candidates. Provider quotes remain an explicit optional override.
Source cadence, query lookback, ranges and offsets retain millisecond precision. A real 100 ms workload now reaches compilation/cost comparison without rounding the source cadence. Existing temporal storage-family capability checks still apply; this does not claim every pane implementation supports fractional windows.
Design:
docs/design_docs/evidence-dependent-candidates.md. The Planner dependency inherited from #768 includes ProjectASAP/ASAPPlanner#460. Scoped HLL evidence can opt into its classic 64-bit linear-counting confidence bound for complete readout populations of at most 4,096 distinct values, under the independent uniform bucket-hash assumption. The contract covers merged panes and is restricted to the verified local Remote Write estimator. Missing, infeasible or unsupported contracts retain exact execution; ERP resource costs do not substitute for probability guarantees.Verification
cargo clippy --workspace --all-targets --locked -- -D warnings,cargo fmt --all -- --check, and remaining exact-fallback/offline-evidence integrations pass.Acceptance boundary
#759 remains stacked directly on this PR and is the full level-3 acceptance layer.
make benefitnow passes workload compilation/cost validation, then fails the strict local-execution gate becausecompat-query-9requires external exact execution. The inherited level-1 test still reports grouped temporal Sum and quantile-ratio plan-shape failures. No full measured-benefit success is claimed, and these gates have not been relaxed.Final stacked process validation: all 19
asapquery_compatibility_process_e2etests pass, including confidence-sized KLL state transport, bounded classic-HLL execution, and exact routing for uncertified empirical evidence. The KLL regression also passes independently on #749.Final #759 stack checks: all 12 Rust runner tests pass, as do
cargo clippy --workspace --all-targets --locked -- -D warnings,cargo fmt --all -- --checkandgit diff --check.Shared physical operator stack update
The shared runtime, operators and sketch codec are owned by ASAPPlanner #462, built on Planner #461. Backend #770 consumes them, #763 integrates ingestion and #765 integrates queries. The stack uses general semi-join and grouped Sort → Limit, with execution phase on the node. Independent library tests run in Planner; deployment acceptance stays in this backend stack.
Automatic workload costing prices general semi-join/anti-join output cardinality and the supported equality semi-join implementation; no dedicated membership operator remains.