Conversation
This was referenced Sep 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dependency stack: main → #768 → #737 → #749 → #771 → #728 → #770 → #763 → #765 → #761 → #742 → #759
Independent follow-ups to #765: #756 (diagnostics), #766 (runtime controls and overhead inspection).
Before this PR
Shared execution and mathematical kernels lived in the backend, while Planner's IR could change independently. Candidate pruning and grouped ranking still had dedicated backend representations.
After this PR
Consume
asap-physical-operatorsandasap_sketch_codecfrom Planner #462, pinned with Planner IR to revision645cacb04a451e25147f0d23aaddd05da16b4746. Remove the backend copies. The library depends on Planner types and has no backend dependency.The API follows Planner #461: phase belongs to node data state, candidate filtering uses a general semi-join, and grouped ranking composes Sort → grouped Limit. Old operator dispatch and wire compatibility paths are removed. Exact accumulator state is explicitly finalized before value consumers. Temporal exact TopK candidates also lower to grouped Sort → Limit, including integer Count scores.
The independent runtime owns shared producers, bounded delivery, cancellation and retained-output accounting. Native computation includes expressions, relations, window reductions and supported summary operations. The same implementation can run at ingestion time or query time. Custom summary values need not fit Arrow RecordBatch.
Backend #763 integrates ingestion and durable publication; #765 integrates query execution and completes relation/temporal/stored-state computation migration. Sources, storage and protocol conversion remain deployment responsibilities. Local raw Scan is deferred; blocking operators have no spill support.
Architecture and DataFusion comparison. Independent library tests run in Planner; backend tests exercise deployment bindings.
Validation
Planner: 246 shared-library unit tests, 18 library integration tests, one doc test, 432 mapping tests and the integration package passed. Both repositories pass strict workspace/all-target Clippy.
Backend stack: 118 type and 444 control-plane library tests passed. The data-plane run passed 925 tests; its remaining persistence synchronization test was fixed and passed separately. All 19 compatibility process tests and 59 other integration tests passed. Tests cover shared SQL sources, cancellation, stored-state coverage, temporal Sort/Limit, HLL/KLL confidence, and ingestion publication/recovery.
The mandatory #754 gate still fails on grouped-temporal-Sum producer shape and quantile-ratio local execution. Those assertions remain intact. Full #759 performance acceptance is not established. Local raw Scan remains explicitly deferred.
The publication process test verifies that a successor is cold before its own data arrives, rejects old-generation frames and returns the successor's new values; it does not assume cross-version payload reuse.