Skip to content

Execute installed QueryPlan sub-DAGs in the query engine #764

Description

@zzylol

Problem

The split physical-plan contract makes PrecomputePlan responsible for maintenance writes and QueryPlan responsible for stored-summary reads and query-time computation. Serving must execute the installed QueryPlan DAG itself, including every query-time operator exported by ASAPPlanner's post-ASAP IR, rather than reconstructing behavior from request text or helper-specific control flow.

Required behavior

For each request, the query engine must:

  1. Resolve the installed QueryPlanEntry and start from its root.
  2. Traverse only the reachable sub-DAG; reject missing inputs, cycles and inconsistent schemas.
  3. Evaluate dependencies before consumers and memoize shared nodes once per evaluation context.
  4. Read stored summaries only through the installed StoredOutputReference, readiness, format, grouping, window and coverage contract.
  5. Execute readouts, exact leaves, residuals, candidate reranking, scalar/vector operations and relational operations from installed nodes.
  6. Support every standardized Planner relational join kind: inner, left, right, full, cross, semi and anti.
  7. Match ExecutableOperatorPayload exhaustively to its Planner-validated maintenance/query phase. New upstream variants must require an explicit backend ownership decision.
  8. Keep selected_dags as provenance; do not execute them as a second serving program.
  9. Report query ID and node ID on execution failures.

Maintenance-only payloads remain owned by PrecomputePlan/#763. Query serving must reject phase crossing rather than running maintenance work on demand.

Documentation and synchronization

Document request lookup, traversal, memoization, all Planner payload variants and phase ownership, StoredSummary reads, exact/fallback boundaries, concurrency and resource limits. Include a shared-KLL example.

Add CI-gated automation that checks ASAPPlanner main, updates all pinned Planner dependencies together, validates the workspace, and opens or updates a synchronization PR.

Acceptance

  • PromQL/MetricsQL and ClickHouse paths execute installed QueryPlan roots.
  • Shared nodes run once per request/evaluation timestamp.
  • All current post-ASAP physical payload variants have an exhaustive phase owner.
  • Every Planner join kind has tested runtime semantics.
  • Stored-summary reads cannot bypass their installed binding.
  • Invalid graphs and unsupported extensions fail explicitly.
  • Planner synchronization is reviewable and cannot write unverified changes directly to backend main.
  • The design document describes the implementation and current limits.

Implement in a dedicated PR stacked after #749. Stack #761 over this PR because Planner #455's evidence semantics and revision update require its backend admission/cost changes.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions