Skip to content

quantile_over_time with a label selector returns all series, ignoring the filter #732

Description

@SheriffOreo

quantile_over_time carrying a label selector is served from the sketch with the selector dropped, so the client receives every series in the metric instead of the one it asked for. The reply is a well-formed HTTP 200, so nothing distinguishes it from a correct answer except counting the series.

Reproduced on main @ b3259449 and on the published ghcr.io/projectasap/asap-query-engine:v0.5.1 image.

Environment

  • Engine and planner built from a fresh clone at b3259449 using the repo's own asap-query-engine/Dockerfile and asap-planner-rs/Dockerfile.
  • Backend prom/prometheus:v3.9.1, backend.type: prometheus, forward_unsupported_queries: true.
  • Data: 500 cells E00001E00500, metric m1, 1,024 stamps on a 900 s grid from 2025-01-01T00:00:00Z (epoch 1735689600). 1735862400 is an on-grid instant.
  • Provisioned workload includes quantile_over_time(0.99, m1[24h]).

Steps to reproduce

curl -s "$ENGINE/api/v1/query" \
  --data-urlencode 'query=quantile_over_time(0.99, m1{cell="E00001"}[24h])' \
  --data-urlencode 'time=1735862400' \
| python3 -c "import sys,json
print('series:', len(json.load(sys.stdin)['data']['result']))"

Expected

1 series (cell="E00001"), which is what Prometheus returns for the identical query.

Actual

500 series, HTTP 200, destination=asap.

The E00001 value inside the reply is +2.77 % off Prometheus on the source build and −10.41 % on v0.5.1. That part is ordinary KLL approximation and is not the issue here; the issue is the shape of the answer.

Why this looks like a defect rather than intended behaviour

With the same selector, every sibling range function forwards and returns exactly 1 series:

query (all with {cell="E00001"}) destination series returned
quantile_over_time(0.99, …[24h]) asap 500
sum_over_time(…[24h]) prometheus 1
max_over_time(…[24h]) prometheus 1
min_over_time(…[24h]) prometheus 1
stddev_over_time(…[24h]) prometheus 1
avg_over_time(…[24h]) prometheus 1

So quantile_over_time is inconsistent with its five siblings and with the README's "Supported Queries" rule that temporal range functions are accelerated only with no label selectors. The filtered request also costs about the same as the unfiltered 500-cell form, which suggests the selector is being ignored rather than applied and over-returned.

A control without a selector behaves correctly (500 series from both engine and Prometheus), so the fault is specific to the selector path.

Engine log for that request

promql.rs (capability-matching branch)
  WARN  No query_config entry for PromQL query
        'quantile_over_time(0.99, m1{cell="E00001"}[24h])'.
        Attempting capability-based matching.
http.rs (destination log line)
  INFO  query='...' destination=asap asap_latency_ms=57.17

Related

Found while independently verifying ASAPQuery behaviour on a clean install; happy to supply the full harness if useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions