With a range aggregate provisioned, an instant query lands correctly only when it falls exactly on an ingest-grid point. At every other instant the accelerated path returns an empty result set with HTTP 200 and destination=asap, while Prometheus answers the identical query normally. Nothing in the reply signals that the answer is empty because of grid alignment rather than because there is no data.
This does not reproduce on ghcr.io/projectasap/asap-query-engine:v0.5.1, so it appears to be a regression — see below.
Environment
- Engine and planner built from a fresh clone at
main @ b3259449 using the repo's own Dockerfiles.
- Backend
prom/prometheus:v3.9.1, backend.type: prometheus, forward_unsupported_queries: true.
- Data: 500 cells, metric
m1, 1,024 stamps on a 900 s grid from 2025-01-01T00:00:00Z (epoch 1735689600). 1735862400 is an on-grid instant.
- Provisioned workload includes
sum_over_time(m1[24h]).
Steps to reproduce
T=1735862400 # an on-grid instant
for off in 0 60 450 899 900; do
curl -s "$ENGINE/api/v1/query" \
--data-urlencode 'query=sum_over_time(m1[24h])' \
--data-urlencode "time=$((T+off))" \
| python3 -c "import sys,json
print(len(json.load(sys.stdin)['data']['result']))"
done
Expected
500 500 500 500 500 — Prometheus returns 500 series at all five offsets.
Actual
500 0 0 0 500
Extent
Sweeping all 901 one-second offsets across one 900 s grid step:
|
engine |
Prometheus |
| offsets returning 500 series |
2 — exactly +0 s and +900 s |
901 — all of them |
| offsets returning 0 series |
899 — contiguous, +1 s through +899 s |
0 |
So the served answer is empty at 899 of every 900 instants. The same happens for sum_over_time(m1[1h]) and quantile_over_time(0.99, m1[24h]), so it is not specific to one function.
Why this looks like a regression
Comparing like with like — both builds on quantile_over_time(0.99, m1[24h]), a shape both genuinely serve (destination=asap), same data, same Prometheus:
| build |
+0 s |
+60 s |
+450 s |
+899 s |
+900 s |
v0.5.1 |
500 |
500 |
500 |
500 |
500 |
main @ b3259449 |
500 |
0 |
0 |
0 |
500 |
Both are accelerating, so the difference is not "v0.5.1 just forwarded everything" — I checked destination= on each request. That suggests a bisect between the two rather than a design change.
Related
None of those record that the accelerated path returns an empty 200 rather than forwarding, which is the part that makes this silent.
With a range aggregate provisioned, an instant query lands correctly only when it falls exactly on an ingest-grid point. At every other instant the accelerated path returns an empty result set with
HTTP 200anddestination=asap, while Prometheus answers the identical query normally. Nothing in the reply signals that the answer is empty because of grid alignment rather than because there is no data.This does not reproduce on
ghcr.io/projectasap/asap-query-engine:v0.5.1, so it appears to be a regression — see below.Environment
main@b3259449using the repo's own Dockerfiles.prom/prometheus:v3.9.1,backend.type: prometheus,forward_unsupported_queries: true.m1, 1,024 stamps on a 900 s grid from2025-01-01T00:00:00Z(epoch1735689600).1735862400is an on-grid instant.sum_over_time(m1[24h]).Steps to reproduce
Expected
500 500 500 500 500— Prometheus returns 500 series at all five offsets.Actual
500 0 0 0 500Extent
Sweeping all 901 one-second offsets across one 900 s grid step:
So the served answer is empty at 899 of every 900 instants. The same happens for
sum_over_time(m1[1h])andquantile_over_time(0.99, m1[24h]), so it is not specific to one function.Why this looks like a regression
Comparing like with like — both builds on
quantile_over_time(0.99, m1[24h]), a shape both genuinely serve (destination=asap), same data, same Prometheus:v0.5.1main@b3259449Both are accelerating, so the difference is not "v0.5.1 just forwarded everything" — I checked
destination=on each request. That suggests a bisect between the two rather than a design change.Related
[start, end)with Prometheus range selector(start, end]boundaries #434 covers the[start, end)versus(start, end]boundary convention.None of those record that the accelerated path returns an empty 200 rather than forwarding, which is the part that makes this silent.