Repository navigation
feat(cache): restore task results from the remote cache - #756
Merged
Merged
Conversation
wan9chi
force-pushed
the
remote-cache-restore
branch
from
September 24, 2026 19:03
bc8a890 to
f607879
Compare
wan9chi
added this pull request to stack #751
September 24, 2026 19:03
fspy benchmarklinuxmacoswindows |
wan9chi
force-pushed
the
remote-cache-restore
branch
3 times, most recently
from
September 25, 2026 02:28
1935fe9 to
5ff9dff
Compare
wan9chi
force-pushed
the
remote-cache-restore
branch
from
September 25, 2026 02:29
5ff9dff to
75d1019
Compare
wan9chi
removed this pull request from stack #751
September 25, 2026 02:34
wan9chi
added this pull request to stack #759
September 25, 2026 02:35
wan9chi
force-pushed
the
remote-cache-restore
branch
5 times, most recently
from
September 27, 2026 12:24
e440c57 to
ead9cc6
Compare
wan9chi
force-pushed
the
remote-cache-restore
branch
from
September 27, 2026 12:40
ead9cc6 to
1ff7a83
Compare
wan9chi
force-pushed
the
remote-cache-restore
branch
from
September 27, 2026 12:45
1ff7a83 to
ee2b6c4
Compare
wan9chi
added a commit
that referenced
this pull request
Sep 27, 2026
## Motivation The run summary saved each kind of error in its own way. Task errors used a six-variant enum whose message prefixes were rebuilt by hand when rendering. Upload failures picked their details separately for each client error variant. Task communication errors kept only their top message. #756 adds another set of these rules for remote cache read failures. `SavedError` replaces all of them. It keeps an error's message and the messages of its sources, so any error can be saved without its own conversion. `UploadError` now derives `thiserror`, which exposes its sources. The compact summary still shows only the message. `--verbose` and `--last-details` now show each cause on its own line below it: ``` ⚠ Not uploaded to the remote cache: network error ↳ error sending request for url (http://127.0.0.1:0/projects/test/store) ↳ client error (Connect) ↳ tcp connect error ↳ <os error> ``` The `✗` line printed after a failed task still shows the whole chain on one line. The e2e redaction now also hides OS error messages at the start of a cause line. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
wan9chi
force-pushed
the
remote-cache-restore
branch
from
September 27, 2026 13:30
4ddb9ad to
348fc6a
Compare
After a local miss in `read` or `read-write` mode, fetch the entry from the remote cache. An exact entry that passes validation is restored: its output archive is downloaded into the cache directory, checked by decoding it, and the entry is recorded locally before it replays as a hit. Hits never upload. A fallback entry, a failed validation, or a failed read is a miss. When the local cache has no entry for the task, the miss reason comes from the remote cache. Read failures produce no warning. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
An error while validating a remote entry now makes the lookup a miss with the reason "remote cache entry couldn't be validated", instead of failing the task. Local validation is unchanged. Exact fetch responses must include `blob_id`, using null for no blob, so a response without it is a malformed response rather than a hit that restores no outputs. The decision from a fetch result to a hit or miss moves into `remote::resolve`, with unit tests for each outcome. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The client returns the blob's chunks instead of writing them to a file. Each chunk is written to the archive file and passed to the archive check, which reads the chunks on a blocking thread as they arrive, so the archive isn't read back from disk. A failed check stops the download. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Follow #761: a read failure's miss carries the error, and the run summary saves it as a `SavedError`. The miss reason is the error's message, such as `remote cache fetch failed`. `--verbose` and `--last-details` show each cause, such as the network error, on its own line below it. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Fetching and uploading each name every access mode, so adding one fails to compile until both decide what it does. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The archive is downloaded to a `.tmp` file and renamed once the check passes. The chunks flow one way: from the network to the check on a blocking thread, which writes each one to the file as it takes it. The file is closed before it's renamed or removed, even after a network error. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
wan9chi
force-pushed
the
remote-cache-restore
branch
from
September 27, 2026 15:46
0a55996 to
aefe44a
Compare
This was referenced Sep 27, 2026
wan9chi
marked this pull request as ready for review
September 27, 2026 16:31
wan9chi
added a commit
that referenced
this pull request
Sep 27, 2026
## Motivation A review of #755 and #756 found a few small problems in remote cache requests and restores. ## Changes - Redirects aren't followed. A store redirected to a page that answers 200, such as a login page, now fails with the redirect's status instead of counting as an upload. - The system proxy settings are used on macOS and Windows, as with reqwest's default features. Proxy environment variables already worked. - Network errors leave out the request URL, so credentials in the endpoint, such as `?token=…`, aren't saved in the run summary or shown by `--last-details`. - An exact entry whose value has an output archive but no blob ID, or a blob ID but no archive, is a miss with the reason `remote cache entry's blob doesn't match its value`. Before, an entry without a blob ID was recorded and replayed without its outputs. - A download stops as soon as the archive check fails, instead of waiting for the server's next chunk. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
#755 uploads task results to the remote cache, but nothing reads them back. Restoring them lets a task that ran on another machine, or before
vp cache clean, replay instead of running again.Changes
vt_remote_cacheaddsfetch(POST {endpoint}/fetch), which returns an exact match, a fallback match with its key, or no match, anddownload(GET {endpoint}/blob/{blob_id}), which streams the blob's chunks.readorread-writemode,vp runfetches the entry. An exact entry is validated like a local one. Its archive is downloaded to a.tmpfile while being checked from the same stream, then renamed, recorded locally, and replayed as a hit. Hits never upload.remote cache fetch failedordownloaded archive is corrupt. Read failures aren't warnings, and--verboseand--last-detailsshow their causes like other errors (refactor(reporter): save errors as a chain of causes #761). After a failed read, the.tmpfile is removed and nothing is recorded.