Skip to content

fix(ci): repair the weekly model-limits drift check and refresh model data - #136

Merged
justin-carper merged 1 commit into
mainfrom
chore/repair-model-drift
Sep 30, 2026
Merged

justin-carper merged 1 commit into
mainfrom
chore/repair-model-drift

Conversation

@justin-carper

@justin-carper justin-carper commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The scheduled model limits · drift vs Cursor docs job has failed on every weekly run (latest: 2026-09-28). This fixes the job and brings the model data it guards back in line with Cursor's docs.

Root cause

scripts/sync-model-limits.mjs fetchDoc sent accept: text/plain,text/markdown,*/*. cursor.com returns 404 for that header and 200 with default fetch headers, so --check exited 2 (HTTP 404) before reading any data. Dropping the header is the one-line fix; the rest of this PR is what the check then surfaced.

Changes

  • Generator
    • The pricing doc now has two tables with identical columns (Cursor Models pool, then Other Models). The parser reads every matching table. A model with a row in both is still reported as ambiguous, not first-wins.
    • MODEL_IDS now matches the live 40-id catalog. Added: claude-fable-5-1, claude-opus-5-5, claude-sonnet-5-5, gemini-3.7-flash, gemini-3.8-flash, grok-4.6, grok-4.7, muse-spark-1.3. Dropped: glm-5.2 (absent from the live catalog).
    • The grok-4.5 and composer-2.5 $0 "pool" overrides are removed: the docs now list price rows for both. auto-smart, default and composer-2 keep $0 as placeholder rates, with corrected why strings.
    • The NO_AUTO_COMPACTION_INPUT_LIMIT block was hand-inserted into the generated src/model-limits.ts in 27d11bb without updating the template, so a regen would have deleted it. It now lives in the template.
  • src/model-limits.ts: regenerated with npm run sync:model-limits, not hand-edited.
  • Cost changes for existing models (from the docs): claude-sonnet-5 $3/$15 → $2/$10, gpt-5.6-sol $5/$30 → $4/$20, grok-4.5 $0 → $2/$6, composer-2.5 $0 → $0.5/$2.5. No context window changed.
  • vitest.config.ts (new): excludes .cursor/**. The plugin mirrors skills into <cwd>/.cursor/skills/, and default discovery then collects their test files (94 load failures when running npm test in a dogfooded checkout).
  • Tests: added a two-table generate test that fails if the parser goes back to first-table-only; removed two tests that only echoed generated cost numbers and assert the invariant instead.

Verification

  • npm run sync:model-limits -- --check: exit 0, prints up to date (context: 36 from docs, 4 overridden | cost: 36 from docs, 4 overridden | 40 model ids), which is the line ci.yml greps for
  • npm run typecheck, npm run build: pass
  • npm test with no --exclude flags: 42 files / 642 tests pass
  • Six generated values spot-checked against both live docs pages by a reviewer
  • Drift job dispatched on this branch via workflow_dispatch (run 36711158492, re-run on the rebased head 6565d6d): model limits · drift vs Cursor docs success (it had failed on every scheduled run before this)

Independent of #132 (different files apart from CHANGELOG.md [Unreleased]; whichever merges second needs a trivial rebase).

… data

The scheduled `model limits · drift vs Cursor docs` job failed on every run.
cursor.com answers 404 to the script's `accept: text/plain,text/markdown,*/*`
header and 200 to default fetch headers, so the check exited 2 before
reading any data. Drop the custom header.

With the fetch working, the generator also needed updating:
- the pricing doc now has two tables with identical columns (Cursor Models
  pool, then Other Models); read every matching table, and still report a
  row present in both as ambiguous
- MODEL_IDS now matches the live 40-id catalog (adds claude-fable-5-1,
  claude-opus-5-5, claude-sonnet-5-5, gemini-3.7-flash, gemini-3.8-flash,
  grok-4.6, grok-4.7, muse-spark-1.3; drops glm-5.2)
- grok-4.5 and composer-2.5 now have docs price rows, so their $0 pool
  overrides are removed; auto-smart/default/composer-2 keep $0 as
  placeholder rates with corrected reasons
- the NO_AUTO_COMPACTION_INPUT_LIMIT block, hand-inserted into the generated
  file in 27d11bb, moves into the generator template so a regen keeps it

Also add a root vitest.config.ts excluding `.cursor/**`: the plugin mirrors
skills into <cwd>/.cursor/skills/, and default discovery then collects their
test files (94 load failures when running `npm test` in a dogfooded checkout).

Tests: add a two-table generate test; drop two tests that only echoed
generated cost numbers and assert the invariant instead.
@justin-carper
justin-carper force-pushed the chore/repair-model-drift branch from b466a22 to 6565d6d Compare September 30, 2026 11:51
@justin-carper
justin-carper merged commit d5109e6 into main Sep 30, 2026
11 checks passed
@justin-carper
justin-carper deleted the chore/repair-model-drift branch September 30, 2026 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant