Skip to content

feat: add livekit-plugins-thegrid - #7138

Open
CastilloLuis wants to merge 5 commits into
livekit:mainfrom
the-gridai:thegridai/thegrid-llm-plugin
Open

CastilloLuis wants to merge 5 commits into
livekit:mainfrom
the-gridai:thegridai/thegrid-llm-plugin

Conversation

@CastilloLuis

Copy link
Copy Markdown

What

Adds livekit-plugins-thegrid, an LLM plugin for The Grid — an OpenAI-compatible inference marketplace.

Modeled on livekit-plugins-perplexity. The plugin subclasses the OpenAI plugin's LLM and sets the base URL and THEGRID_API_KEY handling; there's no bespoke client.

Registration points, matching how perplexity/cerebras are wired:

  • livekit-plugins/livekit-plugins-thegrid/ (new package)
  • livekit-plugins-thegrid = { workspace = true } in the root pyproject.toml
  • thegrid = ["livekit-plugins-thegrid>=1.8.0"] extra in livekit-agents/pyproject.toml
  • tests/test_plugin_thegrid.py

Left CHANGELOG.md and the version manifests alone per CONTRIBUTING.

One deliberate difference from the perplexity template

Perplexity passes _strict_tool_schema=False. I tested The Grid with strict: true tool schemas (and additionalProperties: false) and it returns well-formed tool calls, so I did not carry that flag over — the default applies.

Verification

Run against the live API, not mocked:

$ pytest tests/test_plugin_thegrid.py --plugin thegrid -q
...                                                                      [100%]
3 passed in 6.34s

That covers chat, a tool call round-trip on agent-standard, and streaming. Also checked ruff format --check and ruff check clean on the new files, that the missing-key path raises, and that GET /v1/models returns 200 directly with no redirect (relevant because _prewarm_impl() calls it with a 5s timeout).

Note on models

The Grid addresses capability tiers — text-standard, code-prime, agent-max each route to a current model for that tier, so an agent keeps working when the underlying model is swapped. models.py lists the tiers plus the lab-pinned *-latest ids; model still accepts any str, and GET /v1/models is authoritative.

Disclosure: I work on The Grid. Happy to adjust anything to house style.

@CastilloLuis
CastilloLuis requested a review from a team as a code owner September 5, 2026 17:18
@CLAassistant

CLAassistant commented Sep 5, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

devin-ai-integration[bot]

This comment was marked as resolved.

@CastilloLuis
CastilloLuis force-pushed the thegridai/thegrid-llm-plugin branch 2 times, most recently from cd232dc to 730f263 Compare September 6, 2026 15:03
The Grid is an OpenAI-compatible inference marketplace that addresses
capability tiers rather than a specific lab's model name. Adds an LLM
plugin wrapping the OpenAI plugin, following livekit-plugins-perplexity.
@CastilloLuis
CastilloLuis force-pushed the thegridai/thegrid-llm-plugin branch from 730f263 to 8180fae Compare September 6, 2026 17:36
@CastilloLuis

Copy link
Copy Markdown
Author

Good catch, and your suggestion was right. Applied, with one extra step it needed.

Guarding the raise on client is None alone was not sufficient: this class still forwarded api_key="" to the OpenAI plugin, and llm.py:152 rejects an explicitly-given empty string with "OpenAI API key is required". So an injected client failed one line further down instead.

It now resolves the key into a local and passes resolved_key or NOT_GIVEN, which leaves the OpenAI plugin free to use the supplied client's own authentication.

Verified all three paths:

supplied client without key: OK, model = text-standard
no client, no key: still raises ValueError OK
env key path still works: text-standard

pytest tests/test_plugin_thegrid.py --plugin thegrid still passes 3/3 against the live API, and ruff check / ruff format --check are clean on 0.15.22.

Separately, I have added the missing uv.lock entry for the new workspace member in this branch, which is what was failing the ruff job (it died at uv sync before lint ran).

@CastilloLuis

Copy link
Copy Markdown
Author

Merged main in; the only conflict was uv.lock.

Resolved by taking upstream's lockfile wholesale and re-running uv lock, so the delta against main is just the new workspace member rather than a hand-merged lockfile:

Resolved 328 packages
Added livekit-plugins-thegrid (dynamic)

Verified after the merge:

  • uv run ruff check on the plugin and its test: all checks passed
  • uv run ruff format --check: 6 files already formatted
  • uv run pytest tests/test_plugin_thegrid.py: 3 passed, run against the live API with a real THEGRID_API_KEY

Worth flagging for whoever reviews: those three tests are live integration tests, so they need a THEGRID_API_KEY secret in CI to run there. Without one they fail at construction with THEGRID_API_KEY is required. Happy to mark them skipif on a missing key if you would rather they no-op in CI, or to supply a key if you have a way to take one.

CastilloLuis and others added 3 commits September 21, 2026 14:04
Two conflicts, both from upstream moving under us.

`livekit-agents/pyproject.toml`: upstream rewrote the optional-dependencies
block and bumped every plugin floor from 1.8.0 to 1.8.2. Took upstream's block
and re-inserted `thegrid` alphabetically at the new floor, so our entry matches
its neighbours instead of pinning a version the release train has left behind.

That bump made our own package inconsistent: the plugin still declared
`__version__ = "1.8.0"` and depended on `livekit-agents[openai]>=1.8.0`, which
would have left the `thegrid` extra unsatisfiable against its own floor. Both
are now 1.8.2, matching the sibling plugins.

`uv.lock`: resolved by taking upstream's file and re-running `uv lock` rather
than hand-merging. The previous lock diff carried unrelated churn from an older
regeneration (bithuman dependency markers); the regenerated one is 16 lines,
all of them ours.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Display names and prose only. Slugs, env vars, hostnames and code
identifiers are unchanged, so nothing about the wire format moves.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…lm-plugin

# Conflicts:
#	livekit-agents/pyproject.toml
#	uv.lock

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants