Skip to content

test(runner): cover CFC capability isolation across invocations - #7225

Open
yang0228 wants to merge 1 commit into
google:mainfrom
yang0228:codex/7224-cfc-invocation-isolation
Open

yang0228 wants to merge 1 commit into
google:mainfrom
yang0228:codex/7224-cfc-invocation-isolation

Conversation

@yang0228

@yang0228 yang0228 commented Sep 21, 2026

Copy link
Copy Markdown

Link to Issue or Description of Change

Problem:

The capability bleed reported in #7224 was fixed by f8282da. Its processor tests construct invocation contexts directly, so they do not exercise the Runner context-creation path that originally mutated the shared agent.

Solution:

Add a parametrized Runner regression test that enables CFC and then disables it while reusing one agent and runner. Cover both independent sessions and reuse of the same session, with no configured executor, a local executor, and an explicitly configured built-in executor. Assert the resulting model-request capability and preservation of the configured executor's identity.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally. See the baseline failures below.

Focused command:

pytest tests/unittests/test_runners.py tests/unittests/flows/llm_flows/extensions/test_code_execution.py -q

Result: 150 passed on Python 3.12, 3.13, and 3.14. All six new cases also passed in each full tox environment.

Regression check: transplant the new test into the unmodified pre-fix source at 43b73e4f (f8282daf^), then run pytest tests/unittests/test_runners.py -k cfc_code_execution -q. Result: 4 failed, 2 passed. Both the no-executor and local-executor cases fail for same-session and cross-session reuse:

E assert [True, True] == [True, False]
E At index 1 diff: True != False

The explicitly configured built-in executor cases pass on both revisions.

Full supported-version suite:

Generated a local, ignored uv.lock because the checkout does not include one and the configured tox runner requires uv sync --locked. Ran the complete suite using:

tox -p 2 -x 'testenv.commands=pytest tests/unittests -n 4'
Python Passed Failed Skipped Xfailed Xpassed
3.10 15,235 5 87 27 2
3.11 15,244 5 86 27 2
3.12 15,234 6 87 27 2
3.13 15,235 5 87 27 2
3.14 15,235 5 87 27 2

The failures are the agent import allowlist check and four test_load_web_page.py cases: public NAT64, private hostname blocking, resolved-IP pinning, and retrying another resolved address. An unmodified d1c64587 source snapshot reproduced the same five failures under Python 3.10: 15,229 passed, 5 failed, exactly six fewer passing cases than this branch. Python 3.12 additionally reports sitecustomize in the runner import allowlist check; both Python 3.12 import failures also reproduce against the unmodified snapshot.

Manual End-to-End (E2E) Tests:

Built the wheel with uv build, installed it into a clean Python 3.12 virtual environment, and ran the local Runner-to-request-processor reproduction from the issue. It uses real ADK objects and requires no model service or credentials:

import asyncio
from google.adk.agents import Agent
from google.adk.agents.run_config import RunConfig
from google.adk.flows.llm_flows.extensions._code_execution import request_processor
from google.adk.models.llm_request import LlmRequest
from google.adk.runners import InMemoryRunner

async def main():
    agent = Agent(name="shared_agent", model="gemini-2.0-flash")
    async with InMemoryRunner(agent=agent, app_name="cfc_regression") as runner:
        capabilities = []
        for session_id, enabled in [("session_a", True), ("session_b", False)]:
            session = await runner.session_service.create_session(
                app_name=runner.app_name, user_id="test_user", session_id=session_id
            )
            context = runner._new_invocation_context(
                session, run_config=RunConfig(support_cfc=enabled)
            )
            request = LlmRequest(model=agent.model)
            async for _ in request_processor.run_async(context, request):
                pass
            capability = any(t.code_execution is not None for t in request.config.tools or [])
            capabilities.append(capability)
            print(f'{session_id}: support_cfc={enabled}, code_execution={capability}, agent.code_executor={agent.code_executor}')
            assert agent.code_executor is None
        assert capabilities == [True, False]
        print('PASS: the shared agent remains unchanged and the second session has no code-execution capability.')

asyncio.run(main())

Observed output from the clean wheel environment:

session_a: support_cfc=True, code_execution=True, agent.code_executor=None
session_b: support_cfc=False, code_execution=False, agent.code_executor=None
PASS: the shared agent remains unchanged and the second session has no code-execution capability.

This validates the local reproduction; it does not exercise a live Gemini connection.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes. The focused suites pass; full-suite baseline failures are documented above.
  • I have manually tested my changes end-to-end through the local reproduction described above.
  • Any dependent changes have been merged and published in downstream modules. No downstream changes are required for this regression test.

Additional context

pre-commit run --files tests/unittests/test_runners.py and the installed commit hooks passed. Hook sources were retrieved from GitHub's official codeload endpoint at the configured tags because this environment could not connect to GitHub's HTTPS git endpoint. The repository hook configuration was used unchanged.

@google-cla

google-cla Bot commented Sep 21, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Exercise Runner context creation when CFC is enabled and then disabled with the same agent, both within one session and across sessions. Preserve configured local and built-in executors while checking the request capability.

Related: google#7224
@yang0228
yang0228 force-pushed the codex/7224-cfc-invocation-isolation branch from e866321 to 9b4647d Compare September 21, 2026 08:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants