Skip to content

report tool asks the agent to vet the logs it attaches, but never shows them #73

Description

@rafaelfiguereod-stack

The gap

The report tool description tells the agent:

there is no review step, so only report real defects and check the attached logs for anything private

I want to be clear up front that I am not questioning the no-review design. You documented it deliberately and it is stated plainly in the tool description. This is about the second half of that sentence.

The agent cannot check the attached logs, because it never sees them. It passes a count:

logs: z.int().min(0).optional().describe(`trailing app log entries to attach (0 to omit; default: ${ISSUE_LOG_TAIL})`),

and the handler resolves that count against the live buffer on its own, at call time:

// apps/desktop/src/dapi/handlers/report.ts
const tail = logs ?? ISSUE_LOG_TAIL;
const lines = tail > 0 ? ctx.logs().slice(-tail).map(formatLogEntry) : [];

So by default 50 entries go into a public issue on this repo, under the user's gh identity, and the only party that saw them before they were published is the handler.

Why calling logs first does not close it

An agent could call the logs tool first and read the tail. That helps, but it does not make the instruction reliable:

  1. Nothing sequences the two calls. The description asks for a check that the tool's own schema does not require, so compliance rests entirely on the model following prose.
  2. Even a diligent agent gets a different set than it reviewed. ctx.logs() is re-read inside report, and the app keeps logging while the agent works. The tail published is the tail at filing time, not the tail at review time.

Point 2 is the part I would not have expected, so I will be explicit that it comes from reading the two call sites rather than from a demonstration. I did not drive a live filing, because that would mean opening a junk issue on your tracker.

Why it matters

Your own description is the argument that the buffer can hold something private, so I will not speculate about what. What I would add is that the buffer is wider than "the agent's own session": logs describes it as page logs, worker logs and uncaught errors, 2000 entries retained across reloads and project switches. A report filed at the end of a working session can carry entries from projects and activity unrelated to the bug.

This is not a vulnerability and I am not filing it as one. Anything that can reach the MCP port can already run gh itself, which is the threat model you set out in dapi/http.ts. It is a case where a stated safety instruction cannot be followed by its intended audience.

Suggested fix

The smallest version that makes the instruction true is to take the logs the agent reviewed instead of a count:

logs: z.array(z.string()).optional().describe("log lines to attach, as returned by the logs tool; omit to attach none"),

The agent then has to have fetched them, what it saw is what gets published, and the race disappears. It costs the agent one extra call, which it is already being asked to make.

Cheaper alternatives if that is too invasive:

  • default logs to 0, so attaching diagnostics is a deliberate act rather than the default
  • return the composed issue body in the tool result, so at least the agent and the person reading the transcript can see what was published
  • keep the count, but return the attached lines alongside the URL

Happy to send whichever you prefer as a PR.

Filed in the open because private vulnerability reporting is not enabled on the repo and there is no SECURITY.md, and because this is a design refinement rather than something an attacker gains from.

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions