Skip to content

fix: put the confirmation token in the preview text - #249

Merged
bardesss merged 3 commits into
bardesss:mainfrom
itapai:fix/confirm-token-in-text
Sep 14, 2026
Merged

bardesss merged 3 commits into
bardesss:mainfrom
itapai:fix/confirm-token-in-text

Conversation

@itapai

@itapai itapai commented Sep 14, 2026

Copy link
Copy Markdown

Problem

A write preview's text says:

To apply this, call request_media again with the same arguments plus confirm set to the token in confirm_token.

but the token itself is only in structuredContent. A client that forwards just the content block to the model never shows it the token, so the handshake cannot complete. Two such clients today:

  • Claude's remote MCP path, which List tools: item payload never reaches clients that ignore structuredContent #234 already documented as not passing structuredContent to the model. That fix covered the list tools; write previews have the same split.
  • Hermes Agent from v2026.9.7 on (commit cee12cc3, a port of kimi-code#3234) drops structuredContent whenever the text block rendered anything usable. Releases up to v2026.8.31 combined both blocks, which is why the same setup works on an August Hermes and fails on a September one.

Observed on 1.26.3 against a live Seerr 3.4.1 from Hermes 2026.9.11: nine request_media previews for the same film, all settled unconfirmed, one confirmation malformed where the model guessed a token, and no request ever reached Seerr. Calling the tool directly over HTTP confirmed the token is absent from content[0].text and present in structuredContent.confirm_token.

Fix

Both preview branches in src/tools/write.ts now state the token in the text, next to the unchanged confirm_token field:

To apply this, call request_media again with the same arguments plus confirm set to v1.….

and on a rejected token:

… A fresh token for this exact operation: v1.….

No response field changes, so this is a fix:. docs/writes.md example updated to match.

Tests

test/writeTool.test.ts: two new cases asserting the text contains structuredContent.confirm_token, for the plain preview and for the rejected-token preview.

The three gates

> arr-mcp@1.26.4 lint
> eslint .

> arr-mcp@1.26.4 typecheck
> tsc --noEmit

      Tests  2678 passed (2678)
   Duration  16.97s

What was exercised

  • On the unpatched 1.26.3, the bug was reproduced against a live Seerr 3.4.1 and a live Hermes 2026.9.11, and the preview response was inspected directly over the MCP HTTP endpoint.
  • The patched build was verified by the harness tests only. It has not been run against a live stack: doing so would have meant copying the instance's API keys to a workstation, which I chose not to do. The change is confined to the two template strings and the token variable they read from, and the tests pin that the text token equals confirm_token.

🤖 Generated with Claude Code

A write preview said "call again with `confirm` set to the token in
`confirm_token`" but only carried the token in structuredContent. A client
that forwards just the text block to the model — Claude's remote MCP path
(bardesss#234), Hermes since 2026.9.7 which drops structuredContent whenever the
text is non-empty — shows the model an instruction naming a field it
cannot see. The model either stops (audit: unconfirmed) or invents a
token (audit: confirmation malformed), and the write never lands.

Both preview branches now state the token itself in the text, alongside
the unchanged confirm_token field. Same shape as the bardesss#234 fix for the
list tools.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@bardesss

bardesss commented Sep 14, 2026 •

Copy link
Copy Markdown
Owner

Thanks, good catch and the diagnosis is right. Same split #235 fixed for the list tools, and the write previews had it too.

Ran the three gates on the branch here, all clean, 2678 passing. Checked the rest of the tree while I was in there: every write tool goes through registerWriteTool, so those two branches really were the only sites still pointing at a structured field, and there's no stale wording left in tools.md, security.md or the README.

One thing I'd like folded in before this goes.

The token is prose now, wrapped in backticks with a period right after it, but verifyAndConsume still does a bare split('.') with no trimming. I measured what your branch does when the extraction isn't byte exact:

exact              applied
with backticks     malformed
trailing period    malformed
leading space      malformed
quoted             malformed
trailing newline   mismatch

Only the exact string applies. And the malformed response hands the fresh token back in the identical shape, backticks and a trailing period, so a client that consistently clips one character past the token never converges. That's the nine previews all settling unconfirmed from your report, surviving the fix. A trim and strip at the top of verifyAndConsume (whitespace, surrounding backticks or quotes, one trailing period) closes it, and the HMAC still rejects anything actually wrong, so it costs nothing.

Smaller, same area: the rejection text interpolates check.remedy, which says "call this tool again without confirm", and then hands over a usable token in the next sentence. Two different next actions in one paragraph. Before your change the trailing sentence pointed at confirm_token, which a text-only client couldn't see, so the remedy was the only reachable instruction and the conflict was latent. Making the token visible surfaces it. Dropping check.remedy from the text on the branches that do reissue would leave one unambiguous instruction, and it's still in confirm_error either way.

Two nits, take or leave:

  • The comment pins "Hermes since 2026.9.7" plus a commit hash. That'll go stale quietly, and writes.md now says the same thing. I'd cut the comment down to a sentence and let the doc carry it.
  • The docs example uses a real token, mu13yno6 decodes to 10:36 this morning. Harmless since the signing key is per process and it died five minutes later, but a made up one reads the same.

Happy to take the extraction tolerance as a follow-up if you'd rather keep this diff tight, but I'd want it in the same release.


Update, a bit later the same day. Ignore the ask above about the trim, I did it myself in #250, now merged. Don't spend time on it.

Splitting it out rather than leaving it with you: it lives in confirm.ts, which your diff never opens, and sending you off to fix a file outside your own change was a daft thing for me to ask. It strips whitespace, wrapping backticks or quotes, and trailing punctuation before the token is parsed. The HMAC still decides everything it decided before, and the stripped form is what the single-use set records, so punctuation is not a replay route.

This branch is BEHIND now. No conflicts though, your three files and my three don't overlap at all, so an update from main is all it wants.

I merged the two locally first to check they actually compose. Lint and typecheck clean, 2681 passing. Then I took the preview text your change produces and clipped the token out of it five ways:

last backticked span   applied
with its backticks     applied
backticks + period     applied
last whitespace word   applied
after "set to "        applied

apply() fired once in each case, so that is the handshake completing from the text alone, which is what your PR set out to do. Good catch, and the writeup made it easy to follow.

Still on your side, all optional:

  • the check.remedy point above, "call this tool again without confirm" sitting immediately before a usable token
  • the two nits, the pinned Hermes version in the code comment and the real token in the docs example

None of it blocks. Update from main and I'll take another look.

@bardesss

Copy link
Copy Markdown
Owner

Edited my review above with an update: the trim landed in #250, so skip that part. Branch just needs an update from main.

@bardesss
bardesss merged commit 316e1b8 into bardesss:main Sep 14, 2026
2 checks passed
bardesss pushed a commit that referenced this pull request Nov 6, 2026
A write preview said "call again with `confirm` set to the token in
`confirm_token`" but only carried the token in structuredContent. A client
that forwards just the text block to the model — Claude's remote MCP path
(#234), Hermes since 2026.9.7 which drops structuredContent whenever the
text is non-empty — shows the model an instruction naming a field it
cannot see. The model either stops (audit: unconfirmed) or invents a
token (audit: confirmation malformed), and the write never lands.

Both preview branches now state the token itself in the text, alongside
the unchanged confirm_token field. Same shape as the #234 fix for the
list tools.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants