Repository navigation
claude-fable-5 EXECUTE-boundary run: sampling-param fix + readout - #24
Merged
Merged
Conversation
claude-fable-5 rejects any explicit temperature parameter (API 400 invalid_request_error: 'temperature is deprecated for this model'), same restriction as Opus 4.7. Add it to the existing shouldOmitAnthropicSamplingParams map so runners send no sampling params for this model. Without this, run-execute-boundary.js fails 25/25 fixtures at the API layer. Note: runs against fable-5 are therefore model-managed sampling, not temperature-0-pinned, and the runner's recorded 'temperature: 0' field reflects the script variable, not the wire (pre-existing quirk shared with the Opus 4.7 path). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pre-registered regression run of the AAR v0.8 EXECUTE-boundary case set against claude-fable-5, both available runner conditions (frozen ZIP skill prompt and script-default eval/SKILL.md). Scorer unmodified. Headlines: skill condition primitiveMatch 20/22 (90.9%) with all zero-target gates clean; key behavioral finding is EB-24B, where fable-5 executed through the planted body/attachment mismatch (both Type 9 pair halves ruled EXECUTE = design INVALID-as-design configuration; sonnet-4-6 had differentiated the pair). All raw/scored SHA-256 anchors, verbatim failure texts, deviations log, and claim boundary are in the memo. Result files themselves stay local per .gitignore policy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
npm install regenerates this 2-line metadata sync for anyone; the lockfile was left at 0.6.1 when package.json was bumped. No dependency changes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
eval/lib/sampling.js(1 line): addclaude-fable-5toshouldOmitAnthropicSamplingParams. Fable 5 rejects any explicittemperature(API 400"temperature is deprecated for this model"), the same restriction the map already handles for Opus 4.7. Without this,run-execute-boundary.jsfails 25/25 fixtures at the API layer.eval/public_notes/aar_v0_8_execute_boundary_fable5_readout.md: full readout of the 2026-06-10 pre-registered AAR v0.8 EXECUTE-boundary regression run againstclaude-fable-5— both runner conditions, per-case strip, verbatim failure texts, SHA-256 audit anchors, deviations log, claim boundary. Raw/scored result files stay local per.gitignorepolicy; the memo's SHAs are the anchors.eval/package-lock.json: 2-line version-field sync to package.json 0.6.2 (npm regenerates this for anyone; no dependency changes).Why
The EXECUTE-boundary case set had only been run against sonnet-4-6. This run adds the fable-5 data point under the frozen v0.8 skill ZIP (SHA verified pre-inference) with the committed scorer, unmodified.
Headline results
vs sonnet-4-6 ZIP run: 82.6%, also 0/0/0. Fable-5 fixes EB-08 and EB-20, reproduces EB-14 (EX-R06-class form mismatch).
What a reviewer should know
temperature: 0field reflects the script variable, not the wire (pre-existing quirk shared with the Opus 4.7 path, disclosed in the memo).scorerParity.js, the cases file, and all regexes are untouched.🤖 Generated with Claude Code