Repository navigation
feat(judge): support crush as a judge CLI [7/9] - #31
Open
LarsArtmann wants to merge 7 commits into
Open
LarsArtmann wants to merge 7 commits into
LarsArtmann wants to merge 7 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The evaluation judge can now run through the
crushCLI, alongsideclaudeandcodex:SupportedCLIsgainscrush; auto-detection and the UI's judge picker pick it upcrush run --quiet --verbosereads the prompt from stdin (large evidence documents never hit argv limits);--verbosestderr names the model that answered, parsed like the codex preamblejudge.IsWorkDirbeltadapter.HomePath(oneMINDWALK_HOMEcontract)Cache.Storeerrors now name the session key, and its temp-file cleanup is errcheck-cleanSealed-subprocess stance is unchanged: no tools, no user config, verdicts still derived mechanically from findings.
Why
Users who only have Crush installed currently cannot evaluate sessions at all; the judge is otherwise CLI-agnostic.
Series
Part 7/9; stacks on #30 (and #25–#29 before it). "Files changed" shows the cumulative stack — this PR's own delta is just the seventh commit,
feat(judge): support crush as a judge CLI.Validation
go test -count=1 ./...green at the tipcrushModelreads the verbose log line and stays empty without one