Skip to content

Add opt-in token CUSUM diagnostics to summary scans - #99

Merged
kwisatzh merged 2 commits into
mainfrom
research/g9-anomaly-core
Nov 16, 2026
Merged

kwisatzh merged 2 commits into
mainfrom
research/g9-anomaly-core

Conversation

@kwisatzh

@kwisatzh kwisatzh commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds scan --token-cusum to show changes in observed input-plus-output token totals alongside the existing scan report. The flag is opt-in; the legacy scan findings and feature-off text/JSON outputs retain their existing behavior.

  • Compose the existing summary reader, coverage checks and snapshot combiner with a one-sided CUSUM using an earlier-window median/MAD reference.
  • Check token accounting independently for each selected producer/epoch snapshot, so errors cannot cancel in aggregate. Preserve canonical higher-sequence replacement and cumulative-regression checks.
  • Add bounded chronological/block-resampling calibration primitives and exact rational contributor intervals for follow-on integration. No runtime dependency changes.

Invariants And Tests

Missing or inconsistent usage remains unavailable. Missing producers, partial intervals, key/accounting changes and unsupported precision cannot become scored zeroes. The tests cover per-source no-attempt tokens, reported/request count mismatch, missing/invalid/conflicting usage, subset violations, missing quality counters, dedup coverage and restart epochs. Replacement tests exercise both input orders and retain regression refusals.

Text/JSON goldens cover the flag on and off. The OFF goldens were independently compared with stdout from the committed base. Numeric tests cover planted changes, causal references, overflow, exact interval arithmetic and share adjustment. The output/error sentinel checks use synthetic inputs.

The required pre-merge regression pins a synthetic case with 4 chronological alarms in 16 scored buckets (above a 1-in-5 target) and 5 block-resampled alarms in 32 buckets (within it). Calibration must reject this candidate. Removing only the chronological-rate safeguard makes the new test fail at that rejection assertion.

Verification

Final candidate: 9bb816170782a245033c45ebeebb8e9df259f8b2, signed and signed off, with Vijay as author and committer. Its follow-up to reviewed b5a7c23 changes only the required regression test; production code and goldens are unchanged. The following local checks passed again on the final exported head using Go 1.26.11:

go test -race -timeout 45m ./...
go vet ./...
sh examples/investigate.sh
sh examples/investigate.sh --missing-usage
sh examples/investigate.sh --two-stacks
sh examples/investigate.sh --sessions
sh examples/demo.sh
python -B -m unittest discover -s docs/media -p 'test_*.py'

Formatting, gocyclo on the new production files, govulncheck (No vulnerabilities found), built-CLI goldens and source-quality predicate mutations also passed. Staticcheck is clean on change, compare and cli. Its repository-wide run reports the same seven pre-existing style findings as the committed base: S1038 in a scenario test and ST1005 in the existing operator-example/Langfuse errors. No unrelated files were changed to clear them.

The first full test attempt used a private umask 077, which changed an existing permission-test fixture from 0644 to 0600. The same failure was reproduced on the unchanged base; the runner now uses 022 for child checks while retaining private outer evidence directories. Credential code is unchanged. An initial offline tool-metadata lookup was retried with the pinned public module metadata available; product dependencies are unchanged.

The final test-only head passed Linux/macOS CI, all four platform build-verification jobs, signatures, DCO, trusted API tests and CodeQL. No local Docker workload was run, and no release is requested.

Limits

The CLI diagnostic is uncalibrated and does not issue CUSUM alerts. It measures observed tokens, not cost; exact recorded-spend reader integration, contributor ranking and held-out detection calibration remain follow-on work. Complete export intervals describe the supplied observations, not proof that every physical request was captured.

Pre-merge review approved merge once this regression was added, without a second round. The test and final-head green CI conditions are met. The ordinary, exact-head squash requests returned HTTP 502; the PR remains open and no protection was bypassed. This increment stays unreleased.

Compose summary coverage and source-quality checks with bounded robust scoring. Add exact contributor interval arithmetic and pin text/JSON output with the feature enabled and disabled.

Signed-off-by: Vijay <evijay@gmail.com>
@kwisatzh
kwisatzh marked this pull request as ready for review November 16, 2026 09:24
Pin a candidate whose chronological replay exceeds its target while seeded block replay passes. The regression fails when the chronological-rate safeguard is removed; production code is unchanged.

Signed-off-by: Vijay <evijay@gmail.com>
@kwisatzh
kwisatzh merged commit 40fbef3 into main Nov 16, 2026
21 checks passed
@kwisatzh
kwisatzh deleted the research/g9-anomaly-core branch November 16, 2026 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant