Repository navigation
Conversation
Separate workflow file (the generated ci.yml is produced by forge_ci and hand edits there are lost on regeneration). One job, one instrumented suite run, two gates: tarpaulin --engine Llvm gates line coverage at a measured floor, and cargo-crap scores CRAP from the same run's lcov. Measured on this branch's base (da9a026), tarpaulin 0.37.5 / ptrace, 21-crate workspace, 1855 tests passing: coverage 52.01% (9463/18194) -> floor 49 (3pp engine-variance margin); worst CRAP after the documented crates/forge_main/src/ui.rs exclusion is 3660 (Renderer::render_event) -> ceiling 4030 (+10% headroom).
|
Hi! I'm the It looks like you correctly set up a CI job that uses the autofix.ci GitHub Action, but the autofix.ci GitHub App has not been installed for this repository. This means that autofix.ci unfortunately does not have the permissions to fix this pull request. If you are the repository owner, please install the app and then restart the CI workflow! 😃 |
First CI run of this gate (llvm engine) measured 50.45% (8709/17263). Align the floor with the intended measured -3pp margin on the engine CI actually runs; the ptrace measurement (52.01%) had understated the margin.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the estate-standard two gates to this repo in one new job, fed by ONE instrumented suite run: cargo-tarpaulin (coverage floor) and cargo-crap (complexity ceiling), both versions pinned (tarpaulin 0.37.5, cargo-crap 0.6.1). Gate state: required-intended — forge is a PUBLIC repo, so ubuntu-latest works and branch protection IS available on this plan; the intent is for this gate to become a required check once green (operator wires it after merge).
What was added
.github/workflows/coverage-crap.yml(namecoverage-crap). It is deliberately a SEPARATE file:.github/workflows/ci.ymlis generated by forge_ci/gh-workflow-gen and hand edits there are lost on the next regeneration. The generated file is untouched.coverage-crap: checkout → toolchain (actions-rust-lang/setup-rust-toolchain@v1, the repo's own convention,toolchain: stable+components: llvm-tools-preview) →arduino/setup-protoc@v3(the repo needs protoc for tonic/prost in forge_app/forge_infra/forge_repo) → lockfile assertion → pinned tarball tool install with in-script PATH export + version assertions → coverage gate → CRAP gate → shell-only run summary.Measured first-run numbers (base da9a026)
Local measurement,
cargo-tarpaulin 0.37.5 --engine Ptrace --workspace --locked --out Lcov -j 2, toolchain pinned to the repo's rust-toolchain.toml (1.92), 21-crate workspace: CI verification (llvm engine) followed in the gate job itself — both runs below.--workspace): 2249 functions scored; worst entries wereUI::handle_subcommands15006 (CC 122) andUI::on_command4556 (CC 67) — both incrates/forge_main/src/ui.rs— thenRenderer::render_event3660 (CC 60, cov 0%,crates/forge_markdown_stream/src/renderer.rs:125).Thresholds + rationale
COVERAGE_FLOOR: "47"— set from the first CI run of this very gate (llvm engine): measured 50.45% (8709/17263 lines) − 3pp estate margin. (Local ptrace measurement was 52.01%; the engine counts a slightly different line set.) Never ships a threshold the repo doesn't pass today.CRAP_THRESHOLD: "4030"— measured max 3660 (Renderer::render_event) + ~10% headroom, after the one documented exclude below.crates/forge_main/src/ui.rs(--exclude 'src/ui.rs', crate-root-relative glob under--workspace; verified it is the only ui.rs in the tree). Rationale: a 3533-line TUI dispatch/event-loop surface, 93 scored functions all at 0% coverage by design (no TTY test harness drives the TUI). Its worst entry forces a ~16507 ceiling that measures nothing, so it is excluded with the same posture as other repos excluding their entry points; the coverage floor still accounts for the file. Glob note: an initialcrates/forge_main/src/ui.rspattern silently matched nothing — the pattern must be crate-root-relative.crates/*(21 workspace members); no orphan crates, no spike/experiment dirs, no non-workspace Rust (benchmarks/ and shell-plugin/ are TS/shell).benchmarks/,scripts/,templates/contain no Rust.Gate proof (both directions, identical exclude set)
cargo crap --lcov lcov.info --workspace --exclude 'src/ui.rs' --threshold 4030 --fail-above→ exit 0 (passes).--threshold 3000→ exit 1, "1/2156 function(s) exceed CRAP threshold 3000" (Renderer::render_event3660). The gate can discriminate, not just pass.Runner choice + why
ubuntu-latest (public repo, follows the repo's existing convention; the generated ci.yml also uses ubuntu-latest). Engine is
--engine Llvmfor estate-wide consistency with the fleet runs even though ptrace would work here; the ptrace engine is blocked in the fleet's sysbox-confined runner containers (measured on dirmacs/skywalkers run 38064108686).DB needs
None — no Postgres/Redis/network-dependent tests (checked for DATABASE_URL/TEST_DATABASE_URL consumers: none). Tests need protoc (installed via arduino/setup-protoc@v3, same as the existing build job).
Caveats
--engine Llvm): coverage 50.45%, CRAP gate green vs ceiling 4030 (two green runs: 38069680519 at floor 49, 38070354043 at floor 47). The floor was re-based after the first run showed the llvm line set differs slightly from ptrace (17263 vs 18194 lines).-j 4) — 21 crates on a 4-core runner is the dominant cost.cargo install-free: both tools install from pinned release tarballs, versions asserted in-script.Ratchet rule