Skip to content

Restore 100% test coverage #201

Closed as not planned
Closed as not planned

Description

@larstalian

Context

pyproject.toml historically set fail_under = 100 for the coverage
gate. The recent rewrite temporarily dropped coverage from 100% to 87%,
so the gate was relaxed to 85% to land that work.
The relaxation is documented but it's debt — the project's quality
bar said 100% and the goal is to get back there.

What this issue tracks

Bring test coverage back up to 100% so fail_under = 100 can be
restored.

Why it matters

The 100% rule is a discipline, not just a number. It catches dead
code, forces edge cases to be tested, and makes deletions safe (if
something's only covered by the test that imports it, deleting the
test catches the unused code). Letting it slide is a one-way ratchet.

Where the gaps are

Run uv run coverage report --skip-covered for the current view.
Headline gaps from the last run (~297 missed statements + 135
partial branches across 24 files):

  • src/openrange/core/episode.py (~70 missed) — advance() until /
    timeout paths, checkpoint/restore/fork, _terminal_state
    branches, NPC error swallowing, close(). Needs new tests after
    the dashboard tests were dropped.
  • src/openrange/dashboard/ (~53 missed) — same root cause: old
    dashboard tests were deleted during the rewrite; replacements not
    yet written.
  • src/openrange/packs/cyber_webapp_offense_v1/ modules — error
    branches in llm_generation (LLM call fail / bad JSON / bad
    source), sampling (validation paths in sample_int /
    weighted_pool), codegen/handlers (each kind branch in
    _default_handler_body, AST splice failure paths), NPC factory
    bad-config paths.
  • Smaller gaps in core/builder.py, core/snapshot.py,
    core/runtime_backing.py (defensive paths).

Where to start

  • pyproject.toml — the fail_under setting + the coverage section.
  • tests/ — the existing tests as model for what to mirror.
  • Run uv run coverage report --show-missing after a test run to
    see exact missed lines.

Acceptance

  • fail_under = 100 restored in pyproject.toml.
  • CI passes.

Notes

This is mechanical but real test-writing work — probably ~200 lines
of test code spread across several files. Good help wanted task
for someone who wants to learn the codebase by exercising every
branch.

Some of the older branches that lost coverage are honestly defensive
code that's never going to fire in practice. For those, a
# pragma: no cover is fine — but don't reach for it as the first
move; write the test if you can.

Activity

  1. added
    help wantedExtra attention is needed
    roadmapTracked on the public roadmap
    coreCore library / runtime / admission
    on May 4, 2026
  2. changed the title [-][roadmap] Restore 100% test coverage[/-] [+]Restore test coverage to 100% (relaxed to 85% during v1 / v0-deletion work)[/+] on May 4, 2026
  3. changed the title [-]Restore test coverage to 100% (relaxed to 85% during v1 / v0-deletion work)[/-] [+]Restore test coverage to 100%[/+] on May 4, 2026
  4. self-assigned this
    on May 4, 2026
  5. changed the title [-]Restore test coverage to 100%[/-] [+]Restore 100% test coverage[/+] on May 4, 2026
  6. larstalian commented on Jun 9, 2026

    @larstalian
    ContributorAuthor

    Current view from a local coverage run -m pytest (640 passed, 2 skipped; total 84%, floor is 80%). The biggest gaps sit squarely on the surface the incoming sim-to-real work will extend, which is worth noting as prioritization:

    Module Cover Note
    openrange_pack_sdk/_harness.py 0% 60 stmts, entirely untested
    src/openrange/core/episode.py 59% the episode loop itself
    packs/cyber_webapp/.../realize.py 59% the realizer — about to grow CONTAINER/backing branches (#252)
    src/openrange/dashboard/runs.py 59%
    cyber_webapp/families/pentest.py 72%
    src/openrange/core/pack.py 71%

    episode.py and realize.py are called out in pyproject.toml's own coverage note (lost the old test_v1_episode.py HTTP round-trip). Closing these two first is high-leverage: they're the modules the backing/fidelity work (#251, #252) and cost metering (#253) will modify, so coverage here makes those changes safe rather than scary.

  7. larstalian commented on Oct 12, 2026

    @larstalian
    ContributorAuthor

    No longer involved with this project, so closing my open issues. Feel free to reopen anything you still want to track.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

coreCore library / runtime / admissionroadmapTracked on the public roadmap

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions