fix(ci): separate CodeQL merge enforcement from SDL scans

[Daniel Meppiel (danielmeppiel)]

fix(ci): separate CodeQL merge enforcement from SDL scans

TL;DR

Add a trusted, workflow-scoped CodeQL policy to the existing required gate,
while preserving default-branch database uploads for external SDL scanning.
Extend coverage to JavaScript/TypeScript and require successful full-tree scans,
current-attempt upload evidence, and no open findings at blocking severity.
The native ruleset remains unchanged until the documented post-merge rollout.

Important

This PR does not remove the live code_scanning requirement.
Keep it until the deployed replacement succeeds on a fresh PR and a merge-queue
run; only then remove CodeQL from that native rule through the API.
The initial PR still executes the old trusted base gate implementation.

Follow-up to #2870 and the maintainer's report of repeated
“Code scanning is still expecting 2 results from CodeQL” messages.

Problem (WHY)

• The reported CodeQL check
  identifies an additional API upload / <default> configuration on main,
  despite successful Python and Actions workflow uploads.
• The repository's native rule requires the tool CodeQL, not a particular
  workflow configuration. The external default-branch producer cannot be
  satisfied by rerunning the repository's PR analysis.
• The prior workflow omitted JavaScript/TypeScript, although the repository
  contains JavaScript. Existing Python and Actions identities must stay stable.
• [!] Job success alone is insufficient: scans may report vulnerabilities,
  processing can lag, and old analyses can survive a same-commit rerun.
• The first live JavaScript scan exposed two existing findings. With the
  maintainer's approval, this PR also removes ambiguous regex repetition from
  smoke-log parsing and makes every governance roster row undergo validation.

The implementation follows the Agent Skills validation loop:
"do the work, run a validator (a script, a reference checklist, or a self-check), fix any issues, and repeat until validation passes."
The first review identified four gaps—diff filtering, rerun provenance,
moving-ref snapshots, and final-gate freshness—which were fixed before publishing.

Approach (WHAT)

• Keep the existing gate check as the single required status-check authority.
• Execute the policy from the immutable base checkout; never run PR artifacts.
• Require all three language jobs and their processed analyses to succeed.
• Bind each analysis to its real SARIF upload ID, workflow run, attempt, ref, and SHA.
• Block all open, undismissed high/critical security or error-level findings
  from this workflow, including pre-existing findings, as approved by the maintainer.
• Leave independent SDL results, database uploads, and the live ruleset intact.

Implementation (HOW)

File	Intent
.github/scripts/ci/codeql_policy.py	Add GET-only policy evaluation: exact event context, successful jobs, bounded artifact parsing, current SARIF evidence, paginated alert instances, and fail-closed errors.
.github/scripts/ci/merge_gate_wait.sh	Evaluate CodeQL before polling other checks and immediately before success, using the existing deadline.
.github/workflows/codeql.yml	Add JavaScript/TypeScript, disable diff-informed filtering, explicitly retain database uploads and processing waits, and publish small per-language/run-attempt evidence artifacts.
.github/workflows/merge-gate.yml	Grant only the additional actions: read and security-events: read permissions needed to inspect evidence.
.github/workflows/ci.yml	Include the new helper in formatting, lint, duplication, and file-length checks.
tests/unit/test_codeql_policy.py	Cover thresholds, dismissals, wrong/missing evidence, reruns, moving refs, pagination, artifact parsing, event identity, and final shell enforcement.
tests/unit/test_security_workflow_contracts.py	Lock down full-tree coverage, all language jobs, evidence uploads, trusted-base execution, read-only permissions, and lint integration.
docs/src/content/docs/contributing/development-guide.md	Document the policy, evidence expiry, staged ruleset migration, preservation of other tools/rules, and rollback ordering.
scripts/daily-release-smoke-signal.cjs	Remove redundant nested regex repetition while preserving failure extraction, ordering, deduplication, and the 30-test limit.
scripts/governance/authority.cjs	Parse every roster row instead of filtering on a URL substring; retain exact anchored GitHub identity validation and existing remit rules.
tests/unit/test_daily_release_smoke_signal.py	Exercise real Node smoke reporting with mocked GitHub I/O, including adversarial input under a 10-second subprocess deadline.
tests/scripts/governance_evidence.test.cjs	Reject malformed/foreign roster identities while preserving whitespace, alignment, and the existing governance scenarios.

No production CLI, dependency manifest, README, or CHANGELOG changes.

Diagrams

The highlighted interactions add current-run merge enforcement without replacing the independent default-branch SDL feed.

sequenceDiagram
    participant W as codeql.yml Analyze jobs
    participant A as GitHub analyses and artifacts
    participant S as External SDL scanner
    participant G as Trusted merge gate
    participant C as Other required CI
    W->>A: Upload Python, Actions, and JavaScript results
    rect rgb(255, 247, 200)
        W->>A: Record SARIF IDs for this run and attempt
    end
    alt Default branch
        W->>A: Publish CodeQL databases
        A->>S: Databases remain available
        S->>A: Return independent API scan results
    else Pull request or merge queue
        rect rgb(255, 247, 200)
            G->>A: Read exact run, attempt, ref, SHA, and alerts
            A-->>G: Successful scans and matching upload evidence
            G->>G: Reject open blocking findings
        end
        G->>C: Wait for existing required checks
        C-->>G: Checks complete
        rect rgb(255, 247, 200)
            G->>A: Revalidate CodeQL before passing gate
            A-->>G: Stable passing evidence
        end
    end
    Note over G,A: Native scanning rule remains until post-merge rollout

Loading

Trade-offs

• Stricter than new-alerts-only. All open scoped findings at the existing
  severity thresholds block, including pre-existing ones. Full-tree scans cost
  more than diff-informed queries; dismissals remain a deliberate human action.
• Bounded evidence retention. Artifacts expire after 14 days. An old PR may
  need a CodeQL rerun rather than reuse evidence that can no longer be verified.
• Separate compliance and merge roles. Private external SDL queries remain
  default-branch compliance scans; this does not claim to run them before merge.
• Deliberate staged activation. Hermetic tests and code review do not replace
  deployed GitHub verification. The first PR uses the old base gate; removing
  native protection before fresh PR/queue verification would create a gap.
• Small standalone CI dependency surface. The helper uses stdlib and gh,
  not APM imports or an editable project install in the trusted gate job.

Benefits

1. Three language analyses, each with current-run proof, must succeed.
2. Missing, stale, malformed, skipped, or failed evidence cannot produce a pass.
3. External default-branch configurations cannot substitute for—or obstruct—the
   workflow-scoped policy after the ruleset migration.
4. The required gate check and existing non-CodeQL protections stay in place.

Validation

Local validation against main at 6f3dd393ce82e4aa3b56d0de292836590af2d59e.
Remote CI and post-merge activation are not claimed complete.

uv run --frozen --extra dev pytest -q tests/unit/test_codeql_policy.py tests/unit/test_security_workflow_contracts.py tests/unit/test_daily_release_smoke_signal.py tests/unit/test_governance_evidence.py tests/quality --tb=short

152 passed in 41.35s

The live first scan completed all three languages and uploaded genuine evidence;
the candidate policy correctly rejected two high-severity JavaScript findings.
Before the fixes, the new regex regression exceeded its 10-second deadline and
the roster regression demonstrated a silently skipped malformed identity.
Both now pass without exclusions, alert dismissals, or weaker thresholds.

Type, lint, and guardrail evidence

uv run --frozen --extra dev mypy --follow-imports=skip .github/scripts/ci/codeql_policy.py

Success: no issues found in 1 source file

The canonical ruff/format/pylint/auth mirror, including the new helper:

All checks passed!
1869 files already formatted
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)
[+] auth-signal lint clean

Remaining CI guards and test-quality checks:

YAML I/O, 2100-line, and portable-path CI guards passed
[+] assertion-quality ratchet clean: AQ001=4, AQ002=12
[+] exact test duplicate ratchet clean: 1228 files, 0 allowed duplicate group(s)

Architecture-boundary lint and git diff --check exited successfully.
The Mermaid diagram passed mmdc and was visually inspected.
Locked dependencies were restored through pip using uv-exported requirements
after uv's package-download transport failed; no dependency files changed.

Scenario Evidence

#	Scenario (user promise)	Principle(s)	Test(s) proving it	Type
1	Our scans can pass without an unrelated SDL result on the PR.	Governed by policy	tests/unit/test_codeql_policy.py::test_clean_exact_commit_with_no_sdl_pr_upload_passes (regression trap for the reported missing-configuration failure)	unit
2	Open blocking findings cannot be hidden by job success, age, or another configuration.	Governed by policy	test_thresholds_block_all_open_findings_including_existing; test_all_instances_are_inspected_not_only_most_recent in tests/unit/test_codeql_policy.py	unit
3	A rerun cannot reuse stale successful results.	Secure by default	tests/unit/test_codeql_policy.py::test_completed_rerun_cannot_reuse_previous_attempt_sarif; test_newer_commit_analysis_cannot_use_historical_alert_snapshot	unit
4	Evidence artifacts cannot execute code or escape the parser's expected file shape.	Secure by default	tests/unit/test_codeql_policy.py::test_artifact_reader_rejects_unexpected_names; test_artifact_reader_accepts_bounded_json	integration
5	A later CodeQL failure still stops the final merge gate.	Governed by policy	tests/unit/test_codeql_policy.py::test_shell_rechecks_codeql_after_other_checks_finish (real Bash orchestration with simulated external checks)	integration
6	Compliance uploads, complete language coverage, and trusted-base execution remain enabled.	Secure by default; OSS / community-driven	tests/unit/test_security_workflow_contracts.py::test_codeql_covers_merge_queue_with_existing_analysis_configurations; test_merge_gate_executes_base_commit_script	integration
7	Newly covered helpers remain safe without suppressing their findings.	Secure by default	tests/unit/test_daily_release_smoke_signal.py::test_failure_report_parses_logs_within_deadline; tests/unit/test_governance_evidence.py::test_governance_node_regressions	integration

How to test

• Run the focused pytest command above; all 152 cases should pass.
• Confirm the PR's three Analyze jobs succeed and each publishes its current-attempt evidence artifact.
• After merge, verify the trusted policy logs on a fresh PR and a merge-queue run; verify default-branch database uploads remain available.
• Only after that verification, back up and re-read ruleset 9294522, remove only CodeQL from parameters.code_scanning_tools, and verify every other rule/tool and required gate remain unchanged.
• For rollback, restore the prior native code-scanning rule before reverting the custom gate.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[github-advanced-security]

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

• The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
• Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
• You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

[Daniel Meppiel (danielmeppiel)]

Docs sync advisory

Verdict: no_change * Pages affected: 0 additional * LLM calls: 2/15 * Took: not recorded

No additional documentation-corpus edits are needed. Rechecked after the JavaScript helper fixes: these are private CI/governance scripts, not APM commands, flags, APIs, or configuration schemas. The contributor development guide already documents the CodeQL policy, newly covered findings, and staged ruleset rollout.

Classifier: high confidence, L1. The isolated classifier could not access its tools; classification was completed directly against the docs index and changed paths. No downstream panel was needed. This advisory does not gate merge.

[Copilot]

Copilot review overview

🟡 Changes recommended

Unresolved critical and moderate findings affect merge-queue enforcement, analysis matching, workflow provenance, freshness, command resolution, and timeout behavior.

Get a fresh assessment by requesting another Copilot review.

Review tier: Lite
Findings: 3 · 1

Open findings (4)

• Normalize merge-group refs before validating the event · New
• Match CodeQL's default analysis category · New
• Bind evidence to a trusted CodeQL workflow definition · New
• Recheck alert findings in the stability guard · New

What changed in this PR

This PR adds workflow-scoped CodeQL enforcement to the merge gate while preserving default-branch SDL uploads.

Changes:

• Adds CodeQL policy evaluation and evidence validation.
• Expands analysis to JavaScript/TypeScript.
• Integrates gate checks, tests, lint coverage, and rollout documentation.

File	Summary and final findings
tests/​unit/​test_security_workflow_contracts.py	Adds workflow contract coverage. No final findings.
tests/​unit/​test_codeql_policy.py	Adds policy behavior and edge-case tests. No final findings.
docs/​src/​content/​docs/​contributing/​development-guide.md	Documents policy, rollout, and rollback procedures. No final findings.
.github/​workflows/​merge-gate.yml	Adds required read permissions. No final findings.
.github/​workflows/​codeql.yml	Expands scans and publishes evidence. Critical · 1 vote: PR-modified workflow output is not bound to a base-owned workflow or verified workflow revision.
.github/​workflows/​ci.yml	Includes the helper in CI checks. No final findings.
.github/​scripts/​ci/​merge_gate_wait.sh	Runs CodeQL checks before and after other checks. Moderate · 1 vote: the initial policy wait can consume the entire shared 55-minute deadline.
.github/​scripts/​ci/​codeql_policy.py	Implements CodeQL policy evaluation. Critical · 1 vote: merge-queue ref validation rejects the real payload shape. Moderate · 3 votes: the final consistency check does not re-read open findings. Critical · 1 vote: analysis-category matching filters out real uploads. Moderate · 1 vote: gh resolution permits PATH shadowing from the project tree.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Daniel Meppiel (danielmeppiel)]

Live CodeQL verification

Verified head 16c1e1443ce7366b97a50e867e9cdd9624170f60, PR merge SHA b4e997da05cdd3c845fa204ea3d6c0875c02c1e3.

• CodeQL run 34711197904: Python, Actions, and JavaScript/TypeScript all succeeded.
• The candidate policy was executed locally against the real GitHub PR metadata, processed analyses, current-run SARIF evidence artifacts, and alert instances. It passed with no blocking workflow-scoped findings.
• Code-scanning alerts 210 and 211 are now fixed, not dismissed. No scan exclusions or severity reductions were introduced.

The initial PR still executes the old trusted-base gate by design. The live main-protection ruleset remains active with its native CodeQL requirement and required gate; deployed PR/queue verification and the documented ruleset migration remain post-merge maintainer steps.