[gmuloc]

Description

Recurring 128/256 KiB memory bands make CodSpeed flag changes that have no demonstrated ANTA memory regression. Earlier native repetitions reproduced the bands on one commit and attributed representative allocations to CPython's asyncio task-registration WeakSet. A controlled reset comparison reduced the two-device peak spread from 132,108/264,037 B to about 1.4 kB.

Move the ordinary CodSpeed memory job to pinned Python 3.14.9, whose native asyncio Tasks use linked-list registration. Pin its current native runner 5.5.0 and use the same CLI/test/benchmark dependency groups as the diagnostic. Keep simulation on Python 3.12. No registry reset is enabled in either job, and the ANTA measured callbacks, cold-inventory setup, scheduling and dynamic fixture coverage are retained.

Qualify this change with a dedicated ARM diagnostic matrix: pinned CPython 3.12.15 and 3.14.9, each running ten isolated/suite pairs on the exact PR head. Each measurement starts a fresh native runner/tracker/pytest process; native warm-up remains enabled. The matrix uses native CodSpeed 5.5.0/memtrack 1.6.1 and captures interpreter/implementation, VM image, installed packages/binaries, workload fingerprints and result statuses. Jobs have independent environments and uniquely named artifacts.

The small summary independently reconstructs allocation-cohort peaks for the isolated two-device target and every one/two/five-device dry-run/full-run suite sample. Each profile is decoded once, with sample markers and tracker event counts validated. These numbers are requested allocation bytes originating within each sample, not RSS or an exported CodSpeed dashboard metric. Raw profiles are retained alongside the small summary, with no diagnostic CodSpeed upload. The earlier four-variant WeakSet experiment remains available only through --include-registry-reset on compatible interpreters.

CPython's task-registry implementation is runtime-specific: non-native third-party tasks on 3.14 still have a WeakSet fallback, and PyPy is unqualified. Compare distributions within each interpreter before interpreting cross-version numbers. The first ordinary dashboard comparison can span a 3.12 baseline and 3.14 head; a new 3.14 baseline is required for meaningful subsequent regression comparisons.

Instructions and limits: tests/benchmark/MEMORY_DIAGNOSTICS.md.

Related: #1773, #1808, #1822.

Validation

• Exact generated local commands passed on Python 3.14.7 for isolated and all six suite cases with PYTHONPATH/CODSPEED_ENV unset. Catalog/status counts match the earlier 3.12 workload; zero registry resets were recorded. These were local walltime fallback runs, not native memory measurements.
• Three representative retained native profiles reproduce the independent peaks and raw event counts exactly. A real six-window suite and hand-calculated synthetic multi-window metrics pass; missing/invalid/empty samples are rejected.
• Mocked driver checks verify default two variants, opt-in four variants, exact selections/reset flags, and retained failure metadata. Workflow checks verify interpreter choices and unique artifact names.
• Targeted and final all-files pre-commit pass, including Ruff, Pylint, Pyright, docs generation and zizmor, with no unrelated generated changes.
• Initial native run 40326834516 passed 40 no-reset processes under CodSpeed 5.4.0/memtrack 1.6.0: 3.14 two-device isolated/suite spreads were both 715 B; 3.12 spreads were 131,834/262,699 B. All six 3.14 suite cases had spread 0–715 B; workloads matched.
• Follow-up targeted and final all-files pre-commit pass. Static checks verify the Python/native pins, official installer checksum and application dependency alignment.

Matching-instrument outcome

Run 40477854645 passed all 40 no-reset processes at c9a6fea5. Both jobs used native CodSpeed 5.5.0/memtrack 1.6.1, the same ARM image and matching workload: 1,132 catalog cases, 2,264 target results, 858 success/1,330 expected fixture failures/76 skipped, no error/unset results. Packages/binaries remained unchanged within each job; cross-job package/version lists match except pip. All 40 result-order fingerprints differ.

Observed peak allocation spreads across ten suite repetitions:

Case	Python 3.12.15 spread (B)	Python 3.14.9 spread (B)
Full run, 1 device	131,094	22
Full run, 2 devices	263,115	1,333
Full run, 5 devices	1,552	1,333
Dry run, 1/2/5 devices	0 / 0 / 0	0 / 0 / 0

Two-device isolated spread is 130,518 B on 3.12 versus 1,146 B on 3.14. No 3.14 target sample contains a ≥128 KiB allocation; the 3.12 targets reproduce the 131,072/262,144-byte allocation bands. The 3.14 two-device suite range is 914,975–916,308 B, approximately 0.146% spread relative to its median. These are observed ranges for this workload, not a universal noise threshold or an RSS/dashboard metric.

The ordinary 3.14 memory job also passed six cases and uploaded performance data. Its logs confirm Python 3.14.9, runner 5.5.0 and memtrack 1.6.1. There are no ANTA/benchmark source differences between diagnostic head and the ordinary job's merge checkout. All PR checks passed.

This supports adopting 3.14 for the memory job without the private WeakSet reset. A new main-branch 3.14 baseline is still required after merge; absolute cross-runtime changes are not ANTA optimizations. PyPy and alternate task implementations remain unqualified.

Small summaries: 3.12, 3.14. Full profiles are retained for 30 days and were uploaded successfully; the matching-run stacks were not independently symbolized.

Prior native evidence

• Run 38914326690 attempt 2: 20 unchanged-commit isolated/suite processes reproduced the allocation bands; representative 131,072/262,144-byte stacks resolve to task-registration set resizing.
• Run 39781256062: 40 baseline/reset processes with matching workload/environment and 140 validated resets. Isolated spread 132,108 → 1,409 B; suite spread 264,037 → 1,341 B. This qualifies the controlled reset on CPython 3.12, not Python 3.14 repeatability.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[codspeed]

Merging this PR will improve performance by 27.27%

1 warning

⚠️ Base and head ran on different runner types

Some benchmarks with significant performance changes were compared across different runner types, so the changes might not come from the code. Run base and head on the same runner type, or use CodSpeed Macro Runners for more stable results.

Learn more about Macro Runners

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 33 untouched benchmarks

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Memory	test_anta[2-devices]	1,138.1 KB	894.3 KB	+27.27%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing gmuloc:ci/memory-repeatability (c9a6fea) with main (496a209)}

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[gmuloc]

closed in favor of #1849