Rerun epiworld-v1 at NH-2, V-1, V-2 (UofUEpiBio/epiworld#365) by gvegayon · Pull Request #35 · UofUEpiBio/epiworld-benchmark

gvegayon · 2026-11-07T09:27:58Z

Pins d571d3e978b54d0e3a8e7a82147dcf39103ba124 (UofUEpiBio/epiworld#365).

The rerun for epiworld's NH-1 and NH-2 (UofUEpiBio/epiworld#363), V-1 (#364) and V-2 (#365), the second half of NH-3 (the docs half is UofUEpiBio/epiworld#366).

Runner change

The five epiworld-v1 runners declare the SEIRH as a natural history on the virus: stages Susceptible, Exposed (carrier), Infected (infectious), Hospitalized (carrier), Recovered; one infection and four rates (the two out of Infected compete, as new_state_update_transition() did). The initial cases and the vaccine are placed with assign::count(), the totals read by stage, and the classes are plain (Model, not Model<>). Scenario 02 reads the transition matrix's stages (a state is now a (status, stage) pair).

make container-check and make container-smoke pass; STAGE=nh-2-v2 make container-v1-stage ran alone (an idle container from another project was up, at 0.2% CPU).

The stage's rows

Stage	epiworld-v1	Scenario	Agents	Simulate (s): 0.x, 1.0	Ratio	Build (s): 0.x, 1.0	Ratio	Peak (MiB): 0.x, 1.0	Ratio	Attack rate: 0.x, 1.0
nh-2-v2	1.0.0+gd571d3e	00	10,000	0.00488, 0.00529	1.08	0.00137, 0.00109	0.79	8, 7	0.81	0.385, 0.385
nh-2-v2	1.0.0+gd571d3e	00	100,000	0.00821, 0.0108	1.31	0.013, 0.01	0.77	38, 33	0.87	0.060, 0.060
nh-2-v2	1.0.0+gd571d3e	01	10,000	0.00253, 0.00257	1.02	0.00127, 0.00104	0.82	8, 7	0.91	0.119, 0.119
nh-2-v2	1.0.0+gd571d3e	01	100,000	0.00859, 0.00838	0.98	0.0123, 0.00977	0.80	43, 39	0.90	0.014, 0.014
nh-2-v2	1.0.0+gd571d3e	02	10,000	0.00236, 0.00258	1.09	0.00126, 0.00103	0.82	9, 8	0.95	0.119, 0.119
nh-2-v2	1.0.0+gd571d3e	02	100,000	0.00893, 0.00885	0.99	0.0132, 0.0103	0.78	44, 40	0.91	0.014, 0.014
nh-2-v2	1.0.0+gd571d3e	03	1,000,000	0.0259, 0.0404	1.56	0.134, 0.112	0.84	352, 294	0.84	0.006, 0.006
nh-2-v2	1.0.0+gd571d3e	04	165,865	0.218, 0.232	1.07	0.0195, 0.0139	0.71	90, 52	0.57	0.622, 0.622

Attack rates agree in every row. The simulation ratio grew by more than 10% from 3d-3b in three rows: scenario_00 at 100,000 (1.03 to 1.31), scenario_03 (0.91 to 1.56) and scenario_04 (0.84 to 1.07). Build and peak-memory ratios did not grow.

Profile (in `docs/epiworld-v1.md`, the stage's note)

Bisected on scenario_03 (10 interleaved replicates): 0.0378 s after #363, 0.0383 s after #364, 0.0381 s after #365, identical epidemics. V-1 and V-2 change nothing; the cost came with the natural-history engine. Against 3d-3b's runner: 0.0253 s versus 0.0375 s.
Callgrind on run(): 444M instructions at 3d-3b, 526M now; reset() (inside run()) went from 79M to 153M, the steps +8M. The reset now makes several passes over the population and allocates vectors of its size at every run (each agent's eligibility for the virus, a label per agent and a pool in assign::detail::draw_counts(), the natural history's per-agent records), plus their page faults.

Per Section 11.7 of epiworld's contact-layers.md, the fix is an epiworld PR into v1-layers before the next C++ phase, followed by another rerun.

🤖 Generated with Claude Code

The runners declare the SEIRH on the virus (stages Susceptible, Exposed (carrier), Infected (infectious), Hospitalized (carrier), Recovered; one infection and four rates, the two out of Infected competing as before), place the initial cases and the vaccine with assign::count(), read the totals by stage, and use Model, Virus and Tool without <>. Pins epiworld d571d3e9 (V-2, UofUEpiBio/epiworld#365). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Both C++ engines at the full design, the stage appended to the history. Attack rates agree; the simulation ratio grew by more than 10% in three rows, profiled to the reset of the natural-history engine (see the stage's note in docs/epiworld-v1.md). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

NH-3 of `natural-history.md` (Section 11): the docs and the model diagrams, written once against the final API (natural histories, variants by id, plain classes). The other half of NH-3, the epiworld-benchmark runners and rerun, is UofUEpiBio/epiworld-benchmark#35: attack rates agree, but the simulation time regressed in three rows; the profile traces it to the natural-history engine's reset (not to V-1 or V-2), to be fixed in an epiworld PR before Phase 4. ## Diagrams drawn from the declarations - `NaturalHistory::draw_mermaid(prefix, stages)`: the stages and transitions as the body of a mermaid flowchart. Infectious, carrier and contact-0 stages get the classes `infectious`, `carrier` and `away`; each transition is an edge labeled with its name (unless the default "From -> To"), its kind and its value (`infection: Transmission probability`, `rate: 1 / Hospitalization period`, `after: lognormal(Incubation period, Incubation period sd), since Rash`, ...), with `-->` for infections and rates, `==>` for durations, `-.->` for external transitions; branches go to each target with its probability; status transitions list their `when()` stages. - `Model::draw_natural_history()`: the whole model, stages and (if any) statuses in their own subgraphs. Needs no run. - `make diagrams` (`script/draw-diagrams.cpp`) writes `include/epiworld/models/<id>.mmd` and `include/measles/<id>.mmd` for every registered model (and SIRLogit, built directly). The hand-drawn files of the old state products are replaced; `surv.mmd` is new. `AGENTS.md` now says to rerun it when a model's declarations change. - Test 12 (diagrams) checks the SEIR's nodes, classes and edges, a statuses subgraph with a `when()` and a `since()`, and that a model without a natural history throws. - The PNGs in `docs/assets/img` are rendered from the new files with the existing `make all` (mermaid.ink); `seirquarantine.png` is new and linked from its header. **`surv.png` is not included**: the render service returned "Service Unavailable" for it; rerun `make -C include/epiworld/models all` to add it. ## Docs - **README**: no longer "a template library" or C++11 (C++17); the features name natural histories, statuses and layers; the two programs' output is the current one (`helloworld.cpp` updated to the code the README shows); "Simulation Steps" and "Agents" describe stages, statuses, the four kinds of transition, the day's order, and one virus per agent. - **`docs/impl`**: - `extending-the-library-with-models.md` rewritten: declaring a natural history (the four kinds, values, durations, branches, samplers, statuses, reactions), the current `ModelSIS`, the run loop, and events (applied in queue order; a transition applies only if the agent is still in a source: not LIFO). - `extending-the-library.md`: custom behaviors as value functions, statuses and reactions; network rewiring through `set_rewiring()`; the global-event and data examples use the current API. - Three pages that were empty are written: `virus-and-tool-distribution.md` (`assign`), `quarantine-isolation-and-contact-tracing.md` (statuses, reactions, layer logs), `events-and-multi-event-handling.md` (what a stage proposes each day, and how competing events resolve). - `core-simulation-models.md`: stages and statuses, diagrams, the registry; the SIRLogit and Surveillance constructors fixed. - `data-collection-and-analysis.md`: counts by (status, stage), `get_today_total_stage()` / `get_today_total_status()`, transitions by name, `get_run_outputs()`; the example that called the database's recorders directly is gone. - `transmission-sampling.md`: which stages are pushed to, in terms of the declarations (no update functions, no RTTI). - `mixing-and-entity-distribution.md` (titled "Mixing by Group"): the overview said a contact matrix's rows sum to 1; it holds expected contacts per day. `distribute_tool_to_groups()` became `assign::group_shares()`. - Template mentions removed from `library-architecture.md`, `performance-optimization.md`, `queueing-system.md`; the models' `@brief`s no longer say "Template for". - Every new code example was compiled and run against these headers. ## Checks `make test WITH_OPENMP=0` in the container: all 137 test cases pass. `helloworld.cpp` and `readme.cpp` run. Runbook (11.6): NH row and "Next" updated; V row marks #365 merged. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

…regression) (#367) Fixes the simulation-time regression found by the NH-3 benchmark rerun (UofUEpiBio/epiworld-benchmark#35), before Phase 4 (runbook 11.7). **Stacked on #366.** Review and merge #366 first; until then this diff also shows #366's commits. Both regenerate `epiworld.hpp`, so stacking avoids conflicts in the amalgam. ## Cause The rerun put the regression in `Model::reset()`, which runs inside `run()`: about 79M instructions at 3d-3b against 153M after NH-2, V-1 and V-2. The daily steps grew only about 8M. Each benchmark run is a fresh process that calls `run()` once, so the extra time came mostly from first touching new memory, more than from the extra instructions. Page faults during `run()` went from 9,350 to 20,348 on scenario 03 (1,000,000 agents). Two sources: - **Per-agent arrays (40 MB at 1M agents).** The natural history kept 10 ints per agent: the day each agent entered each stage (agents × stages), plus six arrays filled at every reset. - **Population-sized vectors at every draw.** Every placement (`assign::*`) made several passes over all agents and allocated vectors the size of the population to place 100 seed cases: - an eligibility mask; - a label for every agent; - a pool of eligible ids built with `push_back`. ## Changes - **Placements write into buffers the model keeps** (`AssignBuffers`, reused between runs and never copied). A draw takes the eligibility mask and appends `(agent, label)` pairs, sorted by agent so they apply in the same order as before. The pool of eligible ids is 32-bit (`runif_index` already takes 32 bits) and is filled without branching. The mixing layer's group draws and `set_initial()` use the same buffers. - **`dist_virus` looks eligibility up in a per-state table** instead of finding each agent's stage. - **Natural-history records are created only when needed.** Each agent's infection count and stage-entry days are kept in a row created the first time it leaves the first stage. An agent without a row has been in the first stage since day 0, with no infection. Dense per-agent state goes from 40 to 8 bytes (`nh_until` and the row index). `get_stage_entered()` and `get_episode()` return the same values as before. - **Status arrays are sized only when the model declares statuses.** `nh_status_since` was written but never read, so it is removed. - **`assign::agents()` removes duplicate ids when the placement is built.** The old per-agent vector removed them implicitly. ## Results Scenario 03 of the benchmark (1,000,000 agents, 100 days). Twelve seeds, the three engines run alternately in the `epiworld-dev` container: | Engine | Reset (instructions) | `run()` (instructions) | Page faults in `run()` | Simulate, median (s) | |---|---|---|---|---| | 3d-3b (07ed7a0) | 78.7M | 444M | 9,350 | 0.028 | | V-2 (d571d3e) | 153.0M | 526M | 20,348 | 0.060 | | This PR | 91.9M | 466M | 10,536 | 0.030 | - **Same epidemics:** final counts are identical across the three engines for every seed. The test suite passes unchanged; internal changes leave model-level results identical. - **Noisy timings:** wall-clock times on this Mac are noisy (V-2 measured 0.046 s to 0.060 s across sessions). The instruction and page-fault counts are exact. - **Not addressed:** the remaining gap is the daily steps (+8M instructions) and a few more passes over the agents at reset. ## Tests - **Test 09** gets one end-to-end case: `assign::agents()` with unsorted, repeated ids gives each agent the tool once. It fails without the deduplication. - **The rest of the suite is unchanged.** It already compares histories, stage-entry days (`52b`), statuses and the mixing groups. - Built and tested in `epiworld-dev` with `WITH_OPENMP=0`; `epiworld.hpp` regenerated. The benchmark rerun pinned to this PR comes next (runbook 11.7). The runbook table is updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Pins 384c89fe (UofUEpiBio/epiworld#367). **Stacked on #35** (the runners as natural histories, and the `nh-2-v2` stage). Merge #35 first; until then this diff also shows its commits. The runners are unchanged; `EPIWORLD_V1_REF` moves to #367's merge. `make container-check` and `make container-smoke` pass. `STAGE=reset-fix make container-v1-stage` ran alone, with all 1,280 runs ok. ## The stage's rows | Scenario | Agents | Simulate (s): 0.x, 1.0 | Ratio (3d-3b, nh-2-v2, now) | Build ratio | Peak ratio | Attack rate | |---|---|---|---|---|---|---| | 00 | 10,000 | 0.00488, 0.00535 | 1.01, 1.08, 1.10 | 0.81 | 0.80 | 0.385, 0.385 | | 00 | 100,000 | 0.00958, 0.0109 | 1.03, 1.31, 1.13 | 0.77 | 0.77 | 0.060, 0.060 | | 01 | 10,000 | 0.00245, 0.00263 | 1.00, 1.02, 1.07 | 0.82 | 0.90 | 0.119, 0.119 | | 01 | 100,000 | 0.0103, 0.0097 | 0.97, 0.98, 0.94 | 0.78 | 0.85 | 0.014, 0.014 | | 02 | 10,000 | 0.00244, 0.00264 | 1.00, 1.09, 1.08 | 0.85 | 0.92 | 0.119, 0.119 | | 02 | 100,000 | 0.011, 0.00998 | 0.96, 0.99, 0.90 | 0.75 | 0.86 | 0.014, 0.014 | | 03 | 1,000,000 | 0.0292, 0.0274 | 0.91, 1.56, 0.94 | 0.76 | 0.72 | 0.006, 0.006 | | 04 | 165,865 | 0.249, 0.231 | 0.84, 1.07, 0.93 | 0.71 | 0.57 | 0.622, 0.622 | - **Results agree.** The attack rates are the same everywhere. - **No ratio grew by more than 10% from the previous stage.** - **1,000,000 agents is fixed:** 1.56 back to 0.94 (3d-3b: 0.91). Page faults during `run()` went from 20,348 to 10,536 (3d-3b: 9,350). - **scenario_00 at 100,000 agents and GeoPops are not fixed.** Their ratios fell only because this run's 0.x times were slower. Their 1.0 times did not move from `nh-2-v2` (0.0109 s; 0.231 s) and stay about 15% above 3d-3b (0.00947 s; 0.202 s). Profiled at 100,000 agents (`docs/epiworld-v1.md`): - interleaved, 0.00922 s at 3d-3b against 0.01026 s now; - `run()` is 121.5M instructions against 130.0M, with fewer page faults; - the difference is the natural history's daily step (`nh_step()`, `nh_propose()`, `Value::eval()`), which grows with the agents that move, not the reset. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Pins 3944abd3 (UofUEpiBio/epiworld#368). **Stacked on #36** (the `reset-fix` stage), itself stacked on #35. Merge #35 and #36 first; until then this diff also shows their commits. The runners are unchanged; `EPIWORLD_V1_REF` moves to #368's merge. `make container-check` and `make container-smoke` pass. `STAGE=step-trim make container-v1-stage` ran alone, with all 1,280 runs ok. ## The stage's rows | Scenario | Agents | Simulate (s): 0.x, 1.0 | Ratio (3d-3b, reset-fix, now) | Build ratio | Peak ratio | Attack rate | |---|---|---|---|---|---|---| | 00 | 10,000 | 0.00473, 0.00507 | 1.01, 1.10, 1.07 | 0.82 | 0.80 | 0.385, 0.385 | | 00 | 100,000 | 0.00958, 0.0108 | 1.03, 1.13, 1.13 | 0.73 | 0.77 | 0.060, 0.060 | | 01 | 10,000 | 0.00242, 0.00254 | 1.00, 1.07, 1.05 | 0.84 | 0.90 | 0.119, 0.119 | | 01 | 100,000 | 0.0112, 0.0101 | 0.97, 0.94, 0.90 | 0.77 | 0.85 | 0.014, 0.014 | | 02 | 10,000 | 0.00246, 0.0026 | 1.00, 1.08, 1.06 | 0.84 | 0.92 | 0.119, 0.119 | | 02 | 100,000 | 0.011, 0.00995 | 0.96, 0.90, 0.90 | 0.78 | 0.86 | 0.014, 0.014 | | 03 | 1,000,000 | 0.0382, 0.0339 | 0.91, 0.94, 0.89 | 0.63 | 0.72 | 0.006, 0.006 | | 04 | 165,865 | 0.269, 0.257 | 0.84, 0.93, 0.95 | 0.79 | 0.57 | 0.622, 0.622 | - **Results agree.** The attack rates are the same everywhere. - **No ratio grew by more than 10% from the previous stage.** The largest change is GeoPops', 0.93 to 0.95. - **This run cannot see #368's gain.** #368 removed about 4% of `run()`'s instructions at 100,000 agents in scenario_00, within a stage's noise: - The 1.0 time there did not move (0.0109 to 0.0108 s), and the ratio stays 1.13 (3d-3b: 1.03). - On GeoPops both engines were slower in this run. - Compared directly (16 interleaved seeds), the runners take 0.00893 s at 3d-3b and 0.00926 s at #368 on scenario_00 at 100,000 agents, and 0.156 s and 0.179 s on GeoPops. - **The remaining cost is per infected agent, so it doesn't grow with density or layers.** Callgrind on `run()` with a 32 KiB L1 and an 8 MiB last-level cache, #368 against 3d-3b: - GeoPops: +3.5% instructions. - A synthetic population of the same size with twice GeoPops' mean degree: +2.2% instructions and about +4% data misses, as one network and as three network layers. This is a check outside the benchmark. - The extra work is the same in absolute terms on both networks (about 300 instructions per infection), so GeoPops, sparse with a 62% attack rate, is the worst case. - Details are in `docs/epiworld-v1.md`. ## Also - **#36 is fixed too.** It had committed `results/environments/` and `run-manifest.json` from the smoke run (14 runs) in place of the published full-run records (2,200). `make container-smoke` rewrites those files, and `v1-stage` then restores the smoke version. #36 now restores them from `v1-layers` (8714725), and this PR leaves them untouched. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

gvegayon and others added 2 commits November 7, 2026 01:51

This was referenced Nov 7, 2026

NH-3: docs, and model diagrams drawn from the declarations UofUEpiBio/epiworld#366

Merged

Reset without population-sized allocations (fixes the NH-3 benchmark regression) UofUEpiBio/epiworld#367

Merged

This was referenced Nov 7, 2026

Rerun epiworld-v1 at the reset fix (UofUEpiBio/epiworld#367) #36

Merged

Rerun epiworld-v1 at the step trim (UofUEpiBio/epiworld#368) #37

Merged

gvegayon merged commit 3b4a5ad into v1-layers Nov 9, 2026
2 checks passed

gvegayon deleted the port-v2-no-tseq branch November 9, 2026 02:36

gvegayon added a commit that referenced this pull request Nov 9, 2026

Merge v1-layers (#35 squash-merged) into port-reset-no-realloc

d266751

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Navigation Menu

Repository navigation

Rerun epiworld-v1 at NH-2, V-1, V-2 (UofUEpiBio/epiworld#365) - #35

Rerun epiworld-v1 at NH-2, V-1, V-2 (UofUEpiBio/epiworld#365)#35
gvegayon merged 2 commits into
v1-layersfrom
port-v2-no-tseq

gvegayon commented Nov 7, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

gvegayon commented Nov 7, 2026

Runner change

The stage's rows

Profile (in docs/epiworld-v1.md, the stage's note)

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Profile (in `docs/epiworld-v1.md`, the stage's note)