Repository navigation
Conversation
Model::draw_natural_history() draws a model's stages and statuses (a mermaid flowchart) from its declarations, through NaturalHistory::draw_mermaid(). `make diagrams` (script/draw-diagrams.cpp) writes the .mmd file of every registered model with it; the hand-drawn diagrams of the old state products are replaced, and the PNGs re-rendered (surv.png still to render). Test 12 checks the drawing. The README and docs/impl are written against the final API: natural histories (stages, the four kinds of transition, statuses, reactions), assign, variants by id and plain classes. Three empty pages are written (placements, statuses and tracing, events); stale examples, constructor signatures and the contact-matrix overview are fixed. AGENTS.md: rerun `make diagrams` when a model's declarations change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## v1-layers #366 +/- ##
============================================
Coverage ? 77.07%
============================================
Files ? 70
Lines ? 8005
Branches ? 1569
============================================
Hits ? 6170
Misses ? 1336
Partials ? 499
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Member
Author
|
Review finding: docs/assets/img/measlesmixing.png is not a PNG; it is a 19-byte plain-text response, "Service Unavailable". This makes the documentation render a broken image. Please rerun image rendering after the service is available (or omit the generated asset until then) and commit a valid PNG. |
gvegayon
added a commit
that referenced
this pull request
Nov 7, 2026
…regression) (#367) Fixes the simulation-time regression found by the NH-3 benchmark rerun (UofUEpiBio/epiworld-benchmark#35), before Phase 4 (runbook 11.7). **Stacked on #366.** Review and merge #366 first; until then this diff also shows #366's commits. Both regenerate `epiworld.hpp`, so stacking avoids conflicts in the amalgam. ## Cause The rerun put the regression in `Model::reset()`, which runs inside `run()`: about 79M instructions at 3d-3b against 153M after NH-2, V-1 and V-2. The daily steps grew only about 8M. Each benchmark run is a fresh process that calls `run()` once, so the extra time came mostly from first touching new memory, more than from the extra instructions. Page faults during `run()` went from 9,350 to 20,348 on scenario 03 (1,000,000 agents). Two sources: - **Per-agent arrays (40 MB at 1M agents).** The natural history kept 10 ints per agent: the day each agent entered each stage (agents × stages), plus six arrays filled at every reset. - **Population-sized vectors at every draw.** Every placement (`assign::*`) made several passes over all agents and allocated vectors the size of the population to place 100 seed cases: - an eligibility mask; - a label for every agent; - a pool of eligible ids built with `push_back`. ## Changes - **Placements write into buffers the model keeps** (`AssignBuffers`, reused between runs and never copied). A draw takes the eligibility mask and appends `(agent, label)` pairs, sorted by agent so they apply in the same order as before. The pool of eligible ids is 32-bit (`runif_index` already takes 32 bits) and is filled without branching. The mixing layer's group draws and `set_initial()` use the same buffers. - **`dist_virus` looks eligibility up in a per-state table** instead of finding each agent's stage. - **Natural-history records are created only when needed.** Each agent's infection count and stage-entry days are kept in a row created the first time it leaves the first stage. An agent without a row has been in the first stage since day 0, with no infection. Dense per-agent state goes from 40 to 8 bytes (`nh_until` and the row index). `get_stage_entered()` and `get_episode()` return the same values as before. - **Status arrays are sized only when the model declares statuses.** `nh_status_since` was written but never read, so it is removed. - **`assign::agents()` removes duplicate ids when the placement is built.** The old per-agent vector removed them implicitly. ## Results Scenario 03 of the benchmark (1,000,000 agents, 100 days). Twelve seeds, the three engines run alternately in the `epiworld-dev` container: | Engine | Reset (instructions) | `run()` (instructions) | Page faults in `run()` | Simulate, median (s) | |---|---|---|---|---| | 3d-3b (07ed7a0) | 78.7M | 444M | 9,350 | 0.028 | | V-2 (d571d3e) | 153.0M | 526M | 20,348 | 0.060 | | This PR | 91.9M | 466M | 10,536 | 0.030 | - **Same epidemics:** final counts are identical across the three engines for every seed. The test suite passes unchanged; internal changes leave model-level results identical. - **Noisy timings:** wall-clock times on this Mac are noisy (V-2 measured 0.046 s to 0.060 s across sessions). The instruction and page-fault counts are exact. - **Not addressed:** the remaining gap is the daily steps (+8M instructions) and a few more passes over the agents at reset. ## Tests - **Test 09** gets one end-to-end case: `assign::agents()` with unsorted, repeated ids gives each agent the tool once. It fails without the deduplication. - **The rest of the suite is unchanged.** It already compares histories, stage-entry days (`52b`), statuses and the mixing groups. - Built and tested in `epiworld-dev` with `WITH_OPENMP=0`; `epiworld.hpp` regenerated. The benchmark rerun pinned to this PR comes next (runbook 11.7). The runbook table is updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
gvegayon
added a commit
to UofUEpiBio/epiworld-benchmark
that referenced
this pull request
Nov 9, 2026
Pins d571d3e978b54d0e3a8e7a82147dcf39103ba124 (UofUEpiBio/epiworld#365). The rerun for epiworld's NH-1 and NH-2 (UofUEpiBio/epiworld#363), V-1 (#364) and V-2 (#365), the second half of NH-3 (the docs half is UofUEpiBio/epiworld#366). ## Runner change The five `epiworld-v1` runners declare the SEIRH as a natural history on the virus: stages Susceptible, Exposed (carrier), Infected (infectious), Hospitalized (carrier), Recovered; one infection and four rates (the two out of Infected compete, as `new_state_update_transition()` did). The initial cases and the vaccine are placed with `assign::count()`, the totals read by stage, and the classes are plain (`Model`, not `Model<>`). Scenario 02 reads the transition matrix's stages (a state is now a (status, stage) pair). `make container-check` and `make container-smoke` pass; `STAGE=nh-2-v2 make container-v1-stage` ran alone (an idle container from another project was up, at 0.2% CPU). ## The stage's rows | Stage | epiworld-v1 | Scenario | Agents | Simulate (s): 0.x, 1.0 | Ratio | Build (s): 0.x, 1.0 | Ratio | Peak (MiB): 0.x, 1.0 | Ratio | Attack rate: 0.x, 1.0 | |---|---|---|---|---|---|---|---|---|---|---| | nh-2-v2 | 1.0.0+gd571d3e | 00 | 10,000 | 0.00488, 0.00529 | 1.08 | 0.00137, 0.00109 | 0.79 | 8, 7 | 0.81 | 0.385, 0.385 | | nh-2-v2 | 1.0.0+gd571d3e | 00 | 100,000 | 0.00821, 0.0108 | 1.31 | 0.013, 0.01 | 0.77 | 38, 33 | 0.87 | 0.060, 0.060 | | nh-2-v2 | 1.0.0+gd571d3e | 01 | 10,000 | 0.00253, 0.00257 | 1.02 | 0.00127, 0.00104 | 0.82 | 8, 7 | 0.91 | 0.119, 0.119 | | nh-2-v2 | 1.0.0+gd571d3e | 01 | 100,000 | 0.00859, 0.00838 | 0.98 | 0.0123, 0.00977 | 0.80 | 43, 39 | 0.90 | 0.014, 0.014 | | nh-2-v2 | 1.0.0+gd571d3e | 02 | 10,000 | 0.00236, 0.00258 | 1.09 | 0.00126, 0.00103 | 0.82 | 9, 8 | 0.95 | 0.119, 0.119 | | nh-2-v2 | 1.0.0+gd571d3e | 02 | 100,000 | 0.00893, 0.00885 | 0.99 | 0.0132, 0.0103 | 0.78 | 44, 40 | 0.91 | 0.014, 0.014 | | nh-2-v2 | 1.0.0+gd571d3e | 03 | 1,000,000 | 0.0259, 0.0404 | 1.56 | 0.134, 0.112 | 0.84 | 352, 294 | 0.84 | 0.006, 0.006 | | nh-2-v2 | 1.0.0+gd571d3e | 04 | 165,865 | 0.218, 0.232 | 1.07 | 0.0195, 0.0139 | 0.71 | 90, 52 | 0.57 | 0.622, 0.622 | **Attack rates agree** in every row. **The simulation ratio grew by more than 10%** from 3d-3b in three rows: scenario_00 at 100,000 (1.03 to 1.31), scenario_03 (0.91 to 1.56) and scenario_04 (0.84 to 1.07). Build and peak-memory ratios did not grow. ## Profile (in `docs/epiworld-v1.md`, the stage's note) - Bisected on scenario_03 (10 interleaved replicates): 0.0378 s after #363, 0.0383 s after #364, 0.0381 s after #365, identical epidemics. **V-1 and V-2 change nothing**; the cost came with the natural-history engine. Against 3d-3b's runner: 0.0253 s versus 0.0375 s. - Callgrind on `run()`: 444M instructions at 3d-3b, 526M now; `reset()` (inside `run()`) went from 79M to 153M, the steps +8M. The reset now makes several passes over the population and allocates vectors of its size at every run (each agent's eligibility for the virus, a label per agent and a pool in `assign::detail::draw_counts()`, the natural history's per-agent records), plus their page faults. Per Section 11.7 of epiworld's `contact-layers.md`, the fix is an epiworld PR into `v1-layers` before the next C++ phase, followed by another rerun. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
gvegayon
added a commit
that referenced
this pull request
Nov 10, 2026
…ain diagrams, NH-5 next (#370) Closes the natural-history plan after NH-4 (all four wrappers merged on fe2385a) with the decisions of 2026-11-09, makes the policy change they call for, and simplifies the model diagrams. ## Decisions recorded (`natural-history.md`, 16-21) - **16. `rule()` and the deadline calendar are deferred.** No model needs either, and adding them later breaks nothing; the calendar joins #265. - **17. Accessors by margin** (`Margin::pair | stage | status`, `get_agents_stages()` / `get_agents_statuses()`, transitions per strain) replace the ones that still expose the engine's cells. Built in NH-5, with the database's split by disease. - **18. Measles quarantine exemption by tool name** (open question 1). Implemented here (below). - **19. PEP acts on the current episode** (open question 3), and its tool stays. PEP's nested `events_run()` stays too: it attaches the tool PEP just gave before the efficacy draw. - **20. Rash onset stays the measles landmark** (open question 2). The names stay `NaturalHistory` and `Statuses` (open question 4). - **21. Several diseases per model, one infection per agent, in 1.0.0 (NH-5, before Phase 4).** Its plan is the next PR. Settled so far: a stage per disease per agent and one status; one database with histories kept per disease (long tables carry a `disease` column; matrices and per-agent stages need the disease when there are several); an explicit `strain()`; tools and stage references name their disease when there are several; pull only with several diseases; a removal in one disease is a built-in "Removed (other disease)" stage in the others. ## Code - **`ModelMeasles::set_quarantine_exempt_tools()` / `get_quarantine_exempt_tools()`**, default `{"MMR", "PEP MMR"}`. The three measles models exempt a traced or exposed contact only if they hold one of these tools, whether or not it protects them: documented MMR, routine or as PEP, is presumptive evidence of immunity, and immunoglobulin is not (CDC). Before this, holding any tool exempted a contact, IG recipients included. **Results change** for runs in which agents hold tools other than MMR (PEP with IG, or tools the user adds). `ModelSEIRQuarantine` keeps "any tool". - **Diagrams without transition details.** `NaturalHistory::draw_mermaid()` (and so `Model::draw_natural_history()` and `make diagrams`) draws one unlabeled arrow per pair of stages or statuses that some transition joins. Kinds, values and names are left out: they are the model's parameters. `draw_mermaid()` loses its second argument. All `.mmd` files are regenerated. - **The render targets** (`include/measles/Makefile`, `include/epiworld/models/Makefile`) no longer use `xargs` (macOS refuses the long command line) or `base64 -w0`, fail on an HTTP error instead of saving it as a PNG (how the 19-byte `measlesmixing.png` got into #366), retry, and pause between requests. - `add_virus()`'s error for a second natural history now points to NH-5. ## Tests - New `18k-measles-mixing-quarantine-exempt.cpp`: with default settings, contacts holding a tool that is not on the list are quarantined; once their tool joins the list, nobody is. - `12-diagrams.cpp` rewritten for the unlabeled arrows (two transitions between the same statuses draw one arrow). - `make test WITH_OPENMP=0` in the devcontainer: every suite passes. The two tests above were also built standalone with `-DEPI_DEBUG`. ## Runbook (`contact-layers.md`) - §11.6: NH-4 is merged everywhere (#231, #44, #30, #72), as are #369 and the benchmark reruns #35-#37. Next: the NH-5 plan, then its engine, database and wrapper ports, then epiworldRShiny, then Phase 4. - Phase 6 gains a release item: measles requires the released epiworldR 1.0.0, with NH-4, before it is published (measles#72 keeps `epiworldR (>= 1.0.0)`). ## Left for the maintainer - **The pictures in `docs/assets/img` are stale.** `measlesmixing.png` here is a real image again, but it was rendered from the labeled diagram, as were the others; `surv.png` was never rendered. Rendering sends the diagrams to mermaid.ink, which the agent's session was not allowed to do: run `make -C include/measles all` and `make -C include/epiworld/models all`, and commit the PNGs. ## Wrappers - measles and epiworldjs bind `set_quarantine_exempt_tools()` in their next port. - **The `downstream` jobs for epiworldR and epiworldpy fail, as expected from the diagram change** (one test each; a port fixes both, no C++ API change needed): - epiworldR `test-model-diagram.R:197` expects `s1 ==>` (the arrow durations had); every arrow is now `-->`. - epiworldpy `test_natural_history.py:133` expects the transition name "Vaccination" in the diagram; names are no longer drawn. - measles and epiworldjs pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
NH-3 of
natural-history.md(Section 11): the docs and the model diagrams, written once against the final API (natural histories, variants by id, plain classes). The other half of NH-3, the epiworld-benchmark runners and rerun, is UofUEpiBio/epiworld-benchmark#35: attack rates agree, but the simulation time regressed in three rows; the profile traces it to the natural-history engine's reset (not to V-1 or V-2), to be fixed in an epiworld PR before Phase 4.Diagrams drawn from the declarations
NaturalHistory::draw_mermaid(prefix, stages): the stages and transitions as the body of a mermaid flowchart. Infectious, carrier and contact-0 stages get the classesinfectious,carrierandaway; each transition is an edge labeled with its name (unless the default "From -> To"), its kind and its value (infection: Transmission probability,rate: 1 / Hospitalization period,after: lognormal(Incubation period, Incubation period sd), since Rash, ...), with-->for infections and rates,==>for durations,-.->for external transitions; branches go to each target with its probability; status transitions list theirwhen()stages.Model::draw_natural_history(): the whole model, stages and (if any) statuses in their own subgraphs. Needs no run.make diagrams(script/draw-diagrams.cpp) writesinclude/epiworld/models/<id>.mmdandinclude/measles/<id>.mmdfor every registered model (and SIRLogit, built directly). The hand-drawn files of the old state products are replaced;surv.mmdis new.AGENTS.mdnow says to rerun it when a model's declarations change.when()and asince(), and that a model without a natural history throws.docs/assets/imgare rendered from the new files with the existingmake all(mermaid.ink);seirquarantine.pngis new and linked from its header.surv.pngis not included: the render service returned "Service Unavailable" for it; rerunmake -C include/epiworld/models allto add it.Docs
helloworld.cppupdated to the code the README shows); "Simulation Steps" and "Agents" describe stages, statuses, the four kinds of transition, the day's order, and one virus per agent.docs/impl:extending-the-library-with-models.mdrewritten: declaring a natural history (the four kinds, values, durations, branches, samplers, statuses, reactions), the currentModelSIS, the run loop, and events (applied in queue order; a transition applies only if the agent is still in a source: not LIFO).extending-the-library.md: custom behaviors as value functions, statuses and reactions; network rewiring throughset_rewiring(); the global-event and data examples use the current API.virus-and-tool-distribution.md(assign),quarantine-isolation-and-contact-tracing.md(statuses, reactions, layer logs),events-and-multi-event-handling.md(what a stage proposes each day, and how competing events resolve).core-simulation-models.md: stages and statuses, diagrams, the registry; the SIRLogit and Surveillance constructors fixed.data-collection-and-analysis.md: counts by (status, stage),get_today_total_stage()/get_today_total_status(), transitions by name,get_run_outputs(); the example that called the database's recorders directly is gone.transmission-sampling.md: which stages are pushed to, in terms of the declarations (no update functions, no RTTI).mixing-and-entity-distribution.md(titled "Mixing by Group"): the overview said a contact matrix's rows sum to 1; it holds expected contacts per day.distribute_tool_to_groups()becameassign::group_shares().library-architecture.md,performance-optimization.md,queueing-system.md; the models'@briefs no longer say "Template for".Checks
make test WITH_OPENMP=0in the container: all 137 test cases pass.helloworld.cppandreadme.cpprun.Runbook (11.6): NH row and "Next" updated; V row marks #365 merged.
🤖 Generated with Claude Code