Skip to content

[PROBE: energy] Surface temperature and upward longwave consistency - #39

Open
xinlan-technology wants to merge 15 commits into
Flood-Lab:mainfrom
xinlan-technology:probe/energy-radiation-consistency
Open

[PROBE: energy] Surface temperature and upward longwave consistency#39
xinlan-technology wants to merge 15 commits into
Flood-Lab:mainfrom
xinlan-technology:probe/energy-radiation-consistency

Conversation

@xinlan-technology

@xinlan-technology xinlan-technology commented Sep 11, 2026

Copy link
Copy Markdown

Closes #22

What this probe asserts

For a uniform, opaque, snow-free gray surface with the emissivity supplied in static.json, the surface temperature a model reports and the upward longwave it reports must describe one surface at every hourly instant:

r = rlus - [eps * sigma * ts^4 + (1 - eps) * rlds]
abs(r) <= max(0.005 * abs(rlus), 0.5 W m-2)

The bound is 0.5 percent of the model's own reported upward flux, not net radiation: two outputs of one model are held to one equation, and for a 290 K surface under a 300 W m-2 sky with eps 0.98, the reflected sky is about 1.5 percent of the upward flux, so a model that drops it passes at 5 percent and fails here. Every scored step must pass, so nothing cancels across hours; interval means are out of scope because mean(T^4) is not mean(T)^4. Net radiation stays a prescribed forcing and is not reconciled with the longwave components, as the proposal scoped. Passing establishes consistency between two outputs, not temperature accuracy.

The three interface questions in the proposal are answered as follows. ts is declared under a new optional diagnostics output category (schemas, spec.py, protocol.py), so a temperature is required like a flux but never summed into a water budget as a state would be. Emissivity comes only from static.json, drawn per seed in [0.95, 0.99] and held fixed. rlds is a generator column like rn.

needs_forcing/needs_static declare mandatory inputs; uses_forcing/uses_static declare inputs consumed whenever supplied. Probe requirements accept either declaration. The three references use optional declarations with documented defaults, so they also remain compatible with other suitable probes.

Discrimination

Reference model Expected Criterion that catches it
reference_radiative PASS n/a
reference_air_emitter FAIL radiative_identity
reference_no_reflection FAIL radiative_identity

reference_radiative is reference_coupled with a skin: the temperature is diagnosed from the sensible heat flux through a fixed bulk conductance, and the upward longwave is that skin's emission plus the reflected sky. Its water and energy columns are reference_coupled's exactly. The two negatives are copies of it with one line changed, emitting at the air temperature or dropping the reflected sky, and tests/test_radiation_references.py pins that they differ from it in rlus alone.

Margins, measured through the real adapter path by scripts/radiation_margins.py and judged seed by seed: on the five gate seeds, twenty additional seeds, one drawn seed near the emissivity ceiling, and the gate seeds with eps forced to 0.99 and 0.95, the positive control's worst step is below 1e-12 of the tolerance (rounding, of order 1e-13 or smaller, varying with the numpy and pandas installed) and each negative control's worst step is at least 1.95 times it (minimum over seeds of the per-seed maximum; the floor is the forced 0.99 group). reference_air_emitter violates 679 to 690 of 720 steps per seed, reference_no_reflection all 720. The probe README carries the table.

Validation

  • Merged upstream/main at 12cbaaa (N/A verdicts, summa, cwatm, lisflood, mass/human-abstraction, LF line endings): ht validate 21 probes, 38 models.
  • pytest -q: 500 passed locally, including the line-ending check; the GitHub run on the new head awaits a maintainer's approval.
  • ht gate: passes locally for all 21 probes; the new probe's gate is 15 adapter invocations in about a second.
  • ht verify-adapter passes for the three reference models, 768 rows each.
  • scripts/radiation_margins.py: 36 cases, all separated as declared, exit 0.
  • ht run --gate-seeds --csv models/result.csv archived the nine evaluated models as N/A (INCOMPLETE) for missing rlus and ts, so no standing changes. No physical-violation claim is made for any of them.
  • README, ROADMAP, CONTRIBUTORS, CITATION.cff, AGENTS.md (the rlus and ts rows), the probe-writing guide and the three-language site, flowchart included, are synchronised; tests/test_docs_in_sync.py passes.

Checklist

  • There is an accepted proposal issue and this PR closes it
  • authors in probe.yaml names every author with name, affiliation and orcid, matching the proposal; CONTRIBUTORS.md and CITATION.cff are updated
  • ht validate passes
  • ht gate --probe energy/radiation-consistency passes
  • The generator is deterministic given a seed and commits no data
  • The probe runs in under a minute on a two-core runner
  • Tolerance and denominator are justified in probe.yaml
  • I constructed models that report plausible temperatures and radiation without describing one surface, and radiative_identity catches both

…s LF

Upstream landed wflow_sbm (Flood-Lab#36), the antecedent storm-window fix (Flood-Lab#35), LF
line endings for every text file with a workflow check that enforces them
(Flood-Lab#37, Flood-Lab#38) and the docker build fix (Flood-Lab#42). The conflicts were the three
places that count probes: the README's models table, the site's models rows
in three languages, and the archive. wflow_sbm's rows take the new probe
into their totals, 14 of 20, and its energy wording widens from the three
heat-flux probes to the four probes that need an energy output, since it
reports neither heat fluxes nor a surface temperature. Its INCOMPLETE row
on energy/radiation-consistency is appended by ht run like the others. The
archive keeps upstream's renormalised rows followed by this branch's, and
the index carries no CRLF.
The three reference adapters take the 644 mode of the coupled family they
copy. probe.yaml says again why the tolerance is 0.5 percent of the
reported upward flux, which the pull request template asks to be justified
there and a trimming pass had reduced to one line. The README's probes
paragraph counts four of the five energy probes as needing an energy
output, since pet-consistency needs none. The overflow guard in
radiative_identity keeps both of its checks, now with a comment that says
what each one catches: a finite but absurd temperature overflows the
fourth power, and a finite but absurd rel_tol overflows the allowance, and
either would otherwise slip through the comparison as a pass.
The probe README no longer counts the archived models, which the merge had
made six; wflow_sbm's own results table takes the INCOMPLETE row this
branch archives for it, with its energy wording widened to the four probes
that need an energy output. radiative_identity drops a step-mean residual
that nothing read and that sat oddly in a criterion whose point is that
steps are not averaged. The criterion's unit tests lose a timestep argument
no case set and a registration test that ht validate already performs.
The mean absolute residual returns to radiative_identity's diagnostics: it
never decides a verdict, but it goes into every report and is the first
number to look at when a model fails on a handful of steps. The overflow
comment now names the two ways a bad value would pass rather than fail: an
infinite allowance makes every slack zero, and an infinite residual over an
infinite allowance is NaN. probe.yaml's tolerance note states the surface
and sky the 1.5 percent example depends on, and says what the floor does,
which is to stop the bound shrinking towards zero, not to keep it finite.
Upstream merged mass/human-abstraction (Flood-Lab#32) with its author, its reference
model and its archive rows. The conflicts were again the places that count
probes: the README's probes paragraph, models table and status line, the
site's models rows in three languages, the archive, and CITATION.cff, where
both branches appended an author. Every count now reads twenty-one, the
evaluated models' denominators move with it, the energy wording stays at the
four probes that need an energy output, the citation keeps its order of
first merged probe, and the archive carries upstream's rows followed by this
branch's. The suite is 437 tests, 21 probes and 35 models, all passing.

@licm13 licm13 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @xinlan-technology, thanks for the support.

The radiative identity, CF variable semantics, numerical guards, and the construction of the positive and negative controls are sound. I reproduced the probe gate, adapter verification, validation, and the 36-case margin scan. The positive control passes and both negative controls fail with adequate margins.

I am requesting changes because the probe’s compatibility contract does not currently include the boundary inputs on which the verdict is conditional. The criterion evaluates native ts and rlus against case-provided rlds and eps, but the probe declares only the two required outputs. A model that emits both outputs while using an internal downward-longwave estimate or its own emissivity is therefore considered compatible and can receive VIOLATION against values it never consumed. That is an input incompatibility, not evidence of a radiative-identity violation.

Please extend the probe/model compatibility contract so this probe requires declared support for rlds and for the case-provided static emissivity, and add a regression test showing that a model which emits ts and rlus but does not accept those inputs is classified as INCOMPATIBLE.

Non-blocking documentation issue: the README reports an exact positive-control maximum of 3e-14, whereas the margin script produces approximately 1.8e-132.1e-13 under other dependency versions allowed by pyproject.toml. The result remains safely negligible, but the table should use a stable bound such as <1e-12 or state the environment used to generate the numbers.

…be re-archived

Upstream stopped scoring a probe that cannot be put to a model: its verdict
is N/A and a model's passes are counted out of the probes that scored it.
That leaves every standing untouched by this probe, which every evaluated
model is N/A on, so the README and site rows are upstream's with the probe
rows and the flowchart entry laid back on top. The archive rows this branch
had written as FAIL (INCOMPLETE) are regenerated by ht run under the new
vocabulary for all eight evaluated models, summa and cwatm included, at the
versions upstream archived last. wflow_sbm's card takes the N/A row for
this probe. The diagnostics test expects NOT_SCORED where it expected FAIL.
Review of Flood-Lab#39 found a gap in the compatibility contract: the criterion
judges a model's ts and rlus against the case's rlds and eps, but the probe
only required the two outputs, so a model that emitted both while using its
own downward longwave or its own emissivity counted as compatible and could
be scored VIOLATION against values it never read. A probe can now name the
case-supplied inputs its verdict rests on, requires.forcing and
requires.static, and a model declares what it consumes in needs_forcing and
a new needs_static; one that lacks a required declaration is N/A with
reason INCOMPATIBLE. The radiation probe requires rlds and eps, its three
reference models declare them, and a regression test runs the positive
control with those declarations removed and gets INCOMPATIBLE, not a
verdict. Existing probes require nothing, so nothing else changes.

The probe README bounds the positive control's residual by 1e-12 rather
than quoting 3e-14, which the reviewer measured at 2e-13 under other
dependency versions the project allows.
…them

Review of the declaration contract found two edges. A model that declares
it reads a static.json key the case does not supply used to reach the
adapter and die there with a KeyError, reported as FAIL (ERROR); the
compatibility check now asks for static keys what it already asked for
forcing columns, and such a case is N/A (INCOMPATIBLE) before anything
runs. And verify-adapter had started refusing reference_coupled on the
radiation probe because the scoring requires rlds and eps to be declared,
though the adapter runs on that case perfectly well; the smoke test now
asks only whether the model can be fed, and leaves what a verdict requires
to run_probe. Both edges carry a regression test, and the probe's archive
rows are regenerated by the final harness.
… card

The final review found the contract's wording and its use disagreeing:
the schema, AGENTS.md and the template defined needs_static as every
static.json key an adapter reads, while the three reference manifests
declare only the emissivity, as every other manifest declares nothing. The
wording now says what the key is for: the static.json keys a probe may
require a model to have read. The README's verdict vocabulary names the
new cause of INCOMPATIBLE, the smoke-test paragraphs name static inputs
beside forcing, the probe template's guidance names the requires keys the
branch added, and the cwatm and summa cards, which arrived with the second
merge, take the N/A row and the sentence wflow_sbm's card already had.
The probe README gives the positive residual as an order of magnitude.
@xinlan-technology

xinlan-technology commented Sep 12, 2026

Copy link
Copy Markdown
Author

@licm13 Thanks for the review, both points are addressed in 17c2fbb.

Compatibility contract. The probe now names the case inputs its verdict rests on, requires.forcing: [rlds] and requires.static: [eps], matched against the manifest's needs_forcing and a new needs_static. A model that emits ts and rlus without declaring those inputs is N/A (INCOMPATIBLE), never a VIOLATION candidate; test_a_model_that_does_not_consume_the_sky_or_the_emissivity_is_not_judged pins that by stripping the declarations from the positive control. The check also runs the other way (a declared static key the case lacks is INCOMPATIBLE before the adapter runs), and ht verify-adapter asks only whether the model can be fed, so reference_coupled still smoke-tests here. Existing probes require nothing, so nothing else changes.

Residual. The README now bounds the positive control by < 1e-12 and notes that the value is rounding that varies with the numpy and pandas installed.

The branch is merged with main at 89f2f14; the eight evaluated models are archived as N/A (INCOMPLETE) on this probe, so no standing changes. pytest -q 486 passed, ht validate 21 probes / 37 models, ht gate passes for every probe, scripts/radiation_margins.py exits 0.

@licm13 licm13 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @xinlan-technology, impressively efficient turnaround — both requested changes have been addressed!

The probe now explicitly declares that its verdict depends on the case-provided rlds and eps, and models must declare those inputs through needs_forcing and needs_static. Models that emit ts and rlus but do not consume those boundary inputs are now classified as N/A (INCOMPATIBLE) rather than receiving a potentially false VIOLATION. The reverse static-input check and the separation between adapter smoke testing and verdict eligibility are also covered by regression tests.

The README now reports the positive-control residual using the stable < 1e-12 bound and explains its dependence on the installed NumPy/Pandas versions.

I rechecked 17c2fbb: the targeted tests pass, the full test suite is clean apart from platform-specific skips, validation reports 21 probes and 37 models, the radiation gate separates all three reference models as declared, and the 36-case margin scan passes.

No further blocking findings. Approved.

Minor non-blocking note: the PR description still says “1e-14 to 1e-13,” while the updated README more accurately says “of order 1e-13 or smaller.” Consider aligning the PR description with the README.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Probe-required diagnostic names are not validated, allowing malformed specifications to pass validation.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds the energy/radiation-consistency probe, supporting diagnostic outputs and required static inputs across the harness.

Changes:

  • Implements the radiation identity criterion, generator, and reference controls.
  • Extends manifests and protocol with diagnostics and static-input declarations.
  • Synchronizes tests, documentation, site content, credits, and archived results.
File summaries
File Description
tests/test_radiation_references.py Tests reference-control separation.
tests/test_radiation_probe.py Tests generation and compatibility.
tests/test_radiation_consistency.py Tests criterion behavior and edge cases.
tests/test_harness.py Tests diagnostics and static inputs.
tests/test_docs_in_sync.py Includes diagnostic variables in docs checks.
templates/probe.template.yaml Documents new requirement categories.
src/hydroturing/spec.py Adds diagnostics and static-input metadata.
src/hydroturing/protocol.py Adds diagnostics to adapter requests.
src/hydroturing/harness.py Enforces declared case inputs.
src/hydroturing/criteria/radiation.py Implements the radiation identity.
src/hydroturing/criteria/__init__.py Registers the criterion.
site/index.html Publishes translated probe details.
scripts/radiation_margins.py Reproduces control margins.
schemas/probe.schema.json Defines new probe requirements.
schemas/model.schema.json Defines diagnostics and static needs.
ROADMAP.md Marks the probe merged.
README.md Updates probe and model documentation.
probes/energy/radiation-consistency/README.md Documents scope and validation.
probes/energy/radiation-consistency/probe.yaml Declares the probe specification.
probes/energy/radiation-consistency/generate.py Generates hourly radiation cases.
models/wflow_sbm/README.md Records the new N/A result.
models/summa/README.md Records the new N/A result.
models/result.csv Archives evaluated-model outcomes.
models/reference_radiative/model.yaml Declares the positive control.
models/reference_radiative/ht_adapter.py Implements consistent radiation output.
models/reference_no_reflection/model.yaml Declares the reflection-negative control.
models/reference_no_reflection/ht_adapter.py Omits reflected longwave.
models/reference_air_emitter/model.yaml Declares the temperature-negative control.
models/reference_air_emitter/ht_adapter.py Emits using air temperature.
models/cwatm/README.md Records the new N/A result.
models/_template/model.yaml Documents new model fields.
docs/writing-a-probe.md Documents the criterion and controls.
docs/adapting-a-model.md Documents static-input compatibility.
CONTRIBUTORS.md Credits the probe author.
CITATION.cff Adds the author citation.
AGENTS.md Documents new variables and contracts.
Review details
  • Files reviewed: 36/36 changed files
  • Comments generated: 1
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/hydroturing/spec.py
Upstream merged LISFLOOD (Flood-Lab#52). The conflicts were the README's models
table, which takes upstream's lisflood row with its energy wording widened
to the four probes that need an energy output, and the archive, which keeps
upstream's rows followed by this branch's. lisflood is archived N/A
(INCOMPLETE) on energy/radiation-consistency by ht run, and its card counts
four N/A probes instead of three. The suite is 21 probes and 38 models.
Copilot's review of Flood-Lab#39 noticed that load_probe took any name under
requires while load_model checks every emission against the known
variables. A probe with diagnostics: [skin] passed ht validate and would
have made every model INCOMPLETE for an output none could declare, and a
flux asked for as a state passed as well. The probe now fails to load with
the same message load_model gives, and a test tries both mistakes.
lisflood's card had one sentence left that excluded three N/A probes from
its standing where the rest of the card, and the archive, say four, and
its comparison with the .2 rows now names the radiation probe as the
fourth; summa's card said three energy probes supply net radiation where
there are now four. The required-name test's docstring and the comment
beside the check say what they reject, an unknown output and a variable
in the wrong category, rather than claiming both would have made every
model INCOMPLETE: a flux asked for as a state is met by the flux.
@xinlan-technology

Copy link
Copy Markdown
Author

@licm13 Thanks for the approval. Updated in ec1d1a8: merged main at 12cbaaa, archived LISFLOOD as N/A on this probe, and added validation for unknown or misclassified required outputs.

The PR description is now aligned with the README. All 488 tests, the full gate, and the 36-case margin scan pass locally.

@licm13 licm13 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked the follow-up through ec1d1a8. The new validation correctly rejects unknown required outputs and variables placed in the wrong output category, with regression tests covering both cases. The PR description is also now aligned with the README’s stable residual bound.

No new blocking findings; my approval stands.

@chrimerss

Copy link
Copy Markdown
Contributor

@xinlan-technology Thanks for this, and for turning @licm13's review around so quickly. The probe is in good shape. I merged it with current main locally, and ht validate, pytest, the full ht gate, scripts/radiation_margins.py and the container job all pass. A separate sweep over the gate seeds plus seeds 0–199 reproduces your margins. Before merging I'd like one change to the new declaration contract, plus some documentation.

1. needs_forcing/needs_static now serve two roles (requested change)

In compatibility_issues (src/hydroturing/harness.py:332-355), these lists now answer two different questions:

  • Can the adapter run on this case? A declared input the case does not supply makes the model INCOMPATIBLE.
  • Does the model consume this input? requires.forcing: [rlds] and requires.static: [eps] are checked against the same lists.

Only this probe supplies rlds or eps. So a model that declares them is INCOMPATIBLE on the other 20 probes, and one that doesn't is INCOMPATIBLE on this one. The references show it:

$ ht run --model reference_coupled --probe energy/surface-energy-closure --seed 4242
PASS
$ ht run --model reference_radiative --probe energy/surface-energy-closure --seed 4242
N/A (INCOMPATIBLE): forcing does not provide rlds; static does not provide eps

A real model that reports ts and rlus would face the same choice: be scored on this probe or on the rest of the suite, not both with one manifest. No merged model reads rlds or eps yet, so now is the cheap moment to separate the two meanings. One way to do it, though the design and naming are up to you:

  • Keep needs_forcing/needs_static meaning "cannot run without", checked against the case as today.
  • Add an optional key for inputs an adapter reads when a case supplies them (say uses_forcing/uses_static). requires.forcing/requires.static would match against the union of both keys.
  • Move rlds/eps in the three reference manifests to the new key.
  • Add a test that a model declaring the new key is compatible with a case that lacks those inputs and still satisfies this probe's requires. Keep the existing test that a model declaring neither key is INCOMPATIBLE here.
  • Describe the distinction in AGENTS.md, schemas/model.schema.json and the model template.

2. The interface questions from #22

My comment on #22 didn't answer your three interface questions, so for the record, I'm happy with how the PR answers them:

  • ts goes under the new diagnostics output category, which budgets never integrate.
  • Emissivity comes only from static.json. A model that uses its own land-cover emissivity therefore can't be scored unless its adapter takes the case's value; please add a sentence saying so under "Limits".
  • rlds is a forcing column, like rn.
  • The linearized Stefan–Boltzmann control stays out of the gate.

3. Documentation

  • Timing of ts and rlus. AGENTS.md defines them "at the sampling instant" without saying which instant. The probe README means the row's timestamp. But row i's pr, pet and rn are means over the following hour, so a model could reasonably report its end-of-step values in row i. A model that does, with its rlus computed from the end-of-step sky, is self-consistent. Yet it fails 103 of 719 steps on gate seed 892092983 (worst 1.87× the tolerance), so it would get VIOLATION. Please state in the AGENTS.md rows, and in the request notes, that row i's ts and rlus are at row i's time, the same instant as row i's rlds.
  • Limits. Two model errors pass on some seeds and are worth a sentence each:
    • A blackbody model (rlus = σ·ts⁴) passes any seed with eps ≥ about 0.987: 11 of 205 drawn seeds, and every seed at eps 0.99. All five gate seeds are below that, so it is caught today.
    • Emission linearized around the previous step's ts, as CLM-style solvers do, scores 0.88–1.80× the tolerance, and 8 of 205 seeds pass. It fails all five gate seeds (1.25–1.59×). Whether it passes depends on the hourly skin-temperature jumps, which reach 12.6 K in an hour.

Once item 1 is in, I'll approve the CI run and take a final look before merging. Thanks again; this is a careful piece of work.

@xinlan-technology

Copy link
Copy Markdown
Author

@xinlan-technology Thanks for this, and for turning @licm13's review around so quickly. The probe is in good shape. I merged it with current main locally, and ht validate, pytest, the full ht gate, scripts/radiation_margins.py and the container job all pass. A separate sweep over the gate seeds plus seeds 0–199 reproduces your margins. Before merging I'd like one change to the new declaration contract, plus some documentation.

1. needs_forcing/needs_static now serve two roles (requested change)

In compatibility_issues (src/hydroturing/harness.py:332-355), these lists now answer two different questions:

  • Can the adapter run on this case? A declared input the case does not supply makes the model INCOMPATIBLE.
  • Does the model consume this input? requires.forcing: [rlds] and requires.static: [eps] are checked against the same lists.

Only this probe supplies rlds or eps. So a model that declares them is INCOMPATIBLE on the other 20 probes, and one that doesn't is INCOMPATIBLE on this one. The references show it:

$ ht run --model reference_coupled --probe energy/surface-energy-closure --seed 4242
PASS
$ ht run --model reference_radiative --probe energy/surface-energy-closure --seed 4242
N/A (INCOMPATIBLE): forcing does not provide rlds; static does not provide eps

A real model that reports ts and rlus would face the same choice: be scored on this probe or on the rest of the suite, not both with one manifest. No merged model reads rlds or eps yet, so now is the cheap moment to separate the two meanings. One way to do it, though the design and naming are up to you:

  • Keep needs_forcing/needs_static meaning "cannot run without", checked against the case as today.
  • Add an optional key for inputs an adapter reads when a case supplies them (say uses_forcing/uses_static). requires.forcing/requires.static would match against the union of both keys.
  • Move rlds/eps in the three reference manifests to the new key.
  • Add a test that a model declaring the new key is compatible with a case that lacks those inputs and still satisfies this probe's requires. Keep the existing test that a model declaring neither key is INCOMPATIBLE here.
  • Describe the distinction in AGENTS.md, schemas/model.schema.json and the model template.

2. The interface questions from #22

My comment on #22 didn't answer your three interface questions, so for the record, I'm happy with how the PR answers them:

  • ts goes under the new diagnostics output category, which budgets never integrate.
  • Emissivity comes only from static.json. A model that uses its own land-cover emissivity therefore can't be scored unless its adapter takes the case's value; please add a sentence saying so under "Limits".
  • rlds is a forcing column, like rn.
  • The linearized Stefan–Boltzmann control stays out of the gate.

3. Documentation

  • Timing of ts and rlus. AGENTS.md defines them "at the sampling instant" without saying which instant. The probe README means the row's timestamp. But row i's pr, pet and rn are means over the following hour, so a model could reasonably report its end-of-step values in row i. A model that does, with its rlus computed from the end-of-step sky, is self-consistent. Yet it fails 103 of 719 steps on gate seed 892092983 (worst 1.87× the tolerance), so it would get VIOLATION. Please state in the AGENTS.md rows, and in the request notes, that row i's ts and rlus are at row i's time, the same instant as row i's rlds.

  • Limits. Two model errors pass on some seeds and are worth a sentence each:

    • A blackbody model (rlus = σ·ts⁴) passes any seed with eps ≥ about 0.987: 11 of 205 drawn seeds, and every seed at eps 0.99. All five gate seeds are below that, so it is caught today.
    • Emission linearized around the previous step's ts, as CLM-style solvers do, scores 0.88–1.80× the tolerance, and 8 of 205 seeds pass. It fails all five gate seeds (1.25–1.59×). Whether it passes depends on the hourly skin-temperature jumps, which reach 12.6 K in an hour.

Once item 1 is in, I'll approve the CI run and take a final look before merging. Thanks again; this is a careful piece of work.

@chrimerss Thanks for the detailed review. Addressed in bec106f: separated mandatory needs_* from optional uses_* inputs, with regression tests. All three references also pass energy/surface-energy-closure.

The docs now clarify row-timestamp sampling and the emissivity, blackbody and linearization limits. I also fixed overflow in the mean-residual diagnostic; scoring formulas and tolerances are unchanged.

All 500 tests, the full gate and the 36-case margin scan pass locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

probe-proposal A proposed new probe

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[PROBE] energy/radiation-consistency: surface temperature and upwelling longwave

4 participants