feat(https-outcalls): cover pay-as-you-go pricing and flexible outcalls - #387
Conversation
Skill Validation ReportValidating skill: /home/runner/work/icskills/icskills/skills/https-outcallsStructure
Frontmatter
Tokens
Markdown
Tokens
Content Analysis
References Content Analysis
Contamination Analysis
References Contamination Analysis
Result: 1 warning Project Checks |
b07fcb8 to
779139e
Compare
The skill taught pricing version 1 as if it were the only model, and its Rust example imported `ic_cdk::management_canister`, which ic-cdk 0.20 removed. The version 1 material is still correct and stays: it is what Motoko does today, and what Rust does on the older crate lines. Added, as two reference files because SKILL.md is at the 500-line limit: - references/pricing-version-2.md — what version 2 charges, the four with_expected_* setters and their defaults, why they fund one pooled per-node budget rather than four independent limits, why a larger attachment can raise the bill, the reject messages for an under-funded call, and what to check before migrating. - references/flexible-outcalls.md — flexible_http_request: replication counts derived from subnet_self_node_count, a verified working example with reconciliation, the four global_error values, and the two separate size limits. In SKILL.md: - A support matrix, following the vetkeys/encrypted-maps pattern, stating that version 2, flexible outcalls and subnet_self_node_count are Rust only, and which of three Rust dependency worlds a project is in. - An outcall modes section. `is_replicated` appeared in both examples but non-replicated mode was never explained. - Rust example rewritten to the ic-cdk-management-canister 0.2 builder. Verified by compiling the extracted block against ic-cdk 0.20.3 and the crate's 0.2.0. - Motoko example switched to `mo:ic/Types` rather than `ic:aaaaa-aa`, so there is one pinned surface, and labelled explicitly as version 1. - Pitfalls 1, 2, 7, 8 scoped by mode and pricing version; three new pitfalls for the version 2 reservation defaults, keeping pricing_version and the attached amount in agreement, and flexible replication counts. - The cycle cost section split by version, with version 1 kept verbatim. - The "over-budgeting is safe but not free" advice corrected for version 2. The attachment beyond the base fee is the per-node allowance, and get_adapter_limits derives the response deadline and transform instruction limit from it, so a margin is permission to spend rather than idle reserve and can be billed for. Version 1 cannot do that, since its charge is fixed when the call is made. The size, header-limit and double-enforcement pitfalls are unchanged, and the two-timeout pitfall gains only a version 2 caveat: they are version independent and were already right. Seven output evals added, covering the pooled budget, that an under-funded version 2 call is accepted rather than rejected, that version 1 is unreachable through the 0.2 crate, that Motoko has no version 2 path, the version/funding mismatch, flexible replication counts, and why a default version 2 call reserves billions. Two existing evals updated: the version 1 cost question is pinned to "the legacy version 1 pricing" now that two versions exist, and the over-attaching question now asks whether the answer depends on the pricing version, since under version 2 it does.
779139e to
9329242
Compare
Review: sound — with two things to fix and two to considerI checked this independently rather than taking the verification section at face value: compiled the code, recomputed the numbers from the replica's own fee functions, and ran the evals the PR body is missing. Verdict: the substance holds up, and unusually well. Every crate and replica claim I checked is accurate, both Rust examples compile, the Motoko example type-checks, and all seven headline cycle figures reproduce to the cycle. The evals confirm the improvement is real — several baselines confidently assert the opposite of the truth. Two fixes below (1 and 2), two judgement calls (3 and 4), and some eval follow-ups. What I verified independentlyCode compiles / type-checks.
Crate claims — all confirmed from crate source and CHANGELOG. Replica claims — all confirmed against The numbers. This is the part I expected to find soft, and it is the opposite. Working from
That the three-row table falls out exactly implies the 200-byte request and the Not verified: that mainnet runs a replica containing these paths, or the claim that the pay-as-you-go flag was enabled on 2026-09-02 across every subnet. I checked source, not deployment. Issues1. [fix] The 500 KB worked example quotes charge figures as reservations —
4.8B and 19M are
The quoted pair is exactly right for a non-replicated call (where Relatedly, "roughly 15x" (9,490/650) is the charge ratio; on the reservation it is ~21x (13,708/650). Pitfall 12 in 2. [fix] The Rust POST example now contradicts itself.
Given "code must be copy-paste correct" is a house rule, I would either drop 3. [consider] The version 1 warning got diluted. Lower confidence than the two above, but I think it is real.
Read alone, "genuinely idle" says a version 1 margin is costless — which is what the pre-PR "over-budgeting is safe but not free" plus the concurrency cap existed to prevent. In I ran eval 4 three times, because one run is not evidence. On aggregate the skill wins — mean 5.3/7 with vs 4.0/7 baseline (4/7 vs 5/7, 5/7 vs 3/7, 7/7 vs 4/7). So this is not a regression overall, and I am not claiming one. But the two version 1 behaviors are the weak spot:
The version 2 half works, cleanly and every time. The version 1 half underperforms a no-skill baseline in 2 of 3 runs, and a representative failing answer is:
n=3 is thin, so treat the numbers as suggestive; what makes me raise it anyway is that they point the same way as the text. Nothing needs backing out — just stop conceding the version 1 half, e.g. "…so a margin is not itself billed — but it is still held for the duration of the call, so it still caps how many outcalls can be in flight." Eval 13 shows the same framing miscalibrating in the other direction: asked why a default call reserves billions, the with-skill answer said an inflated attachment "can make the call more expensive, not just wasteful" — in a case where the charge is identical and only the hold differs. Both files do make the "for a call that completes within either reservation the charge is the same" point, but it sits under the louder "attaching more can cost more" headline. Version 1 reads too permissive, version 2 too alarmist; one fix covers both, which is keeping "reserved" and "charged" lexically distinct wherever they appear together. 4. [consider] The Motoko version floors look overstated. The support matrix says Non-blocking
EvalsThe PR body has no eval results, so I ran them. Every changed or added case, with baseline, per CONTRIBUTING. Where a run is listed more than once I re-ran it — for eval 4 to test variance, elsewhere because a baseline died on the runner's 120s timeout. Output evals — 9 changed/added cases
Trigger evals: should-trigger 6/6, should-not-trigger 1/1 — including all three new queries ( The improvement is real. Evals 8, 10 and 11 are the clearest: baselines there do not merely omit things, they confidently assert the opposite. Eval 8's baseline claims an under-funded call is rejected up front and the nodes are "never contacted — no fetch, no replication, no consensus round". Eval 10's baseline invents a Three things the eval run surfaced that are worth acting on:
One process note: 4 of 9 baselines died on |
Addresses marc0olo's review. All four items, plus the three eval follow-ups and the token-budget note. [fix 1] The 500 KB worked example quoted charge figures in a paragraph about the reservation. For a fully replicated call `max_consensus_fee` scales the delivery fee by `n / canister_http_threshold(n)`, which is 13/9 on a 13-node subnet, so 501,024 transformed bytes reserve 6.87B rather than the 4.75B they settle at, and 2,000 bytes reserve 27.4M rather than 19.0M. Recomputed independently and matched the review to the cycle. The file now gives both figures and explains the scale-up, and notes it does not apply to non-replicated calls. The "roughly 15x" raw-to-delivered ratio is the charge ratio; the reservation ratio is about 21x, and both are now labelled. [fix 2] The Rust POST example contradicted itself after `.non_replicated()` was added: the idempotency-key comment still said "across replicas", and `transform_post` still justified discarding the body by per-replica variance. Dropped `.non_replicated()` and kept the example replicated, so both comments are true again, with a comment pointing at the alternative and at pitfall 8. [fix 3] "A margin is genuinely idle" under version 1 read as "costless", which is what the original "safe but not free" existed to prevent, and the review's eval runs show the version 1 half underperforming a no-skill baseline. It now says a version 1 margin is never billed but is still held, so it still caps outcalls in flight. The version 2 side is qualified in the other direction, since eval 13 showed it reading as alarmist: a larger attachment raises the charge only where it lets the call consume more. [fix 4] The Motoko floors were the dfinity/examples pins, not the package's requirements. `ic` 4.2.0 declares `core = "2.5.0"` and `[requirements] moc = "1.4.0"`; the matrix said 2.6.1 and 1.14.1. Eval follow-ups: - Eval 8 wanted reject messages that only existed in the reference. The `Debugging Outcall Failures` block now carries `Insufficient cycles` and the full `Out of cycles: ...` string, and notes that the version 1 insufficient-cycles reject is version 1 only. - Eval 9's miss was the `from_args` overwrite, which the skill never stated. Added: it is the documented migration path and it silently overwrites `pricing_version` with 2. - Eval 12 hardcoded `total_requests: 5`. Pitfall 14 now carries the derivation as a code block rather than inline prose. On the token budget: the review was right that the 500-line limit had been met by denser prose rather than by moving content out. The typed JSON parser and the normalizing transform are now `references/json-responses.md`, neither being specific to outcalls. SKILL.md goes 499 -> 474 lines and 4,656 -> 4,645 words, so it is below the reviewed state on both counts despite the additions above. Not done: `skills/cloud-engine-canisters/SKILL.md` still names `ic_cdk::management_canister::http_request` in three places. Out of scope here, as the review says; worth its own issue. Verification: the SKILL.md and flexible blocks compile clean for wasm32-unknown-unknown against ic-cdk 0.20.3 + ic-cdk-management-canister 0.2.0, and the json-responses blocks compile clean composed with the SKILL.md example, which is where `fetch_price` is defined. `node scripts/check-project.js` passes. The evals could not be re-run: no `claude` CLI in this environment.
|
Thanks — recomputing the figures and running the evals rather than reading the verification section is what caught the one I would have defended. All four items are addressed in 0a6ca6f, plus the three eval follow-ups and the token note. 1. [fix] Reservation vs charge in the 500 KB example — fixedYou're right, and I recomputed it from
I had quoted the charge column in a paragraph whose entire subject was the reservation. The file now gives both, explains the scale-up, and says it does not apply to non-replicated calls — which also resolves the internal disagreement you noted, since the table above it is replicated. "Roughly 15x" is likewise the charge ratio; the reservation ratio is about 21x. Both are now labelled where they appear, and pitfall 12 no longer quotes a charge ratio inside reservation prose. 2. [fix] The POST example contradicting itself — fixed by revertingI took the first of your two options and dropped Your third sub-point stands on its own and I'd missed it: the example sets a transform and declares its instruction expectation, right after pitfall 1 says no transform is the cheapest configuration. That is now consistent, because with the example replicated a transform is required rather than optional. 3. [consider] The version 1 warning diluted — acceptedThe eval runs are what convinced me; I'd have called this a wording preference otherwise. "Genuinely idle" now reads:
And I took your point that eval 13 shows the same framing failing in the opposite direction, so the version 2 side is qualified too: a larger attachment raises the charge only where it lets the call consume more than a tighter budget would have allowed, and where the call would have completed either way the charge is identical and only the hold differs. 4. [consider] Motoko floors overstated — fixedConfirmed: Eval follow-ups — all three
I could not re-run the evals — no Token budgetYour diagnosis was right, and the word count is the metric that shows it: 2,971 → 4,656 while lines went 485 → 499. I met the limit with denser prose instead of moving content out. The typed JSON parser and the normalizing transform are now Filling the gap you flaggedYou noted you'd checked source, not deployment, for the 2026-09-02 flag claim. I checked that separately and can back it: the flip is |
Re-review of
|
| # | Case | before | after |
|---|---|---|---|
| 4 | over-attaching, and why v2 changes the answer | 4/7, 5/7, 7/7 | 4/7, 4/7, 7/7 |
| 8 | under-funded v2 accepted, not rejected | 3/5 | 4/5 |
| 9 | v1 unreachable through 0.2 | 3/4, 3/4 | 3/4 |
| 12 | flexible committee size | 3/5 | 4/5 |
| 13 | why a default v2 call reserves billions | 4/5, 4/5 | 5/5 |
Landed: eval 8 (the two reject names now surface) and eval 13 (5/5 — the reservation-vs-charge distinction reads correctly now).
Did not land, and worth knowing rather than re-fixing blind:
- Eval 9 still misses on
from_args, now 3 failures out of 3 attempts across both commits. The sentence is present, but it is inside the long dependency-worlds paragraph at L32, which is not where a model answering "how do I get version 1 out of 0.2?" is looking. It may need to sit next to that question, or in a pitfall. - Eval 12 improved to 4/5, but on a different behavior — it still hardcodes
total_requests: 5. The code block did not change that. Worth noting the malformed fence sits in exactly this pitfall, so the raw markdown a model loads has the derivation followed by a fence line with prose on it; fixing the fence may be a fair precondition before judging this one.
One thing to look at, flagged not asserted
Behavior level on eval 4, three runs each side:
| Behavior | before | after |
|---|---|---|
| "over-attaching is still not free" | 1/3 | 1/3 |
| "does not endorse the 1B round number" | 1/3 | 1/3 |
| "v2: attachment becomes the per-node allowance" | 3/3 | 1/3 |
So [3]'s version 1 half did not measurably move, and the version 2 qualification that fixed eval 13 may have cost the version 2 allowance behavior. I am flagging rather than claiming: n=3, the distribution is bimodal (two identical runs plus one clean sweep), and I over-read a small sample earlier in this review. Worth a look at whether "do not read this as 'a buffer makes every call dearer'" can be softened without blunting the allowance mechanism.
Unchanged from last time
Still not verified by me: that mainnet runs a replica containing these paths, or that the pay-as-you-go flag was enabled 2026-09-02 across every subnet. I checked source at dfinity/ic master@2c2c7c9, not deployment.
Agreed cloud-engine-canisters is out of scope — happy to file that issue separately.
Re-review of 0a6ca6f. The blocker plus the nit, and two follow-ups from the eval results. [blocker] Pitfall 15's closing fence had prose on the same line, so CommonMark never closed the block and the rest of the pitfall rendered as literal Rust on the built site. Confirmed in dist/ before and after: the guidance now sits after `</pre>`, `**any**` renders as `<strong>`, and no backticks or raw markup leak. `npm run validate` passed either way, since the fence check counts fences without applying the alone-on-its-line rule, so this would not have been caught by CI. Audited every fence in the skill and its references; no others. [nit] The POST comment said "the transform below"; `transform_post` is defined above `post_data`. Now names it instead of placing it. Eval follow-ups: - Eval 9 kept missing the `from_args` overwrite because the sentence sat inside the long dependency-worlds paragraph, which is not where a model answering "how do I get version 1 out of 0.2?" looks. It is now its own pitfall 13, phrased as that question, with the two real escape hatches. Pitfalls 13 and 14 renumbered to 14 and 15. - Eval 4's version 2 allowance behavior regressed 3/3 to 1/3, plausibly because the qualification added for eval 13 led with the charge and buried the mechanism. The paragraph now leads with the allowance and what it sets, then states the charge consequence and its limit as plain fact rather than as a "do not read this as" negation. Eval 13's distinction is preserved. Also repointed the 60-second timeout cross-reference from "pitfall 12" to references/pricing-version-2.md. It was wrong before the renumbering: pitfall 12 is about reservations, and the under-funded timeout mechanism is in the reference. SKILL.md is 477 lines / 4,700 words. Still below the 499 / 4,656 of the reviewed state on lines, and 44 words above it, for one new pitfall.
|
Fixed in Blocker: fence — fixed, and verified in
|
marc0olo
left a comment
marc0olo
left a comment
There was a problem hiding this comment.
Approving. Fence fixed — verified in dist/: guidance sits outside the block, **any** renders as <strong>, no markup leaks. Also re-checked the renumbered cross-references (all three still resolve), every claim in the new pitfall 13 against the crate source, and that all Rust in SKILL.md plus both references still compiles against ic-cdk 0.20.3 + ic-cdk-management-canister 0.2.0.
Evals on b6ac285
| Eval | before | now | baseline |
|---|---|---|---|
| 9 — v1 unreachable through 0.2 | 3/4, 3/4 | 4/4 | 0/4 |
| 13 — why a default v2 call reserves billions | 4/5, 4/5 | 5/5 | 0/5 |
| 4 — over-attaching | 4,5,7 /7 | 5,4,5 /7 | 5,3,4 /7 |
| 12 — flexible committee size | 3/5 | 2,3,3 /5 | 1/5 |
Eval 9's move to its own pitfall worked, and eval 13 held through the version 2 rewrite.
Two follow-ups, not blockers
Eval 4's allowance behavior is 0/3, down from 3/3 on the original draft. All three judge reasons are identical: "describes the allowance correctly but frames the downside purely as a concurrency limiter, never states that a larger attachment can be charged more." It tracks where the bold sits — 9329242 bolded "attaching more can cost more" (3/3), 0a6ca6f bolded it with an inline hedge (1/3), b6ac285 bolds the mechanism and leaves the consequence unbolded and twice-qualified (0/3). Keeping the mechanism-first order but restoring a short bolded consequence line before the qualification should recover it.
Eval 12: my precondition guess was wrong. You were right to leave it untouched so it could be measured cleanly — the fence is fixed and total_requests is still hardcoded in 3/3 runs. It needs different treatment, not a cleaner render.
Both are prominence tuning on content that is correct and complete, and everything is still well above baseline. Happy to file them as one issue.
Still unverified by me, unchanged from earlier: that mainnet runs a replica with these paths, and the 2026-09-02 flag-enablement date. Source-checked at dfinity/ic master@2c2c7c9, not deployment.
Both follow-ups from the approving review of b6ac285, now measured rather than guessed: the claude CLI is available here. **Eval 12 (hardcoded total_requests): 3/5 -> 5/5.** Your precondition guess about the fence was wrong, as you found, and describing the correct derivation was not enough either. What worked was naming the wrong code beside it, so pitfall 15 shows the hardcoded `ReplicationCounts { total_requests: 5, ... }` marked WRONG above the derived form. The behavior that had failed 3/3 across both previous commits now passes, and so do the other four. **Eval 4 and eval 13 are not in tension after all.** I first concluded they were and that the prose could only trade one against the other. That was wrong. They are the same rule applied in opposite directions: - Eval 4 loosens a budget that binds: 200M -> 1B lets a slow endpoint run further, so the bill can rise. - Eval 13 tightens one that does not bind: 5.3B -> 112M, where the call was never going to use five billion instructions, so the bill is unchanged and only the hold moves. Both follow from one proposition: the allowance is a ceiling on consumption, the charge follows consumption, so the attachment changes the bill only when the ceiling binds. The previous wording stated a claim and then hedged it, which forces a model to pick a side, and whichever it picked the other eval failed. It now states the rule once with both consequences as symmetric cases, neither subordinate. The version 1 paragraph also leads with the cost rather than the absolution. Measured, with the caveat that both evals are bimodal and these are small samples: eval 4 before: 3/7, 4/7, 5/7, 5/7, 6/7 after: 7/7, 6/7, 6/7 eval 13 before: 4/5, 5/5 after: 4/5, 4/5, 5/5 Eval 4's "attachment becomes the per-node allowance ... and can therefore be charged more" passed for the first time, after 8 consecutive failures across your runs and mine. Eval 13 is unchanged, so the gain did not come out of it. One deliberate redundancy: the bolded sentence restates the allowance mechanism the paragraph above already establishes. I removed it as sloppy writing and eval 4 measured 4/7 and 6/7 against 7/7 and 6/7 with it in. The samples overlap so the difference is not established, but the duplicated form never scored below 6 and the de-duplicated one did, so in a file whose reader may load only part of it the restatement stays. Note for reading these numbers: eval 13 gave 4/5 here on text byte-identical to what you measured at 5/5, so anything under three runs on this pair is suggestive at best.
|
Both follow-ups addressed in Eval 12: 3/5 → 5/5Your precondition guess was wrong, as you found, but so was the assumption behind my original fix: describing the correct derivation wasn't enough either. What worked was naming the wrong code beside it. Pitfall 15 now reads: // WRONG, even when you want five nodes: 5 may exceed the subnet size.
let replication = ReplicationCounts { total_requests: 5, min_responses: 3, max_responses: 5 };
// RIGHT: ask for five, but never more than the subnet has.
let total_requests = subnet_self_node_count().min(5);
let min_responses = total_requests / 2 + 1;The behavior that failed 3/3 across both previous commits now passes, and so do the other four. Eval 4 and eval 13 are not in tension — I had that wrongMy last reply claimed the pair was irreducibly opposed and that prose could only trade one against the other. That was wrong, and worth correcting explicitly since you might otherwise have filed it as such. They are the same rule applied in opposite directions:
Both follow from one proposition: the allowance is a ceiling on consumption, the charge follows consumption, so the attachment changes the bill only when the ceiling binds. My wording stated a claim and then hedged it, which forces a model to pick a side — and whichever it picked, the other eval failed. It now states the rule once, with both consequences as symmetric bullet cases, neither subordinate. The version 1 paragraph also leads with the cost rather than the absolution.
Eval 4's allowance-and-charged-more behavior passed for the first time, after 8 consecutive failures across your runs and mine. Eval 13 is unchanged, so the gain did not come out of it. |
2 participants
The skill taught pricing version 1 as if it were the only model, and it has no coverage of
flexible_http_request. Its Rust example also importsic_cdk::management_canister, which ic-cdk 0.20 removed.This is additive. The version 1 material stays, because it is what Motoko does today and what Rust does on the older crate lines. What it needed was labelling, so an agent knows which pricing regime it is in.
Added
Two reference files, since
SKILL.mdwas already at the 500-line limit CONTRIBUTING sets:references/pricing-version-2.md— which resources each version charges for, the fourwith_expected_*setters and their defaults, why they fund one pooled per-node budget, why a larger attachment can raise the bill, which expectations are safe to declare and why, the four ways an under-funded call fails, and what to check before migrating.references/flexible-outcalls.md—flexible_http_request: replication counts derived fromsubnet_self_node_count, a compiling example, reconciliation strategies and their threat model, the fourglobal_errorvalues, the two separate size limits and the block-budget cap on the transformed default, why a local run never shows disagreement, and whether a transform pays for itself.Changed in SKILL.md
vetkeys/encrypted-mapsper-language table pattern. Version 2, flexible outcalls andsubnet_self_node_countare Rust only; the Motoko cells say so with the reason.is_replicatedappeared in both code examples but non-replicated mode was never explained anywhere in the file.ic-cdk-management-canister0.2 builder.import IC "ic:aaaaa-aa"tomo:ic/Types, so the wrapper and the types come from one pinned surface, and labelled explicitly as version 1.pricing_versionand the attached amount in agreement, and flexible replication counts.Deliberately unchanged: the size, header-limit and double-enforcement pitfalls. They are version independent and were already right. The two-timeout pitfall gains only a version 2 caveat.
Corrections to advice that version 2 invalidates
Four places where carrying version 1 intuition across gives the wrong answer. These are the substance of the change, more than the new feature coverage.
"Over-budgeting is safe but not free." Under version 2 the attachment beyond the base fee becomes the per-node allowance, and
get_adapter_limitsderives the operating limits from it. A margin is permission to spend, not idle reserve: bytes are safe (the cap is also bounded by the caller's ownmax_response_bytes), but the deadline and the transform instruction limit scale with the allowance and are billed per unit consumed. Attaching more can cost more. Version 1 cannot do that, since its charge is fixed when the call is made.Which narrowing matters most flips with the cap. With
max_response_bytes = 4_000the transform-instruction default is nearly the whole 5.34B reservation, and declaring it reaches ~112M. With the cap unset the same call reserves ~34B, mostly blockspace, and the same declarations only reach ~28.8B — a 1.2x improvement rather than 48x.The ~1 KB delivery floor is in the quote, not the charge. It lives in
max_consensus_fee, because "whatever was asked for, a reject of this size may be delivered in its place". The settle path isconsensus_fee(proof.metadata.content_size, subnet_size)on the actual delivered size, unfloored. So stripping bytes always lowers the bill in proportion, while shrinking an already-small response does not lower the reservation further. A practical consequence:get_cost()shows the reservation, so it understates what a transform saves on a small response.The two byte expectations are not alike.
raw_response_bytesis the server's to decide, and declaring below what it sends is the narrowing that fails late — leave it at its default.transformed_response_bytesis bounded by your own transform, and it is the expensive one: a delivered byte costs ~9,490 cycles against ~650 for a raw byte downloaded, roughly 15x. Because it defaults tomax_response_bytes + 1_024, a transform that shrinks a large response leaves the delivery reserve sized for the raw one. For a 500 KB response reduced to a 2 KB extract,max_response_bytesmust stay ~500 KB so the cap is not available as a lever, and the default reserves on the order of 4.8 billion cycles for delivery where declaring 2,000 reserves about 19 million.Why the Rust example had to change rather than have its import path edited
I compiled the existing block in four configurations:
ic-cdk = "0.19"(as published)ic-cdk = "0.20", path unchangederror[E0432]: unresolved import ic_cdk::management_canisteric-cdk = "0.20"+ic-cdk-management-canister = "0.2", path swappedhttp_requestwas removed in 0.2.0, plusmissing field pricing_versiontwiceic-cdk = "0.20"+ic-cdk-management-canister = "0.1", path swappedSo the published skill is not broken, but it is pinned to a line two minors old, and an agent applying it inside a project on 0.20 emits code that does not compile. A path edit is not enough, because
ic-management-canister-types0.10 addedpricing_versionand the example writes exhaustive struct literals. Hence the builder, and hence the matrix telling an agent which of the three worlds it is in.Evals
Seven output evals added to
evaluations/https-outcalls.json, aimed at what an agent is most likely to get wrong:with_expected_roundtrip_time_msdoes not impose that deadlinewith_pricing_versionto invent)ic4.x has nopricing_versionfield to setpricing_versionand the attached amount have to agreemin_responsesandmax_responsesTwo existing evals updated, both because the questions became version-dependent rather than because the skill changed: the version 1 cost question is now pinned to "the legacy version 1 pricing that is still the replica default", and the over-attaching question now asks whether the answer depends on the pricing version. Flagging these explicitly, since editing an eval alongside the content it grades should be deliberate rather than incidental.
Verification
Every factual claim was checked against a source rather than carried over:
dfinity/ic): all nine reject strings exist word-for-word in non-test code, and the code labels check out —Out of cyclesisCanisterReject,No consensusisSysTransient,Http body exceedsmaps viaLimitExceeded => SysFatal. Flexible defaults are(2n)/3 + 1,N,N; the combined limit is2 * 1024 * 1024; the refund interval is 60 s."Timeout expired"has no hits in replica code: it is tonic'sTimeoutExpiredDisplay surfacing the adapter's 30 s client timeout, and the existing[SysFatal]label is right becauseclient.rsmaps every gRPC code exceptUnavailable/InvalidArgumenttoSysFatal.subnet_self_node_countandcost_http_request_v2landed;cost_http_requestremains for version 1 callers; the builder's replication constraint, defaults andflexible_transformed_default_capmatch their doc comments.IC.HttpRequestArgs,IC.HttpRequestResult,#getandCall.httpRequest(args : IC.HttpRequestArgs)all exist in the publishedic4.x; the snake_case names the skill used do not.13dividing transform instructions is a reference subnet size rather than the node count, and flexible outcalls run locally but every response comes from one node.ic-cdk 0.20.3+ic-cdk-management-canister 0.2.0.node scripts/check-project.jspasses for 31 skills (3 warnings, all pre-existing skills with no eval file).npm run buildcompletes, 35 pages, no errors. This is the real frontmatter gate via the zod schema insrc/content.config.ts.SKILL.mdis 499 lines, under the documented limit. The one intra-document link resolves: the built page emits<h2 id="outcall-modes">.skill-validatoris not installed in my environment, so that check is left to CI.Note on scope
Motoko support for version 2 and flexible outcalls is not blocked on this PR. It needs a
mocrelease carrying two new primitives, and then a publishedicpackage built against it. The Motoko cells in the matrix say "not available, needs an unreleasedmoc" rather than naming a version, and the version 1 guidance below them is the supported path meanwhile.