Repository navigation

Test and tune the planner's web search and fetch use - #712

Merged

hanna-paasivirta merged 21 commits into

OpenFn:mainfrom

IZO-Ong:search-test

Nov 5, 2026

Merged

Test and tune the planner's web search and fetch use#712
hanna-paasivirta merged 21 commits into
OpenFn:mainfrom
IZO-Ong:search-test

Commits on Oct 8, 2026
- test: add web tool trace builder for planner behaviour tests
  IZO-Ong
  committed
  0b6c268
- test: add web tools experiment metrics and winner selection
  IZO-Ong
  committed
  37b6adb
- test: add recording planner and variants for web tools experiment
  IZO-Ong
  committed
  5e8ba66
- test: add calibration to estimate cost and token size
  IZO-Ong
  committed
  03b1a24
- test: pair code execution results in the web tool trace
  IZO-Ong
  committed
  a52caf8
- test: add web tools behaviour scenarios
  IZO-Ong
  committed
  4b1cf04
- test: add staged web tools experiment runner
  IZO-Ong
  committed
  b312eff
- test: add strict web tools behaviour suite for the planner
  IZO-Ong
  committed
  6924306
- test: report how often the truncated source page was fetched
  IZO-Ong
  committed
  07a9dab
- test: add judged acceptance specs for web tool answers
  IZO-Ong
  committed
  9181f11
- test: let variants remove prompt text, add 1d experiment variant
  IZO-Ong
  committed
  c8c6472
- feat: tell planner to search before fetching
  IZO-Ong
  committed
  3be01a6
- test: drop 1a from the fingerprint test
  IZO-Ong
  committed
  daa0911
- test: allow the first remembered URL refusal in the strict test suite
  IZO-Ong
  committed
  e405392
- test: pair every server tool result in the web tool trace
  IZO-Ong
  committed
  d622d59
- test: stop failed variants winning and fold curly apostrophes in web tool metrics
  IZO-Ong
  committed
  45fc6ed
- test: key the web tools run cache by prompt and scenario, retry failed runs
  IZO-Ong
  committed
  b0b8f8f
Commits on Oct 30, 2026
- Merge branch 'main' into search-test
  IZO-Ong
  committed
  00af8a1
- test: fold web tools scenarios into the strict suite
  IZO-Ong
  committed
  821d483
- feat: record each planner web call in Langfuse
  IZO-Ong
  committed
  227476e
Commits on Nov 4, 2026
- Add prompt line to guide agent to parse json result of web_search and web_fetch
  IZO-Ong
  committed
  9ca699f

Commits

Commits on Oct 8, 2026

Commits on Oct 30, 2026

Commits on Nov 4, 2026