Skip to content

[js/web] Rebind external tensors on every run when graph capture is enabled - #33985

Open
naft-san (Nominy) wants to merge 4 commits into
microsoft:mainfrom
Nominy:fix-web-graph-capture-rebind
Open

naft-san (Nominy) wants to merge 4 commits into
microsoft:mainfrom
Nominy:fix-web-graph-capture-rebind

Conversation

@Nominy

@Nominy naft-san (Nominy) commented Nov 8, 2026 •

Copy link
Copy Markdown

Description

With enableGraphCapture, run() binds inputs and outputs only on the first run of a session, but the WebGPU EP unregisters the GPU buffers of external tensors at the end of every run. Any later run that executes kernels (an uncaptured gpu_graph_id: '-1' run, or capturing another graph id) then binds released buffers and fails with Required member is undefined in createBindGroup.

This keeps the binding under graph capture, since ORT-allocated outputs must stay bound for replay, but re-binds inputs and pre-allocated outputs on every run. Because the binding is kept, a graph-capture session must request the same outputs, each pre-allocated or not, on every run. A run that changes this used to return stale or empty outputs; it now throws before binding anything, and the session keeps working.

Adds a unittest for the WebGPU EP build (JSEP rejects re-registering a different external buffer under graph capture). CI does not run unittests for that build; run it with npm test --webgpu-ep -- unittest -b=webgpu. It fails without the fix.

ORT-allocated outputs whose shape changes between graph ids are still unsupported: they returned a stale output before, and now raise an error.

Motivation and Context

Running a captured decode step after an uncaptured prefill (gpu_graph_id: '-1', then '1') fails on the second run.

Copilot AI balanced review requested due to automatic review settings November 8, 2026 15:56
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Clearing outputs breaks graph replay with GPU-buffer inputs and no preallocated outputs, causing subsequent runs to fail in _OrtGetTensorData rather than return captured outputs.

1 open issue
What changed in this PR

This PR updates ONNX Runtime Web’s WebGPU binding lifecycle to address stale GPU-buffer references across runs when graph capture is enabled.

Changes:

  • Clears bound outputs and resets the IO-bound state after successful runs, including graph-capture runs, so subsequent runs rebind their tensors.
  • Adds a comment explaining why bindings must not outlive per-run GPU-buffer registrations.
File Description
js/​web/​lib/​wasm/​wasm-core-impl.ts Removes the graph-capture exception from binding cleanup and documents the buffer-lifetime rationale.

Review effort: Balanced

Comment thread js/web/lib/wasm/wasm-core-impl.ts Resolved
@Nominy naft-san (Nominy) changed the title [js/web] Rebind IO after every run when graph capture is enabled [js/web] Rebind external tensors on every run when graph capture is enabled Nov 8, 2026
@Nominy
naft-san (Nominy) marked this pull request as draft November 8, 2026 16:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Output-binding state persists incorrectly across runs, causing graph replay to retrieve unregistered output handles and changed output requests to fail with output-count mismatches.

4 open issues
1 resolved since last review

Review effort: Max

Comment thread js/web/lib/wasm/wasm-core-impl.ts Resolved
Comment thread js/web/lib/wasm/wasm-core-impl.ts Resolved
…pose test outputs, skip under proxy

With graph capture the IO binding is kept across runs, so a run that requests a different output set, or switches an output between pre-allocated and ORT-allocated, used to return stale or empty outputs. Such a run now throws before binding anything, and the session keeps working with its original configuration.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block the graph-capture I/O binding fix.

No open issues 🎉

4 resolved since last review

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants