Repository navigation
[js/web] Rebind external tensors on every run when graph capture is enabled - #33985
Open
naft-san (Nominy) wants to merge 4 commits into
Open
naft-san (Nominy) wants to merge 4 commits into
naft-san (Nominy) wants to merge 4 commits into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Clearing outputs breaks graph replay with GPU-buffer inputs and no preallocated outputs, causing subsequent runs to fail in _OrtGetTensorData rather than return captured outputs.
1 open issue
What changed in this PR
This PR updates ONNX Runtime Web’s WebGPU binding lifecycle to address stale GPU-buffer references across runs when graph capture is enabled.
Changes:
- Clears bound outputs and resets the IO-bound state after successful runs, including graph-capture runs, so subsequent runs rebind their tensors.
- Adds a comment explaining why bindings must not outlive per-run GPU-buffer registrations.
| File | Description |
|---|---|
js/web/lib/wasm/wasm-core-impl.ts |
Removes the graph-capture exception from binding cleanup and documents the buffer-lifetime rationale. |
Review effort: Balanced
… only external tensors
naft-san (Nominy)
marked this pull request as draft
November 8, 2026 16:12
…ple graphs (WebGPU EP)
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Output-binding state persists incorrectly across runs, causing graph replay to retrieve unregistered output handles and changed output requests to fail with output-count mismatches.
4 open issues
1 resolved since last review
Review effort: Max
…pose test outputs, skip under proxy With graph capture the IO binding is kept across runs, so a run that requests a different output set, or switches an output between pre-allocated and ORT-allocated, used to return stale or empty outputs. Such a run now throws before binding anything, and the session keeps working with its original configuration.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Description
With
enableGraphCapture,run()binds inputs and outputs only on the first run of a session, but the WebGPU EP unregisters the GPU buffers of external tensors at the end of every run. Any later run that executes kernels (an uncapturedgpu_graph_id: '-1'run, or capturing another graph id) then binds released buffers and fails withRequired member is undefinedincreateBindGroup.This keeps the binding under graph capture, since ORT-allocated outputs must stay bound for replay, but re-binds inputs and pre-allocated outputs on every run. Because the binding is kept, a graph-capture session must request the same outputs, each pre-allocated or not, on every run. A run that changes this used to return stale or empty outputs; it now throws before binding anything, and the session keeps working.
Adds a unittest for the WebGPU EP build (JSEP rejects re-registering a different external buffer under graph capture). CI does not run unittests for that build; run it with
npm test --webgpu-ep -- unittest -b=webgpu. It fails without the fix.ORT-allocated outputs whose shape changes between graph ids are still unsupported: they returned a stale output before, and now raise an error.
Motivation and Context
Running a captured decode step after an uncaptured prefill (
gpu_graph_id: '-1', then'1') fails on the second run.