Skip to content

connect-node: reject gRPC responses truncated by RST_STREAM(NO_ERROR) with Code.Unavailable - #1752

Open
rwieruch wants to merge 1 commit into
connectrpc:mainfrom
rwieruch:fix/rst-no-error-truncation
Open

rwieruch wants to merge 1 commit into
connectrpc:mainfrom
rwieruch:fix/rst-no-error-truncation

Conversation

@rwieruch

@rwieruch rwieruch commented Jul 28, 2026

Copy link
Copy Markdown

Note from myself, because the issue/PR were crafted by Fable 5: If you do not accept such Issues/PRs, feel free to close both. However, our team ran into this issue five days ago and I investigate the issue yesterday with Fable 5 and a backend peer from our team who has control over Envoy.

Fixes #1751.

When a peer (typically a proxy such as Envoy) resets a stream with RST_STREAM code=0 (NO_ERROR) before the response completed, Node's http2 client (pre nodejs/node#63249) surfaces the truncation as a clean, error-free end-of-stream with rstCode 0. The protocol layer then fails parsing the truncated body with misleading wire-format errors — [invalid_argument] protocol error: incomplete envelope (cut mid-message) or [internal] protocol error: missing status (cut before trailers) — blaming the server and reading as non-retryable, when the actual failure is a transport-level abort. See #1751 for the full evidence (Envoy edge logs, instrumented client traces, controls).

What this PR does

In h2Request()'s close handler: when the request declared TE: trailers (as gRPC requests always do), and the stream closes with rstCode 0 without a 'trailers' event and without a trailers-only response (grpc-status in the initial HEADERS), reject with Code.Unavailable:

http/2 stream closed with error code NO_ERROR (0x0) before trailers were received

This mirrors grpc-js, which reports the same case as Received RST_STREAM with code 0 (Call ended without gRPC status) and ignores NO_ERROR resets only when the status already arrived.

Design notes

  • Why the TE: trailers signal: pre-#63249, no public Node API distinguishes this truncation generically — the 'trailers' event is the only deterministic signal (stream.closed-at-'end' is a timing artifact, endAfterHeaders does not help). The TE header is how the request itself declares that trailers must terminate the response, so the guard is scoped precisely to protocols that need it. If you'd prefer an explicit option plumbed down from the transports instead of reading the request header, happy to rework.
  • Relationship to Reject unary calls when the http/2 stream closes with NO_ERROR before the response #1747: that PR handles the complementary case — RST(NO_ERROR) before response headers (currently a hang). This PR deliberately does not overlap it: for TE: trailers requests the new guard also catches the pre-response close (reject instead of hang), while requests without TE: trailers keep today's behavior in that case, which remains Reject unary calls when the http/2 stream closes with NO_ERROR before the response #1747's scope. The two compose cleanly if both land.
  • Conservative by construction: a NO_ERROR reset after a complete response (incl. trailers) still succeeds — per RFC 9113 §8.1, "Clients MUST NOT discard responses as a result of receiving such a RST_STREAM" — and a genuinely complete-but-trailerless HTTP response (e.g. a proxy's HTML error page) still surfaces its HTTP status error, not the new guard. Both are pinned by tests.
  • Not covered: gRPC-web (trailers travel in the body, so the http/2 layer cannot know they are missing); on post-#63249 Node the new branch becomes harmless dead code, since the reset then surfaces as an explicit stream error first.

Testing

  • Package tests: 113/113. 5 new failure-mode tests fail without the fix (mid-message cut, cut before trailers, pre-response close — at the universal-client level and through createTransport from protocol-grpc with real error-message assertions); 4 new non-regression tests (post-trailers reset succeeds, trailers-only response succeeds, trailerless non-TE response succeeds, HTTP error response without trailers keeps its HTTP status error).
  • Conformance: 6258/6258 client and 4718/4718 server.
  • The fix was additionally verified against a live Envoy-fronted environment exhibiting the bug: the misleading protocol errors became the new Unavailable on the identical failing streams (details in connect-node: RST_STREAM(NO_ERROR) mid-response yields misleading incomplete envelope / missing status instead of a transport error #1751).

… with Code.Unavailable

When a peer (typically a proxy such as Envoy) resets a stream with
RST_STREAM code NO_ERROR before the response completed, Node's http2
client (pre nodejs/node#63249) surfaces the truncation as a clean,
error-free end-of-stream. The protocol layer then fails parsing the
truncated body with misleading errors: "protocol error: incomplete
envelope" (invalid_argument) or "protocol error: missing status"
(internal).

Track whether a request that declared "TE: trailers" (as gRPC requests
always do) received its terminating trailers or a trailers-only
response, and reject with Code.Unavailable from the close handler when
the stream ended with rstCode 0 without either.

Verified: package tests 113/113 (5 new failure-mode tests fail without
the fix, 4 new non-regression tests), conformance 6258/6258 client and
4718/4718 server.

Signed-off-by: Robin Wieruch <hello@rwieruch.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

connect-node: RST_STREAM(NO_ERROR) mid-response yields misleading incomplete envelope / missing status instead of a transport error

1 participant