Skip to content

Keep the expression decoder's frames small when optimized (Covered by… - #653

Merged
brianegge merged 1 commit into
morganstanley:mainfrom
brianegge:claude/happy-carson-gbigyg
Oct 4, 2026
Merged

brianegge merged 1 commit into
morganstanley:mainfrom
brianegge:claude/happy-carson-gbigyg

Conversation

@brianegge

Copy link
Copy Markdown
Contributor

OSS-Fuzz issue 569129860: stack-overflow in hobbes::decode, reached from fuzz-type-decode through a TExpr. The expression decoder splits each node kind into its own function so that the frame that recurs per nesting level only holds that kind's locals, but each of those functions has a single caller, and an optimizing build inlined them all back into decode(). Under the OSS-Fuzz build (-O1 with ASan) that frame was 3.6KB, so the 2000 levels maxDecodeNesting allows needed about 7MB of stack, and ClusterFuzz ran out before the bound was reached.

Mark the per-kind decoders noinline. decode() is now 224 bytes of ASan frame and the recurring path well under 1KB a level, so a description at the bound decodes in under 1MB of stack in that build.

Adds the ClusterFuzz testcase to the type-decode corpus, and a test that decodes a description at the nesting bound on a thread with a small stack (2MB, or 4MB under ASan); it overflows on the unfixed code in the OSS-Fuzz build configuration.

Also documents in AGENTS.md how to set up a build on Ubuntu 24.04 (including cloud agent containers) and macOS, and how to reproduce an OSS-Fuzz crash.

Old bug, present since the first OSS-Fuzz build; not a regression.

Claude-Session: https://claude.ai/code/session_018Y1u5Q3rXMvtPYL8eGktgn

… dco/Brian_Egge.md)

OSS-Fuzz issue 569129860: stack-overflow in hobbes::decode, reached from
fuzz-type-decode through a TExpr. The expression decoder splits each node
kind into its own function so that the frame that recurs per nesting level
only holds that kind's locals, but each of those functions has a single
caller, and an optimizing build inlined them all back into decode(). Under
the OSS-Fuzz build (-O1 with ASan) that frame was 3.6KB, so the 2000 levels
maxDecodeNesting allows needed about 7MB of stack, and ClusterFuzz ran out
before the bound was reached.

Mark the per-kind decoders noinline. decode() is now 224 bytes of ASan
frame and the recurring path well under 1KB a level, so a description at
the bound decodes in under 1MB of stack in that build.

Adds the ClusterFuzz testcase to the type-decode corpus, and a test that
decodes a description at the nesting bound on a thread with a small stack
(2MB, or 4MB under ASan); it overflows on the unfixed code in the OSS-Fuzz
build configuration.

Also documents in AGENTS.md how to set up a build on Ubuntu 24.04 (including
cloud agent containers) and macOS, and how to reproduce an OSS-Fuzz crash.

Old bug, present since the first OSS-Fuzz build; not a regression.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Y1u5Q3rXMvtPYL8eGktgn
@brianegge
brianegge requested a review from bingenito October 4, 2026 10:16
@brianegge
brianegge enabled auto-merge October 4, 2026 10:16
@brianegge
brianegge requested review from dawa79 and kelliott55 October 4, 2026 10:17
@brianegge
brianegge merged commit 11e924d into morganstanley:main Oct 4, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants