Repository navigation
[DSM] Encode AWS / Kinesis pathway context with a single Base64 layer - #9595
Open
robcarlan-datadog wants to merge 5 commits into
Open
robcarlan-datadog wants to merge 5 commits into
robcarlan-datadog wants to merge 5 commits into
Conversation
SQS, SNS and Kinesis producers Base64-encoded dd-pathway-ctx-base64 a second time, so other-language consumers could not read .NET pathways. Consumers have accepted both encodings since #9289. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BenchmarksBenchmark execution time: 2026-11-10 18:24:38 Comparing candidate commit 34bb518 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 72 metrics, 0 unstable metrics, 4 known flaky benchmarks, 121 flaky benchmarks without significant changes.
|
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (9595) and master. ✅ No regressions detected |
Kinesis now injects like EventBridge, so the adapter no longer needs a binary header path. This drops the legacy dd-pathway-ctx header on Kinesis, which no consumer reads there: the .NET Kinesis consumer does not extract a pathway and other tracers prefer dd-pathway-ctx-base64. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
robcarlan-datadog
marked this pull request as ready for review
November 9, 2026 19:03
robcarlan-datadog
requested review from
tabgok
and removed request for
a team
November 9, 2026 19:03
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![datadog-official[bot]](https://avatars.githubusercontent.com/in/1358509?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
More details
The single-Base64 encoding change preserves SQS/SNS legacy header encoding. The documented rollout constraint remains: SQS/SNS consumers older than v3.55.0 must be upgraded before producers to retain DSM pathway linkage.
🤖 Bits Code Review · Commit 319c792
tabgok
approved these changes
Nov 9, 2026
tabgok
left a comment
tabgok
left a comment
There was a problem hiding this comment.
This is fine by me, as long as DSM is fine with the breakage across versions
robcarlan-datadog
changed the title
bouwkast
reviewed
Nov 10, 2026
bouwkast
left a comment
bouwkast
left a comment
Collaborator
There was a problem hiding this comment.
I forgot to post the review 😅
Looks good to me overall, we were talking today and are wondering if we can get a config to re-enable the old double encode produce to give a way to migrate and update
Opt-in escape hatch that restores the double Base64 encoding on SQS/SNS producers while consumers older than 3.55.0 are still deployed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
robcarlan-datadog
requested review from
MilanGarnier
and removed request for
a team
November 10, 2026 17:09
Contributor
There was a problem hiding this comment.
More details
Single encoding is consistent with the shared propagator and receiving adapters. SQS/SNS consumers older than v3.55.0 still require consumer-first upgrades or the legacy producer encoding setting.
🤖 Bits Code Review · Commit 189c101
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
Stop double Base64 encoding
dd-pathway-ctx-base64on AWS producers (SQS, SNS, Kinesis). The value is now written exactly as the shared DSM propagator produces it, matching every other tracer.dd-pathway-ctxheader is unchanged (one Base64 layer, since it's raw bytes going into JSON).dd-pathway-ctxheader. Kinesis shouldn't write it: the context lives in the record's JSON.Reason for change
Follow-up to #9289, which made .NET consumers accept both single and double encoded pathways. With that in place, producers can switch to the single encoding, so other-language consumers can correctly read .NET-produced pathway contexts.
Breaking change
.NET SQS/SNS consumers older than v3.55.0 (the first release with #9289) can't read single-encoded pathways and will lose DSM pathway linkage from upgraded producers. Upgrade consumers before producers, or set
DD_DATA_STREAMS_LEGACY_ENCODING_BEHAVIOR_ENABLED=trueon SQS/SNS producers to keep the old double encoding until they are upgraded.Kinesis has no such concern: the .NET Kinesis consumer doesn't extract the pathway context.
Implementation details
AwsMessageAttributesHeadersAdapters.StringBuilderJsonAdapter.Add: write the Base64 header as-is instead of re-encoding it.Kinesis
ContextPropagation: callInjectPathwayContextAsBase64String; delete the now-unusedIBinaryHeadersCollectionimplementation fromKinesisContextAdapter.New
DD_DATA_STREAMS_LEGACY_ENCODING_BEHAVIOR_ENABLEDsetting (defaultfalse) restores the old SQS/SNS encoding. Kinesis doesn't honor it: no .NET Kinesis consumer extracts the pathway, so there's nothing to stay compatible with.Test coverage
Inject_EncodesPathwayOnce(wasInject_PreservesExistingDotnetWireFormat) now asserts single encoding.InjectTraceIntoDatatest decodes the injected value once and checks the pathway hash.🤖 Generated with Claude Code