Keep each reply's live reasoning apart in every SDK

.claude/skills/sdk-swift/SKILL.md

@@ -256,7 +256,8 @@ working. What the router writes is what it renders:
 - A reasoning step stores only its opening. The whole of it comes a window at a time on the
   live updates' `reasoning` field; `LiveReasoning` reassembles it, fed from the client's
   `MessageUpdatedEvent`s (the channel's message list can fold updates together), and goes to
-  `StreamingReasoningView(part:text:)`.
+  `StreamingReasoningView(part:text:)`. Look a step up by its reply and its id: every reply
+  numbers its steps from `r1`.
 - A call that asks first is `awaiting_approval` with an `approval`: `AIToolApprovalView`, with
   an `AIToolApprover` whose `decide` is `session.decide`. On a call there is no channel; show
   `session.approvals` with `AIToolApprovalCard`.

.claude/skills/sdk/SKILL.md

@@ -594,6 +594,7 @@ The `voice` request fields (`CreateSessionRequest`, `AgentConfigRequest`, `Updat
 `AgentConfig`, `AgentConfigRequest`, `AgentConfigPatch` and `SyncAgentRequest` take an optional `role` (`AgentRole`, one value: `agent_moderator`), and `listAgentConfigs` takes a `role` query parameter that answers the app's one live config in that role, test copies left out (AI-1472). A second live config in the same role is a 409 `role_taken`; left out on an update, a patch or a sync, the stored role stays. Go has the regenerated client and sends `role` from agent.yaml (`agents.Settings.Role`); JavaScript has the regenerated types and Python (`plugins/stream`) the regenerated models. JavaScript, Python, .NET, Ruby, Rust and PHP need to read `role:` from agent.yaml and send it in their folder sync, and .NET, Ruby, Rust and PHP need their generated models regenerated. Swift, Kotlin and Dart need nothing, since configs are server-side only.
 
 `createRun` is capped per app (AI-1478): at most `ROUTER_RUNS_CONCURRENT_PER_APP` runs in flight, 20 by default, counted in Redis across routers. A run over it is a 429 `concurrency_limited` with `Retry-After`, which a caller retries after that wait. A spent budget moves from 429 to 403 `budget_exceeded`, which a caller must not retry until the budget resets. `ErrorDetail.code` gains `concurrency_limited`. Only descriptions changed in the spec. Go has the regenerated client and JavaScript the regenerated types. .NET, Ruby, Rust, PHP and Python pick up the descriptions when they regenerate, and a hand-written run method should retry a 429 after `Retry-After` and treat a 403 `budget_exceeded` as final. Swift, Kotlin and Dart need nothing, since the operation is server-side only.
+`LiveReasoning` keeps each reply's thinking apart (no spec or router change). The router names a reply's reasoning steps `r1`, `r2`, ... per reply message, so every reply has an `r1`; keyed by step id alone, a later reply's round was dropped when shorter than an earlier reply's `r1`, run on after it when longer, and the earlier reply's thinking showed in its place. A step is now looked up by its reply's message id and its own id, which `read` takes from the message: Swift `reasoning[message: message.id, step: step.id]`, JavaScript `reasoning.get(message.id, step.id)`, Kotlin `reasoning[message.id, step.id]` (and `add(text, offset, messageId, stepId)`), Dart `reasoning[(message: message.id, step: step.id)]`. No other SDK has a `LiveReasoning`.
 `ModelCallTiming` (the call timeline's `model_calls`) gains an optional `error_code` (AI-1466): why a model call did not run to its end, left out when `success` is true. `cancelled` is a call its caller closed first (a superseded flow decision, a losing hedge, a barge-in): no failure, and its tokens are still billed. Any other code is a failure. The session socket's `model_call` frame carries the same `error_code`, and its `success` is now false for a cancelled call (it was true). Go has the regenerated client, JavaScript the regenerated types and Python (`plugins/stream`) the regenerated model (only this change's hunk); Python's latency log prints `CANCELLED` for one. Kotlin's generated code was regenerated too, which also brings in the `ErrorType.timeout` and `ErrorDetail.code` docs AI-1471 left for it. .NET, Ruby, Rust and PHP need their generated `ModelCallTiming` regenerated and should show a cancelled call as cancelled rather than failed; none needs another code change. Swift, Kotlin and Dart need nothing else, since they have no `ModelCallTiming` type.
 `Connector` (`GET /v1/agents/connectors` and `/{id}`) gains an optional read-only `sandbox` (`ConnectorSandbox` `{env, hosts}`, AI-1573): the env var code in an agent's sandbox reads a placeholder from, and the hosts the sandbox's egress puts the connection's credential in its place for. Only `github` has one (revision 6: `GITHUB_TOKEN`, `api.github.com` and `uploads.github.com`); it is left out for every other connector. Go has the regenerated client, JavaScript the regenerated types and Python (`plugins/stream`) the regenerated `Connector` and `ConnectorSandbox` (only this change's hunks). .NET, Ruby, Rust and PHP need their generated `Connector` regenerated; none needs a code change. Swift, Kotlin and Dart need nothing, since the catalog is server-side only.
 

CHANGELOG.md

@@ -2359,6 +2359,7 @@ own `StreamVideo` is left as it is.
 - A line the agent says on a call (`say`, and the greeting) is in its transcript and its responses, and the model knows it was said. It went straight to the voice with no reply announced, so the Chat channel and `responses.items` never had it, and the caller's answer to a greeting's question reached the model and the flow controller with no question before it. A line that opens the call is told to the model in its instructions rather than as a first turn, which chat templates that need a user turn first refuse; a conversation is not named after its greeting (#1040).
 - The agent's spoken replies in a call's Chat channel carry `ai_generated`, as replies in a conversation held in writing already did, so Stream's AI components render them as the assistant's (#1040).
 - What a caller says after an answer, on a transcriber that keeps the run of speech open (Deepgram Flux resuming a turn), is heard as only the new words. The whole run was passed on each time, so every later turn repeated the earlier ones to the model, the flow controller, in `heard` and in the transcript, more each turn. Words added to something being read out ("order 1 2" then "3 4", "one two" then "three", letters spelled out) are still heard whole (#1040).
+- `LiveReasoning` keeps each reply's reasoning apart, in the Swift, JS, Kotlin and Dart SDKs. It looked a step up by its id alone, and every reply numbers its steps from `r1`, so a later reply's live thinking was dropped when its round was shorter than an earlier reply's, or joined onto the earlier text when it was longer, and the earlier reply's thinking showed in its place. A step is now read by its reply and its id: `reasoning[message: message.id, step: step.id]` in Swift, `reasoning.get(message.id, step.id)` in JS, `reasoning[message.id, step.id]` in Kotlin (whose `add` now takes the message id before the step id), and `reasoning[(message: message.id, step: step.id)]` in Dart. (#1051)
 - The router's agent answers a question whose turn decision timed out. The decision call is given 3 s; when its stream had opened but no token came by then, the call was closed and no ruling was sent, so the question was never answered nor stored. That ruling now carries the timeout, and the agent answers a finished question whose ruling failed, as it already did when the call timed out before it opened (AI-1460).
 - A person on a voice call with no display name goes under `Speaker 1`, `Speaker 2` and on, numbered in the order such people first spoke, in what the reply model reads (`Speaker 1: what about me`) and in what the flow controller is asked. Their user id, or their participant id when they had no user, went ahead of their words instead, so the model read the app's internal ids. Their name, once it is known, still replaces the label, and named people read as before (AI-1458).
 - A phone's tools work on the call of a person's saved conversation. Every tool call there needed a typed question's request id, which a spoken question and the vision skill's request for camera frames never have, so on a call each was refused with `session: tool call has no durable command binding` and the agent said the tool did not work. A request id is now needed only in a conversation held in writing, as taking the results already was.

examples/voice_agents/android_demo/app/src/main/kotlin/io/getstream/visionagents/android/demo/ChatScreen.kt

@@ -197,7 +197,7 @@ private fun MessageRow(message: Message, chat: AgentChat, approver: AIToolApprov
                 val step = part.reasoning
                 if (step != null) {
                     StreamingReasoning(
-                        text = version.let { chat.reasoning[step.id] } ?: step.preview.orEmpty(),
+                        text = version.let { chat.reasoning[message.id, step.id] } ?: step.preview.orEmpty(),
                         isThinking = step.isStreaming,
                         durationSeconds = step.durationSeconds,
                         summary = step.summary,

examples/voice_agents/swift_assistant/app/SwiftAssistant/MessageRow.swift

@@ -36,7 +36,7 @@ struct MessageRow: View {
                         // The step keeps only the opening of its thinking; the whole of it
                         // came with the live updates, while the reply was being written.
                         if let step = part.reasoning {
-                            StreamingReasoningView(part: step, text: reasoning[step.id])
+                            StreamingReasoningView(part: step, text: reasoning[message: message.id, step: step.id])
                         } else {
                             AIMessagePartView(part: part, approver: approver)
                         }

examples/voice_agents/swift_demo/app/SwiftDemo/ChatView.swift

@@ -149,7 +149,7 @@ private struct MessageRow: View {
                         // The step keeps only the opening of its thinking; the whole of it
                         // came with the live updates, while the reply was being written.
                         if let step = part.reasoning {
-                            StreamingReasoningView(part: step, text: reasoning[step.id])
+                            StreamingReasoningView(part: step, text: reasoning[message: message.id, step: step.id])
                         } else {
                             AIMessagePartView(part: part, approver: approver)
                         }

sdks/dart/chat/lib/src/chat.dart

@@ -52,12 +52,15 @@ extension SessionChat on AgentSession {
 /// The whole of it reaches people watching the reply live: each update carries the next
 /// window of the round being streamed, in the message's `reasoning` field. [read] every update
 /// into one of these, from the client's `message.updated` events rather than the channel's
-/// message list, which can fold several updates into one, and show a step by its id.
+/// message list, which can fold several updates into one, and show a step by its reply and its
+/// id, as `reasoning[(message: message.id, step: step.id)]`: every reply numbers its steps
+/// from `r1`.
 final class LiveReasoning {
-  final _steps = <String, ({String text, int end})>{};
+  final _steps = <({String message, String step}), ({String text, int end})>{};
 
-  /// The thinking of reasoning step [stepId] so far, or null before any of it arrived.
-  String? operator [](String stepId) => _steps[stepId]?.text;
+  /// The thinking of reasoning step `step` of reply `message` so far, or null before any of it
+  /// arrived.
+  String? operator [](({String message, String step}) key) => _steps[key]?.text;
 
   /// Reads the window a live update carries, if it carries one.
   void read(stream.Message message) {
@@ -66,24 +69,24 @@ final class LiveReasoning {
       'offset': final num offset,
       'text': final String text,
     }) {
-      _add(text, offset.toInt(), id);
+      _add(text, offset.toInt(), (message: message.id, step: id));
     }
   }
 
-  /// Adds what [text], which starts [offset] code points into step [id]'s thinking, has that
-  /// is not held yet. A window that repeats recent thinking, for somebody who joined midway,
-  /// overlaps what is held; one that starts past it means updates were missed, and is kept
-  /// after a paragraph break rather than run on.
-  void _add(String text, int offset, String id) {
-    final held = _steps[id] ?? (text: '', end: 0);
+  /// Adds what [text], which starts [offset] code points into the thinking of the step [key]
+  /// names, has that is not held yet. A window that repeats recent thinking, for somebody who
+  /// joined midway, overlaps what is held; one that starts past it means updates were missed,
+  /// and is kept after a paragraph break rather than run on.
+  void _add(String text, int offset, ({String message, String step}) key) {
+    final held = _steps[key] ?? (text: '', end: 0);
     final runes = text.runes;
     final end = offset + runes.length;
     if (end <= held.end) {
       return;
     }
     final gap = offset > held.end && held.text.isNotEmpty ? '\n\n' : '';
     final fresh = String.fromCharCodes(runes.skip(held.end > offset ? held.end - offset : 0));
-    _steps[id] = (text: '${held.text}$gap$fresh', end: end);
+    _steps[key] = (text: '${held.text}$gap$fresh', end: end);
   }
 }
 

sdks/dart/chat/test/chat_test.dart

@@ -67,22 +67,29 @@ void main() {
   group('LiveReasoning', () {
     late LiveReasoning reasoning;
 
-    void window(String text, int offset, [String id = 'r1']) => reasoning.read(
-      stream.Message(
-        text: 'ignored',
-        extraData: {
-          'reasoning': {'id': id, 'offset': offset, 'text': text, 'length': offset + text.length},
-        },
-      ),
-    );
+    void window(String text, int offset, [String id = 'r1', String message = 'reply']) =>
+        reasoning.read(
+          stream.Message(
+            id: message,
+            text: 'ignored',
+            extraData: {
+              'reasoning': {
+                'id': id,
+                'offset': offset,
+                'text': text,
+                'length': offset + text.length,
+              },
+            },
+          ),
+        );
 
     setUp(() => reasoning = LiveReasoning());
 
     test('windows in order make the whole thinking', () {
       window('The order was ', 0);
       window('delivered unopened.', 14);
 
-      expect(reasoning['r1'], 'The order was delivered unopened.');
+      expect(reasoning[(message: 'reply', step: 'r1')], 'The order was delivered unopened.');
     });
 
     test('a repeat of recent thinking adds only what is new', () {
@@ -91,36 +98,63 @@ void main() {
       window('the policy, which allows 30 days', 9);
       window('policy', 13);
 
-      expect(reasoning['r1'], 'Checking the policy, which allows 30 days');
+      expect(
+        reasoning[(message: 'reply', step: 'r1')],
+        'Checking the policy, which allows 30 days',
+      );
     });
 
     test('a missed window leaves a paragraph break rather than running on', () {
       window('First thought.', 0);
 
       window('Much later.', 40);
 
-      expect(reasoning['r1'], 'First thought.\n\nMuch later.');
+      expect(reasoning[(message: 'reply', step: 'r1')], 'First thought.\n\nMuch later.');
     });
 
     test('somebody joining midway starts from the window they get', () {
       window('the rest of it', 300, 'r2');
 
-      expect(reasoning['r2'], 'the rest of it');
-      expect(reasoning['r1'], isNull);
+      expect(reasoning[(message: 'reply', step: 'r2')], 'the rest of it');
+      expect(reasoning[(message: 'reply', step: 'r1')], isNull);
     });
 
     test('positions count code points as the router does', () {
       window('café ', 0);
 
       window(' 🌸 bloom', 4);
 
-      expect(reasoning['r1'], 'café 🌸 bloom');
+      expect(reasoning[(message: 'reply', step: 'r1')], 'café 🌸 bloom');
+    });
+
+    // Every reply numbers its steps from `r1`, so a later reply's first round is not the
+    // earlier one's, whether it thinks for less or for more.
+    test('each reply keeps the thinking of its own steps', () {
+      window('Looking up the order history and the return policy.', 0, 'r1', 'first');
+
+      window('Refund it.', 0, 'r1', 'second');
+      window(
+        'Checking which of the two orders arrived damaged, then the refund window.',
+        0,
+        'r1',
+        'third',
+      );
+
+      expect(
+        reasoning[(message: 'first', step: 'r1')],
+        'Looking up the order history and the return policy.',
+      );
+      expect(reasoning[(message: 'second', step: 'r1')], 'Refund it.');
+      expect(
+        reasoning[(message: 'third', step: 'r1')],
+        'Checking which of the two orders arrived damaged, then the refund window.',
+      );
     });
 
     test('a message with no window changes nothing', () {
-      reasoning.read(stream.Message(text: 'Hello'));
+      reasoning.read(stream.Message(id: 'reply', text: 'Hello'));
 
-      expect(reasoning['r1'], isNull);
+      expect(reasoning[(message: 'reply', step: 'r1')], isNull);
     });
   });
 }