Streaming overlap: drain listener tail, fast duration estimates, seam-safe chunk scheduling

[WinstonLiyt]

Summary

Fixes and speed-ups for the streaming overlap pipeline (python -m streaming.overlap: streaming TTS on the speaker side, streaming ASR + tree updates on the listener side).

metric (4 motions, 24 speeches)	before	after
playback starvation events (chunk arrives late)	5 (19.9 s total)	0
next chunk ready ahead of need, median / min	54.7 s / -4.4 s	85.3 s / +4.1 s
refinement timed out (non-first chunks)	11 / 98	0 / 108
audio at the end of each speech never transcribed for the opponent, median	11.8 s	0 s
single-chunk speeches (no streaming at all)	2	0
FastSpeech duration estimate, median	4.91 s	0.02 s
LLM rewrite call, median	8.2 s	1.6 s
TTS pipeline wall clock per speech, median	67.9 s	21.6 s
first-chunk latency, median / max	60 s / 121 s	41 s / 53 s
max abs budget remaining	40.6 s	4.0 s
seam silence between chunks, p90 / max	720 / 1240 ms	390 / 425 ms
ffmpeg decode warnings from the read/write race	46	0

Opening/rebuttal audio totals stay within 232-238 s of the 240 s budget in both versions; overrun is unchanged (2.7 s -> 2.5 s median).

What was wrong

1. The listener dropped the last 8-14 s of every speech. It transcribes in 15 s windows and had no drain at the end of a turn; since overlap turns are marked tree_via_streaming, the full-text analysis was skipped too, so the closing sentences of every speech never reached the opponent's tree.
2. Every duration estimate reloaded FastSpeech2 (checkpoint + vocoder + 206k-line lexicon + G2p), 4.9 s per call and ~20 GB GPU per process.
3. The rewrite LLM ran with default reasoning effort, ~10 s per call for a length-constrained rewrite.
4. Chunks arrived late because the refinement deadline ignored the work that happens after selection (worker joins, optional speed re-synthesis, decode, file write, bridge copy).
5. Closings without paragraph breaks became one chunk, with no streaming or refinement and up to 40 s of unused budget; chunk 0's in_range was hard-coded True.
6. Seam silence varied from 0.2 s to 1.2 s because tts-1 trailing silence is not constant.
7. continuous_audio.mp3 was rewritten in place while the listener read it; a Whisper request with no timeout once hung the listener for the rest of a turn; the playback simulator inserted ~1 s of bookkeeping between chunks.
8. The system prompt's word targets (520 / 260) correspond to 211 s of speech at the measured 2.46 words/s of tts-1 echo, so refinement was mostly expanding text (median 1.16x); with 580 / 290 it is 1.03x.

Changes by file

• src/utils/fs_wrapper.py, src/utils/time_estimator.py: shared FastSpeech2 instance, duration-only forward (no decoder/vocoder), cached lexicon/G2p, vectorized position table. Bit-identical durations.
• src/tts_streaming.py: reasoning_effort="minimal" for rewrites; sentence-packing chunker; 2 s deadline margin; non-blocking worker cleanup; gated speed adjustment; seam silence normalization; atomic chunk writes; honest chunk-0 in_range; optional calibration ablation (TTS_STREAM_CALIB=1, off by default, documented negative result).
• src/streaming/env.py: final drain of the remaining audio at turn end; decode cache keyed by mtime/size; Whisper call with 30 s timeout and retries.
• src/streaming/overlap.py: atomic write of continuous_audio.mp3; bookkeeping time charged against playback sleep.
• src/utils/prompts/others.py: word targets 520/520/260 -> 580/580/290.

Not changed, worth knowing

• First-chunk latency (~41 s median) is almost entirely tree planning and speech writing before TTS-Start, not the TTS pipeline.

How to reproduce

cd src
export NLTK_DATA=<conda-env>/nltk_data
CUDA_VISIBLE_DEVICES=0 python -m streaming.overlap --config configs/test2/case1/overlap_deepseek-chat.yml

Per-chunk timing lands in log_files/<id>_outputs/*_chunks/chunk_profile.csv; playback and listener events in log_files/<id>.log.