[NicoDeVeaux]

fix(bam): unmapped sweep returns only records with no reference

For PR #118 (noodles migration). Base: #118's head 0b00cf1 (perf(bam): stream CRAM reads via self-referential cursors), the newest at the time of writing.

What

IndexedReader::fetch(Unmapped) now returns exactly the records with no reference sequence ID, which is what main's htslib path returns (FetchDefinition::Unmapped = HTS_IDX_NOCOOR):

• BAM (BamCursor::Unmapped): keep a record only when reference_sequence_id() is None, instead of whenever it carries the UNMAP flag.
• CRAM (query_unmapped): the same filter on the returned records.

Why

On the parallel path (--threads > 1 with an index, the path nf-core/rnaseq uses), each reference is read with fetch(tid) and then the unplaced reads with fetch(Unmapped).

• BAM: the sweep seeks to index.last_first_record_start_position(), the first record overlapping the last reference's final 16 kb linear-index window, and keeps every UNMAP-flagged record from there on. A placed unmapped mate (flag 0x4 with a tid, as aligners write for pairs with one mate unmapped) in that window was already returned by its reference's fetch, so it is counted twice. Aligners that place unmapped mates (STAR/RSEM and HISAT2 BAMs in rnaseq) hit this whenever such a mate falls in the last contig's final window.
• CRAM: query_unmapped starts at the slice the index lists for unplaced reads. In a multi-reference slice that slice also holds placed records, and the placed unmapped mate is again returned twice.

Minimal input (BAM, chr1/chr2 of 100 kb; 6 records):

r0  0    chr1  1001   60  50M  *  0      0
r1  0    chr1  1201   60  50M  *  0      0
r2  73   chr2  99001  60  50M  =  99001  0
r3  133  chr2  99001  0   *    =  99001  0
u1  4    *     0      0   *    *  0      0
u2  4    *     0      0   *    *  0      0

	in total	primary	read2	mapped
samtools 1.22.1	6 + 0	6 + 0	1 + 0	3 + 0 (50.00%)
main bea5571, -t 4	6 + 0	6 + 0	1 + 0	3 + 0 (50.00%)
#118 0b00cf1, -t 1	6 + 0	6 + 0	1 + 0	3 + 0 (50.00%)
#118 0b00cf1, -t 4	7 + 0	7 + 0	2 + 0	3 + 0 (42.86%)
this branch, -t 4	6 + 0	6 + 0	1 + 0	3 + 0 (50.00%)

How tested

• Unit tests in src/rna/bam/io.rs: bam_unmapped_sweep_returns_only_unplaced_records (noodles-written BAM of the records above; the sweep must return exactly u1, u2, and the per-reference fetches plus the sweep must return each record once) and cram_unmapped_sweep_returns_only_unplaced_records (two samtools 1.22.1 CRAMs of the same records under tests/data/unmapped_sweep/, one with per-reference slices and one with a single multi-reference slice; see the README there). On 0b00cf1 both fail with the sweep returning r3, u1, u2.
• Integration test test_parallel_unmapped_sweep_counts_placed_unmapped_mate_once: flagstat of rustqc rna at --threads 1 and 4 must equal samtools 1.22.1's; on 0b00cf1 -t 4 gives 7 + 0 in total.
• cargo test --release --locked --offline: 206 unit, 12 bin, 19 integration, 2 doc tests pass. cargo fmt --check passes. cargo clippy -- -D warnings reports the same three needless_range_loop errors as unmodified main with the local toolchain (clippy 0.1.90); with that lint allowed, --all-targets is clean.
• Formal-review harness (pr118_confirm.py, baseline main bea5571): on 0b00cf1, W1 and W4 count one record twice at -t 4 (4 -> 5, 6 -> 7) as predicted; on this branch every world matches main and -t 1 (W1 4, W2 5, W3 5, W4 6). The script itself now exits 1, because it asserts the PR's predicted double count.

Not changed, and why

• The BAM sweep still seeks to the last reference's final linear-index window and skips the placed records there; seeking to the end of the last reference's chunks would skip less but needs more index plumbing for no change in output.
• A related CRAM problem is left for a separate change: in a multi-reference slice, fetch(tid) also returns the other references' records, because noodles' query filters on the interval only. With the multi-reference fixture at -t 4 this branch prints 10 + 0 in total (0b00cf1: 11, main: 6). The cram_unmapped_sweep test checks only the sweep for that file for this reason. A follow-up branch, fix/pr118-cram-multiref-fetch, stacked on this one, fixes it.

🤖 Generated with Claude Code