Skip to content

Mask the blind spots a lift between distant anchors used to drop - #11

Merged
Paururo merged 1 commit into
mainfrom
fix/blindspot-lift
Sep 23, 2026
Merged

Paururo merged 1 commit into
mainfrom
fix/blindspot-lift

Conversation

@Paururo

@Paururo Paururo commented Sep 23, 2026

Copy link
Copy Markdown
Member

What changed and why

With --mask_blindspots and --blindspot_liftover, the H37Rv Illumina blind spots are lifted onto each run's reference. The lift interpolates between shared unique k-mers and drops a position wherever the anchors around it are far apart. Blind spots are repeats, where anchors are scarce, so on other MTBC assemblies a third of them were never masked and went to the variant caller.

Checked against minimap2's alignment of each of the cohort's two references to H37Rv, with the MTBC-ancestor FASTA the container ships (same SHA-256):

Now (--sample 10) With --align-gaps
E1ASM0035: blind spots masked at minimap2's place 84,457 bp (63%) 130,854 bp (97.4%)
E1ASM0035: blind spots left unmasked 49,837 bp 3,440 bp
E1ASM0057: blind spots masked at minimap2's place 87,485 bp (65%) 131,708 bp (98.2%)
E1ASM0057: blind spots left unmasked 46,695 bp 2,472 bp
Bases masked elsewhere ~300 bp 700–1,100 bp

Sampling is not the cause: with every anchor kept, interpolation alone masks 66%. PREPARE_REFERENCE now lifts the blind spots with --align-gaps, the alignment between two anchors from #9. About 15% of the H37Rv blind spots are stretches these assemblies do not have, with nothing to mask.

Filling each blind-spot stretch between its lifted ends was also tried. It masked 1.4 points more but doubled the bases masked elsewhere, so it is not included.

Outputs and defaults

  • The per-reference exclusion list gains the blind spots the lift used to drop. On an MTBC assembly that is about 45 kb more masked positions.
  • The step takes about 15 s and 1 GB per reference, within PREPARE_REFERENCE's 8 GB.
  • Nothing changes with --mask_blindspots false (the default) or --blindspot_liftover false.

How it was verified

tests/run_tests.sh unit   # 2,187 passed, 39 skipped
tests/run_tests.sh lint   # clean
mkdocs build --strict     # passes

The numbers above come from lifting assets/H37Rv_blindspots.bed exactly as PREPARE_REFERENCE does, onto the two real references, and comparing every position with minimap2. Lifted onto H37Rv itself, 99.4% of the blind spots land on their own position. The rest are at the two ends of the genome, where there is nothing to interpolate from, and scattered inside long repeats. --blindspot_liftover false is still the setting for a reference in H37Rv coordinates.

Checklist

  • tests/run_tests.sh passes locally (unit and lint; the pipeline leg in CI)
  • ruff check . is clean, and I did not restyle code the change does not touch
  • New or changed behaviour has a test, and tests pin what the code does today (--align-gaps is covered in test_pathotypr_liftover.py)
  • A new process in modules/ has a stub: block (not applicable: no new process)
  • tests/data/ was not edited by hand (fixtures come from make_test_data.py)
  • User-visible changes have a CHANGELOG.md entry and, where relevant, a docs update

The H37Rv blind spots are lifted onto the run's reference by
interpolating between shared unique k-mers, and a blind spot was
dropped wherever the anchors around it were far apart. Blind spots are
repeats, where anchors are scarce: on two MTBC assemblies of the
185-sample cohort, the mask covered 63% and 65% of the blind spots
minimap2 places on each, and 46,700 to 49,800 bp of problematic sites
went to the variant caller unmasked.

PREPARE_REFERENCE now lifts them with --align-gaps, which aligns the
stretch between two anchors where it cannot interpolate: 97% and 98%
are masked at minimap2's place, with 700 to 1,100 bp masked elsewhere,
in about 15 s and 1 GB. About 15% of the H37Rv blind spots are
stretches those assemblies do not have, with nothing to mask.
@Paururo
Paururo merged commit be3ece0 into main Sep 23, 2026
7 checks passed
@Paururo
Paururo deleted the fix/blindspot-lift branch September 23, 2026 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant