Repository navigation
Mask the blind spots a lift between distant anchors used to drop - #11
Merged
Merged
Conversation
The H37Rv blind spots are lifted onto the run's reference by interpolating between shared unique k-mers, and a blind spot was dropped wherever the anchors around it were far apart. Blind spots are repeats, where anchors are scarce: on two MTBC assemblies of the 185-sample cohort, the mask covered 63% and 65% of the blind spots minimap2 places on each, and 46,700 to 49,800 bp of problematic sites went to the variant caller unmasked. PREPARE_REFERENCE now lifts them with --align-gaps, which aligns the stretch between two anchors where it cannot interpolate: 97% and 98% are masked at minimap2's place, with 700 to 1,100 bp masked elsewhere, in about 15 s and 1 GB. About 15% of the H37Rv blind spots are stretches those assemblies do not have, with nothing to mask.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed and why
With
--mask_blindspotsand--blindspot_liftover, the H37Rv Illumina blind spots are lifted onto each run's reference. The lift interpolates between shared unique k-mers and drops a position wherever the anchors around it are far apart. Blind spots are repeats, where anchors are scarce, so on other MTBC assemblies a third of them were never masked and went to the variant caller.Checked against minimap2's alignment of each of the cohort's two references to H37Rv, with the MTBC-ancestor FASTA the container ships (same SHA-256):
--sample 10)--align-gapsSampling is not the cause: with every anchor kept, interpolation alone masks 66%.
PREPARE_REFERENCEnow lifts the blind spots with--align-gaps, the alignment between two anchors from #9. About 15% of the H37Rv blind spots are stretches these assemblies do not have, with nothing to mask.Filling each blind-spot stretch between its lifted ends was also tried. It masked 1.4 points more but doubled the bases masked elsewhere, so it is not included.
Outputs and defaults
PREPARE_REFERENCE's 8 GB.--mask_blindspots false(the default) or--blindspot_liftover false.How it was verified
The numbers above come from lifting
assets/H37Rv_blindspots.bedexactly asPREPARE_REFERENCEdoes, onto the two real references, and comparing every position with minimap2. Lifted onto H37Rv itself, 99.4% of the blind spots land on their own position. The rest are at the two ends of the genome, where there is nothing to interpolate from, and scattered inside long repeats.--blindspot_liftover falseis still the setting for a reference in H37Rv coordinates.Checklist
tests/run_tests.shpasses locally (unit and lint; the pipeline leg in CI)ruff check .is clean, and I did not restyle code the change does not touch--align-gapsis covered intest_pathotypr_liftover.py)modules/has astub:block (not applicable: no new process)tests/data/was not edited by hand (fixtures come frommake_test_data.py)CHANGELOG.mdentry and, where relevant, a docs update