Skip to content

Deliver indels, and read two SNPs of one codon as one change - #13

Merged
Paururo merged 2 commits into
mainfrom
feat/indels-mnv
Sep 24, 2026
Merged

Paururo merged 2 commits into
mainfrom
feat/indels-mnv

Conversation

@Paururo

@Paururo Paururo commented Sep 24, 2026

Copy link
Copy Markdown
Member

What changed and why

Two things the pipeline did not deliver: its indels, and the amino-acid change of a codon that carries more than one SNP.

Indels. The main VCF holds SNPs only, and of the indels only the homozygous ones reached a filtered file (the legacy var.homo.indel.vcf). A minority indel was in none, and a minority frameshift is how resistance through Rv0678 often starts. Indels now get the definition a SNP has: each sample's <sample>.<ref>.indels.ann.vcf.gz keeps every indel FreeBayes made, normalised, with FILTER PASS where it meets the hom or het rule and LowSupport where not, so a minority call keeps its fraction instead of vanishing. <samplesheet>_indel_matrix.tsv holds the run's indels the way the SNP matrix holds its SNPs, and the report's matrix panel gains an Indels view with frameshifts marked.

Codon-level changes, with get_MNV. Two SNPs in one codon change one amino acid, and annotated a base at a time they name two changes that are not there. GET_MNV runs get_MNV 1.1.5 on each sample's passing SNPs and indels with the BAM they were called on, and reads every codon whole on the reads that span it. get_MNV also writes a combined change for two SNPs of one codon on different molecules, which no read carries, so <samplesheet>_mnv.tsv keeps only the codons at least --mnv_min_reads reads carry whole. The SNP matrix gains codon_change / codon_change_samples, and the report marks those SNPs and their cells.

On reads simulated from an H37Rv window (rpoB + Rv0678), called with FreeBayes and annotated with SnpEff:

Codon Per-base annotation says Read whole Reads carrying it Kept
rpoB 445 (CAC>GAG) His445Asp + His445Gln His445Glu 103 / 104 yes
rpoB 460 (GAG>TTG), minority Glu460* (stop) + Glu460Val Glu460Leu (MNV-masked) 23 / 99 yes
rpoB 430, two SNPs on different molecules Leu430Met + Leu430Leu Leu430Ile 0 / 115 no

The same run's indel matrix: Rv0678 c.144dupC p.Glu49fs, c.301delG p.Ala101fs at AF 0.1852, and c.402_404delACG p.Arg135del at AF 0.8061; the sample without them reads AF 0 at the anchor's depth.

A report fix on the way. The report read single-base records only, so every SNP FreeBayes wrote as an MNP (CAC>GAG in one record) was in the SNP matrix and in none of the report's tables. It now reads such a record as the SNPs it is made of.

Outputs and defaults

  • New per sample: <sample>.<ref>.indels.ann.vcf.gz (+ .tbi; indels.vcf.gz with --annotate_main_vcf false), var.het.indel(.ann).vcf in the legacy split set, and mnv/ with get_MNV's TSV, VCF, summary and run manifest.
  • New per run: <samplesheet>_indel_matrix.tsv and <samplesheet>_mnv.tsv.
  • Changed: <samplesheet>_snp_matrix.tsv gains two columns after aa_change, codon_change and codon_change_samples, when get_MNV runs. Anything that reads that file by column position needs to account for them; with --run_mnv false the layout is unchanged.
  • New parameters, on by default: --make_indel_matrix, --run_mnv, --mnv_min_reads 2, --mnv_gff_features CDS, --mnv_translation_table 11, --mnv_container.
  • New processes: GET_MNV (per sample, 2 CPUs, 2 GB), MNV_TABLE, INDEL_MATRIX, each with a stub.
  • Container: GET_MNV runs in quay.io/biocontainers/get_mnv 1.1.5, pinned by digest, until the pipeline image carries it. The Dockerfile now installs get_mnv=1.1.5, but this PR publishes no image (docker-publish runs on a version tag); after the next release, bump params.container and empty --mnv_container. The Singularity profiles keep the docker:// scheme and the docker profile strips it, as for the main image.
  • Garnatxa: the launcher runs main, so the first launch after the merge runs GET_MNV and pulls its image from quay.io.
  • A gzipped GFF, which the samplesheet accepts, is decompressed for get_MNV, which reads plain text only.

How it was verified

tests/run_tests.sh        # ruff clean · unit 2,218 passed, 39 skipped · JS 120 passed · pipeline 39 passed, and the container test added since, 7 of 7 on its own
mkdocs build --strict     # passes
  • New tests: the indel matrix (10), the run's codon table on rows from a real get_MNV run (7), the report's indel view and codon marks including the MNP expansion (7), the SNP matrix's codon columns and its unchanged layout without them (4), the indel soft filter against a real bcftools (1), the GET_MNV script with a gzipped and a plain GFF (2), and the get_MNV image's scheme per profile (1). -profile test_full reaches the three new processes.
  • End to end on simulated data: FreeBayes, SnpEff, get_MNV 1.1.5 and the new scripts, with the results above.
  • Not run: the full pipeline on real data, and the get_MNV container itself (no Docker daemon locally; the same get_MNV version was run from bioconda).

Checklist

  • tests/run_tests.sh passes locally (all four legs)
  • ruff check . is clean, and I did not restyle code the change does not touch
  • New or changed behaviour has a test, and tests pin what the code does today
  • A new process in modules/ has a stub: block, and -profile test_full reaches it
  • tests/data/ was not edited by hand (fixtures come from make_test_data.py)
  • User-visible changes have a CHANGELOG.md entry and, where relevant, a docs update

The main VCF holds SNPs only, and of the indels only the homozygous
ones reached a filtered file, the legacy var.homo.indel.vcf. A minority
indel was in none, and a minority frameshift is how resistance through
Rv0678 often starts. CALL_FREEBAYES now writes each sample's indels,
normalised, with FILTER PASS where they meet the hom or het rule a SNP
has and LowSupport where not; annotated, they are
<sample>.<ref>.indels.ann.vcf.gz. INDEL_MATRIX puts the run's indels in
<samplesheet>_indel_matrix.tsv: every indel a sample calls with a PASS,
its length, gene, effect and HGVS, and per sample the fraction, the
depth and the filter, with AF 0 at the anchor's depth where a sample
was read without it. The report's matrix panel gains an Indels view,
and the legacy split set gains var.het.indel.vcf.

Two SNPs in one codon change one amino acid, and annotated a base at a
time they name two changes that are not there. GET_MNV runs get_MNV
1.1.5 on each sample's passing SNPs and indels with the BAM they were
called on, and reads every codon whole on the reads that span it.
MNV_TABLE keeps in <samplesheet>_mnv.tsv the codons at least
--mnv_min_reads reads carry whole: get_MNV also writes a combined
change for two SNPs of one codon on different molecules, which no read
carries. With it, the SNP matrix gains codon_change and
codon_change_samples. On reads simulated from H37Rv, rpoB codon 445
reads His445Glu on 103 of 104 reads (base by base, His445Asp and
His445Gln), a pair in codon 460 on 23% of the reads reads Glu460Leu
where its first base alone says a stop, and two SNPs of codon 430 on
different molecules stay two changes. get_MNV runs in its bioconda
image, pinned by digest, until the pipeline image is rebuilt with it;
the Dockerfile now installs it.

The report read single-base records only, so every SNP FreeBayes wrote
as an MNP (CAC>GAG: two changes of one codon on the same reads) was in
the SNP matrix and in none of the report's tables. It now reads such a
record as the SNPs it is made of.
The first real run of GET_MNV failed on every sample. get_MNV writes
its VCF and indexes it with tabix, which the bioconda image it runs in
does not carry, so it warned, skipped the index and exited 0, and
Nextflow failed the task over the .tbi it had been told to expect. A
stub run touches every declared output, so only a real run could see
it; an end-to-end run did.

The VCF is now published bgzipped and unindexed (tabix -p vcf indexes
it as it is), and the codon table the run reads never used the index.
@Paururo
Paururo merged commit 6b9d170 into main Sep 24, 2026
7 checks passed
@Paururo
Paururo deleted the feat/indels-mnv branch September 24, 2026 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant