vcf-liftover-wrapper

High-throughput clinical VCF coordinate liftover engine (GRCh37 ⇄ GRCh38 via UCSC chain files), in Rust.

Status: Phases 1–6 complete

The full parse → decompose → normalize → lift → audit → write pipeline runs and is covered by an end-to-end integration test. The Phase 6 validation harness (scripts/) is in place; its diff/gate logic is self-tested, but the real accuracy comparison against bcftools +liftover must be run by you on a machine with bcftools + downloaded data — the harness fabricates no numbers. See scripts/README.md.

Workspace layout

Cargo.toml                         # workspace
crates/
  vlw-core/                        # library — logic, unit-testable w/o IO
    src/byte_span.rs   ✅ ByteSpan + Bytes (zero-copy field model)
    src/reason.rs      ✅ ReasonCode taxonomy (Picard-mirrored)
    src/typestate.rs   ✅ VariantRecord<State> + consuming transitions
    src/normalizer.rs  ✅ decomposition + indel left-alignment (Tan 2015)
    src/liftover.rs    ✅ chain projection via chainfile + coitrees
    src/auditor.rs     ✅ FASTA cross-check, swap/strand/NW-realign, STR
    src/lib.rs         ✅ crate root / re-exports
  vcf-liftover-wrapper/            # binary — CLI + streaming pipeline
    src/parser.rs      ✅ zero-copy mmap/gzip input + VCF line splitting
    src/reference.rs   ✅ in-memory FASTA reference (noodles-fasta)
    src/pipeline.rs    ✅ parser → worker pool → order-preserving writer
    src/lib.rs         ✅ library surface (testable pipeline)
    src/main.rs        ✅ clap CLI over pipeline::run

What's delivered (verified green)

• cargo test (56 tests incl. proptests + an e2e integration test), cargo clippy --all-targets -- -D warnings, and cargo fmt --check all clean.
• ByteSpan / Bytes — zero-copy offsets into the shared mmap buffer, with an owned escape hatch for allele rewrites.
• Typestate pipeline — Raw → Normalized → Mapped → Audited as sealed marker types with self-consuming transitions; "no stage skipped" is a compile-time guarantee.
• normalizer — multi-allelic decomposition + Tan/Abecasis/Kang 2015 left-alignment.
• liftover — chainfile-backed projection; MultiBlock / NoChain surfaced, not merged.
• auditor — target FASTA cross-check; REF/ALT swap (SNV and indel) with INFO reversal + genotype repair; strand inversion; Needleman-Wunsch chain-gap realignment; STR-ambiguity detection; every reject carries a ReasonCode.
• Streaming pipeline — mmap/gzip input, hand-rolled zero-copy VCF splitting, a worker pool over bounded crossbeam-channels with backpressure, and a writer that restores input order and emits a rejected-variants report.
• CI workflow (.github/workflows/ci.yml) running fmt/clippy/test.

Running the accuracy validation

The harness under scripts/ diffs our drops against bcftools +liftover and enforces the acceptance gate (our indel drop rate must not exceed bcftools's):

python scripts/test_compare_liftover.py   # verify the diff logic (synthetic)
scripts/fetch_data.sh                      # download real fixtures (large)
scripts/run_validation.sh                  # run both tools + gate (needs bcftools)

Real chain/FASTA/1000G fixtures must be sourced (not fabricated) — URLs are in scripts/fetch_data.sh and BLOCKERS.md.

Build

cargo build
cargo test
cargo clippy --all-targets -- -D warnings
cargo fmt --check