Skip to content
BranchesTags

Repository files navigation

RefCompat

RefCompat is an open-source Python tool for determining whether heterogeneous genomic resources can share a coherent reference-coordinate context for a stated use case, and for explaining the evidence, conflicts, conditions, and unresolved questions behind that conclusion.

The project is currently in early development and should not yet be treated as production software.

The problem

Genomic workflows commonly combine files that appear to belong to the same reference but differ in ways that matter operationally:

  • assembly or reference-distribution differences such as GRCh37, hg19, b37, or hs37d5;
  • chr1 versus 1 and other sequence-name namespaces;
  • primary-only references versus ALT/decoy/patch-inclusive distributions;
  • stale .fai or sequence-dictionary artifacts;
  • VCF REF alleles that do not match the supplied FASTA;
  • BAM/CRAM sequence dictionaries that do not reconcile with the intended reference;
  • GTF/GFF annotations that require missing sequences or exceed sequence bounds;
  • bundles whose files carry inconsistent provenance or were generated from different reference assets.

Existing tools solve important parts of this problem. RefCompat is intended to provide the reasoning layer above those parts: inspect a collection of resources, represent what each resource requires and provides, evaluate the constraints against a coherent reference context, and produce a scoped, traceable verdict.

Core position

RefCompat does not define genomic sequence identity.

  • Individual sequence identity is delegated to GA4GH refget Sequences.
  • Sequence-collection identity and comparison are delegated to GA4GH Refget Sequence Collections (SeqCol).
  • RefCompat consumes those standardized identities and relationships as evidence for higher-level interoperability reasoning.

Conceptually:

refget Sequences
"What exact sequence is this?"
        |
        v
SeqCol
"What exact collection is this?"
"How do collections relate?"
        |
        v
RefCompat
"Are the requirements of these heterogeneous resources jointly
satisfied for the stated reference-coordinate use case?"

Compatibility model

Compatibility is treated as constraint satisfaction, not similarity.

RefCompat is being designed around these principles:

  • strong content evidence cannot be overridden by many weak similarities;
  • compatibility can be directional: one resource may require a subset of what another provides;
  • a result is always scoped to a stated evaluation context;
  • INDETERMINATE is a first-class result when evidence is insufficient;
  • input/analysis status is separate from compatibility status;
  • provenance claims remain distinguishable from verified identity;
  • conditions must come from explicit scope or profile rules, not from RefCompat guessing what the user considers irrelevant;
  • no silent scientific repair or semantic transformation.

Implemented top-level verdicts are:

  • COMPATIBLE
  • COMPATIBLE_WITH_CONDITIONS
  • INCOMPATIBLE
  • INDETERMINATE

Planned analysis status remains separate from compatibility verdicts: COMPLETE, PARTIAL, or INVALID_INPUT.

Research basis

The current design was informed by a purposively collected and reviewed corpus of 200 real compatibility-related incidents from public genomics support forums and issue trackers. Two independent 100-case batches were used to test whether the problem taxonomy and proposed feature priorities remained stable under a different source mix.

The corpus is design evidence, not a prevalence study. The row-level incident records are not distributed with RefCompat; validation uses small synthetic or clearly redistributable fixtures derived from observed failure patterns.

Planned first substantial release

The initial implementation scope is intentionally narrow:

  • FASTA inspection and local refget/SeqCol identity;
  • FASTA ↔ .fai integrity;
  • FASTA ↔ SAM/Picard-style .dict integrity;
  • FASTA ↔ BAM/CRAM reference-context checks;
  • VCF header/reference-context checks (implemented);
  • exhaustive VCF REF ↔ FASTA verification;
  • verified VCF sequence-name binding from uniquely matched contig MD5 identity;
  • BCF2 parity through the same VCF logical compatibility path (Milestone 8 complete after internal review, external milestone review, provider-error remediation, and targeted external confirmation);
  • explicit BED3–BED9/BED12 layout and first-class BED report identity (Milestone 9 Slice 2; inspection and coordinate reasoning pending);
  • GTF/GFF3 sequence-name and coordinate-bounds checks;
  • whole-bundle compatibility reasoning;
  • verified alias handling as shared evidence infrastructure;
  • human-readable and machine-readable reports.

Milestone 9 Slices 1–2 now pin the BED coordinate-compatibility contract and implement its atomic layout/resource/schema foundation. BED inspection and coordinate reasoning remain pending. Liftover, broad workflow validation, automatic repair, and complex ecosystem profiles are intentionally deferred.

Design documents

Start with:

Project status

Milestones 1–8 are complete. Milestone 8 added native BCF2 parity through the existing VCF logical model, including strict declared/detected encoding checks, one-based logical POS preservation, exhaustive RCHECK-050 reuse, scoped/profile/reporting parity, exact stable schema 2.0.0, retained 1.0.0/1.1.0 immutability, and internal/external review. The external review found one provider-close error-boundary defect; the correction preserves normalized mid-stream parse failures, normalizes close-only failures, adds real corrupted-BGZF BCF/VCF.gz coverage plus the negative BCF UCSC-alias case, and the targeted external follow-up independently reproduced the fix and assessed M8 SAFE TO CLOSE. Milestone 4 includes BAM/CRAM header observation, core contract projection, conservative M5-backed sequence binding, descriptive header-dictionary relationship reasoning, and an explicit offline CRAM reference policy: RefCompat copies parser-visible @HD, ordered @SQ, and @PG metadata into immutable domain values, projects declared @SQ names, lengths, and M5 values into mandatory format-neutral requirements, and can resolve cross-name SN differences only when declared M5 identity uniquely and length-consistently identifies an in-scope FASTA sequence. It can separately distinguish exact declared identity, verified naming-only differences, reorder, subset/superset/overlap/disjoint membership, M5 conflicts, and unresolved dictionary relationships without creating a second alignment-specific verdict. Header-only CRAM reasoning never triggers reference retrieval; the planner either selects the explicitly chosen local FASTA anchor or defers, and any future decoder adapter must preserve that no-ambient/no-network fallback boundary explicitly. It still does not scan reads, trust AN alternate names, or treat declared M5 values as anchor authority. The alignment path is read-only and diagnostic: verified bindings and relationship summaries describe correspondence but never rename or reheader data, and RefCompat does not remap or realign records. The completed VCF work includes context observation, exhaustive direct REF-to-FASTA validation, verified cross-name binding from uniquely matched contig MD5 identity, projection of those facts into format-neutral requirements and Tier-A evidence, explicit anchor-owned ingestion of pair-derived REF evidence into whole-bundle reasoning, and threshold-free isolated/localized/distributed/systematic REF-conflict interpretation. Milestone 5 core implementation, integration/adversarial exit coverage, internal hardening, external milestone-boundary review, circular-landmark correction, and targeted follow-up are complete; the follow-up confirmed the correction and closed the milestone: RefCompat preserves sparse GTF/GFF3 seqid/coordinate usage, raw and decoded GFF3 identifiers, validates ordinary feature and ##sequence-region bounds, uses relevant embedded FASTA content as annotation-owned identity evidence, accepts independently established annotation-owned content identities for conservative GTF/GFF3 binding or exhaustive full-anchor absence proof, and applies the GFF3 circular-origin exception only from a unique structural circular region landmark with a proven single-wrap representation and one literal Is_circular=true control. It does not build gene-model hierarchy, infer naming aliases, turn GTF metadata into sequence identity, or let annotation metadata replace the selected FASTA anchor. RefCompat also has explicit anchor-driven evaluation requests/scope, typed sequence presence/length/identity/order/reference-base requirements and capabilities, separate compatibility constraints/evaluations, qualitative traceable evidence aggregation without numeric scoring, structured issue/unresolved findings plus explicit-scope conditions, an explicit FASTA ReferenceContext with content-verified SequenceBinding plus whole-bundle orchestration, categorical COMPATIBLE / COMPATIBLE_WITH_CONDITIONS / INCOMPATIBLE / INDETERMINATE aggregation over mandatory constraints, and compact decisive conflict-core extraction. Milestone 6 is complete: ucsc-preflight uses explicit native UCSC database selection, deterministic provenance-bearing provider snapshots, authoritative naming evidence only after an independent content bridge to the selected FASTA anchor, representative VCF/BAM/CRAM reuse, deterministic offline behavior, adversarial exit coverage, and completed internal/external milestone reviews. Milestone 7 now pins the stable CompatibilityReport and workflow boundary: analysis completeness remains separate from compatibility, the immutable report root and explicit deterministic projections avoid recursively mirroring internal dataclasses, and internal scientific/API review hardened cross-object trace validation, global requirement/capability IDs, local artifact-path portability, and condition ordering. Slice 4 froze the first stable core schema at 1.0.0; Slice 5 advanced the M7 stable schema additively to 1.1.0 while retaining exact 1.0.0, and added report-owned resource observations, BAM/CRAM dictionary relationship context, and UCSC provider/source/profile provenance without leaking provider implementation objects. Slice 6 now adds deterministic human rendering plus a stable whole-bundle workflow exit policy that distinguishes positive complete results, incompatibility, invalid input, indeterminate results, partial execution, and pre-report operational failure; the provisional Milestone 1 diagnostic commands remain unchanged. M8 Slice 2 made exact schema 2.0.0 current at the M8 boundary because the closed ResourceKind enum gains bcf, while retained exact 1.0.0/1.1.0 schemas remain unchanged; draft output advances to revision 4. Slice 7 exercises scoped VCF, BAM/CRAM relationship, annotation incompatible/indeterminate, and UCSC-profile VCF results end to end through stable JSON/schema validation, human rendering, and workflow exits. The Slice 8 internal adversarial/backward-compatibility review is now complete: it hardens direct profile-context capability trace retention, pins a UCSC content-conflict report, and proves additive 1.1.0/retained-1.0.0 compatibility invariants without changing scientific verdict or workflow semantics. The required external M7 milestone-boundary review reproduced the full gate, found no MAJOR issue, and assessed the repository SAFE TO CLOSE M7; accepted MINOR follow-up hardens general referential closure/additivity coverage, portable provenance-locator obligations, deterministic human context ordering, and concrete PARTIAL/INVALID_INPUT reporting paths without changing scientific verdict or stable JSON shape. M8 Slice 2 adds ResourceKind.BCF, strict declared-versus-detected VCF/BCF encoding validation, shared logical VCF context/REF observation for BCF, real-pysam one-based POS coverage, and the exact 2.0.0 report boundary. Slice 3 proves that generated BCF2 and equivalent textual VCF traverse the same declared-MD5 binding, exhaustive REF validation, conflict-pattern, generic contract/evidence, and whole-bundle verdict path without adding BCF-specific scientific reasoning. Slice 4 now carries BCF through scoped conditional, incompatible, indeterminate, UCSC-profile, stable JSON 2.0.0, human rendering, workflow-exit, and declared/detected encoding-mismatch paths while retaining the existing report and scientific semantics. The Slice 5 internal review found no production-science defect, pinned byte-exact retained M7 report assets and the exact 1.1.0→2.0.0 schema delta, exercised BCF draft revision 4, and hardened suffix-independent encoding detection. The independent external M8 review confirmed those scientific/schema claims but found one provider-close error-boundary defect; the correction preserves a normalized mid-stream parse failure when close also fails, normalizes close-only failures, and adds real corrupted-BGZF plus negative UCSC-alias regression coverage. The targeted external follow-up independently reproduced the remediation against real corrupted BCF and VCF.gz inputs, found no remaining MAJOR issue, and assessed M8 SAFE TO CLOSE. A GeneratorExit close-failure edge case during deliberate early iterator abandonment remains non-blocking post-M8 hardening; no current production caller abandons the iterator early.

Milestone 9 Slices 1–2 are complete. Standard BED3–BED9/BED12 layouts and ResourceKind.BED are explicit, current stable output uses exact schema 3.0.0, provisional output uses revision 5, and retained 1.0.0/1.1.0/ 2.0.0 assets remain unchanged. The minimal BED known answer records the declared layout as a generic observation without fabricating feature claims. Slice 3 will add native zero-based half-open streaming observation.

Current CLI diagnostics

RefCompat has not yet published a stable package release. From a source checkout configured with uv, the current CLI diagnostics can be run directly with:

uv sync
uv run refcompat inspect-fasta reference.fa
uv run refcompat check-fai reference.fa reference.fa.fai
uv run refcompat check-dict reference.fa reference.dict

Add --format json to any command for provisional machine-readable output. These local diagnostics intentionally do not emit a whole-bundle compatibility verdict; see docs/diagnostic-output.md.

For whole-bundle workflows, Milestone 7 exposes the stable process-status policy through refcompat.reporting.workflow_exit_code(report). That API is separate from the provisional diagnostics above; Slice 6 does not change their existing exit behavior.

Development

RefCompat currently supports Python 3.10–3.14; Python 3.11+ is recommended for new environments and uses uv for project environments and dependency locking.

uv sync --all-groups
uv run --locked pytest
uv run --locked ruff check .
uv run --locked ruff format --check .
uv run --locked mypy
uv build

See docs/development.md for details.

License

RefCompat is licensed under the Apache License 2.0. See NOTICE for project provenance and THIRD_PARTY_NOTICES.md for redistributed test-fixture notices.

Citation

RefCompat was created by Greg Roe. If you use RefCompat in published research, please cite the software using the metadata in CITATION.cff.

About

RefCompat checks whether genomic references, variants, alignments, annotations, and derived resources share a compatible reference-coordinate context using GA4GH refget/SeqCol identity and evidence-backed compatibility reasoning.

Topics

bambioinformaticscompatibility-checkercramdata-validationfastaga4ghgenomegenome-assemblygenomic-datagenomicsgff3gtfinteroperabilitypythonreference-genomerefgetseqcolvariant-analysisvcf

Resources

Readme
Apache-2.0 license

Contributing

Contributing
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages