Skip to content

v0.7.6: --parquet-only and --parquet -c - #7

Merged
flalom merged 1 commit into
mainfrom
feat/parquet-only-0.7.6
Sep 19, 2026
Merged

flalom merged 1 commit into
mainfrom
feat/parquet-only-0.7.6

Conversation

@flalom

@flalom flalom commented Sep 19, 2026

Copy link
Copy Markdown
Owner

Closes #6.

@tetedange13 asked for either a parquet-only mode or for -c to work with --parquet. This ships both.

  • --parquet -c is now accepted: the text file is gzipped, the parquet keeps its un-gzipped name (X_reformatted.tsv.gz + X_reformatted.tsv.parquet).
  • New --parquet-only: implies --parquet, writes no text file at all. Rejects -c (nothing to compress).
  • Parquet output is byte-identical across --parquet, --parquet-only and --parquet -c on a real VEP file.
  • validate_release.sh on main still pointed at the pre-rename data/ paths; fixed.
  • tests/test.rs split into tests/{annotations,cli,maf,parquet,reports,tsv}.rs + tests/common/mod.rs.

Copilot AI lite review requested due to automatic review settings September 19, 2026 16:10

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Restore removed public APIs, prevent header output in --parquet-only, and make release validation portable.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 4 High severity · 1 Medium severity

Open (5)
What changed in this PR

Adds --parquet-only and --parquet -c support, reorganizes integration tests, and updates release validation for v0.7.6.

Changes:

  • Adds Parquet-only output and compression support.
  • Refactors and expands integration tests.
  • Updates validation scripts, documentation, and package metadata.
  • Removes several public compatibility APIs.
File Description
tests/​tsv.rs TSV tests.
tests/​test.rs Replaced by focused test modules.
tests/​scripts/​validate_release.sh Release validation updates.
tests/​scripts/​compare_tsv_bcftools.py TSV comparison utility.
tests/​reports.rs Report tests.
tests/​parquet.rs Parquet behavior tests.
tests/​maf.rs MAF tests.
tests/​common/​mod.rs Shared test helpers.
tests/​cli.rs CLI tests.
tests/​annotations.rs Annotation tests.
src/​summary.rs Summary refactoring.
src/​reformat_vcf.rs Streaming output changes and removed APIs.
src/​read_vcf_gz.rs VCF reading refactoring.
src/​parquet_writer.rs Streaming Parquet writer changes.
src/​main.rs New CLI modes and output handling.
src/​extract_sample_info.rs Sample extraction updates.
src/​essentials_fields.rs MAF conversion updates and removed API.
README.md Documents v0.7.6 and new options.
Cargo.toml Version metadata update.
Cargo.lock Locked version update.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/essentials_fields.rs
Comment on lines +144 to 147
/// Convert a single-ALT record to a MAF row, reading the depth columns from explicitly named
/// samples. `tumor` is the sample the t_* columns describe; `normal` populates the
/// matched-normal columns. Passing `None` for either keeps the historical behaviour of
/// using the first sample that declares `DP`.
Comment thread src/parquet_writer.rs
Comment on lines 82 to 83
fn build_tsv_schema(headers: &[String]) -> Schema {
let fields: Vec<Field> = headers
Comment thread src/reformat_vcf.rs
}

/// The column line. Streaming writers call this once, before the first batch.
pub fn write_tsv_header<W: Write>(writer: &mut W, headers: &[String]) -> std::io::Result<()> {
Comment thread src/summary.rs
sort_chromosomes(counts)
}

/// Sort chromosome keys in natural order: 1-22, X, Y, M/MT, then others alphabetically.
Comment thread src/main.rs
Comment on lines 795 to 801
println!();

// The text file's own name plus .parquet, so both output formats read
// X_reformatted.<tsv|maf>.parquet. --parquet and --compress are mutually
// exclusive (checked at startup), so there is no .gz case.
// X_reformatted.<tsv|maf>.parquet (a -c .gz suffix is dropped first).
#[cfg(feature = "parquet_out")]
if let Some(sink) = parquet_sink.take() {
match sink.close() {
@flalom
flalom merged commit 1586c81 into main Sep 19, 2026
5 checks passed
@flalom
flalom deleted the feat/parquet-only-0.7.6 branch September 19, 2026 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Reduce (or nullify) size of text file written with --parquet option ?

2 participants