SeqMux

SeqMux is a fast, portable FASTQ demultiplexer written in Rust.

It demultiplexes single-end and paired-end FASTQ using a sample barcode table (dual or single barcode), with quality/adapter trimming and multi-threaded streaming I/O. No Python, Conda, pigz, or SLURM required.

Production workflow (I395 / I464)

Validated on a 16-CPU allocation (qcpu_18i). Use exact matching, default adapter trimming, and gzip level 1. -t 12 had the lowest median wall time among 8, 12, and 16 on the 2-million-pair I464 production path.

seqmux demux \
  -i mix_1.fq.gz \
  -I mix_2.fq.gz \
  -b sample_barcodes.csv \
  -o results \
  -p seqmux \
  -t 12 \
  --compression-level 1

Setting	Value
Threads	12 workers (R1/R2 decompress concurrently)
Mismatches	--mismatches-1 0 --mismatches-2 0 (defaults)
Adapter	on (Illumina R1/R2 defaults); --no-adapter to disable
Compression	--compression-level 1 (CLI default remains 6)
Orientation	both, canonicalized so output R1 carries Barcode1

Observed full-dataset throughput (gzip in, gzip level 1 out, 16-CPU allocation):

Dataset	Pairs	Wall	Throughput	Assigned	Samples vs Python
I464	140,771,720	6 min 47 s (406.80 s)	346,047 pairs/s	87,477,095	all 35
I395	60,523,000	2 min 57 s (177.03 s)	341,880 pairs/s	39,455,648	all 18

Assignment counts match the lab Python exact demux. Representative outputs passed gzip integrity, paired counts, paired IDs, order, and canonical orientation checks. Gzip files were not shown to be byte-identical to another run. Details: docs/REAL_DATA.md.

Features (v0.3)

• Single-end and paired-end FASTQ (.fastq / .fq / .gz)
• SeqMux sample table CSV (SampleNumber, Barcode1, Barcode2, …)
• Dual-barcode demux (PE: both R1/R2 orientations; SE: R1 5′ + R1 3′)
• Single-barcode demux (Barcode1 only at R1 5′)
• Optional N bases in barcodes as UMI (written to header as rbc:)
• Hamming-distance matching with mismatch tolerance; ties → unassigned
• Quality trimming (BWA-style) and 3′ adapter trimming
• Multi-threaded chunked pipeline with ordered per-sample output
• Gzip output without shelling out to external tools
• Summary TSV + stderr report

Install

git clone https://github.com/Caizhaohui/SeqMux.git
cd SeqMux
cargo build --release
# binary: target/release/seqmux

Requirements: Rust stable (1.74+). No external runtime tools.

Quick start

Paired-end (recommended for dual barcode):

seqmux demux \
  -i mix_R1.fastq.gz \
  -I mix_R2.fastq.gz \
  -b sample_barcodes.csv \
  -o results \
  -t 8

Single-end:

seqmux demux \
  -i reads.fastq.gz \
  -b sample_barcodes.csv \
  -o results \
  -t 8

Validate FASTQ:

seqmux validate -i reads.fastq.gz
seqmux validate -i r1.fastq.gz -I r2.fastq.gz

Sample barcode table (required format)

SeqMux reads a CSV with a header row. This is the only supported barcode input.

Minimal example

SampleNumber,Barcode1,Barcode2
I464469-A1,AAGTCCAA,GGAGTACT
I464469-A2,GACCTGAA,GGACTTGG
I464469-A3,GGCTTAAG,TGGATCGA

Full example (extra columns are ignored)

Same layout as experimental tables such as I464-469erdai_barcode_and_name.csv:

SampleNumber,Barcode1,Barcode2,PCR_product,rawdata1,rawdata2,library_round,F_primer,R_primer
I464469-A1,AAGTCCAA,GGAGTACT,atgcca...,mix_1.fq.gz,mix_2.fq.gz,A,fwd...,rev...
I464469-A2,GACCTGAA,GGACTTGG,agcgcg...,mix_1.fq.gz,mix_2.fq.gz,A,fwd...,rev...

A trimmed example file is included at examples/sample_barcodes.csv.

Required columns

Column	Required	Description
SampleNumber	Yes	Sample ID / output name (unique)
Barcode1	Yes	First barcode (DNA: A/C/G/T/N)
Barcode2	Optional*	Second barcode for dual-index demux

*If any row has Barcode2, all rows must have Barcode2.
If Barcode2 is omitted entirely, SeqMux runs in single-barcode mode.

Column name aliases (case-insensitive)

Logical field	Accepted headers
Sample	SampleNumber, Sample, Sample_Name, Sample_ID, Name
Barcode1	Barcode1, Barcode_1, BC1, Index1, i7
Barcode2	Barcode2, Barcode_2, BC2, Index2, i5

Extra columns (PCR_product, rawdata1, rawdata2, library_round, F_primer, R_primer, …) are ignored and may remain in the file for laboratory records.

Rules

• Header row is required
• Sample names must be unique
• Barcode combinations (Barcode1, Barcode2) must be unique
• Bases: only A/C/G/T/N (auto-uppercased); N = UMI position
• Empty lines are skipped via the CSV reader

How barcodes are matched

Mode	When	Matching
Dual barcode + PE	Barcode2 present and -I given	Both orientations: Barcode1@R1 5′ + Barcode2@R2 5′, and Barcode2@R1 5′ + Barcode1@R2 5′. Swapped mates are rotated so output R1 has Barcode1 (unless --no-canonicalize).
Dual barcode + SE	Barcode2 present, no -I	Barcode1 @ R1 5′, Barcode2 @ R1 3′
Single barcode	no Barcode2 column / all empty	Barcode1 @ R1 5′ only

After assignment (unless --keep-barcodes):

• PE dual: trim Barcode1 from R1 5′ and Barcode2 from R2 5′
• SE dual: trim Barcode1 from 5′ and Barcode2 from 3′
• Single: trim Barcode1 from 5′

Mismatch thresholds: --mismatches-1 / --mismatches-2. Best unique score wins; ties go to unassigned. Default is exact match (0). Use 1 only after checking docs/MISMATCH_QC.md.

Barcode matching uses the original 5′ sequence, then barcodes are trimmed, then quality/adapter trim. 5′ quality trimming therefore cannot eat the barcode. 3′ Illumina adapter trimming is on by default; use --no-adapter to turn it off.

Orientation vs the lab Python demux: SeqMux canonicalizes so output R1 carries Barcode1. The I395/I464 Python scripts wrote the Barcode2-bearing mate as R1. Assignment counts match; R1/R2 file contents are swapped relative to those scripts unless --no-canonicalize is set. See docs/COMPATIBILITY.md.

Common options

Option	Description
-i / --input	R1 or single-end FASTQ
-I / --input2	R2 FASTQ
-b / --barcodes	Sample barcode table CSV
-o / --out-dir	Output directory
-p / --prefix	Output prefix (default seqmux)
-t / --threads	Worker threads
--max-reads	Stop after N reads/pairs (0 = all)
--skip-reads	Skip N reads/pairs before processing
--counts-only	Count only; do not write FASTQ
--mismatches-1	Allowed mismatches for Barcode1
--mismatches-2	Allowed mismatches for Barcode2
--orientation	Dual PE: both (default), canonical, swapped
--no-canonicalize	Do not rotate swapped mates to Barcode1-on-R1
--keep-barcodes	Do not trim barcode bases
--discard-unassigned	Drop unassigned reads
-a / --adapter-r1	3′ adapter for R1 (default Illumina AGATCGGAAGAGCACACGTCTGAA)
--adapter-r2	3′ adapter for R2 (default Illumina AGATCGGAAGAGCGTCGTG)
--no-adapter	Disable 3′ adapter trimming
-q / --quality-cutoff-3	3′ Phred quality cutoff
-l / --min-length	Minimum length after trim
--force	Overwrite existing outputs
--no-gzip	Write plain FASTQ

Run seqmux demux --help for the full list.

Output layout

Single-end:

<prefix>_<SampleNumber>.fastq.gz
<prefix>_unassigned.fastq.gz
<prefix>.summary.tsv

Paired-end:

<prefix>_<SampleNumber>_R1.fastq.gz
<prefix>_<SampleNumber>_R2.fastq.gz
<prefix>_unassigned_R1.fastq.gz
<prefix>_unassigned_R2.fastq.gz

Example: sample I464469-A1 → seqmux_I464469-A1_R1.fastq.gz.

Design notes

• Pipeline: Reader → bounded chunks → N workers → ordered writer
• Workers never open output files; writer alone finishes gzip streams
• No temporary _tmp_thread_* files and no final cat merge

Development

cargo fmt --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all
cargo build --release

License

MIT — see LICENSE and NOTICE.md.

Acknowledgements

Pipeline and trimming ideas were informed by demultiplexing practice in the NGS community, including Ultraplex.