Skip to content

Omit blank BioProject references from submission XML instead of writing "Not Provided" - #367

Open
developer-rpai wants to merge 1 commit into
CDCgov:masterfrom
developer-rpai:fix/omit-blank-bioproject-362
Open

developer-rpai wants to merge 1 commit into
CDCgov:masterfrom
developer-rpai:fix/omit-blank-bioproject-362

Conversation

@developer-rpai

Copy link
Copy Markdown

What

Fixes #362: when ncbi-bioproject is left blank in the metadata, the generated NCBI submission XML currently carries BioProject references with the placeholder text Not Provided. NCBI rejects such submissions, and tostadas then hangs waiting on the rejected submission.

Root cause

XMLSubmission.safe_text() substitutes "Not Provided" for any blank/NaN value, and the three BioProject call sites wrote that placeholder straight into the XML:

  • BiosampleSubmission.add_action_block() → <BioProject><PrimaryId db="BioProject">Not Provided</PrimaryId></BioProject>
  • SRASubmission.add_attributes_block() → AttributeRefId/BioProject with the placeholder
  • GenbankSubmission.xml_create_wgs() → AttributeRefId/BioProject with the placeholder

Fix

  • Added XMLSubmission.bioproject_id(), which returns the stripped BioProject accession, or None when the value is missing, blank, NaN, or the "Not Provided" placeholder.
  • All three BioProject sites now omit the element/AttributeRefId entirely when bioproject_id() is None and log a warning ("ncbi-bioproject is blank; omitting the BioProject reference from the ... submission XML"), instead of emitting a known-bad placeholder.
  • A real BioProject accession is still written exactly as before.

Reproduction evidence (sandbox, pristine master @ 654854d)

A repro script driving add_action_block() / add_attributes_block() with a blank (NaN, as pandas reads a blank Excel cell) ncbi-bioproject produced <PrimaryId db="BioProject">Not Provided</PrimaryId> in both BioSample and SRA XML on pristine code, and omits the BioProject reference after the fix; a real accession (PRJNA123456) is preserved in both cases.

Tests

  • New tests/test_submission_helper.py: 13 tests covering blank (NaN/""/whitespace/"Not Provided"/None), missing-key, and real-accession cases across the BioSample and SRA paths, plus the bioproject_id() helper.
  • Negative control: 12/13 fail on pristine code (the 13th is the no-regression check that a real accession is still written, which passes on both).
  • With the fix: 13/13 pass (pytest tests/test_submission_helper.py -q).

Notes / limitations

  • I could not run the repo's Nextflow test suites in this sandbox (no Nextflow/conda bioinformatics stack), so CI status is unverified — I did not observe CI passing.
  • The table2asn source TSV (create_source_file) still writes the raw ncbi-bioproject value; a blank cell serializes as empty there, which is unchanged behavior and out of scope for this fix.
  • The bug exists identically on the dev branch; this PR targets master (the default branch).

…ng "Not Provided"

NCBI rejects submissions that carry a placeholder value such as
"Not Provided" as a BioProject PrimaryId, which causes tostadas to hang
waiting on the rejected submission. When ncbi-bioproject is blank, missing,
or a placeholder, the BioProject element/AttributeRefId is now omitted
entirely and a warning is logged, per the suggestion in CDCgov#362.

Fixes CDCgov#362.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Internal][Bug] BioProject = Not Provided

1 participant