Skip to content

Bring optimised Jaccard code into main - #72

Merged
johnlees merged 8 commits into
mainfrom
jaccard_simd
Jul 31, 2026
Merged

johnlees merged 8 commits into
mainfrom
jaccard_simd

Conversation

@johnlees

Copy link
Copy Markdown
Member

Closes #68

@codecov

codecov Bot commented May 17, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.15166% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.32%. Comparing base (4072a47) to head (5bd17d0).

Files with missing lines Patch % Lines
src/sketch/mod.rs 85.05% 13 Missing ⚠️
src/inverted.rs 61.53% 5 Missing ⚠️
src/sketch/multisketch.rs 73.68% 5 Missing ⚠️
src/hashing/aahash_iterator.rs 66.66% 1 Missing ⚠️
src/lib.rs 80.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #72      +/-   ##
==========================================
- Coverage   90.65%   90.32%   -0.34%     
==========================================
  Files          17       17              
  Lines        3232     3359     +127     
==========================================
+ Hits         2930     3034     +104     
- Misses        302      325      +23     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread src/sketch/mod.rs Resolved
Comment thread src/sketch/mod.rs Resolved
Comment thread src/sketch/mod.rs Resolved
Comment thread src/sketch/mod.rs Resolved
Comment thread src/distances/jaccard.rs Resolved
Comment thread src/distances/jaccard.rs Resolved
Comment thread src/lib.rs Resolved
Comment on lines +1 to +2
14412_3#82.contigs_velvet.fa.gz 14412_3#84.contigs_velvet.fa.gz 0.011049058 0.17300157
14412_3#84.contigs_velvet.fa.gz 14412_3#82.contigs_velvet.fa.gz 0.011049058 0.17300157

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These diffs are larger

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting ~2/3% diff in underlying Jaccard

old: 0.3221537, 0.34842753, 0.37201434, 0.3902269
new: 0.31687897, 0.3392715, 0.3803742, 0.39818868

Which is amplified by regression

Likely a resolution change as wider bins, different hashes

14412_3#84.contigs_velvet.fa.gz 5 10 0 0
TIGR4.fa.gz 0 0 7 10
14412_3#82.contigs_velvet.fa.gz 10 6 0 0
R6.fa.gz 0 0 10 9

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Surprised this changes

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a larger sketch size:

14412_3#82.contigs_velvet.fa.gz 10000   6599    3       3
R6.fa.gz        3       2       10000   6813
14412_3#84.contigs_velvet.fa.gz 6599    10000   2       2
TIGR4.fa.gz     3       2       6813    10000

I think this is just quite noisy at s=10 (and as MOD_SIGN changed, effectively using different hashes)

@johnlees
johnlees marked this pull request as draft June 17, 2026 13:26
@johnlees johnlees linked an issue Jul 24, 2026 that may be closed by this pull request
@johnlees
johnlees marked this pull request as ready for review July 27, 2026 14:38
@johnlees johnlees mentioned this pull request Jul 27, 2026
@johnlees
johnlees merged commit c4bf09f into main Jul 31, 2026
11 of 13 checks passed
@johnlees
johnlees deleted the jaccard_simd branch July 31, 2026 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Benchmarks of POC bitpacked code vs original Random matches in larger sketches

1 participant