Skip to content

perf(query): allow writes during snapshot searches - #23

Merged
zhenghaoz merged 1 commit into
mainfrom
optimize/query-write-concurrency
Sep 19, 2026
Merged

zhenghaoz merged 1 commit into
mainfrom
optimize/query-write-concurrency

Conversation

@zhangzhenghao

Copy link
Copy Markdown
Collaborator

Summary

  • pin immutable collection query snapshots so Query, MultiQuery, and GroupByQuery release the collection read lock before long-running search work
  • keep segment runtimes alive with query/cache ownership until the last lease is released
  • make Close and Destroy wait for active query leases and retry/report delayed runtime close errors
  • capture schema in the same snapshot generation as documents and runtime indexes

This follows Alibaba zvec's separation between short collection-handle locking and stable segment ownership, implemented with pure-Go lifecycle management.

Tests

  • go test ./... -count=1 -timeout=20m
  • go test -race ./... -count=1 -timeout=30m
  • go vet ./...
  • concurrency and runtime lifecycle tests repeated with -count=20/-count=50
  • git diff --check

Scope

This change addresses query/write lock contention and safe runtime reclamation. It does not change the on-disk format, writer serialization, or ANN algorithms.

@codecov

codecov Bot commented Aug 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.36111% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.69%. Comparing base (2e6c9a0) to head (f9d90d3).

Files with missing lines Patch % Lines
collection.go 91.85% 11 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #23      +/-   ##
==========================================
+ Coverage   82.65%   82.69%   +0.04%     
==========================================
  Files         140      140              
  Lines       27883    27983     +100     
==========================================
+ Hits        23046    23140      +94     
- Misses       4837     4843       +6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@zhangzhenghao
zhangzhenghao force-pushed the optimize/query-write-concurrency branch from f2bfd4f to 5d15a1d Compare August 21, 2026 15:13
@zhangzhenghao
zhangzhenghao force-pushed the optimize/query-write-concurrency branch from 5d15a1d to f9d90d3 Compare September 19, 2026 10:03
@zhenghaoz
zhenghaoz merged commit 86ec1a9 into main Sep 19, 2026
8 checks passed
@zhenghaoz
zhenghaoz deleted the optimize/query-write-concurrency branch September 19, 2026 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants