Speed up the proximity brute-force kernel (#3740) - #3744
Open
brendancol wants to merge 2 commits into
Open
Conversation
_process_numpy_bruteforce serves allocation(), direction(), every GREAT_CIRCLE call and proximity() without scipy, on numpy and per chunk on dask+numpy. It called _distance for every pixel/target pair, which branched on the metric and (for GREAT_CIRCLE) ran four range checks, a sqrt and an asin per pair, re-read the target coordinates through the row/col index arrays each time, and its prange over rows was serial because @ngjit has no parallel=True. Gather the target coordinates into flat arrays once, give each metric its own inner loop, and compile the pixel loop with parallel=True. The inner loops compare a monotone proxy of the distance (squared distance, |dx|+|dy|, the haversine term) and only take the sqrt/asin and the float32 rounding when the proxy beats the running best. The strict < still runs on the float32 distance, so the lowest-flat-index tie-break at float32 precision (#3689) is unchanged. GREAT_CIRCLE validates the coordinate grids once up front and raises the same messages the per-pair guards raised. The parallel kernel launch is serialized behind a module-level lock, same as convolution and terrain (#3141), because the dask path calls it from worker threads. 300x600 raster, 1000 random targets, 20-core host, median of 5: EUCLIDEAN/PROXIMITY 496 ms -> 87 ms (1 thread), 8.6 ms (20) EUCLIDEAN/ALLOCATION 501 ms -> 91 ms (1 thread), 10.2 ms (20) MANHATTAN/DIRECTION 488 ms -> 95 ms (1 thread), 15.5 ms (20) GREAT_CIRCLE/PROXIMITY 3563 ms -> 1136 ms (1 thread), 87.6 ms (20) Results are bit-identical to the previous kernel across all three metrics, all three modes, bounded and unbounded max_distance, explicit and default target_values, and NaN cells in the image.
brendancol
commented
Sep 4, 2026
brendancol
left a comment
Contributor
Author
There was a problem hiding this comment.
PR Review: Speed up the proximity brute-force kernel (#3740)
Blockers (must fix before merge)
- none
Suggestions (should fix, not blocking)
-
xrspatial/proximity.py:557,:575,:599: each inner loop assignsbest_dist = dunconditionally after the proxy test, relying ond <= best_distfrom monotonicity. That holds as long asdis a number. If the rounded distance ever comes out NaN (for great circle, a haversine term a hair above 1.0 sendsarcsin(sqrt(a))to NaN),best_distbecomes NaN and every laterd < best_distis False, so no later target can win. The old kernel skipped a NaN candidate and carried on. I could not construct an input that pushesapast 1.0 in float64, so this may be unreachable, butbest_dist = d if better else best_distcosts nothing and removes the assumption. The proxy update can stay unconditional.
Nits (optional improvements)
-
xrspatial/tests/test_proximity.py:822: the concurrency test hammers a 3x3 raster, so each launch finishes in microseconds and the eight threads rarely overlap inside the kernel. A raster on the order of the 40x40 tie-break fixture would make the launches actually contend for the lock. -
xrspatial/proximity.py:667: the kernel takestlons,tlats,tcoslatsas empty arrays for the Euclidean and Manhattan metrics. A one-line comment on the call site saying they are placeholders would save the next reader from looking for where they are filled.
What looks good
- The proxy-then-round argmin keeps the float32 tie-break exactly and the PR proves it with a case where the squared distances differ by a whole unit (36000000 vs 36000001) yet both round to float32(6000.0). The A/B against
mainis bitwise over 54 metric/mode/range/target/NaN combinations. _nearest_great_circlereproduces the arithmetic ofgreat_circle_distancein the same evaluation order, and the target terms are precomputed with numba'snp.radians/np.cosrather than numpy's, so no ulp drift between the two sides.- The range check reproduces the order the per-pair guards fired in and the messages are compared byte for byte against
great_circle_distance. - The lock follows the convolution/terrain pattern and the dask path (
_process_daskmapping_process_numpy) is covered because the lock lives inside_process_numpy_bruteforce. test_bruteforce_kernel_compiled_parallelguards the exact regression the issue describes (prange without parallel=True).- Benchmarks
Proximity,Allocation,Directionalready cover this path; theperformancelabel is on the PR.
Checklist
- Algorithm matches reference (bitwise A/B against the previous kernel)
- All implemented backends produce consistent results (numpy, dask+numpy, cupy, dask+cupy pass locally)
- NaN handling is correct (NaN image cells and NaN halo coordinates behave as before; see the suggestion on a NaN rounded distance)
- Edge cases are covered by tests
- Dask chunk boundaries handled correctly (unchanged; kernel is per chunk)
- No premature materialization or unnecessary copies
- Benchmark exists
- README feature matrix: not applicable, no new function
- Docstrings present and accurate
…#3740) Update best_dist only when the candidate wins so a NaN rounded distance (a haversine term a hair past 1.0) is skipped instead of poisoning every later comparison, as the old kernel did. Comment the placeholder great circle arrays on the non-great-circle call path. Hammer the lock test with a 40x40 raster so the concurrent launches overlap.
brendancol
commented
Sep 4, 2026
brendancol
left a comment
Contributor
Author
There was a problem hiding this comment.
PR Review: Speed up the proximity brute-force kernel (#3740), follow-up pass
Re-reviewed commit 27d6771 against the first pass.
Blockers (must fix before merge)
- none
Suggestions (should fix, not blocking)
- none
Nits (optional improvements)
- none
Disposition of the first pass
- Suggestion, NaN rounded distance poisoning the running best: fixed. All three inner loops now do
best_dist = d if better else best_dist, and the shared comment block above them explains why.best_proxystill updates unconditionally, which is fine: any later candidate with a proxy at or above a NaN-producing one would produce NaN too and could not win under the old kernel either. A/B againstmainis still bitwise over the same 54 cases. - Nit, tiny concurrency raster: fixed. The lock test now hammers a 40x40 raster with 50 random targets.
- Nit, placeholder great-circle arrays: fixed with a comment at the call site.
What looks good
- The follow-up touches only the three select lines, one comment, and the test; no behaviour change outside the NaN edge case. 609 tests pass locally including the cupy and dask+cupy backends.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3740
_process_numpy_bruteforceis the CPU kernel behindallocation(),direction(), everyGREAT_CIRCLEcall, andproximity()without scipy, on numpy and per chunk on dask+numpy. It called_distancefor every pixel/target pair, so the metric branch ran per pair and great circle paid four range checks, asqrtand anasinper pair. It re-read target coordinates through the row/col index arrays each time. Itsprangeover rows was serial because@ngjithas noparallel=True._nearest_euclidean,_nearest_manhattan,_nearest_great_circle), and compile the pixel loop withparallel=True.|dx|+|dy|, the haversine term) and only takes thesqrt/asinand the float32 rounding when the proxy beats the running best. The strict<still runs on the float32 distance, so the lowest-flat-index tie-break at float32 precision (allocation/direction: nearest-target tie evaluated at different float precision per backend, diverging on non-lattice grids #3689) is unchanged. A candidate whose proxy does not beat the running best has a float32 distance at or above it and could never have won.GREAT_CIRCLEvalidates the coordinate grids once before the loop and raises the same messages the per-pair guards ingreat_circle_distanceraise, in the order they would have fired._PARALLEL_KERNEL_LOCK, the same pattern asconvolution.pyandterrain.py(Streaming reproject thread pool aborts the process when numba parallel kernels run concurrently #3141), since_process_daskmaps_process_numpyover chunks from worker threads.The public
euclidean_distance,manhattan_distance,great_circle_distanceand_distancefunctions, the cupy kernel and the cKDTree paths are untouched.Timings
300x600 raster, 1000 random targets, 20-core host,
time.perf_counter, median of 5 after one warmup:Results are bit-identical to the kernel on
main(np.array_equal(old, new, equal_nan=True)) on that raster across all 54 combinations of the three metrics, the three modes,max_distanceat inf and at a finite value that leaves some pixels NaN, explicit and defaulttarget_values, and NaN cells in the image. The great-circle grid used lon in [-10, 10] and lat in [40, 45]. The single-thread gain is larger than the spike in #3740 predicted because the proxy comparison also skips the float32 rounding on most pairs.The argmin is not a pure select on the proxy, which differs from the sketch in #3740. A raw float64 proxy comparison picks the float64-closer target on float32 near-ties, which breaks
test_tie_break_float32_precision_nonlattice_gridand the CPU/GPU tie parity the CUDA kernel was aligned to. Gating the float32 rounding on the proxy keeps the old ordering exactly and only pays for it on record-breaking candidates.Backends: numpy and dask+numpy run the new kernel. cupy and dask+cupy are unchanged.
Tests
pytest xrspatial/tests/test_proximity.py(609 passed, including cupy and dask+cupy on this box)test_dask_task_names.py,test_accessor.py,test_dataset_support.py,test_balanced_allocation.py(178 passed)allocationanddirectionGREAT_CIRCLErange messages matchgreat_circle_distancebyte for byte_bruteforce_kernelis compiled withparallel=Trueallocationcalls from 8 threads match the serial result