Repository navigation
fix: keep the CPU torch index out of git installs of wsidata - #76
Merged
Merged
Conversation
PR #69 routed torch and torchvision to the PyTorch CPU index on Linux via tool.uv.sources, and its comment said this only affects `uv sync`/`uv lock` in this repo. That is wrong for git and path installs: uv applies a dependency's tool.uv.sources to the project that installs it. A project depending on `wsidata @ git+https://...` got Linux torch as a `+cpu` build from download.pytorch.org, while torchvision, which wsidata lists only in its dev group, came from PyPI as a CUDA build. The mismatched builds fail with "RuntimeError: operator torchvision::nms does not exist", which broke 7 lazyslide-tutorials notebooks, and any Linux GPU user installing wsidata from git with uv silently got a CPU-only torch. Scope both sources to the dev group and repeat torch there, since a group-scoped source only applies to requirements declared in that group. Dependents only resolve wsidata's project dependencies, so they now get torch from PyPI. uv locks one torch per platform, so resolved versions in this repo's lock do not change: CI, Read the Docs and local Linux environments keep the CPU build, with or without the dev group.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
#69 routed torch and torchvision to the PyTorch CPU index on Linux through
tool.uv.sources. Its comment says that only affectsuv sync/uv lockin this repo, which is wrong for git and path installs, because uv applies a dependency'stool.uv.sourcesto the project that installs it.rendeirolab/lazyslide-tutorials depends on
wsidata @ git+...@main. Its CI got Linux torch2.14.0+cpufrom download.pytorch.org. torchvision is only in wsidata'sdevgroup, so its source never applies downstream, and it came from PyPI as a CUDA build. The builds did not match, and 7 notebooks failed withRuntimeError: operator torchvision::nms does not exist. Any Linux GPU user who installs wsidata from git with uv also gets a CPU-only torch without noticing.Fix
Both sources get
group = "dev", and torch is added to thedevgroup, because a group-scoped source only applies to requirements declared in that group. A dependent only resolves wsidata'sproject.dependencies, so it now gets torch from PyPI.groupon sources has been in uv since 0.5.4.uv locks one torch per platform, so no resolved version in this repo's
uv.lockchanges. The diff is metadata only:requires-distdrops the index, anddevgains torch. CI, Read the Docs and local Linux environments keep the CPU build, whether or not thedevgroup is installed.Alternatives considered:
prune-cache(on by default) drops pre-built wheels before saving.uv pip installafteruv sync --lockedstill downloads the CUDA build first. Avoiding that download means passing--no-install-packagefor everynvidia-*package and using--no-syncfrom then on. It would also leave RTD on CUDA.Verification
Scratch project with
wsidata @ git+https://github.com/rendeirolab/wsidata@<ref>andtorchvision, thenuv lock:main2.14.1PyPI +2.14.1+cpuCPU index (Linux)0.29.1PyPIf4f39f6)2.14.1PyPI0.29.1PyPIIn this repo,
uv lock --checkpasses.uv exportof the CI set (--no-default-groups --group dev --group cucim), the RTD set (--all-extras --group docs) and the base set (--no-default-groups) all givetorch==2.13.0+cpu/torchvision==0.28.0+cpuon Linux. None of them pulls the CUDA stack; the onenvidia-*package,nvidia-nvimgcodec-cu12, comes from cucim, as before.Follow-up
After this merges, the
override-dependencies = ["torch>=2.1"]workaround in rendeirolab/lazyslide-tutorials#5 can go, along with auv lock --upgrade-package wsidatathere.