Skip to content

test: replace module-level skips with requires marker - #996

Open
liopeer wants to merge 3 commits into
test/requires-markerfrom
test/requires-marker-module-skips
Open

liopeer wants to merge 3 commits into
test/requires-markerfrom
test/requires-marker-module-skips

Conversation

@liopeer

@liopeer liopeer commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

What has changed and why?

Phase 1 of the CI speedup: replace module-level pytest.skip(..., allow_module_level=True) with pytestmark = pytest.mark.requires(...).

Stacked on #995. Merge that first, then change this PR's base to main.

  • torchmetrics: requires("torchmetrics>=1.5") in the 10 _metrics modules and requires(f"torchmetrics>={TORCHMETRICS_MIN_VERSION}") in the template tests. test_classwise_metric_collection.py imported torchmetrics classes at module level; those imports moved into the test.
  • test_train_task.py: requires("albumentations>=1.4.0", "torchmetrics>=1.5", "transformers").
  • tests/_models/{rfdetr,super_gradients,timm,ultralytics}: package imports moved from module level into the tests, plus a module-wide requires(...). test_get_model in test_super_gradients_package.py parametrizes class names instead of classes.

Markers only apply after a module is imported, so optional packages must not be imported at module level.

Only remaining module-level skip: ffmpeg in test_extract_video_frames.py (system binary, not a Python package).

How has it been tested?

  • With torchmetrics 0.8.2 (uv run --isolated --with torchmetrics==0.8.2): tests/_metrics and test_train_task.py collect without errors, 102 skipped with Requires torchmetrics>=1.5.
  • Without rfdetr / super-gradients / timm / ultralytics: the four directories collect without errors, 102 skipped with Requires <package>.
  • With the packages installed: 80 passed in the four directories.
  • Collected test IDs are identical to main, except that the super_gradients tests are now collected and reported as skipped when super-gradients is missing (they were hidden by the module-level skip before).
  • mypy tests passes.

Did you update the documentation?

  • Yes
  • Not needed (internal change without effects for user)

🤖 Generated with Claude Code

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-24T20:03:20.109499Z 78da16c PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 78da16ccab

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread tests/conftest.py Outdated
"_models/timm/*": "timm",
"_models/ultralytics/*": "ultralytics",
}
collect_ignore_glob = [

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve skips for explicitly selected optional-package tests

When an optional package is absent, these globs only suppress files found while recursively collecting a directory; pytest still collects a file or node ID passed explicitly. Consequently, commands such as pytest tests/_models/timm/test_timm.py now fail during import instead of reporting skips, while invoking one of these directories alone exits with “no tests collected” (status 5). This is reproducible with pytest 9.0.3 and breaks the documented workflow for running a specific test file, so the optional-package handling needs to cover explicit selections as well.

Useful? React with 👍 / 👎.

…ctories

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@liopeer liopeer changed the title test: replace module-level skips with requires marker and directory exclusion test: replace module-level skips with requires marker Oct 24, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant