inspect-sr-skill

An agent-oriented implementation of INSPECT-SR for systematic reviewers who need a second pair of eyes on potentially problematic RCTs.

An agent skill that encodes the INSPECT-SR trustworthiness framework into a structured workflow. Point the agent at any study document (such as a PDF, registry entry, protocol, abstract, or notice) to systematically evaluate post-publication alerts, conduct and governance, text, figures, and results, without conflating trustworthiness with risk of bias, fraud detection, or generalisability.

---

_{Animation made with the huashu-design skill}

---

What it does

INSPECT-SR was developed by Jack Wilkinson and colleagues to answer one important question for systematic reviewers: is this RCT sufficiently trustworthy to include in evidence synthesis? It is not a risk-of-bias tool, not a fraud detector, and not a conflict-of-interest scorer.

Important

Not an official project! This repository is an independent community-led implementation of the INSPECT-SR framework. It is not affiliated with or endorsed by the official INSPECT-SR authors and developers.

This skill packages the INSPECT-SR framework to ensure consistent, reproducible execution by LLM agents, keeping the trustworthiness assessment strictly isolated from adjacent evaluations.

This skill is designed to assist, not replace, the systematic reviewer. The agent proposes, but the human decides. Every INSPECT-SR assessment produced by this skill must be reviewed, critiqued, and signed off by a qualified human before it influences any review decision. The agent records its sources and reasoning so you can verify each check independently. Please always treat the output as a first draft, not a verdict.

What's new

v0.1.1

• Refined assessment rules: Pre-reconciliation study identity checks and sequential arm-increment auditing (Check 4.2).
• Governance guardrails: Mandatory human checkpoints prior to study exclusions and codified red-line boundaries.

v0.1.0

• Aligned with INSPECT-SR Guidance v1.1.2: Updated guidance reflecting the latest official release, including critical nuances around rounding intervals for continuous p-values and proper scoping for statistical result verification.
• Modular per-domain architecture: Replaced monolithic guides with dedicated per-domain reference documents (references/domains/) containing official check wording, specific pitfalls to avoid, and worked examples from the official guidance.
• Local computational verification (scripts/): Integrated local R scripts covering arithmetic checks so the agent computes exact bounds rather than falling back to Unclear:
  • Mean/SD integer consistency via GRIM and GRIMMER (scrutiny)
  • Continuous p-value rounding consistency intervals and categorical test reproduction
  • Multi-variable baseline table scanning (reappraised)
  • Subgroup mean recombination and percentage denominator checks
• Calibrated assessment procedure: Codified explicit Step 0 evidence boundaries ("No is a claim you have to earn"), early stopping when serious concerns is justified, and standardized reporting deliverables.

Installation

The skill is built on the open Agent Skills standard. It works in any skills-compatible AI agent runtime.

Option 1: One-liner (recommended, cross-runtime)

npx skills (from Vercel Labs) reads this repo, finds skills/inspect-sr-skill/, and drops it into the right folder for whatever agent it detects:

npx skills add emberwhirl/inspect-sr-skill

Alternatively, open your preferred skills-compatible agent (e.g., Hermes, Claude Code, Codex, etc.) and tell it:

Install this skill for me: https://github.com/emberwhirl/inspect-sr-skill

Option 2: Manual clone

git clone https://github.com/emberwhirl/inspect-sr-skill.git

Then point your agent at the global or project-level skill directory (skills/inspect-sr-skill/).

Usage

Once installed, the skill activates automatically when you prompt your agent in natural language about topics defined in the SKILL.md frontmatter. Any query concerning potentially problematic RCTs, post-publication notices, statistical discrepancies, or study trustworthiness will trigger the workflow.

Example prompts you can paste:

"Perform INSPECT-SR on the Strasser 2023 trial."

Notes on R and computational dependencies

The skill includes R scripts under scripts/ for running mathematical checks locally:

• pval_check.R and consistency.R rely only on base R.
• grim_grimmer.R and baseline_scan.R require the R packages scrutiny and reappraised.

Environment setup:

• R runtime: Ensure R is installed in your system or runtime environment.
• Package installation: When running in an environment where the agent has command execution permissions, the agent will install missing R packages (scrutiny, reappraised) automatically when needed. If your agent is operating without terminal permissions, or if you prefer to manage dependencies manually, please install these packages in your R environment prior to running assessments.

License

This skill is released under the MIT License.

The original INSPECT-SR tool and guidance are copyrighted by their authors and licensed under CC-BY 4.0. When you use this skill to support review methods or reporting, please preserve attribution to the official INSPECT-SR guidance and cite Wilkinson et al. (see Citation).

Repository structure

inspect-sr-skill/
  ├── README.md                                       
  ├── assets/
  │   └── walkthrough.webp                            
  └── skills/inspect-sr-skill/                        
      ├── SKILL.md                                    
      ├── LICENSE                                     
      ├── references/
      │   ├── domains/
      │   │   ├── 1-post-publication-notices.md
      │   │   ├── 2-conduct-governance-transparency.md
      │   │   ├── 3-text-and-figures.md
      │   │   └── 4-results-in-the-study.md
      │   ├── table-template.md                       
      │   └── Wilkinson_MedRxiv_40950444_MEDLINE.txt  
      └── scripts/
          ├── README.md
          ├── baseline_scan.R
          ├── consistency.R
          ├── grim_grimmer.R
          └── pval_check.R

Citation

If this skill supports your review or methods write-up, please cite the original INSPECT-SR preprint:

Wilkinson J, Heal C, Flemyng E, et al. INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials. medRxiv. 2025:2025.09.03.25334905. doi:10.1101/2025.09.03.25334905. PMID: 40950444; PMCID: PMC12424918.

A copy of the MEDLINE (txt) format citation is bundled at skills/inspect-sr-skill/references/ for convenience when managing bibliographies.

Acknowledgements

• The INSPECT-SR team for the elegant tool
• Ian Hussey (@ianhussey), maintainer of the INSPECT-SR guidance repository
• Lukas Jung (@lhdjung) for scrutiny
• Mark Bolland for reappraised