Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 23 additions & 12 deletions .claude/skills/verify-pyprocar/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ pyprocar is a Python library, not a server. The user surface is Python calls in

- **Object API (current):** `ElectronicBandStructurePath/Mesh.from_code`, `DensityOfStates.from_code`, `FermiSurface.from_code` → `BandStructurePlotter` / `DOSPlotter` / `FermiPlotter` / `FermiSlicePlotter`. See `examples/general/new_*_examples.py`.
- **Legacy one-call functions:** `pyprocar.bandsplot`, `bandsdosplot`, `dosplot`, `fermi2D`, `FermiHandler`, `BandStructure2DHandler`, which the `examples/*/*.ipynb` notebooks use. They are mid-migration. The per-feature status is in `features/README.md`.
- **File utilities:** `pyprocar.bandgap`, `kpath`, `filter`, `repair`, `cat`, `generate2dkmesh`. These read and write VASP files instead of plotting; see `features/utilities.md`.
- **File utilities:** `pyprocar.bandgap`, `kpath`, `filter`, `repair`, `cat`, `generate2dkmesh`, the `scripts/procar.py` CLI and the `pyprocar-download` CLI. These read and write files instead of plotting; see `features/utilities.md`.

"Driving" means: write a small driver script that makes those calls on a real calc dir, run it through the harness, and read the evidence.

Expand Down Expand Up @@ -105,19 +105,29 @@ $H run bands-plain data/examples/bands/non-spin-polarized .claude/skills/verify-
```

Per-feature recipes are in `features/`. Each feature has a ready driver at `scripts/examples/<feature>.py` that exercises every entry point the feature file lists. It uses `scripts/lib/verify_steps.py`:
- `@step(name)` runs one entry point in isolation and records `{ok, ...facts}` or `{ok: false, error, where}` into `summary.json`, so one crash doesn't hide the rest.
- `finish()` exits 1 if any step failed. A step that records a known crash fails the run too, so read `summary.json` per step and compare it against the feature file's proven end state.
- `@step(name)` runs one entry point in isolation and records `{ok, ...facts}` or `{ok: false, error, where}` into `summary.json`, so one crash doesn't hide the rest. `where` is the innermost frame in this checkout's `pyprocar/`, as `pyprocar/<path>:<line>`, else the driver's own line as `driver.py:<line>`. It reads the same in every worktree and run.
- `finish()` exits 1 if any step failed. A step that records a known crash fails the run too, so the exit code alone says nothing. Check the run with `compare`:

To drive the whole map:
```bash
$H compare data/verify-runs/<run>...
```

It finds the `scripts/examples` driver whose bytes equal the run's `evidence/driver.py`, reads the run's `summary.json`, `side_effects.txt` and exit code against that driver's rows in the `## Expected step status` tables, prints `matches` or `DEVIATES` with one line per difference, and exits 1 when any run deviates. A deviation is a step whose status, error type or crash site differs from its row, a step missing from either side, an unexpected side-effect file, or an exit code other than 1 with a known defect and 0 without. Triage each one: a fix or a new crash in the library, or a stale row. A one-off driver, or a run of a driver that has changed since, matches no driver, so compare reports it as a deviation; read its `summary.json` instead, or rerun the current driver.

To drive the whole map, then compare every run:

```bash
E=.claude/skills/verify-pyprocar/scripts/examples
E=.claude/skills/verify-pyprocar/scripts/examples tag=map$(date +%H%M%S)
for f in bands:bands/non-spin-polarized dos:dos/non-spin-polarized fermi3d:fermi3d/non-spin-polarized \
fermi2d:fermi2d/non-spin-polarized bs2d:bands/2d-bands parsers:bands/non-spin-polarized \
utilities:bands/non-spin-polarized; do
$H run ${f%%:*} data/examples/${f#*:} $E/${f%%:*}.py; done
utilities:bands/non-spin-polarized bands_plain:bands/non-spin-polarized \
ref_unfold:bands/unfolding ref_orbits:fermi3d/van-alphen ref_spin_ibz:fermi2d/bisb_monolayer; do
$H run $tag-${f%%:*} data/examples/${f#*:} $E/${f%%:*}.py; done
$H compare data/verify-runs/*-$tag-*
```

The loop runs one driver at a time, which keeps the Fermi-surface builds apart.

Headless rules:
- Matplotlib: always `savefig` into `EVIDENCE`. Pass `show=False` to legacy functions.
- PyVista: `pv.Plotter(off_screen=True)` + `screenshot(EVIDENCE/...)`. It works on this machine. The `vtkEGLRenderWindow ... OpenGL 3.2` WARN line is noise. A blank screenshot has about 1 distinct color; `distinct_colors()` checks for that.
Expand All @@ -130,7 +140,7 @@ Headless rules:
Evidence lives at `data/verify-runs/<run>/evidence/` and survives cleanup. That directory is gitignored via `/data`. A proof includes:
- **The rendered image.** Open it with the Read tool and look at it. Check for correct k-path labels, sensible energy window and non-empty curves. A blank axes still has a nonzero PNG size.
- **`summary.json` with numbers that tie the image to the data.** Examples: array shapes, number of plotted artists, axis limits, tick labels, and projection sums.
- **`side_effects.txt`.** Since #245, `from_code` and the legacy plotting functions write nothing into the calc dir without `use_cache=True`; the bands, dos, fermi3d, fermi2d and bs2d drivers listed nothing at d6d4aaa7. An `ebs.pkl` or any other file listed there is a regression. Only the file utilities write files, as `features/utilities.md` documents.
- **`side_effects.txt`.** Since #245, `from_code` and the legacy plotting functions write nothing into the calc dir without `use_cache=True`; the bands, dos, fermi3d, fermi2d, bs2d and parsers drivers listed nothing at c13166ce. An `ebs.pkl` or any other file listed there is a regression. Only the file utilities write files, as `features/utilities.md` documents.
- **Exit code and `run.log`**, including warnings.

Proof standards:
Expand All @@ -145,7 +155,7 @@ Proof standards:
$H clean data/verify-runs/<run>
```

This removes only that run's `work/` scratch copy (including its `TMPDIR`) and keeps `evidence/`. It refuses paths outside `data/verify-runs/`. Clean every run you made before hand-back, and leave other agents' runs alone. There are no processes to kill.
This removes only that run's `work/` scratch copy (including its `TMPDIR`) and keeps `evidence/`. It refuses paths outside `data/verify-runs/`. Clean every run you made before hand-back, and leave other agents' runs alone. Name your own run dirs, or glob on a tag only you used, such as the `map<HHMMSS>` tag above: a pattern such as `*-base-*` also matches other agents' runs. There are no processes to kill.

```bash
$H gc [hours] # default 24
Expand All @@ -155,13 +165,14 @@ This removes the `work/` copy of every run that started more than `hours` ago, f

## Helpers

- `scripts/verify.sh`: `doctor | fetch <relpath>... | run <name> <fixture> <driver.py> | clean <run-dir> | gc [hours] | worktree-setup | exec <cmd>...`
- `scripts/verify.sh`: `doctor | fetch <relpath>... | run <name> <fixture> <driver.py> | compare <run-dir>... | clean <run-dir> | gc [hours] | worktree-setup | exec <cmd>...`
- `scripts/compare.py`: what `$H compare` runs; it parses the `## Expected step status` tables in `features/*.md`.
- `scripts/examples/bands_plain.py`: the minimal single-call template, which exits 0. Copy it for a one-off driver.
- `scripts/examples/{bands,dos,fermi3d,fermi2d,bs2d,parsers,utilities}.py`: full per-feature drivers. Run them as shown under Drive.
- `scripts/lib/verify_steps.py`: `step`, `png`, `distinct_colors`, `writable_copy` and `finish` for multi-step drivers. It is importable because the harness puts `scripts/lib` on `PYTHONPATH`.
- `scripts/lib/references/`: independent references (unfolding weights, reduced spin mesh, tiled cut orbits), each with an analytic `validate()` and a `ref_*.py` driver. Compare against one of these before you write your own; `features/README.md` lists them.

Known repo issues that affect verification (as of dev @ d6d4aaa7):
- `pyprocar.download_from_hf(relpath, output_path=".")` crashes when given a str; it needs a `Path`. The harness passes a `Path`.
Known repo issues that affect verification (as of dev @ c13166ce):
- `pyprocar.download_from_hf(relpath, output_path=".")` and the `pyprocar-download` CLI crash when the output path is a str; it needs a `Path`. The harness passes a `Path`. `utilities.md` has the details.

A PR that changes a documented side effect updates this skill in the same PR. basedpyright type-checks `scripts/`, so a PR that breaks a driver's import or call fails CI's typecheck job until it fixes the driver.
26 changes: 17 additions & 9 deletions .claude/skills/verify-pyprocar/features/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,28 @@

One file per user-facing feature. Each lists every entry point a user can reach it by; a proof that covers one entry point is incomplete when the file lists others.

Each feature has a reusable driver in `scripts/examples/<feature>.py`. It runs every entry point as an isolated step, records each step's result or crash in `summary.json`, and exits 1 if any step failed. Read `summary.json` per step instead of trusting the exit code alone.
Each feature has a reusable driver in `scripts/examples/<feature>.py`. It runs every entry point as an isolated step, records each step's result or crash in `summary.json`, and exits 1 if any step failed. Each feature file ends with an `## Expected step status` table, one row per driver step: `ok`, or `known-defect` with the error type and crash site that `verify_steps` records as `where`. A `side_effects.txt` row lists the files the run must leave in the calc copy. `$H compare <run-dir>...` checks runs against these tables and exits 1 on any deviation, so read its output instead of the driver's exit code. When a deviation is a real change, update the row and the prose in the same edit.

| Feature | File | Driver / fixture | Status (dev @ d6d4aaa7, re-driven 2026-10-04) |
| Feature | File | Driver / fixture | Status (dev @ c13166ce, re-driven 2026-10-05) |
|---|---|---|---|
| Band structure along a k-path | [bands.md](bands.md) | `bands.py` / `data/examples/bands/non-spin-polarized` | Object API plain/parametric/scatter/`plot_quiver`/flip and legacy `bandsplot` plain/parametric/scatter/overlay_species and `bandsdosplot` work; `plot(vectors_mode=)` and `plot_overlay` crash |
| Density of states | [dos.md](dos.md) | `dos.py` / `data/examples/dos/non-spin-polarized` | Object API and legacy `dosplot` plain/parametric work; other `dosplot` modes not driven |
| 3D Fermi surface | [fermi3d.md](fermi3d.md) | `fermi3d.py` / `data/examples/fermi3d/non-spin-polarized` | Object API and legacy `FermiHandler` plain/parametric/fermi_speed work |
| 2D Fermi slice | [fermi2d.md](fermi2d.md) | `fermi2d.py` / `data/examples/fermi2d/non-spin-polarized` | Legacy `fermi2D` plain/parametric and `FermiSlicePlotter` with and without scalars work; `plot(scalars_data=<Property>)` draws uncolored lines |
| 2D band structure surface | [bs2d.md](bs2d.md) | `bs2d.py` / `data/examples/bands/2d-bands` | Object API and legacy `BandStructure2DHandler` work |
| Code parsers (VASP, QE, Elk, Abinit, Siesta, Lobster, BXSF, FRMSF) | [parsers.md](parsers.md) | `parsers.py` / `data/examples/bands/non-spin-polarized` + `data/codes/qe` | VASP, QE and Elk bands proven; others not driven |
| File utilities (`bandgap`, `kpath`, `filter`, `repair`, `cat`, `generate2dkmesh`) | [utilities.md](utilities.md) | `utilities.py` / `data/examples/bands/non-spin-polarized` | All six work |
| Band structure along a k-path | [bands.md](bands.md) | `bands.py` / `data/examples/bands/non-spin-polarized` | Object API plain/parametric/scatter/`plot_quiver`/flip/`plot_overlay` with an array and legacy `bandsplot` plain/parametric/scatter/overlay_species/overlay_orbitals and `bandsdosplot` work; `plot(vectors_mode=)` and `plot_overlay`/`plot_quiver` given `ebs.bands` crash |
| Density of states | [dos.md](dos.md) | `dos.py` / `data/examples/dos/non-spin-polarized` | Object API and every legacy `dosplot` mode work; `stack` without `items` crashes |
| 3D Fermi surface | [fermi3d.md](fermi3d.md) | `fermi3d.py` / `data/examples/fermi3d/non-spin-polarized` | Object API and legacy `FermiHandler` plain/parametric/fermi_speed, `save_3d` and the box widget work |
| 2D Fermi slice | [fermi2d.md](fermi2d.md) | `fermi2d.py` / `data/examples/fermi2d/non-spin-polarized` | Legacy `fermi2D` plain/plain_bands/parametric and `FermiSlicePlotter` with and without scalars work; `plot(scalars_data=<Property>)` draws uncolored lines |
| 2D band structure surface | [bs2d.md](bs2d.md) | `bs2d.py` / `data/examples/bands/2d-bands` | Object API with and without the zone and legacy `BandStructure2DHandler` work |
| Code parsers (VASP, QE, Elk, Abinit, Siesta, Lobster, BXSF, FRMSF) | [parsers.md](parsers.md) | `parsers.py` / `data/examples/bands/non-spin-polarized` + `data/codes` | VASP, QE (collinear and non-collinear), Elk and Abinit bands work; Siesta, Lobster, BXSF and FRMSF have no fixture |
| File utilities, test-data download and the `procar.py` CLI | [utilities.md](utilities.md) | `utilities.py` / `data/examples/bands/non-spin-polarized` | `bandgap`/`kpath`/`filter`/`repair`/`cat`/`generate2dkmesh` and `procar.py cat`/`filter` work; `spin_asymmetry` is a stub; `download_from_hf` with a str path and `procar.py bandgap`/`generate2dkmesh` crash |

Update the Status column whenever a run changes what is known.

## Entry points outside the map

`import pyprocar` and `pyproject.toml` expose these too. No feature file covers them:
- `pyprocar.pyposcar` (POSCAR reading, defects, clusters, RDF) is in `docs/source/api/pyposcar/`, with fixtures in `data/examples/pyposcar/`. It needs `import pyprocar.pyposcar`; `import pyprocar` does not load it. Not driven.
- `calculate_band_velocity`, `calculate_band_speed` and `calculate_avg_inv_effective_mass` are what `get_property("bands_velocity" | "bands_speed" | "avg_inv_effective_mass")` calls, which `bands.md`, `bs2d.md` and `fermi3d.md` drive.
- `welcome` prints the banner that the legacy functions print. `Settings` has no caller.
- `scripts/poscar.py`, `scripts/dftb+2procar.py` and `scripts/tmp.py` are uninstalled developer scripts.

## Independent references

A reference computes an expected value without pyprocar, so a verifier compares the library against it instead of writing a new one. Each lives in `scripts/lib/references/`, imports only numpy, scipy, pyvista or spglib, and has a `validate()` that checks it on an analytic case and rejects a wrong-convention control. Its driver runs `validate()` as the step `analytic_validation` before it touches the fixture. Run all validations without a fixture:
Expand Down
Loading
Loading