Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .claude/skills/verify-pyprocar/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ Headless rules:
Evidence lives at `data/verify-runs/<run>/evidence/` and survives cleanup. That directory is gitignored via `/data`. A proof includes:
- **The rendered image.** Open it with the Read tool and look at it. Check for correct k-path labels, sensible energy window and non-empty curves. A blank axes still has a nonzero PNG size.
- **`summary.json` with numbers that tie the image to the data.** Examples: array shapes, number of plotted artists, axis limits, tick labels, and projection sums.
- **`side_effects.txt`.** Since #245, `from_code` and the legacy plotting functions write nothing into the calc dir without `use_cache=True`; the bands, dos, fermi2d and bs2d drivers listed nothing at e9599a2f. An `ebs.pkl` or any other file listed there is a regression. Only the file utilities write files, as `features/utilities.md` documents.
- **`side_effects.txt`.** Since #245, `from_code` and the legacy plotting functions write nothing into the calc dir without `use_cache=True`; the bands, dos, fermi3d, fermi2d and bs2d drivers listed nothing at d6d4aaa7. An `ebs.pkl` or any other file listed there is a regression. Only the file utilities write files, as `features/utilities.md` documents.
- **Exit code and `run.log`**, including warnings.

Proof standards:
Expand Down Expand Up @@ -141,8 +141,9 @@ This removes the `work/` copy of every run that started more than `hours` ago, f
- `scripts/examples/bands_plain.py`: the minimal single-call template, which exits 0. Copy it for a one-off driver.
- `scripts/examples/{bands,dos,fermi3d,fermi2d,bs2d,parsers,utilities}.py`: full per-feature drivers. Run them as shown under Drive.
- `scripts/lib/verify_steps.py`: `step`, `png`, `distinct_colors` and `finish` for multi-step drivers. It is importable because the harness puts `scripts/lib` on `PYTHONPATH`.
- `scripts/lib/references/`: independent references (unfolding weights, reduced spin mesh, tiled cut orbits), each with an analytic `validate()` and a `ref_*.py` driver. Compare against one of these before you write your own; `features/README.md` lists them.

Known repo issues that affect verification (as of dev @ e9599a2f):
Known repo issues that affect verification (as of dev @ d6d4aaa7):
- `pyprocar.download_from_hf(relpath, output_path=".")` crashes when given a str; it needs a `Path`. The harness passes a `Path`.

A PR that changes a documented side effect updates this skill in the same PR. basedpyright type-checks `scripts/`, so a PR that breaks a driver's import or call fails CI's typecheck job until it fixes the driver.
30 changes: 23 additions & 7 deletions .claude/skills/verify-pyprocar/features/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,30 @@ One file per user-facing feature. Each lists every entry point a user can reach

Each feature has a reusable driver in `scripts/examples/<feature>.py`. It runs every entry point as an isolated step, records each step's result or crash in `summary.json`, and exits 1 if any step failed. Read `summary.json` per step instead of trusting the exit code alone.

| Feature | File | Driver / fixture | Status (dos-rewrite @ 1f36aae1, re-driven 2026-10-02) |
| Feature | File | Driver / fixture | Status (dev @ d6d4aaa7, re-driven 2026-10-04) |
|---|---|---|---|
| Band structure along a k-path | [bands.md](bands.md) | `bands.py` / `data/examples/bands/non-spin-polarized` | Object API plain/parametric/scatter/`plot_quiver`/flip work; `plot(vectors_mode=)` and `plot_overlay` crash; every legacy `bandsplot` mode and `bandsdosplot` crash |
| Density of states | [dos.md](dos.md) | `dos.py` / `data/examples/dos/non-spin-polarized` | Object API works; legacy `dosplot` crashes in every mode |
| 3D Fermi surface | [fermi3d.md](fermi3d.md) | `fermi3d.py` / `data/examples/fermi3d/non-spin-polarized` | Object API works; legacy `FermiHandler` parametric/fermi_speed work, plain crashes |
| 2D Fermi slice | [fermi2d.md](fermi2d.md) | `fermi2d.py` / `data/examples/fermi2d/non-spin-polarized` | `FermiSlicePlotter` with scalars works, without scalars draws nothing; legacy `fermi2D` unreachable in every mode |
| 2D band structure surface | [bs2d.md](bs2d.md) | `bs2d.py` / `data/examples/bands/2d-bands` | Object API works; legacy `BandStructure2DHandler` crashes |
| Band structure along a k-path | [bands.md](bands.md) | `bands.py` / `data/examples/bands/non-spin-polarized` | Object API plain/parametric/scatter/`plot_quiver`/flip and legacy `bandsplot` plain/parametric/scatter/overlay_species and `bandsdosplot` work; `plot(vectors_mode=)` and `plot_overlay` crash |
| Density of states | [dos.md](dos.md) | `dos.py` / `data/examples/dos/non-spin-polarized` | Object API and legacy `dosplot` plain/parametric work; other `dosplot` modes not driven |
| 3D Fermi surface | [fermi3d.md](fermi3d.md) | `fermi3d.py` / `data/examples/fermi3d/non-spin-polarized` | Object API and legacy `FermiHandler` plain/parametric/fermi_speed work |
| 2D Fermi slice | [fermi2d.md](fermi2d.md) | `fermi2d.py` / `data/examples/fermi2d/non-spin-polarized` | Legacy `fermi2D` plain/parametric and `FermiSlicePlotter` with and without scalars work; `plot(scalars_data=<Property>)` draws uncolored lines |
| 2D band structure surface | [bs2d.md](bs2d.md) | `bs2d.py` / `data/examples/bands/2d-bands` | Object API and legacy `BandStructure2DHandler` work |
| Code parsers (VASP, QE, Elk, Abinit, Siesta, Lobster, BXSF, FRMSF) | [parsers.md](parsers.md) | `parsers.py` / `data/examples/bands/non-spin-polarized` + `data/codes/qe` | VASP, QE and Elk bands proven; others not driven |
| File utilities (`bandgap`, `kpath`, `filter`, `repair`, `cat`, `generate2dkmesh`) | [utilities.md](utilities.md) | `utilities.py` / `data/examples/bands/non-spin-polarized` | `kpath`/`filter`/`repair`/`cat`/`generate2dkmesh` work; `bandgap` crashes |
| File utilities (`bandgap`, `kpath`, `filter`, `repair`, `cat`, `generate2dkmesh`) | [utilities.md](utilities.md) | `utilities.py` / `data/examples/bands/non-spin-polarized` | All six work |

Update the Status column whenever a run changes what is known.

## Independent references

A reference computes an expected value without pyprocar, so a verifier compares the library against it instead of writing a new one. Each lives in `scripts/lib/references/`, imports only numpy, scipy, pyvista or spglib, and has a `validate()` that checks it on an analytic case and rejects a wrong-convention control. Its driver runs `validate()` as the step `analytic_validation` before it touches the fixture. Run all validations without a fixture:

```bash
$H exec python .claude/skills/verify-pyprocar/scripts/lib/references/check.py
```

| Reference | Driver / fixture | Checks | Feature file |
|---|---|---|---|
| `unfold.py`: Popescu-Zunger weights from the raw PROCAR phases and POSCAR | `ref_unfold.py` / `data/examples/bands/unfolding` | `ebs.unfold` weights | [bands.md](bands.md) |
| `spin_ibz.py`: symmetry-reduced non-collinear mesh (spglib operations plus time reversal) and the axial spin transform of each unfolded k | `ref_spin_ibz.py` / `data/examples/fermi2d/bisb_monolayer` | spin under `ibz2fbz` | [fermi2d.md](fermi2d.md) |
| `orbits.py`: closed orbits of a plane through a 16-cell tiled marching-cubes surface | `ref_orbits.py` / `data/examples/fermi3d/van-alphen` | cross-section area and van Alphen frequency text | [fermi3d.md](fermi3d.md) |

A reference whose `validate()` fails is not evidence. Fix the reference before you compare against it.
26 changes: 17 additions & 9 deletions .claude/skills/verify-pyprocar/features/bands.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,14 @@ Plot E(k) along a high-symmetry path from a non-SCF bands calculation. Optionall
- Bands + DOS side by side: legacy `pyprocar.bandsdosplot(bands_settings=dict(...), dos_settings=dict(...))`.
- Other legacy modes and notebooks: `ipr`, `atomic`, `overlay*`, `autobandsplot`; fixtures `bands/{ipr,atomic_levels,auto,compare_bands}`. Not driven.
- Band unfolding: `pyprocar.unfold(code="vasp", dirname=<bands/unfolding/supercell copy>, mode=, unfold_mode=, transformation_matrix=np.diag([2, 2, 2]), fermi=5.2182, show=False, savefig=...)`, or `ebs.unfold(transformation_matrix=, structure=ebs.structure)` for the weights alone (#277). Check the weights against `bands/unfolding/primitive` (the same Γ M K Γ path, 150 k-points): every k has 24 supercell bands below -20 eV whose weights sum to 3.00 (the three primitive Mg 2p bands), and every supercell band with weight above 0.25 between -10 and 12 eV lies within 0.5 eV of a primitive band. The primitive run was self-consistent on its line-mode k-points, so the two calculations differ by 0.1 to 0.4 eV. Convention trap: adding a k-dependent phase exp(i k.r) to the weight breaks this (4 eV mismatches, negative weights). `tests/pyprocar/scripts/test_unfold.py` runs these checks.
- Unfolding reference: `scripts/lib/references/unfold.py` computes Popescu-Zunger weights from the raw PROCAR phases and POSCAR with numpy only. Compare it with `ebs.unfold` and the primitive bands:
```bash
$H run ref-unfold data/examples/bands/unfolding .claude/skills/verify-pyprocar/scripts/examples/ref_unfold.py
```
- `validate()` unfolds plane-wave states in supercells of a two-atom cell under diag(2,2,2) and three sheared matrices, one of them not normal, to exactly 0 or 1 (error 1.1e-16, tolerance 1e-12). The same states with an exp(-2 pi i k.t) factor miss by at least 0.259.
- At d6d4aaa7 the reference matches `ebs.unfold` to 8.9e-16 over all 150 x 80 weights, and the `INDEPENDENT_WEIGHTS` literals to 2.1e-6.
- Against `primitive/EIGENVAL`, the Mg 2p weights sum to between 2.999 and 3.002 at every k. No band with weight above 0.25 lies more than 0.375 eV from a primitive band.
- With the k-phase factor, 806 such bands miss by up to 4.2 eV.

## How to get to it (user POV)

Expand All @@ -25,22 +33,22 @@ $H run bands data/examples/bands/non-spin-polarized .claude/skills/verify-pyproc
$H run bands-plain data/examples/bands/non-spin-polarized .claude/skills/verify-pyprocar/scripts/examples/bands_plain.py # single proven step, exits 0
```

`bands.py` runs every object-API call and the legacy calls as separate steps. At 1f36aae1 it exits 1 because of the crashes listed below.
`bands.py` runs every object-API call and the legacy calls as separate steps. At d6d4aaa7 it exits 1 because of the two object-API crashes listed below.

The proven end state (SrVO3, non-spin-polarized) is:
- `obj_plain`: 30 artists, xticklabels `Γ M Γ R X`, ylim about `[-32, 13.6]` eV, meaning energies are already Fermi-shifted (by the parsed `ebs.fermi`, 4.9992). The PNG shows bands crossing 0 eV near Γ–M.
The proven end state (SrVO3, non-spin-polarized, d6d4aaa7) is:
- `obj_plain`: 32 lines, xticklabels `Γ X M Γ R X`, ylim about `[-32.0, 13.6]` eV, meaning energies are already Fermi-shifted (by the parsed `ebs.fermi`, 4.9992). The PNG shows bands crossing 0 eV near Γ–M.
- `obj_parametric` / `obj_scatter`: 20 colored collections plus a colorbar axis. V-d weight is highest in the bands above 5 eV.
- `obj_quiver_plot_quiver`: 20 quiver collections and a colorbar. The x axis shows raw k-distance, with no high-symmetry tick labels.
- `side_effects.txt` lists `./ebs.pkl`.
- `legacy_bandsplot_plain`: 33 lines. `parametric` and `scatter`: 20 collections each. `overlay_species`: 60 collections, one band of O, Sr and V weight per band.
- `legacy_bandsdosplot`: the band panel with ticks `Γ X M Γ R X` and a total-DOS panel on the same energy axis.
- `side_effects.txt` is empty.

## Gotchas

These crash at 1f36aae1. They are product gaps: record them as failures, and don't route around them.
- **Legacy `bandsplot` plain** and **`bandsdosplot`**: `AttributeError: 'KPath' object has no attribute 'metadata'` at `plotter/bs_plot.py:346`.
- **Legacy `bandsplot` parametric/overlay_species** and **object `plot_overlay`**: `AttributeError: 'Property' object has no attribute 'ndim'` at `plotter/bs_plot.py:1551`.
- **Legacy `bandsplot(mode="scatter")`**: `ValueError: Invalid mode: scatter`, even though the error message lists `scatter`. The enum member is misspelled `SACATTER` in `scripts/scriptBandsplot.py`.
These crash at d6d4aaa7. They are product gaps: record them as failures, and don't route around them.
- **Object `plot_overlay`**: `AttributeError: 'Property' object has no attribute 'ndim'` at `plotter/bs_plot.py:986`. The legacy `overlay_species` mode works.
- **`plot(..., vectors_data=..., vectors_mode="quiver")`**, as written in `new_bands_examples.py`: `vectors_mode` falls through to `Line2D.set()` and raises `AttributeError`. Use `plot_quiver` instead.

Other notes:
- `BandStructurePlotter.plot` draws on the current matplotlib figure. Grab it with `plt.gcf()` to save.
- `from_code` always writes `ebs.pkl` into the calc dir. The fixture also ships a prebuilt `ebs.pkl`/`kpath.pkl`, which only `use_cache=True` would read.
- The fixture ships a prebuilt `ebs.pkl`/`kpath.pkl`, which only `use_cache=True` reads.
12 changes: 5 additions & 7 deletions .claude/skills/verify-pyprocar/features/bs2d.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,11 @@ $H run bs2d data/examples/bands/2d-bands .claude/skills/verify-pyprocar/scripts/

The fixture dir holds `graphene/` (Fermi -0.795606) and `bisb_monolayer/`. The driver uses `graphene/`.

The proven end state (1f36aae1):
- `obj_surface`: `n_points` is 800 at a 20×20 grid, with about 900 distinct colors. The screenshot shows two separated sheets (π and π*). They are jagged at this coarse grid.
- `side_effects.txt` lists `./graphene/ebs.pkl`.
The proven end state (d6d4aaa7):
- `obj_surface`: `n_points` is 800 at a 20×20 grid, with about 8.9k distinct colors. The screenshot shows two separated sheets (π and π*) colored by energy from -6.5 to 4.9 eV. They are jagged at this coarse grid.
- `legacy_handler_plain` and `legacy_handler_plain_notebook_kwargs`: about 1.2k distinct colors each. The screenshot shows the two sheets inside the hexagonal BZ prism with an `E - E_F (eV)` axis.
- `side_effects.txt` is empty.

## Gotchas

- **Legacy `BandStructure2DHandler.plot_band_structure` crashes** at 1f36aae1. This is a product gap.
- With plain kwargs it raises `PyVistaAttributeError: Attribute 'brillouin_zone' does not exist ... 'BS2DPlotter'` at `plotter/bs_2d_plot.py:364`.
- With the notebook's kwargs (`add_fermi_plane`, `fermi_plane_size`, `energy_lim`) it raises `TypeError: Plotter.__init__() got an unexpected keyword argument` at `bs_2d_plot.py:76`.
- `BS2DPlotter.plot(show_brillouin_zone=True)` hits the same `brillouin_zone` attribute error. Pass `False`.
- `BS2DPlotter.plot(show_brillouin_zone=True)` raised `PyVistaAttributeError: Attribute 'brillouin_zone' does not exist` at 1f36aae1. The legacy handler draws the zone at d6d4aaa7, but the driver passes `False`, so that call is not re-driven.
6 changes: 3 additions & 3 deletions .claude/skills/verify-pyprocar/features/dos.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,14 @@ Plot total and projected DOS versus energy. The `dos-rewrite` branch is rebuildi
$H run dos data/examples/dos/non-spin-polarized .claude/skills/verify-pyprocar/scripts/examples/dos.py
```

The proven end state (SrVO3, non-spin-polarized, 1f36aae1):
The proven end state (SrVO3, non-spin-polarized, d6d4aaa7):
- `obj_total`: `total_shape` is `[6000, 1]` and `total_max` is 62.91. The energy range is about `[-31.1, 12.6]` eV, already Fermi-shifted. The PNG shows a non-zero curve with peaks near -29, -13 and 0 eV.
- `obj_total_colored_by_projection`: `proj_le_total` is true, with 1 colored collection plus a colorbar axis.
- `obj_vertical_projected_legend`: the legend reads `Total`, `O_{2-4}-(p)`.
- `obj_new_files_in_calc` is `[]`. `from_code` writes `dos.pkl` only when `use_cache=True`.
- `legacy_dosplot_plain` and `legacy_dosplot_parametric`: xlim `[-6, 4]`, with 3 and 2 lines. The driver does not run the other seven modes.
- `side_effects.txt` is empty.

## Gotchas

- **Legacy `dosplot` crashes in every mode** at 1f36aae1, with `AttributeError: property 'energies' of 'DensityOfStates' object has no setter` at `scripts/scriptDosplot.py:338`. This is a product gap.
- Before that crash, legacy `dosplot` with the default `use_cache=False` **deletes and rewrites** the calc dir's `dos.pkl` and `structure.pkl`. They show up in `side_effects.txt`. This is one reason the harness works on a copy.
- The fixture ships `dos.pkl`, `structure.pkl` and PDFs from old runs. Ignore them; they are not evidence.
Loading
Loading