Skip to content

fix(core): check projection axes against bands and structure, and give every parser its orbital names - #298

Merged
lllangWV merged 36 commits into
devfrom
293-projection-layout
Oct 4, 2026
Merged

lllangWV merged 36 commits into
devfrom
293-projection-layout

Conversation

@lllangWV

@lllangWV lllangWV commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

Why

A user who loaded a QE non-collinear band structure got projections on one spin channel and 32 j-basis slots, so spins=[3] (Sz) raised an IndexError and an orbital selection such as V eg picked p states. The Abinit non-collinear DOS also had one channel. check_projected_layout (pyprocar/core/projection.py) counted axes only, so neither these layouts nor swapped spin and atom axes failed it. Five parsers passed no orbital names, and VASP cut a fixed list, so a filtered PROCAR whose header reads o0 was labelled ['s'].

What changed

  • check_projected_layout compares the projection's leading axes with the bands or total DOS, its atom axis with the structure, and requires 1, 2 or 4 spin channels. Its errors name the layout, for example expects (2, 3, 1|4, *, *) from bands of shape (2, 3, 1). Non-collinear projections (total, Sx, Sy, Sz) may go with 1 band channel (QE) or 4 (VASP and Abinit repeat each eigenvalue). ebs.py changes only the constructor call.
  • QE: the new pyprocar/io/qe/projwfc/spinor.py writes each |l j m_j> (spin-orbit) or |l m s_z> shell in the real (l, m) harmonics and spin up and down. It uses QE 7.2's own spinor, sph_ind and rot_ylm coefficients and gives total, Sx, Sy and Sz per orbital. The parser now reads s_z states and species labels with digits (the IndexError in QE Parser: re.findall for j= fails in non-collinear calc without SOC #219 and PyProcar QE parser fails for non-collinear calculations without SOC (s_z inst #220 input).
  • Abinit: a spinor DOS with nspden 1 gets three zero spin channels. Time reversal makes its spin-resolved DOS zero. Every parser passes the names it projects: projwfc.x's labels for QE, the PROCAR header for VASP and Abinit bands, and the lm slots for the Abinit, Elk and Lobster DOS. The QE, Abinit and Elk DOS also pass their structure, so the atom check runs.

Scope

Tradeoffs

  • Two non-collinear band conventions stay. Requiring 4 band channels everywhere would rewrite the 1-channel layouts in the ibz2fbz and unfold tests, which Symmetry unfolding (ibz2fbz) follow-ups: ignored rotations, magnetic time reversal, phases and orbitals under rotation #283 owns. Allowing (1, 4) costs no checking power, because 4 projection channels can only mean non-collinear.
  • QE non-collinear projected_phase is now None. A spinor has no single phase per orbital, and the unfolder indexes phases by projection channel.
  • A filtered PROCAR keeps o0. pyprocar.filter writes that name for a sum over orbitals the user chose, and the header is the only record of what the column holds. s would name an orbital the column is not.

Blast Radius

Every ElectronicBandStructure and DensityOfStates construction runs the stronger check. All parsers on all fixtures under data/codes and data/examples pass it, and the full test suite passes. QE non-collinear users get a new orbital layout (16 real orbitals, 4 channels instead of 32 slots on 1).

Verification

  • 26 new tests in tests/pyprocar/core/test_projected_layout.py, tests/pyprocar/io/qe/test_noncollinear_projections.py, tests/pyprocar/io/test_orbital_names.py and tests/pyprocar/io/abinit/test_dos.py. 22 fail on origin/dev and pass at head; the other 4 guard the spin layouts that must stay accepted. Their expected values come from hand-derived QE spinor amplitudes, projwfc.x's own weights and Löwdin labels, Abinit's PROCAR header, Elk's ELMIREP.OUT and the integrated DOS columns. Full suite at head: pytest tests -n 8 gives 2313 passed, 3 skipped. basedpyright and both ruff_new_violations.py gates pass.

  • QE SrVO3 spin-orbit bands at Gamma: V t2g and eg weights are (0.984, 0.000) for band 41 and (0.000, 0.833) for band 47. These match kpdos.out's j-state sums and the cubic crystal field. The V eg plot before and after:

    before: eg selection picks j-basis p slots, so no band is coloured
    after: the two eg bands above E_F are coloured
    after: Sz of the V d bands, a spin channel that raised IndexError before

  • Independent QE 7.5 runs of bcc Fe with the moment tilted to theta 60, phi 30, with and without spin-orbit coupling. pw.x reports the site moment at theta 60.00, phi 30.00. pyprocar's spin of band 1 at Gamma is (0.750, 0.433, 0.500) of its total in both runs, the direction (sin 60 cos 30, sin 60 sin 30, cos 60). Over all bands and k-points, every Fe d spin above 0.1 lies within 6 degrees of the moment.

Round 2: PROCARs written by pyprocar.filter

The round-1 audit found that a PROCAR filtered by atoms no longer loaded, that a filtered o0 column was still labelled (s), and asked whether selecting an orbital by name survives the header-based VASP names.

  • A PROCAR filtered by atoms loads and plots again. pyprocar.filter(..., atoms=[[0],[1],[2,3,4]]) writes one row per group of atoms, so the PROCAR had 3 rows next to a 5-atom POSCAR and the new atom check raised. The VASP parser (pyprocar/io/vasp/parser.py, ebs) now passes atom_groups, the PROCAR's ion count, when it differs from the structure, and warns. check_projected_layout compares the atom axis with that count, so a swapped atom and orbital axis still raises (test_band_structure_of_atom_groups_checks_the_atom_axis_against_the_group_count). Atom index i selects the i-th group and is labelled by its index. A selection with no atoms sums every group, where origin/dev raised IndexError. Species selections (atoms=['O'], overlay_species, items) raise a ValueError that names the filtered PROCAR, because a group can hold several species. The structure stays on the band structure for the lattice and the symmetry operations. (Corrected in round 3: fermi2D on an atom-filtered PROCAR of irreducible k-points now raises instead of unfolding. See Round 3.) docs/source/user-guide/filter.rst says all of this.
  • Labels use the parser's names. OrbitalIndexer put a shell letter before it read the names (pyprocar/core/atomic_orbital_index.py, _build_label_tokens), so index 0 always read s. The new orbital_shells takes shell l to be the 2l+1 orbitals whose names have angular momentum l. QE, Abinit, Elk (Y1-1) and both VASP spellings keep p and d. A filtered o0 now reads Bi-(o0) on examples/bands/auto (Bi-(s) before). A QE spin-orbit j state no longer reads s.
  • Shell names select by the same names the labels use. orbitals=['p'] and overlay_orbitals used the fixed table s=0, p=1-3, d=4-8 (pyprocar/scripts/_selection.py). They now take shells from orbital_shells too. A shell is either its 2l+1 named orbitals or one column named by its letter, the sum that Elk's task 21 and pyprocar.filter(orbital_names=['s','p','d']) write. On standard VASP, QE, Abinit and Elk names the indices are unchanged. A shell name with no such columns raises hold no whole p shell, where origin/dev raised IndexError or summed the wrong columns. overlay_orbitals on the QE spin-orbit DOS (32 j states) now raises instead of drawing j slots 0, 1-3 and 4-8 as s, p and d.
  • Selecting one orbital by name raises a clear error. orbitals takes indices and the shell names s, p, d and f. A single name such as x2-y2 or dx2 raised KeyError on every file before. It now raises ValueError: orbitals takes orbital indices or the shell names s, p, d, f, not 'dx2'... for both spellings on both header versions, so no selection depends on which spelling a VASP version writes.
  • unfold resolves species names through the selection resolver, as bandsplot now does, so an atom-filtered PROCAR refuses them there too.
  • Policy change in the shared resolver (ProjectionSelectionResolver.resolve): with no structure and no atom or species selection, it now selects every atom row instead of raising Atom indexer is required. Grouped rows need this. (Corrected in round 3: the Lobster DOS parser passes no structure, so real files reach this path too, and at the round-2 head the DOS paths summed no rows there. The resolver now returns explicit row indices for every source. See Round 3.)

Before and after, the O bands of SrVO3. On the left is origin/dev with the unfiltered PROCAR and atoms=[2,3,4]. On the right is this branch with the filtered PROCAR and atoms=[2], a call that raised ValueError at the round-1 head. The two plots are identical.

origin/dev, unfiltered PROCAR, atoms=[2,3,4]
this branch, atom-filtered PROCAR, atoms=[2]

Round-2 verification

  • New tests, all red on origin/dev (3ae5842) and green here:
    • tests/pyprocar/scripts/test_filtered_procar.py (data, 11 tests) goes through pyprocar.filter and bandsplot. An atom group's projection equals the sum of its atoms' rows in the unfiltered PROCAR (to 1e-9). The orbital sum and an orbital-filtered p column match the unfiltered PROCAR the same way. A group label is 0, not Sr_{0}, and the o0 label is o0, not s. Species selections raise. orbitals=['p'] on columns named py pz px draws the same colours as their indices, orbitals=['d'] on columns named s p d the same as index 2, and overlay_orbitals there labels s, p, d. orbitals=['p'] on o0 o1 raises. A single orbital name raises for both header spellings. Red reasons on origin/dev: IndexError: index 3 is out of bounds for axis 3 with size 3, IndexError: index 4 is out of bounds for axis 4 with size 3, 'Sr_{0}' == '0', 's' == 'o0' and KeyError: 'dx2'.
    • tests/pyprocar/core/test_orbital_labels.py covers o0 and QE j states (red: 's' == 'o0', 's,l0_j0.5_m0.5' == ...). Five cases guard the shell letters that must stay (QE, Elk, part of a shell, filtered s/p/d names). They carry guards_existing_behaviour.
    • tests/pyprocar/core/test_projected_layout.py adds 2 atom_groups tests (red: TypeError on the new argument).
  • Live, on private copies: bandsplot on the atom-filtered SrVO3 PROCAR in plain, parametric with and without atoms, and overlay_orbitals works. overlay_species and overlay with items raise the species error. At the round-2 head, fermi2D plain and parametric ran on an atom-filtered fermi2d/non-spin-polarized PROCAR. Round 3 makes that call raise, because the fixture holds irreducible k-points (see Round 3).
  • Gates at 29b25ea (b1a8d63 only adds a guards_existing_behaviour marker): quick suite -n 4 gives 1933 passed and 2 skipped. The -m data suite gives 400 passed and 1 skipped (round 1 had 389). basedpyright passes, after 2 baseline entries were deleted for resolver comparisons the change made necessary. Both ruff_new_violations.py gates and the F821 check pass. CI on b1a8d63: format, lint, test, typecheck and red-green pass, and red-green reports ok for 27 added or changed tests.

Round 3: the round-2 audit and live findings

The round-2 audit failed the PR on finding A. The live review passed it with a test gap, finding B. Round 3 fixes both, closes one sibling of A, and adds a guard for the interaction with #299.

  • A. A DOS without a structure summed no atoms (audit, blocking). At the round-2 head, ProjectionSelectionResolver.resolve returned atoms=() when the source had no structure. ElectronicBandStructure and FermiSurface read () as every row, but DensityOfStates passed it to np.take, which kept no rows. The Lobster DOS parser passes no structure, so dosplot(code='lobster', mode='parametric', orbitals=[1,2,3]) drew an all-zero projection with no error. origin/dev raised Atom indexer is required there.
    • Fix. The resolver now takes the source's row count (n_atom_rows, from the new ProjectionSource.n_atoms, pyprocar/core/projection.py:206). With no atom or species selection it returns every row index explicitly (pyprocar/core/atomic_orbital_index.py:957-958). The EBS, Fermi surface and metadata callers pass selection.atoms as given, so no consumer reads () as "all" any more.
    • An explicitly empty selection raises The selection names no atoms; select at least one atom or species (atomic_orbital_index.py:961). On origin/dev, compute_projected_sum(atoms=[], orbitals=[0]) on a DOS returns zeros with no error (measured).
  • Sibling of A: empty spins. spins=[] raises The selection names no spins; omit spins to select every channel (atomic_orbital_index.py:963). On origin/dev the same DOS call returns an empty array with no error (measured).
  • Label change for sources without a structure. The resolver now names the rows it selects, so a selection without atoms is labelled by them. A 2-row DOS with no structure and orbitals=[4,5,6,7,8] reads 0-1-(d). The round-2 head printed (d), and origin/dev raised Atom indexer is required. Sources with a structure keep their species labels.
  • B. No test pinned the whole-shell rule (live note 1). The mutant that accepts part of a shell (== 2 * ang + 1 changed to >= 1 in orbital_shells) passed every test at the round-2 head. Two tests now pin it. test_two_of_three_p_orbitals_are_not_labelled_p checks that the labels of py and pz read py,pz. test_part_of_a_shell_is_not_that_shell writes a PROCAR with pyprocar.filter(orbital_names=['s','py','pz']) and checks that bandsplot(orbitals=['p']) raises hold no whole p shell. Both fail under the mutant (measured: assert 'p' == 'py,pz' and DID NOT RAISE).
  • Guard for fix(ebs): make ibz2fbz unfold magnetic groups, phases, atoms and orbitals correctly #299: an atom-filtered PROCAR of irreducible k-points no longer unfolds. Since fix(ebs): make ibz2fbz unfold magnetic groups, phases, atoms and orbitals correctly #299, ibz2fbz moves projections between atom rows by each symmetry operation's atom permutation. A row of an atom-filtered PROCAR holds a group of atoms, and the file does not record which ones. So no row permutation is known to be right. ibz2fbz now raises a ValueError if any operation it applies moves an atom (pyprocar/core/ebs.py:2278-2288). The check runs before it copies any property. The message names the two working routes: filter the PROCAR of a full k-grid calculation (ISYM = -1), or select atoms in the plot of the unfiltered PROCAR. filter.rst says the same. On origin/dev (with fix(ebs): make ibz2fbz unfold magnetic groups, phases, atoms and orbitals correctly #299) the same fermi2D call fails with IndexError: index 2 is out of bounds for axis 3 with size 2 (ebs.py:2323). Tradeoff: a group that the symmetry maps onto itself, such as all three O atoms in one row, would unfold correctly. The file cannot show that, so pyprocar refuses instead of guessing.
  • Rebase. Round 3 rebased onto origin/dev 5884b80 (fix(ebs): make ibz2fbz unfold magnetic groups, phases, atoms and orbitals correctly #299). In CLAUDE.md it kept this PR's projections row and fix(ebs): make ibz2fbz unfold magnetic groups, phases, atoms and orbitals correctly #299's documented-calls row. origin/dev has since moved to f243254 (feat(verify-pyprocar): validated references and fresh status tables #297, which changes only the verify-pyprocar skill). git merge-tree against it is clean, so the base stays. The merged tree passes basedpyright, both ruff_new_violations.py gates and the F821 check.

Round-3 verification

  • New tests, red on origin/dev and green at head:

    • tests/pyprocar/core/test_dos.py: test_projected_sum_without_structure_or_atoms_sums_every_atom_row (expects 105.0 per energy) and test_magnetization_without_structure_or_atoms_sums_every_atom_row (expects 13.5). Both are red with Atom indexer is required. test_an_empty_atom_selection_raises_instead_of_summing_nothing and test_an_empty_spin_selection_raises_instead_of_summing_nothing are red with DID NOT RAISE.
    • tests/pyprocar/io/lobster/test_parser.py::test_dosplot_parametric_colours_a_lobster_dos_by_every_atom (p of both atoms over every projection, 0.9). Red with Atom indexer is required.
    • tests/pyprocar/scripts/test_filtered_procar.py (data): test_part_of_a_shell_is_not_that_shell is red with IndexError: index 3 is out of bounds for axis 4 with size 3. test_fermi2d_refuses_to_unfold_an_atom_filtered_irreducible_procar is red with the IndexError above. I ran both against the origin/dev f243254 library with private fixture copies.
    • test_two_of_three_p_orbitals_are_not_labelled_p passes on origin/dev. It carries guards_existing_behaviour, because the fixed table never called py and pz a p shell either.
  • Gates at 625821d, through verify.sh exec:

    Gate Result
    pytest tests -m "not data" -n 4 1959 passed, 2 skipped, exit 0
    pytest tests -m data -n 4 403 passed, 1 skipped, exit 0
    basedpyright (head, and head merged with f243254) 0 errors, 0 warnings, exit 0
    ruff_new_violations.py check origin/dev (merged tree) 32 changed Python files, ok
    ruff_new_violations.py format origin/dev (merged tree) 32 changed Python files, ok
    ruff check --select F821,F822,F823,NPY201 . (merged tree) all checks passed
    red_green.py origin/dev ok: 17 tests fail on base, 4 guard existing behaviour, 17 data tests are not judged
    gh pr checks 298 format, lint, red-green, test and typecheck pass
  • The E501 failure in an earlier CI lint run (a guard reason line) is fixed in cca79e0. CI lint passes at 625821d.

  • fix(warnings): show user-facing warnings through warnings.warn #294 is still open, so the Abinit user_logger.warning call stays. It needs converting if fix(warnings): show user-facing warnings through warnings.warn #294 merges first.

Closes #293. Refs #219, #220.

@lllangWV
lllangWV force-pushed the 293-projection-layout branch 2 times, most recently from ceac04e to 959fe37 Compare October 4, 2026 17:19
@lllangWV

lllangWV commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

VERDICT: FAIL

Reviewed SHA: 38b2524 (base origin/dev bf65596, which is also the merge base, so no trunk merge was needed). Lane: audit + regression. The PR head and dev did not move while I reviewed.

What I ran (detached worktrees, verify.sh exec, reflink copies of every fixture inside each worktree)

  • Full -m data suite. Head: 389 passed, 1 skipped, exit 0. origin/dev: 381 passed, 1 skipped, exit 0. A first head run had 3 xdist workers OOM-killed while the machine was at load 100. A clean rerun passed.
  • Quick suite at head (-m "not data" -n 8): 1924 passed, 2 skipped, exit 0. gh pr checks: format, lint, test and typecheck pass.
  • I ran every parser (get_parser(code, dir).ebs/.dos) on 49 fixture dirs at head and at origin/dev: all of data/codes/{abinit,elk,qe,vasp} except QE non-colinear dos/fermi, plus data/examples/{bands,dos,fermi3d,fermi2d}/*. For each I compared shapes, sha1 hashes of bands/projected/phase/total, orbital names, structure atom counts, selection labels for orbitals [0], [1], [4], [8], [1,2,3], [2,5] and [4..8], warnings, and side effects.
    • The bands, projected, phase and total arrays are identical everywhere except in the two intended places:
      • QE non-colinear bands: (155,50,1,5,32) plus phase becomes (155,50,4,5,16) with phase None.
      • Abinit non-colinear DOS: (9401,1,1,9) becomes (9401,4,1,9). Channel 0 sums to 1604.6301 both times, and the Sx/Sy/Sz channels are zero (the fixture has nspden 1, nspinor 2).
    • No parser errors and no files written into any calc dir at either commit.
    • The VASP "N orbitals ... unknow" warning is gone. DOS selection labels for QE, Elk and Abinit now resolve instead of raising "Atom indexer is required", because those parsers now pass a structure.
  • QE non-colinear dos dir (a Mesh, so ibz2fbz runs, including the spin rotation now that is_non_collinear is True) loads at head: (4096,50,4,5,16).
    • Head: 29 s, peak RSS 11.1 GB. origin/dev: 320 s, 7.3 GB.
  • Norm check on QE non-colinear bands: the head total channel sums to 7651.60, which equals the raw Σ|<state|band>|² of 7456.99 × 155/151 inserted k-points. The spinor basis is unitary for l = 0..3 (max |BBᴴ − I| = 3e-16). origin/dev's j-slot sum, 7984, over-counts.
  • pyprocar.filter in each mode (atoms, orbitals with default and given names, bands, kpoints, ncl spin, collinear spin), then a load. I also ran bandsplot on the atom-filtered file.
  • git merge-tree of this PR against fix(warnings): show user-facing warnings through warnings.warn #294.

Acceptance (#293)

  1. Done. QE non-colinear bands have projected (155,50,4,5,16) (total, Sx, Sy, Sz) and bands keep 1 channel. The norm is conserved against raw atomic_proj.xml. Abinit non-colinear DOS has 4 projection channels and an unchanged total channel. The live lane checks values beyond the norm.
  2. Done. Swapped axes raise, and the error names the layout: the layout (n_kpoints, n_bands, n_spins, n_atoms, n_orbitals) expects (200, 20, 1|4, 5, *).
  3. Done. Names are passed by QE bands and DOS, Abinit DOS (s py pz px dxy dyz dz2 dxz dx2), Elk DOS (Y00..Y33) and Lobster DOS (seen in the dump and in the unit test).
  4. Done. examples/bands/auto gives ['o0']. All other VASP PROCAR names are unchanged, because the VASP 6 header equals the old fixed list. bands/ipr/Bi2Se3 now reads dx2 from its older header.
  5. Tests exist. Red/green and mutation are the live lane's job; I did not check them.

Findings

  • Blocking: an atom-filtered PROCAR no longer loads (pyprocar/core/projection.py:126 via pyprocar/io/vasp/parser.py:255). docs/source/user-guide/filter.rst ("To filter selected atoms") documents pyprocar.filter(..., atoms=[[0],[1],[2,3,4]]). The file it writes has 3 ion rows next to a 5-atom POSCAR.
    • Smallest repro: copy data/examples/bands/non-spin-polarized, run pyprocar.filter('PROCAR','PROCAR-f',atoms=[[0],[1],[2,3,4]]), replace PROCAR with PROCAR-f, then call pyprocar.bandsplot(code='vasp', dirname=..., mode='plain', show=False).
    • origin/dev: plain works, and parametric with atoms=[0,1,2] works.
    • Head: every mode raises ValueError: projected has shape (200, 20, 1, 3, 9); the layout ... expects (200, 20, 1|4, 5, *) from bands of shape (200, 20, 1) and a 5-atom structure. atoms=[[0,1,2,3,4]] fails the same way.
    • The PR handles the filtered-orbital case on purpose (it keeps o0), and its Blast Radius section says all parsers pass. It does not mention atom filtering.
    • Suggested fix: when the PROCAR ion count differs from the POSCAR, the VASP parser drops the structure and warns, or raises an error that names an atom-filtered PROCAR. If that is decided out of scope, record it in the PR and in filter.rst.
  • Note: the label of a filtered o0 column still reads (s) (pyprocar/core/atomic_orbital_index.py:310-316). prefer_groups matches index 0 against the s group before it looks at orbital_names, so the band and legend label for examples/bands/auto is Bi-(s). This contradicts the PR's reason for keeping o0. The same applies to orbitals=[[0],[1,2,3],...] with the default names.
  • Note, likely merge conflict with fix(warnings): show user-facing warnings through warnings.warn #294: the textual merge is clean, but fix(warnings): show user-facing warnings through warnings.warn #294 deletes user_logger = logging.getLogger("user") from pyprocar/io/abinit/parser.py. This PR adds user_logger.warning( at pyprocar/io/abinit/parser.py:258. After both merge, that line is undefined: ruff F821 fails, and an nspinor 2, nspden 4 Abinit DOS raises NameError. Convert the line to warn_user in whichever PR merges second.
  • Note: loading a QE non-colinear Mesh peaks 53% higher (7.3 GB to 11.1 GB on the 405-k SrVO3 dos dir; the fermi dir has the same grid). It is 11x faster. Under load, my first head run on that dir was OOM-killed (exit 137).
  • Note, pre-existing and out of scope: QE collinear spd_phase (pyprocar/io/qe/parser.py, the loop at pyprocar_projections_phase[..., i_atom, i_orbital] += projections[..., i_state]) still adds shells of the same (l, m) with different radial functions coherently, as |Σa|². The new spinor path adds them incoherently. The two QE conventions now differ.
  • Note: VASP DOS names still come from the fixed list (pyprocar/io/vasp/parser.py:306,345) while VASP bands read the PROCAR header. Acceptance item 4 covers only the PROCAR.
  • Note, pre-existing and scoped out by the PR: the QE spin-orbit DOS keeps 1 channel with 32 j slots, and DensityOfStates.is_non_collinear is True through the 32-orbital rule, so spin_projection_names lists 4 components for a 1-channel projection.
  • Audit, clean:
    • Every ElectronicBandStructure and DensityOfStates construction goes through the parsers, get_ebs_from_data or DensityOfStates.interpolate. ibz2fbz, unfold, reduce_*, BS2D and FermiSurface change properties in place and do not re-run the check.
    • Spin selection (resolve_spins, ebs_sum, Fermi surface and BS2D nspins = 1 when non-collinear) handles both 1 and 4 band channels.
    • No callers remain of the removed orbital_names_old, orbital_names_short, labels or _format_soc_label.
    • No narrating comments, no baseline edits, and the new data tests are marked.

@lllangWV

lllangWV commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

VERDICT: PASS+NOTES

Reviewed SHA: 38b2524 (base origin/dev bf65596; the merge-base equals origin/dev, so no merge with trunk was needed). Lane: Live + red/green + gates.

Gates at head (the commands from ci.yml, each run through verify.sh exec)

  • pytest tests -m "not data" -n 8: exit 0.
  • basedpyright --project pyrightconfig.json: exit 0.
  • ruff_new_violations.py check origin/dev: exit 0. format origin/dev: exit 0.
  • ruff check --select F821,F822,F823,NPY201 .: exit 0.
  • Full -m data suite: head 389 passed and 1 skipped (exit 0); origin/dev 381 passed and 1 skipped (exit 0). The first run at -n 8 lost workers to OOM because other agents were loading the machine. I reran it at -n 6 and both runs came out clean.
  • gh pr checks 298: format, lint, test and typecheck all pass.

Live: the QE spinor conversion (acceptance item 1)

  • I wrote my own reference, separate from spinor.py:
    • It builds each |l j m_j> state from sympy Clebsch-Gordan coefficients on scipy complex Y_lm (Condon-Shortley phase).
    • It fits each complex harmonic by least squares onto a re-implementation of QE ylmr2 real harmonics at 400 random directions.
    • It fixes the phase of each state against a transcription of QE 7.2 spinor.f90, sph_ind.f90, rot_ylm (init_us_1.f90) and compute_mj (projwfc.f90), all downloaded from gitlab qe-7.2. The result: QE equals the CG state for j = l+1/2 and minus the CG state for j = l-1/2, for every l from 0 to 3.
    • It groups states into shells by the projwfc labels (atom, 4P), not by wfc numbers.
    • It parses atomic_proj.xml independently.
  • Analytic check of spinor_projections: I fed it psi = (real orbital m) x (spinor along n). This covered l = 0 to 3, every real m, 3 random spin directions each, and both layouts (j halves as separate wfcs, and one wfc). The expected result is total 1 on m and S equal to n. Worst error: 2.2e-15.
  • data/codes/qe/7.2/SrVO3/non-colinear/bands (spin-orbit), through the public API ElectronicBandStructurePath.from_code("qe"):
    • The shape is (155, 50, 4, 5, 16). origin/dev gives (155, 50, 1, 5, 32).
    • Across all 151 k, 50 bands, 5 atoms, 16 orbitals and all 4 channels, max|pyprocar - reference| = 2.4e-15.
    • For each (atom, l), sum over m of the total equals projwfc's raw sum of |amp|^2 to 8e-16. It equals projwfc's own SrVO3.k.projwfc_up weights to 4e-10 for Sr, V and O5. For O3 and O4, projwfc_up averages the two atoms by symmetry (W = (A_O3 + A_O4)/2), and the O3+O4 sum matches to 5e-10. This is QE's symmetrisation of that file, not a pyprocar defect.
    • max(|S| - total) = 2e-16.
  • The owner's QE 7.5 Fe runs (angle1=60, angle2=30), copied privately. pw.x prints total magnetization (1.68, 0.97, 1.12), which is theta 60.0 and phi 30.0.
    • Spin-orbit run (Fe.rel pseudopotential, j basis): pyprocar equals my reference to 2.2e-15. The projected spin of the occupied bands, summed over the path, is (29.41, 16.98, 19.58), which is theta 60.03 and phi 30.01. Of the 179 bands with d weight above 0.3, the median |cos(S_d, n)| is 1.0000 and the minimum is 0.879; that spread is spin-orbit mixing.
    • No-spin-orbit run (s_z basis): matches the reference to 3e-16. Theta 60.00, phi 30.00, and every d band has |cos| = 1.
  • Abinit non-collinear DOS (abinit/9.6/Fe/non-colinear/dos, nspden 1): the shape is (9401, 4, 1, 9), with channels 1 to 3 exactly zero. origin/dev gives (9401, 1, 1, 9).
    • The zero spin channels are correct. Time reversal flips every (atom, real-orbital) spin component, so the BZ-integrated spin PDOS vanishes when nspden = 1.
    • The owner's Abinit 10.8 nspden=4 run writes DOS_AT with no magnetization columns. That supports the PR's warn-and-keep-total path.

Live: the layout check against every parser and every fixture (acceptance items 2 and 3)

  • Every fixture went through verify.sh run: all of data/codes (abinit, elk, qe and vasp, including bands, dos, fermi and fermi_plane) and data/examples/{bands,dos,fermi2d,fermi3d}. That is 83 from_code loads, both ebs and dos.
    • All 83 load at head and at origin/dev.
    • At head, check_projected_layout(proj, names, layout, bands|total, structure) re-runs cleanly on every one.
    • Swapping the spin and atom axes raised on every fixture whose shape changes. Only the 1-spin, 1-atom cases are shape-symmetric (Abinit Fe non-polarized, van-alphen). Swapping the k and band axes raised on every band structure. Each error names the layout.
    • Nothing was written into any calc copy.
  • Orbital names, head against origin/dev:
    • QE bands, fermi and DOS: s pz px py dz2 ... (origin/dev: None).
    • Abinit DOS: s py pz px dxy dyz dz2 dxz dx2 (None).
    • Elk DOS: Y00 Y1-1 ..., 16 names (None).
    • examples/bands/auto: ['o0'] (origin/dev: ['s']), which is acceptance item 4.
    • ipr/Bi2Se3-spinorbit-surface: now dx2 as its PROCAR header says (origin/dev: x2-y2).
  • Lobster has no fixture; its synthetic test covers it.

Red/green (acceptance item 5)

  • I ran head's 4 new or changed test files against origin/dev's library: 22 failed and 32 passed. Every new test fails for the reason it states:
    • "DID NOT RAISE" or the old message for the layout tests.
    • (9401,1,1,9) != (9401,4,1,9) for the Abinit test.
    • (1,4,1,1,32) and (155,50,1,5,32) != (155,50,4,5,16) for the QE tests.
    • None names, and ['s'] != ['o0'] for VASP.
  • The 4 tests that pass on origin/dev are the "accepts each spin layout" positive controls and the existing Abinit tests.
  • Mutations at head (one at a time, new tests rerun each time): all 11 were caught.
    • Spinor: the lower-j sign, the Sy sign, a swapped m_j to m mapping, the rot_ylm sign for k<0, and no j-pairing. Each failed the j-basis synthetic test, and the Gamma data test caught 3 of the 5.
    • The spinor spd returning None.
    • Dropping the structure atom check (2 tests) and dropping the leading-shape check (9 tests).
    • Abinit without padding.
    • VASP with the fixed name list.
    • QE names set to None.
  • Expected values come from outside the code under test: projwfc's kpdos.out weights, PROCAR and ELMIREP headers, the trapezoid integral of DOS_AT, and hand-built QE-convention amplitudes. I confirmed those amplitudes against my reference.

Acceptance

  • (1) Done.
  • (2) Done.
  • (3) Done.
  • (4) Done.
  • (5) Done.

Notes (none blocking)

  1. The QE non-collinear DOS (qe/7.2/SrVO3/non-colinear/dos) is still (7435, 1, 5, 32): one channel with j-resolved l1_j0.5_m-0.5 names. That breaks the "non-collinear = 4 channels" contract, but the frozen acceptance set names only QE bands and Abinit DOS. projwfc's spin-orbit pdos files carry no spin components, so this is the same limit as Abinit with nspden 4. Worth an issue.
  2. An Abinit spinor DOS with nspden 4 gets 1 channel plus a user warning (pyprocar/io/abinit/parser.py, _projected_dos). The warning is loud, not silent, and it is justified by Abinit's DOS_AT format.
  3. Spin-orbit runs with a fully relativistic pseudopotential combine the j = l-1/2 and j = l+1/2 halves into one shell, although their radial functions differ (_shells in spinor.py). That is an approximation. It is exact for scalar-relativistic pseudopotentials in a spin-orbit run, such as the SrVO3 fixture. The Fe run with the rel pseudopotential still lands on 60.03 and 30.01.
  4. VASP names now follow the PROCAR header, so the same orbital can be named dx2 or x2-y2 depending on the file. The data suite passes; I did not check selection by the x2-y2 name on old-header files.

@lllangWV

lllangWV commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

VERDICT: FAIL

Reviewed SHA: b1a8d63 (round 2), on origin/dev 3ae5842, which is also the merge base. Lane: audit + regression. Neither the head nor dev moved while I reviewed.

What I ran

All runs used detached worktrees, verify.sh exec, and reflink copies of every fixture inside each worktree.

  • Full -m data -n 4 suite. Head: 400 passed, 1 skipped, exit 0. origin/dev: 381 passed, 1 skipped, exit 0.
  • gh pr checks. format, lint, red-green, test and typecheck all pass.
  • Selection dump, at head and at origin/dev, on 50 fixture dirs. These are all of data/codes except QE non-colinear fermi (same grid as the dos dir, which I included), plus data/examples/{bands,dos,fermi3d,fermi2d}/*. For each ebs and dos I recorded:
    • orbital_indices for s, p, d, f and [4,5]
    • projection_components for orbitals, species, items={sp:['d']} and items={sp:[0,1]}, as labels and sums
    • resolver atoms for an orbital-only selection
    • labels for the slices all, 0-3 and 9-15
    • compute_projected_sum(orbitals=d) with no atoms
  • Atom-filtered PROCARs written by pyprocar.filter(atoms=[[0],[1],[2,3,4]]), run through bandsplot (plain, parametric with atoms=[2], default, orbitals=['d'] and atoms=['O']; overlay_orbitals, overlay_species and items), fermi2D (plain and parametric) and FermiHandler (plain and parametric), at head and at origin/dev.
  • Constructor axis-swap cases with atom_groups set.
  • Lobster DOS and a structure-less DensityOfStates through compute_projected_sum, projection_components and dosplot.
  • Baseline diff per file.
  • git merge-tree of this PR against fix(warnings): show user-facing warnings through warnings.warn #294 (head 4297e96).

Acceptance (#293)

Items 1 to 4 are still done. I re-ran the regression from round 1: QE non-collinear (155,50,4,5,16), Abinit non-collinear DOS 4 channels, names from every parser, and bands/auto named ['o0']. Red/green belongs to the live lane; CI red-green passes.

The round-1 blocker is fixed

At head, atom-filtered PROCARs load with a warning, and ebs.atom_groups == 3:

  • compute_projected_sum(atoms=[0], d) equals row 0 of the raw array: 23.675.
  • An unselected sum equals all rows: 899.232.
  • The label of atoms=[2] is 2-(d).

Results of the user entry points:

Entry point Head origin/dev
bandsplot plain works works
bandsplot parametric atoms=[2] works works
bandsplot parametric default works IndexError
bandsplot orbitals=['d'] works IndexError
bandsplot overlay_orbitals works IndexError
bandsplot atoms=['O'], overlay_species, items raise the new species error IndexError, except that items happened to hit row 1
fermi2D plain and parametric works works
FermiHandler plain and parametric works works

A genuine swap still raises with atom_groups set, for each of these: spin and atom swapped (2,3,3,1,9) with groups 3; (2,3,4,1,9) with groups 4 and 1 band channel; k and band swapped; groups 4 against 3 rows. The docs in filter.rst match what I observed.

Findings

Blocking: with no structure, a DOS selection without atoms now silently returns zeros.

  • Cause. ProjectionSelectionResolver.resolve (pyprocar/core/atomic_orbital_index.py:950-955) now returns atoms=() when there is no atom indexer. The EBS and FermiSurface callers map an empty tuple to None (all rows). DensityOfStates passes selection.atoms straight to sum_projection_components (pyprocar/core/dos.py:822-835; also 948, 1081 and 1235). There _validate_indices(()) returns an empty index array and np.take keeps no atoms.
  • Effect. origin/dev raised Atom indexer is required .... Head draws an all-zero projection with no error. The PR claims the resolver "selects every row when there is no structure", and that is false for DOS.
  • Repro 1, synthetic. DensityOfStates(energies, total, projected=rand(11,1,2,9), orbital_names=VASP9) without a structure, then .compute_projected_sum(orbitals=[4,5,6,7,8]). Head gives 0.0; the expected value is 60.77. With atoms=[0,1], head gives 60.77.
  • Repro 2, real parser path. The Lobster DOS never gets a structure. Use the DOSCAR_CONTENT of tests/pyprocar/io/lobster with an s p_y p_z p_x block (p sum 1.65), then call dosplot(code='lobster', mode='parametric', orbitals=[1,2,3]).
    • origin/dev raises.
    • Head runs and plots 0.
    • projection_components(dos, 'orbitals') gives s 0.0, p 0.0, d 0.0.
  • Fix. Map an empty selection.atoms to None in the DOS paths, as EBS does, or have the resolver return every row index.

Regression comparison of selections, origin/dev against head, 50 fixtures.

  • For standard VASP, QE collinear, Elk band, Abinit and Lobster names, the selected indices and labels are unchanged. 11 dirs are identical, including QE collinear bands/fermi, Elk bands, and bands/{2d-bands,atomic_levels,compare_bands,ipr,unfolding}.
  • Every other difference falls into one of the cases below. I judge each one acceptable.
    • orbital_indices(['f']) on a 9-orbital source (all VASP, the Abinit bands and bands/auto). origin/dev returned [9..15], which raised IndexError later. Head raises "hold no whole f shell".
    • QE, Elk and Abinit DOS. origin/dev raised "Atom indexer is required", or failed with 'NoneType' ... species for species. Head resolves, because those DOS now carry a structure. Labels: Fe_{0}-(s,p,d) and Sr_{0}V_{1}O_{2-4}-(s,p,d,f). Overlay sums: s+p+d equals the total for Fe, 1604.63.
    • QE non-collinear bands and dos-dir ebs. Values change with the intended 16-orbital spinor layout. For example, overlay d goes from 3337.7 to 1508.3, because origin/dev's indices 4-8 were j slots.
    • QE spin-orbit DOS (j names). orbitals=['s'|'p'|'d'|'f'] and overlay_orbitals now raise. origin/dev mapped them onto j slots [0], [1,2,3], [4..8], which mislabelled them. Labels now show the j names.
    • bands/auto (o0). The label goes from Bi-(s) to Bi-(o0), which fixes my round-1 note. Shell selections and overlay_orbitals raise a clear error instead of IndexError.

Notes:

  • fix(warnings): show user-facing warnings through warnings.warn #294 merge hazard, still open. merge-tree with fix(warnings): show user-facing warnings through warnings.warn #294 at 4297e96 is clean as text. The merged pyprocar/io/abinit/parser.py:258 still calls user_logger.warning( after fix(warnings): show user-facing warnings through warnings.warn #294 has deleted user_logger. That gives ruff F821 and a NameError on an Abinit DOS with nspinor 2 and nspden 4.
  • Baseline commit 885409e is deletion-only. It removes 2 reportUnnecessaryComparison entries in atomic_orbital_index.py: cols 50-71 is atoms_set is not None and cols 47-71 is species_list is not None, both in the old resolve. Both comparisons became necessary once atoms_set and species_list can stay None. It is tied to the resolver change, and CI typecheck passes.
  • The VASP atom check now accepts any count mismatch. pyprocar/io/vasp/parser.py:248-257 sets atom_groups = procar.n_atoms, and the projection is reshaped by the same count, so a stale or wrong POSCAR only warns. filter.rst documents this; it is not blocking.
  • Not exercised: unfold on an atom-filtered PROCAR. pyprocar.filter(atoms=...) itself fails on the only phase-PROCAR fixture (bands/unfolding/supercell), with ValueError: inhomogeneous shape in procarfilefilter.py:257. That failure is the same at origin/dev and at head.
  • Same defect left elsewhere, outside the script paths. ElectronicBandStructurePath.build_overlay_orbitals_weights (pyprocar/core/ebs.py:1455) still uses the fixed pyprocar.utils.info.orbital_names groups, not orbital_shells. No script calls it.

@lllangWV

lllangWV commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

VERDICT: PASS+NOTES

Reviewed SHA: b1a8d63. It is rebased onto origin/dev 3ae5842, which is also its merge-base. Lane: Live + red/green + gates (round 2).

Gates at head (the commands from ci.yml, each run through verify.sh exec, all pytest runs at -n 4)

Gate Result
pytest -m "not data" 1933 passed, exit 0
basedpyright 0 errors, exit 0
ruff_new_violations.py check origin/dev exit 0
ruff_new_violations.py format origin/dev exit 0
ruff --select F821,F822,F823,NPY201 exit 0
full -m data at head 400 passed, 1 skipped, exit 0
full -m data at origin/dev 381 passed, 1 skipped, exit 0
gh pr checks format, lint, red-green, test, typecheck all pass

Live: filtered PROCARs through the one-call functions, head against origin/dev

  • Setup. I made private copies of examples/bands/{non-spin-polarized,spin-polarized,non-colinear} and examples/fermi2d/non-spin-polarized, then ran pyprocar.filter on each:
    • atoms=[[0],[1],[2,3,4]] (Sr, V, O3)
    • atoms=[[0,1],[2,3,4]] (one mixed-species group)
    • orbitals=[[0],[1,2,3],[4..8]], once with the default names o0 o1 o2 and once with orbital_names=['s','p','d']
  • Independent reference. I read the unfiltered PROCAR with my own parser and summed its columns. Parametric colours are the midpoints of the per-k values, exactly.
  • Head plots against my PROCAR sums: max |diff| = 0.0000 for every case.
    • Atom-filtered: atoms=[2], atoms=[1] with orbitals=[4..8], atoms=[2] with orbitals=['d'], no atoms (sum of all groups), and the mixed group atoms=[0].
    • Orbital-filtered: atoms=[1] with orbitals=[1] (default names), and atoms=[1] with orbitals=['p'] (named s p d).
    • Unfiltered: atoms=['O'] with orbitals=['p'].
  • Head plots of filtered files against the equivalent selection on the unfiltered file at origin/dev: max |diff| ≤ 3.6e-15 for every case.
    • bandsplot plain and parametric, and overlay_orbitals (legend s p d).
    • overlay_species with orbitals=[1] against [1,2,3] (legend O Sr V).
    • Overlay items {'O':[1]} against {'O':[1,2,3]}, and {'O':['p']} on the s p d file.
    • The spin-polarized and non-collinear parametric plots for atoms=[2].
    • fermi2D parametric for atoms=[2] with orbitals=[4..8], and for atoms=[1] with orbitals=[2] on the orbital-filtered file; fermi2D plain contour segments are identical.
  • The same calls on origin/dev. Ten of them crash there with IndexError: index 3 is out of bounds: overlay_orbitals, overlay_species, items and atoms=['O'] on the atom-filtered file, and the shell names p/d on the orbital-filtered files.
  • Selections head now refuses, with clear errors:
    • species selections on an atom-filtered file (overlay_species, items, atoms=['O']) raise Cannot select species 'O' ... (a PROCAR filtered by atoms);
    • orbitals=['p'] on an o0 o1 o2 file raises hold no whole p shell;
    • overlay_orbitals on that file raises hold no whole s, p, d or f shell;
    • orbitals=['x2-y2'] and ['dx2'] raise orbitals takes orbital indices or the shell names (origin/dev: KeyError).
  • Load warning. Loading an atom-filtered file warns that the PROCAR has 3 ion rows and the structure 5 atoms. An unfiltered file has atom_groups None.
  • Unfiltered plots are unchanged. All 24 unfiltered cases are identical at head and origin/dev (max |diff| 0): bandsplot modes, fermi2D, and unfold plain, parametric, species ['B'], ['B'] with ['p'], and overlay_species.
  • examples/bands/auto. Head gives names ['o0'] and the label Bi_{0}-(o0) (origin/dev: s). orbitals=[0] gives the same colours as origin/dev. orbitals=['s'] and overlay_orbitals raise hold no whole s shell (origin/dev: s silently picks o0, and overlay crashes with IndexError).

Shell selection by letters and indices, per code

  • I computed the expected indices myself from each code's orbital order:
    • VASP and Abinit: s py pz px dxy dyz dz2 dxz x2-y2|dx2
    • QE: s pz px py dz2 dxz dyz dx2-y2 dxy followed by f
    • Elk: Y00, Y1-1 … Y33
  • Fixtures checked: VASP SrVO3 non-spin-polarized and non-collinear; QE non-spin-polarized, collinear spin-polarized and non-collinear bands, and the non-spin-polarized DOS; Elk bands and DOS; Abinit Fe non-spin-polarized and non-collinear bands, and the DOS.
  • Result: on all of them s, p, d, ['s','p'] and [1,2,3] give [0], [1,2,3], [4..8], [0..3] and [1,2,3]. That matches my computation and origin/dev. f gives [9..15] on QE and Elk. The overlay_orbitals shells match too.
  • Differences from origin/dev, all corrections:
    • f on the 9-orbital codes now raises. origin/dev returned the out-of-range indices [9..15].
    • Mixing letters and indices (['p', 4]) now works. origin/dev: KeyError: 4.
    • QE column 2 is now labelled px. origin/dev labelled it p_z, which is wrong: QE's order is s, pz, px, py.
    • On the QE non-collinear DOS (j-resolved), shell letters now raise. origin/dev silently picked j-state columns as if they were s, p, d.
  • The VASP, Abinit and Elk labels are unchanged.

Spinor reference at the new head

  • QE SrVO3 non-collinear bands: pyprocar against my independent Clebsch-Gordan and QE-convention reference, max |diff| = 2.4e-15. |S| ≤ total.
  • The owner's Fe runs (moment at theta 60, phi 30): spin-orbit matches the reference to 2.2e-15, and the projected occupied spin points at theta 60.03, phi 30.01. Without spin-orbit: 3.3e-16, and theta 60.00, phi 30.00.

Layout-check sweep and the swapped-axis raise

  • 83 from_code loads at head, through verify.sh run, covering all of data/codes and data/examples/{bands,dos,fermi2d,fermi3d}.
    • All 83 load. All 83 pass when I re-run check_projected_layout(..., atom_groups).
    • Swapping the spin and atom axes raised on every fixture whose shape changes; the only shape-symmetric ones are the 1-spin, 1-atom Fe and van-alphen fixtures. Swapping the k and band axes raised on every band structure.
    • Shapes and names are unchanged from round 1, and no files were written into any calc copy.
  • On the atom-filtered nsp, nsp-mixed, sp, ncl and fermi2d variants (atom_groups 3 or 2):
    • the valid array passes with atom_groups and raises without it;
    • a spin/atom swap raises both with and without atom_groups;
    • atom_groups + 1 raises;
    • a k/band swap raises.

Red/green and mutants

  • Head's 7 test files against origin/dev's library: 37 failed, 46 passed.
    • Every new test fails for the reason it states, for example: 'Sr_{0}' == '0', 's' == 'o0', a group-count check that does not raise, IndexError and KeyError instead of the new ValueErrors, and the j-basis label.
    • The tests that pass on origin/dev are the "accepts each spin layout" and "whole shell keeps its letter" positive controls and existing tests.
  • 10 mutants of the atom_groups and orbital_shells logic:
    • Caught by the PR's new tests (8): the VASP parser never sets atom_groups; the check ignores atom_groups; the resolver keeps species for atom groups; orbital_shells ignores names; the label builder uses the fixed groups; no letter-named column; a single orbital name silently dropped; species resolved through the POSCAR in bandsplot.
    • Caught only by the existing data tests in test_one_call_functions (1): dropping the x2-y2 special case, which takes VASP's d shell away. That gave 3 failures.
    • Survives the whole suite (1): accepting partial shells (== 2l+1 changed to >= 1). The behaviour at head is correct; I checked that a file with columns py pz raises hold no whole p shell on orbitals=['p']. No test pins it (see note 1).

Acceptance (frozen set, unchanged)

  • (1) Done. QE non-collinear bands give (155, 50, 4, 5, 16) and match the reference; the Abinit non-collinear DOS gives (9401, 4, 1, 9).
  • (2) Done.
  • (3) Done.
  • (4) Done.
  • (5) Done.
  • The round-1 audit FAIL is fixed: an atom-filtered PROCAR loads and plots with correct values, and species selections on it raise.

Notes (none blocking)

  1. Test gap: no test pins the whole-shell rule in orbital_shells (pyprocar/core/atomic_orbital_index.py, len(...) == 2 * ang + 1). The mutant >= 1 passes all 2333 tests. A one-line test is enough: header py pz plus orbitals=['p'] should raise.
  2. Pre-existing, the same at origin/dev and outside scope:
    • pyprocar.filter(atoms=...) on a PROCAR lm decomposed + phase file (unfolding/supercell) crashes with ValueError: inhomogeneous shape in procarfilefilter.FilterAtoms.
    • An orbital-filtered phase PROCAR fails in unfold with cannot reshape array of size 576000 into shape (150,80,1,24,3).
    • So the atom-group path cannot reach unfold at all.
  3. Carried over from round 1: the QE non-collinear DOS keeps 1 j-resolved channel, and head now refuses shell letters on it (loud, not silent). An Abinit spinor DOS with nspden 4 keeps the total only and warns.
  4. Documented limit (docs/source/user-guide/filter.rst): an atom-filtered PROCAR with exactly as many rows as the POSCAR has atoms cannot be detected. Its rows get POSCAR species labels.

@lllangWV

lllangWV commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

VERDICT: PASS+NOTES

Reviewed SHA: 625821d. I ran everything on that SHA merged with origin/dev f243254 (merge-tree clean, merged tree 3ad04be6). The comparison baseline is origin/dev f243254. Lane: Live + red/green + gates (round 3).

Gates on the merged tree (the ci.yml commands through verify.sh exec, pytest at -n 4)

Gate Result
pytest -m "not data" 1959 passed, 2 skipped, exit 0
full -m data at head 403 passed, 1 skipped, exit 0
full -m data at origin/dev 382 passed, 1 skipped, exit 0
basedpyright 0 errors, 0 warnings, exit 0
ruff_new_violations.py check origin/dev / format origin/dev 32 changed files ok, exit 0 / exit 0
ruff --select F821,F822,F823,NPY201 exit 0
red_green.py origin/dev ok, exit 0 (it does not judge data tests; I ran those below)
gh pr checks 298 format, lint, red-green, test, typecheck all pass

Round-2 blocker: a DOS without a structure. Fixed. I checked it against values I computed with numpy from arrays I built.

  • Synthetic DOS with no structure: 3 atom rows, 2 spins, random weights that vary with energy.
    • compute_projected_sum(orbitals=d) equals the sum over every row: max |diff| 3.6e-15.
    • orbitals=p, spins=[1]: 8.9e-16.
    • atoms=[0,2]: 0.
    • norm_mode='total_projection': 2.8e-16.
    • compute_magnetization(orbitals=p) equals up minus down over every row: 8.9e-16.
    • The same holds without orbital names, and for the total channel of a non-collinear DOS. origin/dev raises Atom indexer is required on all of these.
    • The label reads 0-2-(p). species=['Fe'] raises the new "no species per atom row" error.
  • Lobster dosplot(mode='parametric', orbitals=[1,2,3]) on my own DOSCAR.lobster: 3 atoms (Fe with s, p, dxy, dz2; two O with s, p), weights that vary with energy, both non-polarized and spin-polarized.
    • The colours equal sum(p over all atoms) / sum(every projection), per energy and per spin image: max |diff| 2.2e-16 (spin-polarized, image 1: 0).
    • The colours range from 0.7455 to 0.7625, not zero. origin/dev raises Atom indexer is required.

Selections with no atoms on real fixtures: explicit rows equal the old "all rows".

  • I ran 65 from_code loads, all of data/codes and data/examples/{bands,dos,fermi2d,fermi3d} through verify.sh run, at head and at origin/dev. On each I ran compute_projected_sum with p, d, spins=[0], s with the last spin, and d with norm_mode='max', plus compute_magnetization on the spin-resolved DOS.
  • Head against itself: no atoms against explicit atoms=range(n_rows) gives max |diff| 0 on every fixture. No atoms against my own numpy sum over rows and orbitals gives at most 3.4e-13.
  • Head against origin/dev: 272 selections are bit-identical (sha1 of the float64 values), labels included. Every difference is an intended change:
    • QE non-collinear bands and fermi: 4 channels now, labels [total]/[z].
    • bands/auto: o0 instead of s.
    • QE, Elk and Abinit DOS: origin/dev raised Atom indexer is required, because those DOS had no structure there.
    • QE collinear fermi grids: see note 4.
  • Plots through the public functions, head against origin/dev, same call: max |diff| 0 on all 17 cases.
    • bandsplot parametric with no atoms (d, all, p, p with spins=[1]) and overlay_orbitals.
    • fermi2D parametric with no atoms (d, all), atoms=[1], and plain, on the irreducible SrVO3 grid.
    • fermi2D on full-grid BiSb.
    • unfold parametric with no atoms, atoms=B, B with p, and plain.
  • Total-based DOS paths (compute_magnetization, with and without from_total, compute_normalized_total, compute_cumulative_total, species V with d) on VASP spin-polarized: identical sha1 at head and origin/dev.

Filtered PROCARs

  • bandsplot on atom-filtered spin-polarized SrVO3 ([[0],[1],[2,3,4]]), each against the equivalent call on the unfiltered file:
    • atoms=[2] against [2,3,4]: max |diff| 2.2e-16.
    • No atoms with p: 2.2e-16. origin/dev: IndexError: index 3.
    • Non-polarized, no atoms with d: 0. origin/dev: IndexError.
  • fermi2D on the full grid (BiSb monolayer, ISYM=-1, non-collinear, 3600 k-points):
    • Filtered [[0,1]]: atoms=[0] and no atoms both equal the unfiltered sum, 2.2e-16. origin/dev raises IndexError for no atoms.
    • Filtered [[1],[0]]: atoms=[0] equals unfiltered atoms=[1], 0.
    • Plain contours are identical.
    • This is the route the refusal message names, and it works.
  • fermi2D on the irreducible grid (fermi2d/non-spin-polarized, 496 k-points):
    • On the atom-filtered file, parametric and plain both raise: A PROCAR filtered by atoms cannot be unfolded from the irreducible k-points ... Filter the PROCAR of a full k-grid calculation (ISYM = -1), or select atoms in the plot of the unfiltered PROCAR. origin/dev: IndexError: index 3 is out of bounds for axis 3 with size 3.
    • Orbital-filtered files still unfold: atoms=[1], orbitals=[2] and no atoms with orbitals=[2] equal the unfiltered d selections, 1.3e-15.
  • unfold: see note 6.

Empty selections

  • On a DOS and an EBS, atoms=[], species=[] and spins=[] (with or without atoms) raise The selection names no atoms / no spins, and so do dosplot and bandsplot spins=[].
  • On origin/dev the same calls give zeros (DOS atoms=[]), an empty array, or sums on the EBS (1783.6, 3150.1, 9636.7), or a numpy zero-size array error from the plots.
  • bandsplot atoms=[] raises an IndexError instead (note 1).

Red/green. I ran head's 10 changed test files against origin/dev's library: 47 failed, 95 passed. Each round-3 test fails for the reason it states:

  • The two no-structure DOS tests and the Lobster test fail with Atom indexer is required.
  • The empty-atom and empty-spin tests fail with DID NOT RAISE.
  • test_part_of_a_shell_is_not_that_shell (data) fails with IndexError: index 3 ... axis 4 with size 3.
  • test_fermi2d_refuses_to_unfold_... (data) fails with IndexError: index 2 ... axis 3 with size 2 at ebs.py:2323.
  • The 3 modified test_atomic_orbital_index.py resolver tests fail on origin/dev only with TypeError on the new n_atom_rows argument. The mutants below cover the behaviour of defaults_to_all_atoms.

Mutants. I ran each against the 11 changed test files, data tests included, and restored the source after each; git status is clean. All are caught:

Mutant Failing tests
no atoms returns () again, guard kept 23
no atoms returns () again, guard dropped (the round-2 state) 14, including the Lobster and both no-structure DOS tests
empty-atom guard dropped 1
empty-spin guard dropped 1
n_atom_rows = n_atoms - 1 17
n_atom_rows = n_atoms + 1 22
n_atom_rows from structure.natoms 2, both atom-group tests
ibz2fbz refusal dropped 1
part of a shell counts as the shell (>= 1) 2

Spinor reference at the head

  • QE SrVO3 non-collinear bands, against my independent Clebsch-Gordan and QE-convention reference: max |diff| 2.44e-15. |S| ≤ total.
  • Fe with the moment at theta 60, phi 30:
    • With spin-orbit: 2.2e-15. The occupied projected spin points at theta 60.03, phi 30.01.
    • Without spin-orbit: 3.3e-16, at theta 60.00, phi 30.00.
  • I validated the reference first on an analytic case: a real harmonic times a spinor along n, for l = 0 to 3, all m, 3 random directions and both layouts. Worst error 2.2e-15.

Notes (none blocking)

  1. bandsplot(atoms=[]) raises IndexError: list index out of range from atoms[0] (pyprocar/scripts/scriptBandsplot.py:230), measured in parametric and overlay_species modes. The new clear ValueError never runs there. scriptUnfold.py:185 has the same atoms[0]; I read it but did not run it. origin/dev fails the same way; the PR rewrote the next line in both files. A not atoms check would route both plots to the resolver's message.
  2. orbitals=[] still means every orbital silently (DOS, EBS, bandsplot, items={'O': []}), unlike the new rule for atoms and spins. origin/dev does the same.
  3. compute_magnetization() on a DOS with only a total and no structure now raises The selection names no atoms; select at least one atom or species. The user selected nothing. origin/dev raised Atom indexer is required. It failed before too, but the new wording points the user the wrong way.
  4. A behaviour change the PR body does not mention, and it is a correction. ibz2fbz now receives QE's orbital names, so the per-orbital projections of QE fermi grids change against origin/dev (head and origin/dev differ by up to 1.33; the totals are identical). I compared the unfolded qe/7.2/SrVO3/non-spin-polarized/fermi grid with QE's directly computed band-path points that land on grid points (15 matches, eigenvalues equal):
  5. On an atom-filtered irreducible PROCAR, fermi2D plain mode is refused too, although plain contours do not depend on the rows. origin/dev crashed there, so this is not a regression; unfolding only the bands would serve plain mode.
  6. Unfold of a filtered PROCAR is not reachable. On the LORBIT=12 bands/unfolding/supercell, pyprocar.filter(atoms=...) itself fails (inhomogeneous shape), and an orbital-filtered file fails in unfold with cannot reshape array of size 576000 into shape (150,80,1,24,3). Both fail identically on origin/dev, so this is pre-existing and outside scope.

@lllangWV

lllangWV commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

VERDICT: FAIL

Reviewed SHA: 625821d (round 3). The PR head did not move during the review.
Lane: audit + regression.

Trunk moved during the review. origin/dev went from f243254 to 5d6ae01, which is #294 (user_logger → warn_user). git merge-tree is clean as text. The merged tree still fails four CI gates, all from the known #294 hazard, which is now live. Against f243254 this round is clean: the resolver changes are correct, and default selections leave every fixture's output unchanged.

Blocking: head merged with current dev (5d6ae01) fails CI

Every failure below comes from two lines this PR adds. Each passes on 5d6ae01 alone.

Gate on head + 5d6ae01 Result
ruff check --select F821,F822,F823,NPY201 . 1 error: pyprocar/io/abinit/parser.py:258 F821, undefined name user_logger
basedpyright --project pyrightconfig.json 1 error: the same line, reportUndefinedVariable
ruff_new_violations.py check origin/dev FAILED ({'F821': 1} more than at base)
pytest -m "not data" -n 8 2 failed, 1983 passed (5d6ae01 alone: 1952 passed)
pytest -m data -n 4 1 failed, 402 passed (5d6ae01 alone: 382 passed)

The two quick-suite failures are #294's enforcement tests:

  • test_library_warns_only_through_warn_user flags pyprocar/io/vasp/parser.py:252. That is the atom-groups warnings.warn(..., stacklevel=2) this PR adds.
  • test_tests_never_lower_the_user_logger_to_see_a_warning flags tests/pyprocar/io/abinit/test_dos.py:180, the caplog.at_level(..., 'user') call.

The data failure is test_magnetic_spinor_dos_keeps_only_the_total_and_says_why, which raises NameError: name 'user_logger' is not defined at abinit/parser.py:258.

User-facing repro. Copy codes/abinit/9.6/Fe/non-colinear/dos and set nspden = 4 in abinit.out. Then get_parser("abinit", d).dos raises NameError on the merged tree. At 625821d alone, the same call logs the warning.

Fix.

  • Call warn_user(...) from pyprocar.utils.log_utils in abinit/parser.py:258 and in vasp/parser.py:252.
  • Assert the Abinit warning with tests.utils.user_warning instead of caplog.at_level('user').
  • Re-run the gates on the PR merged into 5d6ae01.

Round-3 audit (on head + f243254, the trunk named at spawn)

Resolver consumers. I grepped every ProjectionSelectionResolver and selection_resolver( site, and every read of selection.atoms and selection.spins. The sites are in ebs.py (809 and 932), fermisurface.py:312, dos.py (822, 948, 1081 and 1235), projection._selection_metadata, scriptBandsplot.py:232 and scriptUnfold.py:187.

  • No consumer still maps () or None to "all".
  • The DOS paths now receive explicit row indices.
  • No caller in pyprocar/, examples/ or docs/ passes atoms=[], spins=[] or species=[].
  • Nothing outside the resolver's own callers reads the metadata atoms key.
  • n_atoms is read eagerly in selection_resolver, but every caller checks for projections first.
  • Every path that rebuilds a band structure (pad, reduce, unfold, BS2D) deep-copies it, so _atom_groups survives.

Synthetic structure-less DOS (2 rows), at origin/dev, the round-2 head (b1a8d63) and the round-3 head:

Call origin/dev Round-2 head Round-3 head
no atoms, orbitals=d raises Atom indexer is required 0.0, labelled (d) 60.7723, labelled 0-1-(d) (expected 60.7723)
atoms=[] 0.0 0.0 raises names no atoms
spins=[] — — raises names no spins
species=[] 0.0 0.0 raises names no atoms

On a VASP DOS that has a structure, origin/dev returns an empty (6000, 0) array for spins=[]. That matches the PR's claim.

Regression dump of every parser on 55 fixture dirs. The dirs are all of data/codes except the QE non-colinear fermi dir, plus examples/{bands,dos,fermi2d,fermi3d} and their subdirs. The dump records layouts, names, array hashes, resolver output and labels, plus compute_projected_sum, projection_components, magnetization and spin texture with default selections.

  • Round-2 head against round-3 head: 0 label or selection differences. The only differences are mesh projections, which come from the rebase onto fix(ebs): make ibz2fbz unfold magnetic groups, phases, atoms and orbitals correctly #299 (ibz2fbz).
  • origin/dev against head: no value or label changes on any VASP fixture. The remaining differences are the intended ones from rounds 1 and 2, already judged in round 2:
    • QE non-colinear bands are (155,50,4,5,16).
    • The Abinit non-colinear DOS has 4 channels.
    • QE, Abinit and Elk DOS selections now resolve.
    • The bands/auto label changes from Bi-(s) to Bi-(o0).
    • The orbital names of ipr/Bi2Se3 change from x2-y2 to dx2.
  • No parse errors, no atom_groups on any unfiltered fixture, and no side effects in any calc dir.

One-call functions with default selections, 183 matplotlib runs and 21 PyVista runs, origin/dev against head:

  • 126 matplotlib runs give identical figure data: line and collection arrays, ticks, legends and texts.
  • The 49 that differ are all intended:
    • 3 are QE non-colinear bandsplot runs.
    • 1 is the bands/auto overlay_orbitals run, which raises a clear error instead of IndexError.
    • 45 are DOS modes on Abinit, Elk and QE, which origin/dev could not plot.
  • The ipr top dir fails the same way at both commits, because it holds subdirs and not a calculation. That was my input, not a regression.
  • All 21 FermiHandler and BandStructure2DHandler screenshots are pixel-identical. The Abinit FermiHandler and fermi2D runs fail at both commits with 'ElectronicBandStructure' object has no attribute 'pad', a pre-existing issue.

ibz2fbz refusal. It fired on no unfiltered irreducible mesh (QE, Abinit, VASP, examples/fermi*), on an orbital-filtered mesh PROCAR, or on an identity grouping [[0],[1],[2],[3],[4]]. The plots of those files equal the unfiltered ones (293.8508 and 271.2844). It does fire for [[0],[1],[2,3,4]] through fermi2D, FermiHandler and BS2D. In those calls origin/dev raises IndexError: index 3 is out of bounds for axis 3 with size 3.

Data suite, -m data -n 4. Head + f243254: 403 passed, 1 skipped, exit 0. f243254 alone: 382 passed, 1 skipped, exit 0.

Gates on head + f243254. basedpyright: 0 errors. Both ruff_new_violations.py gates: ok. The F821 gate: passes. Quick suite: 1959 passed, 2 skipped. gh pr checks: all 5 pass, run on the pre-#294 merge.

#293 acceptance. All 4 items are done.

  • QE non-colinear bands are (155,50,4,5,16), and the Abinit non-colinear DOS is (9401,4,1,9).
  • The swap check: spin and atom swapped, k and band swapped, 3 spins, and a group swap all raise.
  • Every parser passes names: QE s,pz,px,py…, Elk Y00…, Abinit s,py,pz,px….
  • bands/auto is named ['o0'].

PR body round-3 section. Its line references match the code (projection.py:206, atomic_orbital_index.py:957-963, ebs.py:2278-2288). The baseline edit deletes 2 entries and nothing else, tied to the resolver.

Notes (not blocking)

  1. The QE collinear mesh changes at the merge with fix(ebs): make ibz2fbz unfold magnetic groups, phases, atoms and orbitals correctly #299, and the PR body does not mention it. QE now passes orbital names, so ibz2fbz rotates its p and d shells. At origin/dev they were copied unrotated, because the names were orbital i. Independent check, cubic SrVO3: along each axis (a,0,0), (0,a,0), (0,0,a), the two equivalent t2g orbitals and the two equivalent p orbitals have equal weights per degenerate group. Head passes 9 of 9 points; origin/dev fails 6 of 9 (mismatch up to 0.99). Head is correct. The QE f names (fz3, fxz2, …) are not in ORBITAL_ALIASES, so the f shell stays unrotated, with fix(ebs): make ibz2fbz unfold magnetic groups, phases, atoms and orbitals correctly #299's warning.
  2. A PROCAR filtered to one group on an irreducible mesh now raises, though origin/dev plotted it. The PR body says origin/dev fails with IndexError, which is true only for multi-row files. Examples: [[0,1,2,3,4]] gives 293.8508 at origin/dev, which is exact, and [[2]] gives 2.4527 where the true value is 2.4555. filter.rst covers this ("needs the full k grid"), but the error text "moves atoms between its rows" reads oddly for a 1-row file. State it in the PR body.
  3. orbitals=[] still silently selects every orbital, while atoms=[] and spins=[] now raise. origin/dev behaves the same way. This is a sibling of the empty-selection policy that the PR leaves open.
  4. Undisclosed change on EBS and FermiSurface: on origin/dev, atoms=[] and species=[] summed every atom. They now raise. The PR body describes only the DOS behaviour.
  5. The QE spin-orbit DOS spin texture (compute_spin_texture and compute_spin_texture_magnitude) now fails with a bare IndexError: Index out of bounds. origin/dev failed earlier with Atom indexer is required. The layout is documented as out of scope, but a clear error would help.

…ucture

check_projected_layout compared only the number of axes, so swapped
spin and atom axes passed. It now compares the leading axes with the
bands or total DOS, the atom axis with the structure, and requires 1, 2
or 4 spin channels. Non-collinear projections may go with one band
channel, as the QE parser stores them, or four, as VASP and Abinit do.

Refs #293
…orbital

projwfc.x projects a spinor run onto |l j m_j> (spin-orbit) or
|l m s_z> states. The parser summed the j amplitudes into 32 slots on one
spin channel and could not read s_z states. It now writes each shell in
the real (l, m) harmonics and spin up/down with QE's own spinor, sph_ind
and rot_ylm coefficients, and stores total, Sx, Sy and Sz per orbital.
Both QE band structures and DOS now pass projwfc.x's orbital names.

Refs #293, #219, #220
VASP and Abinit read the names from the PROCAR header instead of
cutting a fixed list, so a PROCAR that pyprocar.filter rewrote keeps its
o0 column name. The Abinit, Elk and Lobster DOS now name their orbital
columns too, so check_projected_layout compares them.

Refs #293
…or transform

Also format and lint the #293 changes.

Refs #293
…nels

Abinit's DOS_AT files hold no magnetization components. With nspden 1 the
spin-resolved DOS is zero by time reversal, so the parser adds three zero
channels. With nspden 4 it keeps the total and warns, since the files
cannot give the spin components.

Refs #293
… to every DOS

- QE spin-orbit DOS names reuse OrbitalIndexer.soc_label, the format the
  label builder already renders.
- Abinit DOS names are a constant: the kept lm columns are a fixed slice.
- Procar.n_orbitals counts the header names instead of rescanning.
- The spinor transform uses a matrix product and adds in place.
- The QE, Abinit and Elk DOS pass their structure, so the atom axis is
  checked against it.

Refs #293
…ader does

An atom-filtered PROCAR holds one row per group of atoms, so it loads, plots
each group, labels it by its row and refuses species selections. An
orbital-filtered PROCAR labels its columns o0, o1 as its header does, and a
single orbital selected by name raises whichever spelling the header uses.

Refs #293
Elk's task 21 and pyprocar.filter(orbital_names=['s','p','d']) write one
column per shell, named by its letter. orbitals=['d'] picked columns 4-8
there (IndexError, or a wrong sum when the columns existed). It now
picks the column named d, and overlay_orbitals draws one curve per such
column.

Refs #293
…ell, filtered IBZ PROCARs refuse to unfold

A DensityOfStates without a structure (every Lobster DOS) gave zeros for
compute_projected_sum and compute_magnetization without atoms, and dosplot
parametric coloured a Lobster DOS 0. Columns py and pz must not form a p
shell. fermi2D on an atom-filtered PROCAR of irreducible k-points failed
with an IndexError once ibz2fbz started permuting atoms (#299).

Refs #293
With no structure the resolver returned atoms=(), which the band and
Fermi surface callers read as every row but DensityOfStates passed on as
no rows, so a Lobster DOS (no structure) summed to zero. The resolver now
takes the source's row count and always returns explicit row indices, and
every caller passes them as given. An empty atom selection raises.

Refs #293
…-points

ibz2fbz (#299) permutes atoms under each symmetry operation, so a PROCAR
whose rows are groups of atoms failed with an IndexError in fermi2D and
every other mesh. The file does not record which atoms a row holds, so no
permutation of its rows is known to be right. It now raises an error that
says to filter a full k-grid calculation or select atoms in the plot.

Refs #293
…euse the test helpers

The atom-group guard ran inside _turn_projections, after ibz2fbz had
already replaced the properties, and refused every symmetry operation. It
now runs first and refuses only operations that move an atom. The fermi2D
test builds its filtered calculation with _filtered_calc, and the DOS tests
import VASP_ORBITALS instead of copying it (simplify review).

Refs #293
The resolver returned spins=(), which band callers read as every spin and
DensityOfStates as none.

Refs #293
…warn_user

#294 removed user_logger from the Abinit parser and made warn_user the only
channel for warnings a user must see, so the merge left a NameError at
io/abinit/parser.py and a raw warnings.warn in io/vasp/parser.py. The
Abinit test now asserts the warning names its own line.
@lllangWV
lllangWV force-pushed the 293-projection-layout branch from 625821d to ea94998 Compare October 4, 2026 21:26
@lllangWV

lllangWV commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

VERDICT: PASS+NOTES

Reviewed SHA: ea94998 (head of #298, based on origin/dev 5d6ae01, so head and head merged with dev are the same tree).
Lanes: both lanes (live + red/green + gates, and audit + regression), scoped to the rebase onto 5d6ae01 and the new commit ea94998. The rest of the PR holds the round-3 verdicts at 625821d.

The patch is 625821d's patch plus this one commit. I compared git diff 5884b801 625821d0 with git diff origin/dev ea949982~1:

  • same file list
  • identical +/- lines (a diff of the two change-line sets is empty)
  • the only differences are hunk offsets and context lines that fix(warnings): show user-facing warnings through warnings.warn #294 changed in dev: the CLAUDE.md warn_user row, warn_user replacing user_logger.warning and UserWarning, stacklevel=2 in context, and the QE parser's warn_user import.

ea94998 itself changes three things:

  • pyprocar/io/abinit/parser.py:258: user_logger.warning → warn_user. warn_user is already imported at :23, and no user_logger is left in the file.
  • pyprocar/io/vasp/parser.py:252: warnings.warn(..., stacklevel=2) → warn_user. The import is added, and warnings is dropped with no other use.
  • tests/pyprocar/io/abinit/test_dos.py:171: the caplog test → user_warning(__file__, match=...). logging is still used for logger at :12.

Every gate the audit found red is now green (verify.sh exec at ea94998; the main checkout's pixi.lock differs from dev's, so CI is the final gate):

  • ruff check --select F821,F822,F823,NPY201 .: all checks passed. At ea94998~1 (79efb00) it finds 1 error, the user_logger at abinit/parser.py:258.
  • basedpyright --project pyrightconfig.json: 0 errors, 0 warnings, exit 0
  • ruff_new_violations.py check origin/dev and format origin/dev: 32 changed files, ok
  • pytest tests -n 8 -m "not data": 1985 passed, 2 skipped, exit 0. This includes all 5 of fix(warnings): show user-facing warnings through warnings.warn #294's tests/test_user_warnings.py checks. At 79efb00, test_library_warns_only_through_warn_user and test_tests_never_lower_the_user_logger_to_see_a_warning fail.
  • pytest tests -n 4 -m data: 403 passed, 1 skipped, exit 0. test_magnetic_spinor_dos_keeps_only_the_total_and_says_why (the nspden=4 test) passes at head and fails at 79efb00.
  • CI on ea94998: format, lint, red-green, test and typecheck all pass. The red-green log ends red/green: ok, with data tests "not judged".

Live, through the public API. I used private copies inside my worktree; nothing was written under data/. In each case the warning was recorded with simplefilter("always").

  • Abinit: get_parser("abinit", <copy of data/codes/abinit/9.6/Fe/non-colinear/dos with nspden 4>).dos.
    • At 79efb00 it raises NameError: name 'user_logger' is not defined.
    • At head it returns a DensityOfStates and emits one UserWarning at probe.py:57, the caller's line: "Abinit's DOS_AT files hold no magnetization components, so the projected DOS of this nspden = 4 spinor run is the total only."
  • VASP: a copy of data/examples/bands/non-spin-polarized, filtered by pyprocar.filter(..., atoms=[[0, 1], [2, 3, 4]]) and loaded through get_parser("vasp", calc).ebs. Both 79efb00 and head name the caller's line (probe.py:68).
  • VASP: the same filtered copy through pyprocar.bandsplot(code="vasp", dirname=calc, mode="plain", show=False).
    • At 79efb00, the old stacklevel=2 names pyprocar/core/ebs.py:129, a library line.
    • At head, warn_user names the user's line (probe.py:72).
    • The delta fixes this in practice, not just in style.

Notes (non-blocking):

  • No test asserts the VASP atom-group warning or the line it names. tests/pyprocar/scripts/test_filtered_procar.py loads filtered PROCARs but never checks the warning, so the bandsplot → ebs.py:129 regression above would go unnoticed. A user_warning(__file__, match="ion rows and the structure") around one existing _ebs(...) or bandsplot call would pin it.
  • Before ea94998, test_library_never_warns_through_the_user_logger passed even with user_logger.warning at abinit/parser.py:258, because the name was unbound in that module after fix(warnings): show user-facing warnings through warnings.warn #294. Only F821 caught it. This is a gap in fix(warnings): show user-facing warnings through warnings.warn #294's checker, not in this PR.

@lllangWV
lllangWV merged commit 60fc0c0 into dev Oct 4, 2026
5 checks passed
@lllangWV
lllangWV deleted the 293-projection-layout branch October 4, 2026 21:44
lllangWV added a commit that referenced this pull request Oct 4, 2026
build_overlay_orbitals_weights and build_overlay_weights called .get on
pyprocar.utils.info.orbital_names, a list, so both raised AttributeError.
They now take the shells from orbital_shells(self.orbital_names), the
definition #298 added, so a PROCAR with only s and p orbitals gives s and p,
and a shell letter the orbitals do not hold raises a ValueError.
lllangWV added a commit that referenced this pull request Oct 4, 2026
Since #298 the QE parser names its orbitals as projwfc.x prints them.
ibz2fbz did not know the three f labels fzx2-zy2, fx3-3xy2 and f3yx2-y3,
so it dropped a QE mesh's phases and left f unrotated. Against QE 7.5
nosym runs, V eg weights were off by 0.30 and La f by 1.0.
ORBITAL_ALIASES now maps the three labels.

QE's real harmonics carry the sign (-1)^|m| (upflib/ylmr2.f90), and the
core's carry none. The parser now writes the phases over the core's
harmonics. Without that the kept phases unfold with wrong signs on px,
py, dxz, dyz, fxz2, fyz2, fx3-3xy2 and f3yx2-y3, and phase products are
off by up to 1.4.

Refs #293
lllangWV added a commit that referenced this pull request Oct 4, 2026
build_overlay_orbitals_weights and build_overlay_weights called .get on
pyprocar.utils.info.orbital_names, a list, so both raised AttributeError.
They now take the shells from orbital_shells(self.orbital_names), the
definition #298 added, so a PROCAR with only s and p orbitals gives s and p,
and a shell letter the orbitals do not hold raises a ValueError.
lllangWV added a commit that referenced this pull request Oct 4, 2026
Since #298 the QE parser names its orbitals as projwfc.x prints them.
ibz2fbz did not know the three f labels fzx2-zy2, fx3-3xy2 and f3yx2-y3,
so it dropped a QE mesh's phases and left f unrotated. Against QE 7.5
nosym runs, V eg weights were off by 0.30 and La f by 1.0.
ORBITAL_ALIASES now maps the three labels.

QE's real harmonics carry the sign (-1)^|m| (upflib/ylmr2.f90), and the
core's carry none. The parser now writes the phases over the core's
harmonics. Without that the kept phases unfold with wrong signs on px,
py, dxz, dyz, fxz2, fyz2, fx3-3xy2 and f3yx2-y3, and phase products are
off by up to 1.4.

Refs #293
lllangWV added a commit that referenced this pull request Oct 4, 2026
…S2D grid, overlay weights, Elk lmirep) (#304)

* test(dos): vertical spin-polarized DOS labels and flips each channel

* fix(dos): split spin channels before orienting a vertical DOS plot

DOSPlotter.plot swapped energies and values before it split the spin
channels, so in vertical orientation (bandsdosplot) every channel got the
first channel's label and the down channel was not mirrored.

* test(autobandsplot): write report.txt only to the path the caller names

* fix(autobandsplot): write the report only to the path given as report=

autobandsplot wrote report.txt into the current working directory on
every call. The report now goes to the path the caller passes as
report=, and no file is written by default. Energies and band indexes
print as plain numbers again, so the text matches the reference report
shipped with the bands/auto fixture.

* test(procarparser): read fused fixed-width k-point coordinates

* fix(procarparser): read fused fixed-width k-point coordinates

The legacy ProcarParser split k-point headers on whitespace, so a header
such as 0.50000000-0.25000000 0.00000000 gave two values and readFile
raised. It now uses the per-coordinate regex that #280 put in
io/vasp/procar.py.

* test(cfg): the scatter plot-mode member is spelled SCATTER

* fix(cfg): rename the SACATTER plot-mode member to SCATTER

UnfoldPlotMode and both BandStructureMode enums spelled the scatter
member SACATTER. Users select the mode by its value "scatter", which
does not change, and no public namespace or doc names the member, so no
alias is kept.

* test(docs): cfg pages name existing files and show the config defaults

* docs(cfg): render the config pages from the config classes

The cfg index listed yml files that do not exist, and the dos, unfold
and band-structure-2D pages were generated from yml files that no code
reads and that had drifted from the config defaults (unfold showed
color black instead of #eeeeee). Those pages now use automodule, like
the band-structure and Fermi-surface-3D pages. The yml files and their
generator are removed, and the config docstrings state the real
defaults.

* test(cli): run the filter and cat subcommands of scripts/procar.py

* fix(cli): pass the filter subcommand's own argument names

procar.py filter read args.infile, args.outfile and args.human_atoms,
which its parser never defines (it defines inFile, outFile and --human),
so every filter call raised AttributeError.

* test(examples): saved band notebook outputs name only current plot options

* docs(examples): clear stale saved outputs in the band notebooks

The unfolding notebook kept plots drawn before #282 fixed pyprocar.unfold,
and the Getting Started and Comparing Bands notebooks kept option dumps
that list the removed weighted_color and weighted_width. Those outputs
are cleared rather than re-run, so no new images enter the repo.

* test(bs2d): a hexagonal k-patch is fully covered and each value stays on its own point

* fix(bs2d): lay the 2D band grid over the k-patch and keep values on their points

With as_cartesian=False the k-mesh slice is a parallelogram in Cartesian
space, and the uv grid spanned its bounding box. For graphene half of the
grid fell outside the data, so those points were NaN and the surface edge
became a sawtooth. The grid is now regular in fractional coordinates and
mapped to Cartesian.

The band surfaces went through cast_to_unstructured_grid().extract_surface(),
which renumbers points, while every property stayed in grid order, so
colours were drawn on other points. The surface is now built from quads
in grid order, which also fixes the dimensions of a non-square grid.

* test(ebs): overlay weights sum the shells the orbital names hold

* fix(ebs): build overlay weights from the shells the orbital names hold

build_overlay_orbitals_weights and build_overlay_weights called .get on
pyprocar.utils.info.orbital_names, a list, so both raised AttributeError.
They now take the shells from orbital_shells(self.orbital_names), the
definition #298 added, so a PROCAR with only s and p orbitals gives s and p,
and a shell letter the orbitals do not hold raises a ValueError.

* docs(examples): drop the overlay-builder bug notes the fix resolved

* test(elk): name task-22 characters by the basis Elk wrote them in

* fix(elk): name task-22 characters by the irreducible-representation basis

Elk 10.7.8 and later write task 22's (l,m) band characters in each site's
irreducible-representation basis unless lmirep is .false., so the Y{l}{m}
names were wrong for those runs. The parser reads the version from
INFO.OUT and lmirep from elk.in, and names those columns Y{l}_ir{i}, the
i-th basis function of l in ELMIREP.OUT order; orbital_shells still finds
each shell. Without INFO.OUT, ELMIREP.OUT from a task-22-only run marks the
basis, and a run that also lists task 10 warns.

* style(bs2d): split the fractional-grid mapping into ruff-formatted lines

* docs(verify-pyprocar): record the #285 fixes in the feature map

* refactor(orbitals): resolve shell names in one place for plots and overlay weights

present_shells and orbital_indices move from scripts/_selection.py to
core/atomic_orbital_index.py beside orbital_shells. The overlay weight
builders call them, so they accept the same shell names as
bandsplot(orbitals=[...]), including a single column named by its letter.
The fixed orbital table in utils/info.py, whose last user was the broken
overlay code, is removed.

* refactor(bs2d): slice the plane once for the fractional grid

* refactor(elk): share the elk.in logical parser and read the version line only

* fix(types): type autobandsplot's fermi as a float and tidy the new tests for basedpyright

* chore(basedpyright): drop baseline entries the #285 rewrites removed

* style: wrap the #285 lines that ruff flags as too long or unformatted

* test(bs2d): keep VTK objects out of failing test frames

* test(bs2d): run the non-square grid build in its own process

* test(elk): keep Ylm names for Elk 6.3 headers and for a task-10 ELMIREP.OUT without a version

* fix(elk): read the Elk 6 version header and keep Ylm names when no version says otherwise

* test(cfg): the old SACATTER member still resolves to SCATTER

* fix(cfg): keep SACATTER as a deprecated alias of SCATTER

* docs(changelog): note the autobandsplot report default and the SCATTER spelling

* test(bs2d): cover the Cartesian grid, a near-square lattice, an off-origin skewed plane and the quads

* test(bs2d): a Cartesian plane across the kz layers covers its patch

* fix(bs2d): build the Cartesian-plane grid in the fractional patch and keep edge points inside the data

* refactor: tighten the #285 round-2 grid and version code and format the new tests
lllangWV added a commit that referenced this pull request Oct 4, 2026
Since #298 the QE parser names its orbitals as projwfc.x prints them.
ibz2fbz did not know the three f labels fzx2-zy2, fx3-3xy2 and f3yx2-y3,
so it dropped a QE mesh's phases and left f unrotated. Against QE 7.5
nosym runs, V eg weights were off by 0.30 and La f by 1.0.
ORBITAL_ALIASES now maps the three labels.

QE's real harmonics carry the sign (-1)^|m| (upflib/ylmr2.f90), and the
core's carry none. The parser now writes the phases over the core's
harmonics. Without that the kept phases unfold with wrong signs on px,
py, dxz, dyz, fxz2, fyz2, fx3-3xy2 and f3yx2-y3, and phase products are
off by up to 1.4.

Refs #293
lllangWV added a commit that referenced this pull request Oct 4, 2026
)

* test(qe): QE phases unfold to the coefficients at each grid point

After #299, a QE mesh dropped its phases and left the f shell unrotated,
because the f labels projwfc.x prints are not the core's names; eg weights
then lost their interference. QE's harmonics also carry the sign (-1)^m.

Refs #293

* fix(qe): unfold QE phases and f orbitals with the core's harmonics

Since #298 the QE parser names its orbitals as projwfc.x prints them.
ibz2fbz did not know the three f labels fzx2-zy2, fx3-3xy2 and f3yx2-y3,
so it dropped a QE mesh's phases and left f unrotated. Against QE 7.5
nosym runs, V eg weights were off by 0.30 and La f by 1.0.
ORBITAL_ALIASES now maps the three labels.

QE's real harmonics carry the sign (-1)^|m| (upflib/ylmr2.f90), and the
core's carry none. The parser now writes the phases over the core's
harmonics. Without that the kept phases unfold with wrong signs on px,
py, dxz, dyz, fxz2, fyz2, fx3-3xy2 and f3yx2-y3, and phase products are
off by up to 1.4.

Refs #293

* fix(qe): keep the Bloch factor when the parser has no projwfc.x orbitals

The harmonic sign flip needs projwfc.x's orbital list. When spd_phase is
set without it, position_gauge_phase returned the phases with no
exp(-2 pi i k.tau) factor, so test_ebs_phases_use_bloch_sums_over_atom_positions
failed. Now the parser skips only the flip, and the Bloch factor always
applies. The signs and the Bloch factor are combined first and applied in
one multiply, so the parser no longer makes a second full-size copy of
the phase array.

Refs #293

* test(fermi): the Fermi-surface mesh pads projections but not phases

Refs #293

* fix(ebs): drop projected_phase when padding a mesh

QE meshes keep their phases since the sign fix, and the Fermi-surface
path padded them: 9.2 GiB peak on the spin-polarized SrVO3 fermi
fixture. Nothing reads phases on a padded mesh, and copies at k + G
would be wrong because position-gauge phases are not periodic in k.

Refs #293

* test(qe): unfolded QE bands are eigenvectors of their little group

The data test checked only warnings and that phases were kept, so it
passed with the sign flip reverted. Each nondegenerate band with V
weight must be an eigenvector of the orbital rotation of every operation
that fixes its k-point: max residual 3.6e-4 and 4.2e-4 over 52466 and
104884 checks by non-identity operations, against 1.0 without the flip.

Refs #293

* docs(verify-pyprocar): state the band windows and image energies of the QE reference

The 2.9e-4 LaVO3 figure holds over bands 1 to 31 of 34, and the 1e-9 eV
energy agreement only at the IBZ k-points. Note the pad memory figure.

Refs #293

* test: reuse the sphere mesh helpers and index each little group once

Refs #293

* test(qe): narrow the optional mesh fields for basedpyright

Refs #293

* docs(verify-pyprocar): state the band windows of the IBZ energy figures
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant