Repository navigation
fix: resolve the #285 follow-ups (DOS labels, autobandsplot report, BS2D grid, overlay weights, Elk lmirep) - #304
Conversation
e048e43 to
763713f
Compare
DOSPlotter.plot swapped energies and values before it split the spin channels, so in vertical orientation (bandsdosplot) every channel got the first channel's label and the down channel was not mirrored.
autobandsplot wrote report.txt into the current working directory on every call. The report now goes to the path the caller passes as report=, and no file is written by default. Energies and band indexes print as plain numbers again, so the text matches the reference report shipped with the bands/auto fixture.
The legacy ProcarParser split k-point headers on whitespace, so a header such as 0.50000000-0.25000000 0.00000000 gave two values and readFile raised. It now uses the per-coordinate regex that #280 put in io/vasp/procar.py.
UnfoldPlotMode and both BandStructureMode enums spelled the scatter member SACATTER. Users select the mode by its value "scatter", which does not change, and no public namespace or doc names the member, so no alias is kept.
The cfg index listed yml files that do not exist, and the dos, unfold and band-structure-2D pages were generated from yml files that no code reads and that had drifted from the config defaults (unfold showed color black instead of #eeeeee). Those pages now use automodule, like the band-structure and Fermi-surface-3D pages. The yml files and their generator are removed, and the config docstrings state the real defaults.
procar.py filter read args.infile, args.outfile and args.human_atoms, which its parser never defines (it defines inFile, outFile and --human), so every filter call raised AttributeError.
The unfolding notebook kept plots drawn before #282 fixed pyprocar.unfold, and the Getting Started and Comparing Bands notebooks kept option dumps that list the removed weighted_color and weighted_width. Those outputs are cleared rather than re-run, so no new images enter the repo.
… on its own point
…heir points With as_cartesian=False the k-mesh slice is a parallelogram in Cartesian space, and the uv grid spanned its bounding box. For graphene half of the grid fell outside the data, so those points were NaN and the surface edge became a sawtooth. The grid is now regular in fractional coordinates and mapped to Cartesian. The band surfaces went through cast_to_unstructured_grid().extract_surface(), which renumbers points, while every property stayed in grid order, so colours were drawn on other points. The surface is now built from quads in grid order, which also fixes the dimensions of a non-square grid.
build_overlay_orbitals_weights and build_overlay_weights called .get on pyprocar.utils.info.orbital_names, a list, so both raised AttributeError. They now take the shells from orbital_shells(self.orbital_names), the definition #298 added, so a PROCAR with only s and p orbitals gives s and p, and a shell letter the orbitals do not hold raises a ValueError.
…asis
Elk 10.7.8 and later write task 22's (l,m) band characters in each site's
irreducible-representation basis unless lmirep is .false., so the Y{l}{m}
names were wrong for those runs. The parser reads the version from
INFO.OUT and lmirep from elk.in, and names those columns Y{l}_ir{i}, the
i-th basis function of l in ELMIREP.OUT order; orbital_shells still finds
each shell. Without INFO.OUT, ELMIREP.OUT from a task-22-only run marks the
basis, and a run that also lists task 10 warns.
…erlay weights present_shells and orbital_indices move from scripts/_selection.py to core/atomic_orbital_index.py beside orbital_shells. The overlay weight builders call them, so they accept the same shell names as bandsplot(orbitals=[...]), including a single column named by its letter. The fixed orbital table in utils/info.py, whose last user was the broken overlay code, is removed.
…sts for basedpyright
a145c3a to
1bc6c79
Compare
|
VERDICT: FAIL Reviewed SHA: 2fd6c01 (base origin/dev 4de5cc0, which is the merge base, so no trunk merge was needed). Lane: audit and regression. Independent verifier. Blocking findingThe Elk version check misses the
What I ran
Audit
Notes (non-blocking)
|
|
VERDICT: PASS+NOTES Reviewed SHA: 2fd6c01. The head contains origin/dev 4de5cc0, so no merge was needed. Gates (run at the head,
|
| config | NaN points, dev | NaN points, head |
|---|---|---|
| frac 20x20 | 440/800 | 0/800 |
| frac 90x90 | 8280/16200 | 0/16200 |
- Head grid steps in fractional coordinates are [1/60, 0] and [0, 1/60]: aligned with b1 and b2, with a range of [-44/60, 45/60], exactly the padded patch. The dev grid steps are diagonal, with a range of +-1.48.
- Head at 90x90: all 2x8100 surface points sit on DFT k-points. |z - EIGENVAL| <= 5.2e-7 on bands 4 and 5.
- Surface colours (
get_property("projected")) at those points equal my PROCAR sums over both ions to <= 2e-14, for s and for pz.get_property("bands")equalspoints[:,2]exactly. - Dirac gap at K = (1/3, 1/3): the surface gives 2.511e-5 eV, EIGENVAL gives 2.50e-5 eV. dev gives 0.267 eV at 90x90 and 1.22 eV at 20x20.
as_cartesian=Trueis unchanged from dev: 440/800 and 8280/16200 NaN. Finite points agree with my bilinear reference to the interpolation error (p99 0.07 eV).- Skewed-lattice check, which reaches beyond the PR's tests: a synthetic 16x16x8 mesh with c* having x and y components, an analytic band, planes kz = 0 and 0.25, and grids 40x30 and 30x30. Head: 0 NaN, error within the linear-interpolation bound (max 0.055), face count (nu-1)(nv-1), and every quad on grid neighbours. dev on the 40x30 grid: 1521 faces instead of 1131, garbage indices, then a segfault (exit 139).
- Rendered legacy handler at its default 120x120 grid, at both commits: both look smooth. The 20x20 driver image is still coarse because of the sampling.
scripts/procar.py. All subcommands ran for real inside bwrap, with only a fake root writable. Input modes and sizes were unchanged afterwards.
- dev: all 4
filtervariants raiseAttributeError: 'Namespace' object has no attribute 'infile'. - head
filter -a 0 1 -a 2 3 4and the--humanvariant: every block equals my sums of the input ions, with tot kept. - head
-o 0 -o 1 2 3 -o 4..8 --orbital_names s p d: exact. - head
-b 10 20: 2200 blocks, bands 10-20. catandcat --gz: 400 k-points, both halves identical to the input.fermi2Dplain and parametric: exit 0, and the PNGs look correct.bandsplot,repair,kpathandunfold: exit 0.
Overlay weights on SrVO3. Reference: my PROCAR parse, which equals ebs.projected exactly.
- dev: both functions raise
AttributeError: 'list' object has no attribute 'get'. - head
build_overlay_orbitals_weights()s/p/d: equal tocompute_projected_sum(diff 0) and to my sums (<= 2e-16). - head
build_overlay_weights({"V":["d"],"O":["p"],"Sr":[0]}): equal to the per-species sums. {"V":["f"]}and{"V":["dxy"]}raise a clearValueError.- The bands.md numbers (O s 473.483, p 1153.606, d 0.0) reproduce.
Elk lmirep. I ran the code itself in place of a synthetic directory: Elk 11.2.3, Si with Td sites, tasks 0 and 22. One run used default lmirep, a second used lmirep=.false..
- Task 22 writes ELMIREP.OUT only in the irrep run, in the documented format.
- Head names the irrep run
Y{l}_ir{i}and the Ylm runY{l}{m}. dev names bothY{l}{m}. - Per-column characters differ between the two runs by up to 0.148, while per-l sums agree to <= 3e-6, the 6-decimal print. So shell sums are right and the Ylm names were wrong.
- At Gamma, the valence triplet's d weight sits in
Y2_ir1..3(ELMIREP 288 x3, t2), withir4..5(528 x2, e) at 0, which confirms the ELMIREP order. - Without INFO.OUT the run still counts as irrep. With task 10 added it warns and keeps Ylm.
- Head overlay shells s/p/d/f agree between the two runs to <= 6e-6. On dev, overlay raises.
- The parser writes nothing into the calc dir.
SCATTER. bandsplot(mode="scatter") and unfold(mode="scatter") run at both commits. No SACATTER is left in the repo.
cfg docs and notebooks. No references to the removed yml files remain in docs, the code or make.bat. The index lists the config modules. No notebook output names an option missing from its config: I scanned every examples/**/*.ipynb against BandStructureConfig and DensityOfStatesConfig. The Sphinx build was not run here either.
Mutation tests (separate worktree, one mutant at a time, the PR's tests plus the neighbouring test modules)
-
Killed (16 of 19):
- BS2D: grid reverted to the bounding box; surface reverted to
extract_surface. - DOS: orient before the split; line drawn un-oriented.
- Overlay: hard-coded shell table; empty result; resolver keeps only the first index; species ignored.
- Elk: irrep off;
>=changed to>;lmirepignored; no warning. - Other fixes: PROCAR regex reverted; report always written to the cwd;
--humandropped.
- BS2D: grid reverted to the bounding box; surface reverted to
-
Survived (3), all test gaps, none blocking:
bandstructure2D.py:162: dropping the normaloffset(frac = 0*offset + ...). No test uses an off-origin plane on a skewed lattice. On my skewed mesh at kz=0.25, this mutant gives 98/1200 NaN where the head gives 0.bandstructure2D.py:204: quads indexed in the wrong order (reshape(shape[::-1]).T), which connects non-neighbours on non-square grids.bandstructure2D.py:209:pv.PolyData(points)with no faces at all.
No BS2D test checks the surface cells, only points and values. My live adjacency check shows the head's faces are right.
Findings
None blocking. Every acceptance item checked is done: DOS labels, autobandsplot report, legacy k-point regex, graphene BS2D, procar.py fermi2D/filter/cat, SACATTER, cfg docs, notebook outputs, overlay weights, and Elk lmirep naming.
Notes (non-blocking)
- Test gap: the 3 surviving BS2D mutants above. A face-adjacency assertion and an off-origin skewed-lattice case in
test_bandstructure2d_grid.pywould kill them. - Unchanged path:
BandStructure2D.from_ebsdefaults toas_cartesian=True. That path is unchanged and still leaves 51-55% of graphene grid points NaN.from_codeand the legacy handler useas_cartesian=Falseand are fixed. The PR scopes the fix to that path. - Pre-existing:
procar.py generate2dkmeshraisesTypeError: 'str' object cannot be interpreted as an integerat both commits, because its subparser has notype=. - Pre-existing: VASP
autobandsplot()with nofermi, as the fixture's ownplot.pycalls it, usesfermi=0. Its report then differs from the shippedreport.txt, with a different window and no defect states, at both commits. - Pre-existing: for VASP,
dosplotandbandsdosplotdo not shift energies whenfermiis None.bandsdosplotstill draws its Fermi line at y=0, which is 4.96 eV below E_F ondos/spin-polarized. - Left open: Elk DOS (task 10) columns are in the irrep basis whenever
lmirepis on, the Elk default in every version, but keepY{l}{m}names. The PR documents this in parsers.md. It is outside the frozen task-22 item. - Wording: the PR body's "k41" is 0-based. It is k-point 42 in the file.
- Environment: Elk 11.2.3 (local build) aborts with "double free or corruption" after finishing task 22 with
lmirep=.true.. The outputs are complete, and thelmirep=.false.run exits cleanly.
…EP.OUT without a version
…rsion says otherwise
…rigin skewed plane and the quads
… keep edge points inside the data
|
VERDICT: PASS+NOTES Reviewed SHA: ced180c. The head contains origin/dev 4de5cc0, so no merge was needed. Gates (run at the head,
|
| path | graphene NaN, dev → head | BiSb NaN, dev → head |
|---|---|---|
| from_code frac 20x20 | 440/800 → 0 | 1056/2400 → 0 |
| from_code frac 90x90 | 8280/16200 → 0 | 17436/48600 → 0 |
| from_code cart 20x20 | 440/800 → 0 | 1056/2400 → 0 |
| from_code cart 90x90 | 8280/16200 → 0 | 17436/48600 → 0 |
legacy handler, default 120x120 (graphene parametric, BiSb spin_texture) |
14640/28800 → 0 | 30456/86400 → 0 |
| legacy handler 90x90 | 8280/16200 → 0 | 17436/48600 → 0 |
- Heights. At 90x90 every vertex is a DFT k-point, in both modes and in the handler. Graphene: 2x8100 points. BiSb: 6x8100 points, 6 bands.
- |z - EIGENVAL| <= 5e-7 for graphene and <= 1e-6 for BiSb.
- The handler surfaces are E - E_F. They match once E_F is added back.
- Off-mesh grids match my bilinear reference to the interpolation error (p99 <= 0.033 eV).
- Surface colours and spin texture at those vertices. The colours are the
projectedproperty: the total over ions and orbitals, and pz.- Graphene colours: <= 1e-7.
- BiSb colours: <= 2e-7.
- BiSb spin texture (
projected_sum_spin_texturein the handler'sspin_texturemode, against Sx, Sy, Sz from the ion rows): <= 1.9e-6, with |S| up to 0.475.
- Band values on their own points.
get_property("bands")equalspoints[:,2]exactly at the head. On dev BiSb they differ by up to 0.68 eV, which is the old point renumbering. - Dirac gap at K = (1/3, 1/3).
- 90x90, frac, cart and handler alike: 2.511e-5 eV, against 2.50e-5 eV in EIGENVAL. dev gives 0.267 eV.
- Handler default 120x120: 0.072 eV, because K is not a grid point.
- Oblique planes. Synthetic 24x24x16 meshes, one skewed lattice and one with c* tilted, an analytic band, 30x30 and 40x30 grids. The linear-interpolation bound is 0.055.
-
Exact-fit cases (all head; dev values in brackets):
plane head NaN head max error dev NaN frac, n=(0,0,1), kz=0.25 0 0.000 (grid on mesh) 254/900 frac, n=(1,0,0) 0 0.029 94/900 frac, n=(1,1,0) 0 0.044 130/900 cart, n=z, tilted lattice 0 0.028 318/900 -
Planes whose cut through the padded box is not a parallelogram leave NaN at the head too, from the geometry: frac n=(1,1,1) 435/900 (dev 530), cart n=(1,0,0) on the tilted lattice 372/900 (dev 488), and cart n=(0.3,0.2,1) 435/900 (dev 497).
-
Every finite point is within the interpolation bound. Every quad joins grid neighbours. dev segfaults on the 40x30 grid.
-
Elk lmirep
Inputs: my own Elk 11.2.3 runs of Si (task 22, once with lmirep on and once off), and the real Elk 6.3 fixture (data/codes/elk/6.3/SrVO3). The 6.3 bands directory was given the INFO.OUT that reads "Elk version 6.3.02 started" and the task-10 ELMIREP.OUT from the dos run.
| case | names | warning |
|---|---|---|
11.2.3, lmirep on |
Y{l}_ir{i} |
none |
11.2.3, lmirep=.false. |
Y{l}{m} |
none |
| 11.2.3 irrep run, INFO.OUT removed | Y{l}{m} |
yes |
| 11.2.3 irrep run, version line edited out of INFO.OUT | Y{l}{m} |
yes |
| 11.2.3 irrep run, INFO.OUT edited to 10.7.7 | Y{l}{m} |
none |
| 11.2.3 irrep run, INFO.OUT edited to 10.7.8 | Y{l}_ir{i} |
none |
| Elk 6.3 with INFO.OUT and stale ELMIREP.OUT | Y{l}{m} |
none |
| Elk 6.3 with stale ELMIREP.OUT, no INFO.OUT | Y{l}{m} |
yes |
| Elk 6.3 plain | Y{l}{m} |
none |
- Per-l sums agree between the irrep and Ylm runs to <= 3e-6. Individual columns differ by up to 0.169.
- The parser writes nothing into any calc directory.
SACATTER alias
In all three enums, SACATTER is SCATTER, E["SACATTER"] resolves, and E("scatter") resolves. Iteration lists SCATTER once and never lists the alias. from_str("scatter") works.
Mutation tests (separate worktree, one mutant at a time)
-
Killed (16 of 17):
- grid: plane level set to 0; quad order on non-square grids; no faces; Cartesian normal left untransformed; Cartesian normal transposed; inset removed; inset pointing outward; old bounding-box grid in Cartesian mode; level sign flipped.
- Elk: old regex; version-less directory guessing irrep; warning removed; threshold moved to 10.7.9.
- SACATTER alias removed from each of the 3 enums.
This includes my three round-1 survivors.
-
Survived (1):
across = 2in place of the argmax atbandstructure2D.py:159. No test uses a plane whose normal is not along kz. On my oblique meshes this mutant leaves 900/900 points NaN for frac n=(1,0,0) and n=(1,1,0), where the head leaves 0.
Findings
None blocking. Changes A to E behave as described: Elk version handling, the deprecated alias, CHANGELOG, tests for the three round-1 mutants, and one fractional grid in both modes.
Notes (non-blocking)
- Test gap: the
acrosssurvivor above. A vertical-plane case, for example normal (1,0,0) on the skewed lattice, would kill it. - Expected NaN on oblique planes: planes whose cut through the padded mesh is not a parallelogram still leave about half the grid NaN, because the grid is a rectangle in the two in-plane fractional coordinates. This is fewer than on dev in every case I ran.
- Environment: the local Elk 11.2.3 build again aborts with "double free or corruption" after finishing task 22 with
lmirep=.true.. The outputs are complete. - Pre-existing: the round-1 notes still apply. They are the
generate2dkmeshCLITypeError, VASPautobandsplot()withoutfermi, thebandsdosplotFermi line whenfermiis None, and the Elk task-10 DOS irrep names.
|
VERDICT: PASS+NOTES Reviewed SHA: ced180c. Base origin/dev 4de5cc0 is the merge base, so no trunk merge was needed ( Gates (this head,
|
| case | graphene | BiSb |
|---|---|---|
| frac z 30x30 | 960 / 0 / 0 | 2196 / 6 / 0 |
| cart z 30x30 | 960 / 960 / 0 | 2196 / 2196 / 0 |
| frac z 40x25 | segfault / 0 / 0 | segfault / 6 / 0 |
| cart z 40x25 | segfault / 1056 / 0 | segfault / 2346 / 0 |
| cart z 20x20, padding 15 | 440 / 440 / 0 | 1056 / 1056 / 0 |
| frac and cart, x and y normals | 0 / 0 / 0 | 672-684 / 0-684 / 0 |
| cart (1,1,0) | 0 / 0 / 0 | 684 / 684 / 0 |
| frac (1,1,0) | 0 / 0 / 0 | 690 / 12 / 12 |
- Values on their own points.
get_property('bands')equals z on every head point, in every case. On origin/dev the number of points where they differ ranges from 16 to 594. - Independent check at DFT nodes. On an 80x80 grid with padding 10, every vertex is a 60x60 DFT k-point. Head's grid sits on those nodes to within 8e-11 (graphene) and 2.4e-6 (BiSb, whose lattice has components of order 1e-7). I compared against the parsed EBS, looked up by fractional k mod 1:
- graphene: max |z - DFT| = 4.2e-11 in both modes, and projected_sum differs by at most 5.7e-11
- BiSb: max |z - DFT| = 4.6e-7 in both modes, and the spin texture differs by at most 1.4e-6
- Cases where origin/dev was already NaN-free (graphene with x, y and (1,1,0) normals): after sorting, head's point set equals origin/dev's. Only the order changes, plus the round-1 fix that keeps each value on its own point.
- Fractional mode, head against 2fd6c01: points move by at most 6e-11. This is the inset.
- Save/load gives back identical points in every case.
- Renders: the BS2DPlotter renders and the handler's
parametricandspin_texturemodes at 60x40 and 50x50 produce correct images at head. At origin/dev the handler segfaults (exit 139) on the 60x40 grid. - Inset mutant: with
inset = 0,test_bs2d_on_a_lattice_slightly_off_square_leaves_no_grid_point_empty[cartesian]fails. The inset is covered by a test. - Callers.
compute_plane_infois called only byfrom_ebs.from_codeandBandStructure2DHandler(which passesas_cartesian=False) go throughfrom_ebs.PlaneInfo.u_limitsandv_limits(the Cartesian bounding box) are read only bysaveandload.u_grid,v_gridanduv_grid_pointsare not read outsidebandstructure2D.py. No code consumes them wrongly.
Elk regression
I ran all 4 Elk fixtures (SrVO3 6.3, bands and dos, non-spin-polarized and collinear spin-polarized) and 6 variants. The variants add the task-10 ELMIREP.OUT to the bands directory: with INFO.OUT, without it, and with the Elk 6.3 INFO.OUT copied in.
- For each case I compared orbital names and SHA-1 digests of the bands, projected, total and DOS arrays between origin/dev and head. They are identical in all 10 cases: always
Y00, Y1-1, ..., 16 names. - Head warns only when
ELMIREP.OUTis present and no version is found ("there is no INFO.OUT"). The round-1 blocking repro now gives the Ylm names with no warning. - The local Elk 11.2.3 binary contains the format
("Elk code version ",I0,".",I0,".",I0," started"). Elk 6.3 writesElk version 6.3.02. The new regex matches both.
Enum aliases, CHANGELOG and guards
- Enum aliases, in all three enums:
- Iteration order and
lenare unchanged. Iteration listsSCATTERin the old slot and does not listSACATTER. __members__containsSACATTERas an alias.E("scatter"),E["SACATTER"]andE(E.SACATTER)all returnSCATTER.to_list()and the string values match origin/dev.from_str("SACATTER")raises at both commits.- The cfg
BandStructureModehas no consumer.
- Iteration order and
- CHANGELOG: the format matches the existing sections (
# ...,##### ...,___). - guards_existing_behaviour reasons: all 3 are accurate.
- Ylm-basis Elk: origin/dev never renames.
- The Elk 6.3 data test: it guards against the round-1 regression.
cat: it already worked.
Claims
- Elk 6.3 header fix: confirmed.
- Version-less directory reads as on dev: confirmed (all fixtures and variants are identical).
- SCATTER alias and iteration: confirmed.
- CHANGELOG entries: confirmed.
- The BS2D NaN table in the PR body: confirmed. Every row matches my numbers.
- "Values on their own points" in both modes: confirmed, at DFT nodes against an independent lookup.
- Gate numbers: confirmed.
- Sphinx build: not checked, as the PR body also says.
Notes (none blocking)
- The 1e-12 inset does not always beat the rounding in the data. Smallest repro: BiSb EBS,
from_ebs(normal=(1,1,0), grid_interpolation=(30,30), padding=10, as_cartesian=False).- Result: 12 NaN z points, which are grid points (28,0) and (29,0) on each of the 6 sheets. Those points also get NaN x and y. The result is the same at 2fd6c01, and origin/dev has 690.
- Cause: the fractional slice points carry float32 rounding (for example
-0.55000001and-0.55000002, near-duplicates), so the edge of the hull is jagged by about 1e-8, against an inset of 1e-12 of the span. - The code comment at
core/bandstructure2D.py:154-155(the phrase is on line 155) ("by far more than rounding") andfeatures/bs2d.md("no BiSb grid corner is lost to rounding") overstate the fix. This is not a regression.
- The docstrings of
from_ebs,from_codeand the handler still say "Number of grid points in (u, v) directions". At head,grid_interpolation[0]follows the first in-plane fractional axis(across+1) % 3. Fornormal=(0,1,0)that axis is f2, which is the v direction. Axis 1 runs along -u. At origin/dev, axis 0 was u. The surface itself is correct. - The
# Unreleasedsection is hand-written. Bothpixi run bump-changelog(git-cliff --prepend) andworkflow_generate_changlog_on_recent_tag.pyprepend their output above it. Whoever cuts the next release has to fold it into the release section by hand. - Pre-existing, outside scope:
plotter/bs_2d_plot.py:31get_uv_bands_gridstill builds a Cartesian bounding-box grid, but it has no callers. A NaN z point still gets NaN x and y from the 2π scale transform (0*NaN).
No .pixi/ directory appeared in my worktrees. I removed the worktrees after the run.
…able_copy The check reads tests/ and flags shutil.copy, copy2, copyfile, copytree, copymode, copystat, os.link, Path.hardlink_to, and cp or rsync through subprocess or os.system, when an argument holds a path built from DATA_DIR or the repo root joined with data. It follows names, attributes, defaults, call-site arguments, return values, pytest fixtures and parametrize values, across modules. It flags #304's Elk test and the auto bandsplot fixture. The module-name helpers move from test_user_warnings into tests/utils so both checks share them.
…ixtures stay usable #304's Elk test copied a 0555 bands dir with copytree, then copied ELMIREP.OUT into it, which raised PermissionError on locked fixtures. writable_copy now also copies a single file.
…y.sh inside data/ (#303) * test: copy fixtures with writable_copy so read-only fixtures work copytree keeps a read-only fixture's modes, so a test that writes into its copy fails with PermissionError once fixtures are locked. writable_copy dereferences symlinks and adds user write to the copy only. * test(verify-pyprocar): verify.sh locks only fixtures and unlocks only its own copies Run a copy of the harness in scratch repos (data in the repo, data linked, and a linked worktree) with a stub env. Cover the round-2 escapes: a symlinked fixture given to run, a symlinked work/ given to clean and gc, glob characters given to fetch, and a fixture root that is a symlink. * feat(verify-pyprocar): make fetched fixtures read-only through one path resolver fixture_path turns an argument into the real path of a fixture inside data/ or refuses it: plain names only, so no glob reaches the Hugging Face pattern, and the resolved path must be a fixture path. fetch, run and doctor act only on that path. run copies with cp -RL, so its copy holds no symlink. Every chmod runs through find -P ! -type l, which never follows a symlink. fetch needs the env only to download. * docs(verify-pyprocar): document read-only fixtures and the verify.sh path rule * refactor(verify-pyprocar): name the path helpers for what they check and drop their comments * test: state the writable_copy guard reason as one string * test(verify-pyprocar): verify.sh changes nothing outside data/ under hostile data roots Unenterable data, data at the top of the tree, data linked to a copy, a leading // spelling, a verify-runs link out of data/, hard links to outside files, and unlockable roots in doctor. A sweep runs every command against a sacrificial tree and compares it before and after. download_from_hf fetches only the archive named exactly. Refs #288 * fix(verify-pyprocar): act only strictly below the main checkout's data/ real_path fails on any failed cd and drops a leading //. Every command that touches data/ first requires data to resolve to <main>/data and verify-runs to be a directory in it, compares paths by component, and chmods only through chmod_below_data, which skips symlinks and files with a second hard link. doctor lists roots fetch cannot lock. download_from_hf fetches exactly <relpath>.zip and no longer deletes <output>/.cache; fetch points its HF cache into data/verify-runs. Refs #288 * docs(verify-pyprocar): state the data-root rule and name copies to unlock Refs #288 * test: assert the prefix download first and drop guard marks on tests dev fails Refs #288 * test: download_from_hf leaves the shared Hugging Face cache alone Refs #288 * fix(download): fetch the one archive into a staging dir, not the shared HF cache download_from_hf deleted the dataset's folder from the active Hugging Face cache after every download. snapshot_download with local_dir set to a staging dir beside the target touches no shared cache, and the archive path is spelled once. Refs #288 * refactor(verify-pyprocar): guard deletes like chmods and share the doctor list printer scrub requires a resolved run dir below verify-runs before rm -rf, the lint now covers rm and mv too, fetch no longer redirects HF_HUB_CACHE, and the tests share one env-and-driver setup. Refs #288 * test: take the hub arguments the fake ignores as keywords Refs #288 * test(verify-pyprocar): run refuses a run dir that already exists; doctor exits 2 on a wrong data root Plants a symlink, a real dir, and real dirs with symlinked work, .pid or evidence at the next run names, and snapshots the whole fake tree. The sweep test now gives the fake worktree a pyprocar/ dir, so worktree-setup reaches its mkdir and ln lines. * fix(verify-pyprocar): run writes only into a run dir it has just created run named its dir <timestamp>-<name> and wrote into it with mkdir -p, a .pid redirect and cp -RL without checking it was new, so an entry planted at that name (a symlink, or a run dir whose work, .pid or evidence links out of data/) made run overwrite and then delete files outside data/ before chmod_below_data refused. A plain mkdir now creates the dir and fails on any existing entry, the dir must resolve strictly below verify-runs, its subdirs use plain mkdir, and noclobber guards every redirect. This also stops two same-second runs with one name from sharing a dir. doctor exits 2, like the other data commands, when the data root is wrong. The lint now lists every line of verify.sh that writes, links, moves or deletes, not only chmod and rm. * docs(verify-pyprocar): name the Hugging Face ref a download writes, the run-dir rule and how to refresh a fixture * refactor(verify-pyprocar): scope noclobber to the .pid write and plant one stamped run in the test A date stub fixes the run name, so the planted-run test plants one entry instead of guessing the next 31 seconds, and the post-driver writes into evidence keep their old behaviour. * test(verify-pyprocar): decode verify.sh output as UTF-8 and keep census lines under 100 columns CI runs in an ASCII locale, where mkdir's quoted error message failed to decode. Two long verify.sh lines are split so the census lint pins them without implicit string concatenation. * test(filter): copy the fixture writable in test_filtered_procar so a locked PROCAR can be edited * test(verify-pyprocar): gc keeps the work of a run whose pid is alive * test(download): download_from_hf stages beside the fixture, not in the system tmp * test(verify-pyprocar): fetch downloads with hf_xet log cleanup off, through pixi run --locked in the main checkout * fix(verify-pyprocar): keep hf_xet's log cleanup off in fetch, run pixi --locked, and stop doctor's git status rewriting the index * test(verify-pyprocar): census also flags fd and & redirects, in-place edits, install, tar, unzip and a command held in a variable * docs(verify-pyprocar): list every write outside data/ and unlock a fixture by its directories only * docs(verify-pyprocar): name the staging dir hf_xet creates beside its chunk cache * docs: only worktree-setup refreshes a git index * test(verify-pyprocar): assert gc's kept and removed work, not its message, and mark the census self-test as guarding * test: fail when a test copies a fixture path without tests.utils.writable_copy The check reads tests/ and flags shutil.copy, copy2, copyfile, copytree, copymode, copystat, os.link, Path.hardlink_to, and cp or rsync through subprocess or os.system, when an argument holds a path built from DATA_DIR or the repo root joined with data. It follows names, attributes, defaults, call-site arguments, return values, pytest fixtures and parametrize values, across modules. It flags #304's Elk test and the auto bandsplot fixture. The module-name helpers move from test_user_warnings into tests/utils so both checks share them. * test(elk,autobandsplot): copy fixtures with writable_copy so locked fixtures stay usable #304's Elk test copied a 0555 bands dir with copytree, then copied ELMIREP.OUT into it, which raised PermissionError on locked fixtures. writable_copy now also copies a single file. * test: make the fixture-copy self-test reach writable_copy with a fixture path, so its exemption is exercised * fix(verify-pyprocar): start main-checkout python with PYTHONDONTWRITEBYTECODE=1 py ran pixi run without it, so doctor, a download and a run driver rewrote stale __pycache__ files in the main checkout (measured in an overlay sandbox). The env stub now logs the variable, and the harness clears it so the caller's env cannot satisfy the test. * test(verify-pyprocar): doctor leaves a stat-dirty git index alone Dropping GIT_OPTIONAL_LOCKS=0 from doctor's git status survived the suite. The test makes the index stale, runs doctor, and checks the index bytes; a plain git status afterwards must rewrite it, so the setup is proven stale. * test(verify-pyprocar): census flags "${cmd[@]}", "${@}" and if/while "$w", and ignores [[ ]] and arithmetic comparisons A > inside [[ ]], (( )) or $(( )) is a comparison, not a redirect. * docs: list matplotlib's config and font cache among verify.sh's writes, and add the fixture-copy rule The rule table names tests/test_fixture_copies.py as the enforcer of copying fixtures only through tests.utils.writable_copy. * refactor(tests): inline one-caller helpers in the fixture-copy check, and map call arguments to positional parameters only * fix(tests): type-narrow the fixture-copy scope, keep the Elk calc dir untyped for from_code, and wrap two long self-test lines * refactor(verify-pyprocar): export PYTHONDONTWRITEBYTECODE once for every python verify.sh starts * refactor(tests): simplify the fixture-copy check (derived shell set, hardlink first, fixture registration in the bind loop) and document writable_copy's file form * style(tests): one condition for fixture registration, and keep the exports line under 100 columns
Why
Issue #285 collects small correctness and cleanup follow-ups. This PR fixes every item that is still open. Each fix has a test that fails on
devand passes here.What changed
bandsdosplotdraws, labelled both spin channels "Total - up" and did not mirror the down channel.DOSPlotter.plotswapped energies and values before it split the channels (plotter/dos_plot.py:154). It now splits the channels first and then orients each one.autobandsplotwrotereport.txtinto the current directory on every call (scriptAutoBandsplot.py:87). It now writes the report only to the path you pass asreport=, and writes no file by default. A default inside the calculation directory was rejected, because fix(ebs): stop parsers writing caches into calculation directories #245 stopped pyprocar writing into input directories. The report also prints plain numbers again, so it matches thereport.txtshipped with thebands/autofixture.io/procarparser.py:153split k-point headers on whitespace, so a header such as0.50000000-0.25000000raised. It now uses the per-coordinate regex that fix(io): resolve the #279 parser follow-ups for Abinit, QE, VASP, Elk and Siesta #280 added toio/vasp/procar.py.core/bandstructure2D.py:148). This held in both modes:as_cartesian=False, whichfrom_codeand the legacy handler use, andas_cartesian=True, thefrom_ebsdefault. The grid is now regular in the two fractional coordinates that span the plane, in both modes. For a Cartesian plane the fractional normal isreciprocal_lattice @ normal. The grid edges are pulled in by 1e-12 of the span, because a grid corner that equals a data corner up to rounding can fall just outside the triangulation and come back NaN (BiSb lost its corner point on every sheet).extract_surface()renumbered the surface points while every property stayed in grid order, so colours were drawn on other points. A non-square grid crashed the interpreter. The surface is now built from quads in grid order.filterreadargs.infile,args.outfileandargs.human_atoms, which its parser never defines, so every call raised.catandfermi2Dalready worked.SCATTERinUnfoldPlotModeand in bothBandStructureModeenums.SACATTERstays in each enum as a deprecated alias of the same value, because the renderedband_structureconfig page named it andunfold(mode=UnfoldPlotMode.SACATTER)worked ondev. Iterating an enum lists onlySCATTER.Unreleasedsection records the two user-visible changes:autobandsplotwrites noreport.txtunlessreport=names a path, and theSCATTERspelling with its alias.dos,unfoldandband_structure_2dpages were generated from yml files that no code reads, and their values had drifted (unfold showedcolor: black, not#eeeeee). These pages now useautomodule, like the band-structure and Fermi-surface-3D pages. The yml files and their generator are removed, and the config docstrings now state the real defaults. The index lists the config modules.pyprocar.unfold. Getting Started and Comparing Bands kept option dumps that listweighted_colorandweighted_width. Those outputs are cleared rather than re-run, so the PR adds no images.build_overlay_orbitals_weightsandbuild_overlay_weightscalled.geton a list and raisedAttributeError(core/ebs.py:1467,1527). They now resolve shells with the samepresent_shellsandorbital_indicesthatbandsplotuses. Those two functions move intocore/atomic_orbital_index.py, besideorbital_shellsfrom fix(core): check projection axes against bands and structure, and give every parser its orbital names #298.lmirepis.false.. The l blocks stay, but the columns inside each l are no longer Y_lm. The parser now reads the version fromINFO.OUTandlmirepfromelk.in, and names those columnsY{l}_ir{i}inELMIREP.OUTorder. Shell sums still work. It reads both header forms: Elk 6 writes| Elk version 6.3.02 started |and Elk 11 writesElk code version 11.2.3(writeinfo.f90). Without a version the columns keep theirY{l}{m}names.ELMIREP.OUTalone does not decide it, because task 10 writes it too (withlmirepon, the default), and the usual workflow leaves a task-10ELMIREP.OUTbeside the bands. WhenELMIREP.OUTis present and no version is found, the parser warns and says whetherINFO.OUTis missing or has no version line. This rule never renames columns on a guess, so a version-less directory reads exactly as ondev.Scope
Closes #285. Everything else stays out of scope. The notes below are defects I found but did not fix.
Blast Radius
The orbital resolvers move from
scripts/_selection.pytocore, andpyprocar.utils.info.orbital_namesis removed (it had no other user). The BS2D change affects the grid ofBandStructure2Din both modes and every surface it builds. Elk naming changes only for task-22 runs in the irreducible-representation basis.Verification
Round 2 (fixes for the round-1 verdicts) is in "Round 2" below. The round-1 runs follow it.
Round 2
Elk 6.3 header (blocking finding). Repro from the verdict: a private copy of
data/codes/elk/6.3/SrVO3/spin-polarized-colinear/bandsplus../dos/ELMIREP.OUT, read withElectronicBandStructurePath.from_code("elk", dir).orbital_names[:9].['Y0_ir1', 'Y1_ir1', 'Y1_ir2', ...].['Y00', 'Y1-1', 'Y10', 'Y11', 'Y2-2', 'Y2-1', 'Y20', 'Y21', 'Y22'], with no warning. The same copy withoutINFO.OUTkeeps those names and warns.test_task_22_characters_in_the_ylm_basis_keep_their_l_m_names[6.3-with-task-10-elmirep](the real 6.3 header, CRLF included),test_task_22_basis_that_cannot_be_told_apart_keeps_l_m_names_and_warns[no-info, info-without-version], and the data testtest_real_elk_6_3_bands_beside_a_task_10_elmirep_keep_their_l_m_names(the repro above). The irrep tests now write the box header Elk 11.2.3 writes.Elk code versionregex (2 tests fail), and the old "ELMIREP.OUT without task 10 means irrep" fallback (2 tests fail).SCATTER alias.
test_scatter_mode_is_spelled_scatter_and_the_old_spelling_still_resolveschecksmode_enum["SACATTER"] is mode_enum.SCATTERandmode_enum(mode_enum.SACATTER) is mode_enum.SCATTERfor all three enums. It failed at 2fd6c01 withKeyError: 'SACATTER'and fails ondevbecauseSCATTERis missing.BS2D Cartesian grid and BiSb corner. NaN points from
BandStructure2D.from_ebson private fixture copies (padding 10):as_cartesian=True30x30as_cartesian=True40x25as_cartesian=True30x30as_cartesian=False30x30as_cartesian=Truedefault 20x20, padding 15as_cartesian=Truedefault 20x20, padding 15test_hexagonal_bs2d_surface_covers_the_k_patch_with_the_analytic_bands[cartesian],test_bs2d_on_a_lattice_slightly_off_square_leaves_no_grid_point_empty[cartesian, fractional](BiSb's reciprocal lattice, padding 10),test_cartesian_bs2d_plane_across_the_kz_layers_covers_its_patch(a* tilted out of kz = 0, so a Cartesian kz plane crosses the fractional layers).BS2D mutants that survived round 1, now killed. Each was run one at a time against
tests/pyprocar/core/test_bandstructure2d_grid.py:offset, nowlevel = 0.0): killed bytest_bs2d_plane_off_the_origin_on_a_skewed_lattice_has_the_bands_of_that_plane(kz = 0.25 on a lattice whose c* has x and y parts, heights against the analytic band to 1e-9) and one more test.reshape(shape[::-1]).T): killed bytest_bs2d_on_a_non_square_grid_builds_every_point, which now also checks 29x19 quads whose projected area equals the patch area.Before and after, graphene with the
from_ebsdefaultas_cartesian=Trueat 20x20, iso and top views. The holes and the stair-step edges are gone:The same at 80x80, where the Dirac cones are resolved:
BiSb,
as_cartesian=True, 20x20:The 20x20 default still looks spiky near the cones. Every vertex is right; a 20-point grid over 79 k-steps samples the cone tips coarsely.
-m "not data"2056 passed, 2 skipped.-m data406 passed, 1 skipped (405 at 2fd6c01; the new one is the Elk 6.3 data test). basedpyright 0 errors, 0 warnings. Bothruff_new_violationschecks and the F821 check pass.red_green.py origin/devpasses.Round 1
python .github/scripts/red_green.py origin/dev: 19 added tests. 17 fail ondevfor the reason they name. 2 guard existing behaviour (cat, Ylm-basis Elk names). The 2autobandsplotdata tests fail ondev, run by hand:report.txtlands in the cwd, andreport=is aTypeError.-m "not data"passes (2049 passed), thedatatests of core, io, plotter, scripts and cfg pass (405 here, 403 ondev, no new failure), basedpyright passes (0 errors, 8 stale baseline entries removed in their own commit), bothruff_new_violationschecks pass, and the F821 check passes.verify.sh run:bandsdosploton SrVO3 spin-polarized: legend up/up becomes up/down, with the down channel mirrored.BandStructure2D.from_code: at 20x20, 440 of 800 points were NaN and are now 0. At 90x90, 8280 of 16200 were NaN and are now 0. The apparent Dirac gap at 90x90 goes from 0.267 eV to 2.511e-5 eV, which is the EIGENVAL value at K.procar.py filterandcaton a fixture copy, andprocar.py fermi2Donfermi2d/non-spin-polarized.issues/issue-199/PROCAR(80 k-points, k41 = [0.50877193, -0.01754386, 0]).compute_projected_sumover the same orbitals.bandsdosplot, before and after:
Graphene BS2D at 90x90, before and after:
Not verified
automodulepages was not run locally, because the dev environment has no Sphinx. They use the same directive as the band-structure page.bandstr.f90,genlmirep.f90) and its release notes.