Skip to content

fix(warnings): show user-facing warnings through warnings.warn - #294

Merged
lllangWV merged 26 commits into
devfrom
fix/284-user-warnings
Oct 4, 2026
Merged

lllangWV merged 26 commits into
devfrom
fix/284-user-warnings

Conversation

@lllangWV

@lllangWV lllangWV commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

Why

pyprocar/utils/log_utils.py sets the user logger to ERROR, so all 62 user_logger.warning(...) calls in pyprocar/ printed nothing in a default call. A user never saw bandsplot's "fermi is not set", FermiSlicePlotter's missing field names, or a parser saying it guessed a unit convention. fermi2D also ignored its verbose argument. This PR applies the rule the operator chose: a message the user must see is a UserWarning that names the user's line, and the user logger keeps only verbose progress.

What changed

  • warn_user() in pyprocar/utils/log_utils.py walks the call stack past frames in pyprocar/, the standard library, <frozen ...> modules and generated <string> code (a dataclass __init__). It then calls warnings.warn_explicit with the first frame left, its module name and its __warningregistry__. The warning names the user's own line at any depth: through a helper, a functools.cached_property, FermiHandler, or a module import. Third-party frames in site-packages count as callers. In a thread, whose stack holds no caller frame, it names the nearest frame outside pyprocar/: the fn(*args) line of concurrent/futures/thread.py, or Thread.run.
  • warn_user has 65 call sites:
  • fix(ebs): make ibz2fbz unfold magnetic groups, phases, atoms and orbitals correctly #299's QEParser.symmetries_xml logged "No rotations found" with user_logger.warning. It is now debug, the same decision this PR made for QEParser.rotations: only ibz2fbz uses the rotations, and it logs its own skip.
  • fermi2D calls set_verbose_level(verbose), as dosplot, unfold and FermiHandler do.
  • tests/test_user_warnings.py enforces the rule. The CLAUDE.md row says exactly what it flags:
    • In pyprocar/: .warning, .error, .critical, .exception, .fatal, .warn, or .log at WARNING or above, called on getLogger("user"), on a name bound to it in the same function or module, on a self. attribute bound to it in the same class, or on a name imported from another module that binds it (also through a package __init__). Also warn or warn_explicit from the warnings module, through import warnings as w or from warnings import ..., outside log_utils.py.
    • In tests/: caplog.at_level/set_level with logger "user", setLevel on the user logger, and pytest.warns(UserWarning, ...) outside the helper.
  • tests/utils/user_warning.py wraps pytest.warns and also asserts that the warning names a line in the calling test file. Every warning test uses it, including fix(ebs): make ibz2fbz unfold magnetic groups, phases, atoms and orbitals correctly #299's three ibz2fbz tests. The verify harness records UserWarnings per step.

How warn_user chooses where a warning points (round 3)

Round 1 (skip_file_prefixes=(pyprocar/,)) named functools.py:998 under a cached property. Round 2 (frame walk plus warn_explicit(..., module_globals=...)) raised ValueError: Module globals is missing a __spec__.loader in IPython and Jupyter, and added a DeprecationWarning to every warning in a plain script. Both rounds failed at the hand-off from our frame choice to CPython, not in the frame choice. So I measured every hand-off I could find against one matrix before choosing.

The matrix runs each candidate as a separate interpreter on generated user scripts that call real pyprocar sites (KPath, QEParser.alat, FermiHandler.plot_fermi_surface), plus a fake library for the <string> and import-time cases. Rerunnable: matrix.py WORKTREE OUTDIR [candidate ...] with candidates.py, kept with the round's evidence.

scenario round 2 head round 1 prefix (a) explicit, no module_globals, old walk (a) explicit, no module_globals, new walk (chosen) (b) computed stacklevel (b) naive stacklevel (c) skip_file_prefixes of walked files (c) same, each name minus its last character
direct FAIL pass pass pass pass pass FAIL pass
helper FAIL pass pass pass pass pass FAIL pass
cached_property FAIL FAIL pass pass pass pass FAIL pass
fermi_handler FAIL pass pass pass pass pass FAIL pass
error_filter_in_script FAIL pass pass pass pass pass pass pass
dedup_default_filter pass pass pass pass pass pass FAIL pass
notebook_globals FAIL pass pass pass pass pass FAIL pass
spec_none_sourcefileloader FAIL pass pass pass pass pass FAIL pass
module_filter_ignores pass pass pass pass pass pass FAIL pass
thread_executor FAIL pass FAIL pass pass pass FAIL pass
thread_plain FAIL pass FAIL pass pass pass FAIL pass
dataclass_string_frame FAIL FAIL FAIL pass pass pass FAIL pass
import_time FAIL pass pass pass pass FAIL FAIL pass
plain_default_stderr pass pass pass pass pass pass FAIL pass
cli_W_error FAIL pass pass pass pass pass pass pass
cli_W_error_DeprecationWarning FAIL pass pass pass pass pass FAIL pass
passed 3/16 14/16 13/16 16/16 16/16 15/16 2/16 16/16
  • (b) passes warnings.warn(message, UserWarning, stacklevel=n) and lets CPython build the context. It works only if n counts frames the way CPython does. CPython skips importlib._bootstrap frames without counting them. The naive count sends an import-time warning to sys:1, silently. The corrected count copies that private rule into pyprocar.
  • (c) fails because CPython matches each skip_file_prefixes entry against the filename minus its last character (PyUnicode_Tailmatch(filename, prefix, 0, -1, -1)), so a full filename never matches. Trimming each name works, but it depends on that quirk.
  • (a) passes only plain values to warn_explicit: filename, line, module name and registry, which are its documented parameters. CPython's warnings.warn passes no module_globals either, so the source lookup, the registry, the filters and -W handling are the same code path. (a) and (b) give identical results in every scenario. (a) has no hidden contract with CPython, so I chose it.
  • The cli_W_error row uses python -W error. On any head, import pyprocar itself fails under -W error when it compiles its sources, because splash.py, scriptDosplot.py and others hold invalid escape sequences (a SyntaxWarning, raised as SyntaxError), and download_examples.py sets HF_HUB_ENABLE_HF_TRANSFER, which makes huggingface_hub raise a DeprecationWarning. Both exist on dev. The matrix runs it with a precompiled pycache and the Hugging Face warning ignored. The repo tests use an in-script simplefilter("error") after the import instead.

Scope

Blast radius

Users now see warnings that were always meant for them, in scripts, IPython and Jupyter. In a plain script a warning prints once per call of bandsplot, dosplot or unfold, even from the same line in a loop: matplotlib's Axis.set_ticklabels enters warnings.catch_warnings(), and that resets the warning registry. Other calls print once per calling line, the default Python rule. Code that set logging.getLogger("user") to WARNING to see these messages now gets them from warnings instead. Under an error filter the user gets the UserWarning raised from their own call.

Verification

  • Real Jupyter kernel (jupyter_client, pyprocar.bandsplot(code='vasp', dirname=<copy of data/examples/bands/non-spin-polarized>, show=False)):

    • Round 2 head 4297e96: ERROR ValueError: Module globals is missing a __spec__.loader.
    • This head: bandsplot returned Axes, and .../ipykernel_2574016/1397239802.py:4: UserWarning: \fermi` is not set! ...` names the cell's line.
  • ipython -c with a KPath call that has one name for two segments (__spec__ = None | __loader__ = None):

    • Round 2 head: RAISED ValueError Module globals is missing a __spec__.loader.
    • This head: recorded: [('UserWarning', '<ipython-input-1-ee06d37d50cd>', 1)].
  • Python 3.12, 3.13 and 3.14 (pixi exec -s python=3.1x, log_utils.py loaded on its own with a fake library): direct call, cached property, dataclass __post_init__, import time, notebook globals, script globals, error filter, once-per-line repeat, executor thread and plain thread. All pass on each version.

  • Red then green, new tests (tests/pyprocar/utils/test_warn_user.py). On the round 2 head, 8 of 11 fail: the notebook cell (ValueError), script globals (extra DeprecationWarning), the error filter and the subprocess script (DeprecationWarning raised instead of the UserWarning), both threads (threading.py self._bootstrap_inner()), and generated code (<string>:4). The other 3 (helper, FermiHandler, once-per-line) and the import-time test fail on dev, where no UserWarning is raised. All pass on this head.

  • Mutations of warn_user (8 warning test files, 252 tests):

    mutation failing tests
    registry=None 1 (once per line)
    add module_globals back 5 (notebook, script, both error-filter tests)
    stop skipping <string> 1
    stop skipping <frozen ...> 1 (import time)
    no thread fallback 2
    stop skipping the standard library 9
    no frame walk 29
    also skip site-packages 0 (a judgement call, see the note below)
  • Red then green, the check (test_the_checks_flag_each_loud_call_and_each_lowered_level). On the new synthetic lines, the round 2 resolver flags 24, 36 and 39 (a function-local getLogger(__name__), self.log bound to a package logger in another class, and a lambda argument) and misses 38 (a logger re-exported through pkg/__init__.py). The raw-call check finds [3, 4] of [3, 4, 7, 8, 10] (it misses import warnings as wm; wm.warn, warnings.warn_explicit and from warnings import warn_explicit). This head flags exactly the expected lines.

  • Red then green, fix(ebs): make ibz2fbz unfold magnetic groups, phases, atoms and orbitals correctly #299's tests. With the helper, test_phases_over_orbitals_without_real_harmonic_names_are_dropped_with_a_warning and test_the_unrotated_orbital_warning_lists_the_orbitals_in_numeric_order fail on dev with got ['UserWarning at pyprocar/core/ebs.py:1669']. The time-reversal test passes on dev because ibz2fbz is called directly, so it is marked guards_existing_behaviour.

  • Red-green job. 39 added or changed tests. 32 fail on base 5884b80, and 7 pass on base, each marked guards_existing_behaviour with its reason. red/green: ok.

  • Gates on 305023e (rebased onto dev 5884b80): pytest -m "not data" -n 4 1952 passed, 2 skipped. -m data -n 4 382 passed, 1 skipped. basedpyright 0 errors, 0 warnings. The ruff new-violation gates are ok. F821/F822/F823/NPY201 are clean. CI run 37232173717 passed on 305023e.

Closes #284

@lllangWV

lllangWV commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

VERDICT: FAIL

Reviewed SHA: 4297e96 (round 2). Base origin/dev 3ae5842, which is still the tip of dev and is the merge base, so the merge is clean. Lane: audit + regression.

Blocking finding

1. In IPython and Jupyter, every warn_user call raises ValueError instead of warning. pyprocar/utils/log_utils.py:30 passes module_globals=frame.f_globals to warnings.warn_explicit. To read the source line, CPython runs those globals through _bless_my_loader. In an IPython or Jupyter namespace, __spec__ and __loader__ are both None, so the C warn_explicit raises ValueError: Module globals is missing a __spec__.loader. The user-facing warning becomes a crash.

  • Public repro in a real Jupyter kernel (jupyter_client), on the private copy of data/examples/bands/non-spin-polarized:
    pyprocar.bandsplot(code='vasp', dirname=calc, show=False)
    The head gives ValueError: Module globals is missing a __spec__.loader. origin/dev returns Axes.
  • Smaller repro: ipython -c running pyprocar.core.kpoints.KPath(kpoints=k, segment_names=[("G","X")], ...) on 2 segments raises the same ValueError.
  • Fix, checked by monkeypatch: drop module_globals=. In IPython the same call then records UserWarning at <ipython-input-1-...>:10 and nothing is raised. Add a test that calls a warning site from globals with __spec__ = None, __loader__ = None (for example exec(..., vars(types.ModuleType("__main__")))), and from globals with __spec__ = None and a SourceFileLoader.

1b. Same cause, plain python script.py. A script's __main__ has __spec__ = None and a SourceFileLoader. There, every warn_user call also emits DeprecationWarning: Module globals is missing a __spec__.loader from <frozen importlib._bootstrap_external>:909.

  • In the fixture sweep below, the head raised 13 such DeprecationWarnings, one per UserWarning. origin/dev raised none.
  • Under warnings.simplefilter("error") or -W error, the user gets that DeprecationWarning raised instead of the UserWarning.
  • The repo's tests cannot see it. .config/.pytest.toml ignores DeprecationWarning, and pytest modules have a __spec__.
  • Dropping module_globals= fixes this too.

Round-1 findings

  • Fixed: warnings from cached_property sites now name the caller's line. The head's -m data summary now shows tests/.../elk/test_parser.py:509 and tests/.../test_parser_contract.py:131 where round 1 showed functools.py:998. In the sweep, Elk's "No GEOMETRY.OUT" names DRIVER:44, :49 and :52.
  • Fixed: BandStructure2DHandler now calls warn_user.

What I ran

  • -m data at the head with -n 4: 381 passed, 1 skipped, exit 0. Round 1's origin/dev run gave the same 381 passed, 1 skipped.
  • The same warning sweep as round 1, on private reflink copies of every bands, dos, fermi2d and fermi3d fixture and data/codes/{abinit,qe,elk,vasp}. That is 134 calls each at the head and at origin/dev 3ae5842: parser members, bandsplot and dosplot with and without fermi, fermi2D, FermiHandler, unfold.
    • Status differed between head and dev in 0 of 134 calls.
    • The head raised 13 UserWarnings, and every one names the calling script's line. 6 were bandsplot's "fermi is not set" at DRIVER:49, 3 were dosplot's at DRIVER:54 and 1 was unfold's at DRIVER:64. 3 were Elk's "No GEOMETRY.OUT" at DRIVER:44/49/52, which fire because that fixture has no GEOMETRY.OUT.
    • Each matches, one for one, a user-logger WARNING that dev hid. The only difference is dev's 13 FermiHandler "Fermi Energy not set!" WARNINGs, which moved to info, as in round 1.
    • No call given fermi warns, apart from the Elk fixture's missing file.
    • New at the head: the 13 DeprecationWarnings of finding 1b. The 99 FutureWarnings are the same at head and dev.
  • Frame-walk probes. I pointed _PACKAGE_DIR at a scratch package and called each shape below under simplefilter("always").
    • Correct: direct calls, cached_property, property, generators, contextmanager, a C callback through map or a sorted key, and calls through np.vectorize in site-packages all name the user's line. exec with globals that lack __name__ names <string>:1.
    • A warning in a thread names threading.py:1032, because the walk reaches the top of the thread's stack without leaving library frames. That is acceptable.
    • A pyprocar callback that a site-packages library calls (pyvista, matplotlib) names that library's line. That is by design.
    • The __warningregistry__ handling works. Under default, the same line warns once. A new catch_warnings resets the registry. ignore and error are honoured, and pytest's warns/recwarn record the warnings, since the PR's tests pass.
  • AST resolver on the current tree, across all of pyprocar/ and tests/. It binds only the 21 module-level user_logger = logging.getLogger("user") names. The receivers it resolves are info x72, debug x5, setLevel x4 (log_utils x3, bandsplot's quiet_welcome), and warning x2. The two warnings are in tests/scripts/extract_ideal_test_data.py, a dev script that the library check does not scan. It flags nothing on the head, so I found no false positives on the current tree.
  • gh pr checks 294: format, lint, red-green, test and typecheck all pass.

Acceptance items

  1. Done. All four checklist sites are covered, as in round 1.
  2. Not done. The head names the user's line at every fixture site, but finding 1 turns each warning into a crash in IPython and Jupyter.
  3. Done. The check is widened, the raw warnings.warn ban is in, and the CLAUDE.md row is accurate, apart from the gaps in the notes.
  4. Unchanged from round 1. The red/green lane owns the fail-on-dev proof.

Notes (not blocking)

  • Resolver limits, shown on synthetic files. Both forms are absent from the tree today.
    • It ignores scope. A function-local user_logger = logging.getLogger(__name__) in a module that also has a module-level user logger gets flagged. So does self.log when another class in the same module binds self.log = getLogger("user").
    • It misses a user logger re-exported through a package __init__ (from pkg.sub import user_logger, or from .sub import ...). The module key is pkg.sub.__init__, but the import names pkg.sub.
    • The CLAUDE.md row's "through aliases and imports too" is therefore slightly broader than what the check does.
  • raw_warnings_warn_calls misses import warnings as w; w.warn(...).
  • Package-logger sites that write only to pyprocar.log, scriptVector.py:38's print("WARNING: ..."), and the Abinit-fermi pad and bisb fermi2D failures that dev already has are all unchanged from round 1.

@lllangWV

lllangWV commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

VERDICT: PASS+NOTES

Reviewed SHA: 305023e. Gates ran on that SHA merged with current origin/dev f243254 (#297). The merge is clean: git merge-tree --write-tree origin/dev 305023e8 exited 0, and the local merge commit is cedd2e8e.
Lane: Live + red/green + gates (independent verifier, round 3).

Gates (merged head, verify.sh exec, -n 4)

  • pytest tests -m "not data" -n 4: exit 0, 1952 passed, 2 skipped.
  • basedpyright --project pyrightconfig.json: exit 0, 0 errors, 0 warnings.
  • ruff_new_violations.py check origin/dev and format origin/dev: exit 0 for both, 33 changed files ok.
  • ruff check --select F821,F822,F823,NPY201 .: exit 0.
  • red_green.py origin/dev: exit 0. Of 39 added or changed tests, 33 fail on base and 6 pass on base. All 6 are marked guards_existing_behaviour, and each reason holds.
  • -m data suite: head 382 passed, 1 skipped (exit 0). origin/dev 382 passed, 1 skipped (exit 0).
  • gh pr checks 294: format, lint, red-green, test and typecheck all pass.

Live: fresh interpreters on private fixture copies (head)

Fixture copies: bands/non-spin-polarized, Elk SrVO3 bands (it has no GEOMETRY.OUT), fermi3d/non-spin-polarized, and an empty directory. In every row, the location is the file:line the warning names.

scenario named location DeprecationWarning or exception?
real Jupyter kernel (jupyter_client), bandsplot with no fermi ipykernel_*/1408399939.py:2 (the cell's line) none
same kernel, KPath with 1 name for 2 segments the cell's line 3 none
same kernel, QEParser(empty).alat (cached_property) the cell's line 2 none
same kernel, helper called in a loop 3 times the helper's line, shown once none
same kernel, simplefilter("error") raises UserWarning expected
ipython -c: KPath, then bandsplot <ipython-input-1-…>:4 and :5, source line printed none (__spec__ is None)
python s_bands.py, 2 calls in a loop s_bands.py:5, twice (matplotlib resets the registry, as the PR documents) none
python -W always::DeprecationWarning s_qe.py s_qe.py:5 none
-W error (precompiled pycache, HF warning ignored) UserWarning raised from the user's line for KPath, bandsplot and QE alat expected
-W error::DeprecationWarning (HF warning ignored) s_kpath.py:4, s_bands.py:5, s_qe.py:5 none
-W ignore nothing printed none
-W ignore::UserWarning:__main__ and :mymod suppressed (module attribution matches stdlib) none
Elk from_code without GEOMETRY.OUT (cached_property) s_elk.py:4 none
QE alat on an empty dir (cached_property) s_qe.py:5 none
FermiHandler(...).plot_fermi_surface(mode="no_such_mode") s_fermi.py:5 none
warning while a user module imports mymod.py:4 none
matplotlib draw_event callback that calls pyprocar s_mpl.py:9 (inside the callback) none
ThreadPoolExecutor, user function s_thread.py:7 none
ThreadPoolExecutor, submit(KPath, …) and submit(pyprocar.bandsplot, …) concurrent/futures/thread.py:59 none
#299 sites via the ElectronicBandStructureMesh constructor (phase dropped, unrotated ×2, time reversal) and direct ibz2fbz head: the user's lines (s_ibz.py:24, :36, :37). origin/dev: pyprocar/core/ebs.py:1669 for 4 of the 5 none
fermi2D(verbose=0) vs verbose=1 head: 4 stdout lines vs 47, with ### Parameters ###. origin/dev: 4 lines for both n/a
harness verify_steps records both bandsplot warnings under "warnings" in summary.json. The bands driver gives the same 2 failing steps on head and on dev (they already fail on dev) n/a

Things that are the same on origin/dev, not caused by this PR: python -W error and -W error::DeprecationWarning both fail at import pyprocar. The first comes from SyntaxWarnings when no pycache exists. The second comes from the huggingface_hub HF_HUB_ENABLE_HF_TRANSFER DeprecationWarning.

#299 conversions: the 3 ebs.py sites now name the caller's line, where origin/dev named ebs.py:1669. QEParser.symmetries_xml "No rotations found" is now user_logger.debug. No user_logger.warning and no raw warnings.warn remain in pyprocar/ outside log_utils.py.

Mutations of warn_user (10 warning test files, 280 tests including data)

mutation failing tests
drop the pyprocar/ skip 32
drop the stdlib skip 9 (Elk and BXSF cached_property tests, bandsplot elk, import-time)
drop the <frozen skip 1 (import-time)
drop the <string> skip 1 (generated code)
registry=None 1 (once per line)
registry={} 1
add module_globals back 5 (notebook, script, 3 error-filter tests)
remove the thread fallback 2
use the outermost frame for the fallback 2
co_firstlineno in place of f_lineno 9
drop the _SITE_DIRS exclusion survives
fixed module="pyprocar" survives

Enforcing check: I added 11 forbidden forms to a temporary pyprocar/ module, and tests/test_user_warnings.py flagged all 11. The 9 user-logger forms were the direct call, a module name, .log(logging.WARNING), .log(30), self.lg.error, a name imported from scriptBandsplot, getLogger(name="user"), a local .exception, and .log(level=ERROR). The 2 warnings forms were w.warn and from warnings import warn. The CLAUDE.md row is present.

Acceptance (#284, frozen set)

  • The 4 User-facing warnings are hidden by the ERROR-level user logger #284 sites. Done.
    • bandsplot: checked live.
    • FermiSlicePlotter scalars and vectors: the unit test calls the public class, and the no-pkg-skip mutation is caught.
    • fermi2D verbose: checked live, head vs dev.
    • The FermiHandler box-widget site: converted. No public call reaches it, because from_ebs raises first, and the test pins that error.
  • The rule applied to every user_logger warning site and every raw warnings.warn site in pyprocar/. Done.
  • The enforcing check and the CLAUDE.md row. Done.
  • Public-API tests fail on origin/dev. Done: 33 of 39 are red on base for the stated reason, and the other 6 are marked guards.

Notes (none blocking)

  1. pyprocar/utils/log_utils.py:25-30: the thread fallback keeps the innermost non-pyprocar frame. So ThreadPoolExecutor().submit(pyprocar.bandsplot, code="elk", dirname=<elk dir without GEOMETRY.OUT>, fermi=0.0, show=False) names functools.py:998, the round-1 location, not concurrent/futures/thread.py:59. The comment says "name the line that called into pyprocar". No user frame exists on that stack, so this is not an acceptance failure. A 3-line fix resets the candidate on each pyprocar frame:
    if filename.startswith(_PACKAGE_DIR): nearest_outside = None / elif nearest_outside is None and not filename.startswith("<"): nearest_outside = frame
    With that fix, the case names thread.py:59, and 81 of 81 warning tests (plus 1 skipped) still pass.
  2. Two test gaps, shown by the mutations that survive. No test pins the module= name, which module-scoped -W ...:module filters depend on. I checked that live. No test exercises the _SITE_DIRS exclusion either.
  3. 29 package-logger logger.warning/.error calls write only to pyprocar.log. Some say a member is None, for example VaspParser.ebs, .dos and .structure (io/vasp/parser.py:237,242,313,330). VaspParser(empty).ebs returns None silently, while QEParser(empty).ebs now warns. from_code still raises loudly. This sits outside User-facing warnings are hidden by the ERROR-level user logger #284's user-logger scope, so it is a candidate follow-up.
  4. A behaviour change: fermi2D() with the default verbose=1 now prints the parameter banner (about 43 lines), as dosplot does. It also leaves the user logger at DEBUG for the rest of the process.
  5. Not checked: Python 3.13 and 3.14. The local env is 3.12.12, and requires-python >= 3.12.

@lllangWV

lllangWV commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

VERDICT: PASS+NOTES

Reviewed SHA: 305023e (round 3, based on 5884b80). Since then origin/dev has moved to f243254 (#297, which changes only .claude/skills/verify-pyprocar/). git merge-tree --write-tree origin/dev 305023e8 exits 0. I ran every check below on the head merged with f243254, in a detached worktree. Lane: audit + regression.

What I ran

  • -m data -n 4: on the merged head, 382 passed, 1 skipped, exit 0. On origin/dev f243254, also 382 passed, 1 skipped, exit 0. Every UserWarning in the head's warnings summary names a test line (elk/test_parser.py:509, qe/test_qe_parser.py:736, test_parser_contract.py:131, test_one_call_functions.py:785). On dev, the fix(ebs): make ibz2fbz unfold magnetic groups, phases, atoms and orbitals correctly #299 warnings name pyprocar/core/ebs.py:1669 and the fermi2D/FermiHandler warning names scriptFermiHandler.py:275.
  • Warning sweep: the same driver as rounds 1 and 2. It ran 134 calls per checkout on private reflink copies of every bands, dos, fermi2d and fermi3d fixture and data/codes/{abinit,qe,elk,vasp}. Each call ran under catch_warnings(record=True) with simplefilter("always"), and a handler captured the user logger's records.
    • The status of each call (ok, error, which members are None) differed between head and dev in 0 of 134 calls. The errors match exactly and already exist on dev: Abinit fermi 'ElectronicBandStructure' object has no attribute 'pad' (6), bisb fermi2D "No Fermi surfaces were generated" (1), and 2 bandsplot_fermi ValueErrors.
    • The head raised 37 UserWarnings, and all 37 name the calling script's line (DRIVER:44/49/52/54/59/61/64).
    • DeprecationWarnings: 0 at head and 0 at dev. Round 2's 13 are gone.
    • The FutureWarnings match: 99 at head and 99 at dev.
    • With fermi given, bandsplot_fermi/dosplot_fermi raise no warning, except Elk's missing GEOMETRY.OUT. That warning is correct.
    • Each head warning matches, one for one, a dev UserWarning or a hidden user-logger WARNING. The only difference is dev's 13 FermiHandler "Fermi Energy not set!" records, which are now info. That move is correct, because the handler uses the Fermi energy it found.
  • Frame-walk probes on the merged head:
    • These name the caller's line: a direct call, cached_property, property, a generator, contextmanager, map, a sorted key, an np.vectorize callback, and exec.
    • A thread names threading.py:1012, as the PR body says.
    • default warns once per line. ignore and error are honoured, and a module="__main__" filter applies.
    • A plain script (__spec__ None, SourceFileLoader) gives UserWarning at script_user.py:9 and no DeprecationWarning. Under simplefilter("error") it raises the UserWarning.
    • ipython -c (__spec__ = None | __loader__ = None) records ('UserWarning', '<ipython-input-1-…>', 1) and raises nothing.
  • AST resolver, across all of pyprocar/ and tests/: it binds exactly the 21 user_logger = logging.getLogger("user") names (20 at module level and 1 in set_verbose_level). All 83 receivers it resolves are those names.
    • Methods: info x72, debug x5, setLevel x4. That is log_utils x3 and bandsplot's quiet_welcome, and the test scan does not cover the library.
    • It also finds warning x2 in tests/scripts/extract_ideal_test_data.py. The library check does not scan that file.
    • It leaves 215 logger.* calls (getLogger(__name__)) and 166 self.log.* calls (getLogger("ProcarParser") and similar) unresolved, which is correct.
    • It flags nothing. I found no false positives on the tree.
  • Other checks:
    • The warning test modules (test_user_warnings, test_warn_user, test_user_facing_warnings, test_ibz2fbz, test_kpoints): 125 passed.
    • ruff_new_violations.py check/format origin/dev: ok, 33 files.
    • gh pr checks 294: format, lint, red-green, test and typecheck all pass.

Acceptance items

  1. The four User-facing warnings are hidden by the ERROR-level user logger #284 sites: done. bandsplot is confirmed live on 6 fixtures, each naming the driver's line. The FermiSlicePlotter names and fermi2D's verbose are done. FermiHandler's "No Fermi surface found" uses warn_user, and its test is honestly marked: from_ebs raises first.
  2. The rule applied to every site in pyprocar/: done for the user logger and raw warnings.warn.
  3. The enforcing check and the CLAUDE.md row: done. The row describes the check accurately. Note 2 lists its limits on synthetic files.
  4. Public-API tests that fail on origin/dev: done per the red-green job. I judged the 7 new guards_existing_behaviour reasons, and each is accurate:

PR body claims

  • Confirmed by my runs: the 65 sites and their breakdown, the 6 + 1 moves to info/debug, the thread naming (Thread.run), the IPython and script fixes, -m data 382/1, and CI passing on 305023e.
  • Not checked by me, because the live lane owns them: the 16-scenario matrix, Python 3.13/3.14, the real Jupyter kernel, the mutation table and the red-green counts.
  • "ibz2fbz ... logs its own skip": literally true, but that log goes only to pyprocar.log. See note 1.

Notes (none blocking)

  1. fix(ebs): make ibz2fbz unfold magnetic groups, phases, atoms and orbitals correctly #299's QE "No rotations found" moving to debug hides nothing a user saw before, but nothing visible replaces it.
    • On dev the message was user_logger.warning, which the ERROR level hid by default. At head it shows at verbose>=1, the same as on dev.
    • On a public QE path the user still gets a loud error. With PwXML.rotations patched to None, get_parser("qe", fermi).ebs and FermiHandler(code="qe", ...) raise ValueError: n_kpoints must be equal to np.prod(kgrid) at both head and dev. The cause appears only in pyprocar.log (ebs.py:2220, "No rotations provided, skipping ibz2fbz").
    • The call that docs/source/user-guide/ebs.rst:76-80 documents is silent. ibz2fbz(ebs, kgrid_info=KGridInfo((4,4,4))) on an ebs whose structure has no rotations returns 3 k-points instead of 64, with no warning. dev behaves the same.
    • Suggested follow-up: make ebs.py:2220 a warn_user. Leave the "already the full grid" skip on the package logger, because that result is correct.
  2. Resolver limits on synthetic files. None of these forms is in the tree today.
    • A class-body log = getLogger("user") binds at module scope, so a function's use of a module-level log = getLogger(__name__) is flagged. This is a false positive, and it contradicts the docstring's "as Python scopes it".
    • It misses a global rebinding inside a function, mod.user_logger.warning(...) reached as a module attribute, and an alias passed in a default argument.
    • The raw check misses from warnings import *.
  3. Thread fallback, by reading the code, not run. nearest_outside is the first stdlib frame below pyprocar. A site inside a cached_property, reached from Thread(target=<pyprocar method>), would therefore name functools.py, not threading.py. This is rare and harmless.
  4. Already on dev and unchanged:
    • Abinit fermi raises pad AttributeError, and bisb fermi2D raises "No Fermi surfaces were generated".
    • Under -W error, import pyprocar fails on huggingface_hub's DeprecationWarning, as the body states.
    • Other package-logger logger.warning sites still write only to pyprocar.log.

The user logger is set to ERROR on import, so user_logger.warning() prints
nothing in a default call (#284). These tests fail on dev:

- tests/test_user_warnings.py lists the 62 user_logger.warning calls in
  pyprocar/ and the 4 tests that lower the user logger with caplog to see
  them.
- bandsplot without fermi, FermiHandler with no Fermi surface and
  FermiSlicePlotter with a missing field name emit no UserWarning.
- fermi2D prints its verbose progress for neither verbose=0 nor verbose=1.
…ler line

FermiSurface.from_ebs raises "No Fermi surfaces were generated" before
FermiHandler's empty-surface check runs, so a call with no crossing is
already loud; the test now guards that. A probe of 72 pockets (two
lattices, six centres, six radii) gave a surface or that error, never an
empty surface.

The bandsplot test also checks that the warning names the caller's file.
The user logger is set to ERROR on import, so all 62 user_logger.warning
calls in pyprocar/ printed nothing in a default call (#284).

warn_user() in utils/log_utils.py raises a UserWarning attributed to the
first frame outside pyprocar (skip_file_prefixes), so the message names the
user's own line. Each site is now either:

- warn_user (57 sites): the result differs from what the user asked for,
  or a member comes back None (fermi not set, ignored argument, missing or
  unreadable input, an assumed unit convention, an unwritten file).
- user_logger.info/debug (5 sites): progress with nothing to act on. QE's
  missing scf.in/scf.out and rotations, Siesta's auto-detected .bands file
  (ebs reports its absence), and FermiHandler using the calculation's
  Fermi energy by default.

Tests that lowered the user logger with caplog to see these messages now
assert them with pytest.warns.
fermi2D documented verbose but never called set_verbose_level, so its progress
output followed whatever level the user logger last had. It now calls it
first, as dosplot, unfold and FermiHandler do.
…rness

CLAUDE.md names the rule and tests/test_user_warnings.py as its enforcer.
verify_steps no longer lowers the user logger, which now carries no
warnings; it records each step's UserWarnings in summary.json instead.
…e probe

- fs_plot and fermi2D raised their #281 warnings with warnings.warn and
  stacklevel=2, which blames a pyprocar line when FermiHandler is the
  caller. They now use warn_user like every other site.
- LobsterParser probed the default DOSCAR.lobster and lobsterout in
  __init__ and warned when either was absent, so a bands-only run warned on
  every load. The probe logs at debug; ebs and dos warn when they return
  None, as Siesta's ebs does.
- UserFriendlyFormatter styled WARNING and ERROR records the user logger
  no longer carries; it is gone.
- The verbose test reads caplog instead of a hand-written handler.
Warnings raised inside a functools.cached_property name functools.py:998
instead of the user's line. The new user_warning helper asserts the
warning's file, and every pytest.warns(UserWarning) test now uses it.
BandStructure2DHandler gets the bandsplot fermi-not-set test.
The check now follows annotated and plain aliases, getLogger(name=...),
an imported getLogger alias, a logger imported from another module,
tuple targets and .log() at WARNING or above. In tests it also flags
caplog levels given the logger by position, setLevel on the user
logger, and pytest.warns(UserWarning) without the line check. A raw
warnings.warn in pyprocar/ fails it: clusters.py:279 is the live case.
…ib wrappers

skip_file_prefixes skipped only pyprocar/, so a warning raised inside a
functools.cached_property named functools.py:998. warn_user now walks
the frames itself past pyprocar and the standard library (not
site-packages) and calls warn_explicit with the first caller frame.
Clusters.hydrogenate and BandStructure2DHandler's fermi-not-set message
now go through warn_user too.
The red-green job (#300) lists them: two pin a line that #281's
stacklevel=2 already named, one pins FermiHandler's loud error, and
three test the check itself or scan tests/ this PR already converts.
@lllangWV
lllangWV force-pushed the fix/284-user-warnings branch from 305023e to 5796327 Compare October 4, 2026 21:11
@lllangWV
lllangWV merged commit 5d6ae01 into dev Oct 4, 2026
5 checks passed
lllangWV added a commit that referenced this pull request Oct 4, 2026
…utput

Every zone build called spglib.delaunay_reduce, which warns on each call
while spglib's process-wide OLD_ERROR_HANDLING is on. Users saw the
DeprecationWarning on every Fermi-surface plot, and #294's FermiHandler
line test recorded it. The call now ignores exactly that message; the None
check below it is the error handling spglib asks for.
lllangWV added a commit that referenced this pull request Oct 4, 2026
…warn_user

#294 removed user_logger from the Abinit parser and made warn_user the only
channel for warnings a user must see, so the merge left a NameError at
io/abinit/parser.py and a raw warnings.warn in io/vasp/parser.py. The
Abinit test now asserts the warning names its own line.
lllangWV added a commit that referenced this pull request Oct 4, 2026
…e every parser its orbital names (#298)

* test(core): projections whose axes disagree with bands, total or structure are rejected

Refs #293

* fix(core): check projection axes against the bands, total DOS and structure

check_projected_layout compared only the number of axes, so swapped
spin and atom axes passed. It now compares the leading axes with the
bands or total DOS, the atom axis with the structure, and requires 1, 2
or 4 spin channels. Non-collinear projections may go with one band
channel, as the QE parser stores them, or four, as VASP and Abinit do.

Refs #293

* test(qe): non-collinear band structures carry total, Sx, Sy and Sz per orbital

Refs #293

* fix(qe): give non-collinear band structures total, Sx, Sy and Sz per orbital

projwfc.x projects a spinor run onto |l j m_j> (spin-orbit) or
|l m s_z> states. The parser summed the j amplitudes into 32 slots on one
spin channel and could not read s_z states. It now writes each shell in
the real (l, m) harmonics and spin up/down with QE's own spinor, sph_ind
and rot_ylm coefficients, and stores total, Sx, Sy and Sz per orbital.
Both QE band structures and DOS now pass projwfc.x's orbital names.

Refs #293, #219, #220

* test(io): every parser names the orbitals it projects

Refs #293

* fix(io): pass the orbital names each parser projects

VASP and Abinit read the names from the PROCAR header instead of
cutting a fixed list, so a PROCAR that pyprocar.filter rewrote keeps its
o0 column name. The Abinit, Elk and Lobster DOS now name their orbital
columns too, so check_projected_layout compares them.

Refs #293

* test: type the layout and QE spinor test inputs for basedpyright

Refs #293

* refactor(qe): parse projwfc states into a SpinorState before the spinor transform

Also format and lint the #293 changes.

Refs #293

* docs(claude): state what check_projected_layout now enforces

Refs #293

* style(abinit): name the angular momentum ang

Refs #293

* test(abinit): a non-collinear DOS carries total, Sx, Sy and Sz

Refs #293

* fix(abinit): give a non-magnetic spinor DOS total, Sx, Sy and Sz channels

Abinit's DOS_AT files hold no magnetization components. With nspden 1 the
spin-resolved DOS is zero by time reversal, so the parser adds three zero
channels. With nspden 4 it keeps the total and warns, since the files
cannot give the spin components.

Refs #293

* refactor(io): simplify the #293 parser changes and pass the structure to every DOS

- QE spin-orbit DOS names reuse OrbitalIndexer.soc_label, the format the
  label builder already renders.
- Abinit DOS names are a constant: the kept lm columns are a fixed slice.
- Procar.n_orbitals counts the header names instead of rescanning.
- The spinor transform uses a matrix product and adds in place.
- The QE, Abinit and Elk DOS pass their structure, so the atom axis is
  checked against it.

Refs #293

* docs(verify-pyprocar): record the QE non-collinear projection end state

Refs #293

* test: filtered PROCARs load, group rows and name orbitals as their header does

An atom-filtered PROCAR holds one row per group of atoms, so it loads, plots
each group, labels it by its row and refuses species selections. An
orbital-filtered PROCAR labels its columns o0, o1 as its header does, and a
single orbital selected by name raises whichever spelling the header uses.

Refs #293

* fix(vasp): load a PROCAR filtered by atoms as groups of atoms and label filtered orbitals by name

pyprocar.filter(atoms=...) writes one row per group of atoms, so the PROCAR
has fewer rows than the POSCAR has atoms and check_projected_layout rejected
it. The VASP parser now passes atom_groups, the number of PROCAR rows, when
the counts differ. The layout check compares the atom axis with that count,
so an axis swap still raises. Atom indices select groups, a selection with
no atoms sums every group, and species selections raise, because a group can
hold atoms of several species. The structure keeps the lattice and the
symmetry operations, so Fermi surfaces unfold as before.

Selection labels used the shell letter s for orbital 0 before they read the
parser's names, so a filtered PROCAR's o0 column read (s). A shell letter
now stands for orbitals only when their names make up that whole shell.

Refs #293

* fix(scripts): raise a clear error for a single orbital selected by name

orbitals accepts indices and the shell names s, p, d and f. A single orbital
name such as x2-y2 or dx2, which VASP spells differently by version, raised
KeyError. It now raises a ValueError that names what orbitals accepts.

Refs #293

* chore(basedpyright): drop the two resolver comparisons the atom_groups change made necessary

* docs(filter): say what the rows and columns of a filtered PROCAR mean when plotted

Refs #293

* test(scripts): a shell name selects the columns a filtered PROCAR's header names

Also copy only the four files the VASP parser reads and parse the
unfiltered PROCAR once per module.

Refs #293

* fix(scripts): resolve shell names from the orbital names, as labels do

orbitals=['p'] and overlay_orbitals used the fixed table s=0, p=1-3,
d=4-8, while labels now read the parser's names. Both now take shells
from orbital_shells, so a shell name selects the columns its names make
up and raises when no whole shell is there. unfold resolves species
names through the selection resolver, as bandsplot does, so an
atom-filtered PROCAR refuses them.

Refs #293

* test(scripts): a shell letter selects the column a filtered PROCAR names by it

Refs #293

* fix(scripts): a column named s, p, d or f is that shell for selections

Elk's task 21 and pyprocar.filter(orbital_names=['s','p','d']) write one
column per shell, named by its letter. orbitals=['d'] picked columns 4-8
there (IndexError, or a wrong sum when the columns existed). It now
picks the column named d, and overlay_orbitals draws one curve per such
column.

Refs #293

* docs(verify-pyprocar): record the filtered PROCAR end states

Refs #293

* test: mark the accepted spin layouts as guarding existing behaviour

Refs #293

* test: a selection without atoms sums every row, a part shell is no shell, filtered IBZ PROCARs refuse to unfold

A DensityOfStates without a structure (every Lobster DOS) gave zeros for
compute_projected_sum and compute_magnetization without atoms, and dosplot
parametric coloured a Lobster DOS 0. Columns py and pz must not form a p
shell. fermi2D on an atom-filtered PROCAR of irreducible k-points failed
with an IndexError once ibz2fbz started permuting atoms (#299).

Refs #293

* test: an empty atom selection raises instead of summing nothing

Refs #293

* fix(core): a selection without atoms resolves to every atom row

With no structure the resolver returned atoms=(), which the band and
Fermi surface callers read as every row but DensityOfStates passed on as
no rows, so a Lobster DOS (no structure) summed to zero. The resolver now
takes the source's row count and always returns explicit row indices, and
every caller passes them as given. An empty atom selection raises.

Refs #293

* fix(ebs): refuse to unfold an atom-filtered PROCAR from irreducible k-points

ibz2fbz (#299) permutes atoms under each symmetry operation, so a PROCAR
whose rows are groups of atoms failed with an IndexError in fermi2D and
every other mesh. The file does not record which atoms a row holds, so no
permutation of its rows is known to be right. It now raises an error that
says to filter a full k-grid calculation or select atoms in the plot.

Refs #293

* chore: annotate the resolver row count and type the filter groups in the test

Refs #293

* test: name n_atoms in the guard reason of the selection label test

Refs #293

* test: keep the guard reason within the line limit

Refs #293

* refactor: check atom groups before ibz2fbz copies any property, and reuse the test helpers

The atom-group guard ran inside _turn_projections, after ibz2fbz had
already replaced the properties, and refused every symmetry operation. It
now runs first and refuses only operations that move an atom. The fermi2D
test builds its filtered calculation with _filtered_calc, and the DOS tests
import VASP_ORBITALS instead of copying it (simplify review).

Refs #293

* test: an empty spin selection raises instead of summing nothing

The resolver returned spins=(), which band callers read as every spin and
DensityOfStates as none.

Refs #293

* fix(core): an empty spin selection raises, as an empty atom selection does

Refs #293

* fix: send the abinit spinor-DOS and VASP atom-group warnings through warn_user

#294 removed user_logger from the Abinit parser and made warn_user the only
channel for warnings a user must see, so the merge left a NameError at
io/abinit/parser.py and a raw warnings.warn in io/vasp/parser.py. The
Abinit test now asserts the warning names its own line.
lllangWV added a commit that referenced this pull request Oct 4, 2026
…utput

Every zone build called spglib.delaunay_reduce, which warns on each call
while spglib's process-wide OLD_ERROR_HANDLING is on. Users saw the
DeprecationWarning on every Fermi-surface plot, and #294's FermiHandler
line test recorded it. The call now ignores exactly that message; the None
check below it is the error handling spglib asks for.
@lllangWV
lllangWV deleted the fix/284-user-warnings branch October 4, 2026 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant