Repository navigation
feat(verify-pyprocar): make fetched fixtures read-only and keep verify.sh inside data/ - #303
Conversation
5c125da to
70db097
Compare
|
VERDICT: FAIL Reviewed SHA: 70db097, on origin/dev 5d6ae01. The head is directly on top of dev, so no merge was needed. Commands and resultsI worked in detached worktrees at the head and at origin/dev. I ran env binaries from the main checkout's
Acceptance (#288)
Findings
The census found no other escape. |
|
VERDICT: FAIL Reviewed SHA: 70db097 (base origin/dev 5d6ae01; Safety. Every verify.sh run (head, round 1, dev and mutants) ran inside Finding (blocking)A pre-existing entry at the next run name sends
Smallest repro, in a fake main checkout with
Result: exit 2. Variant: a real run dir at that name whose
Acceptance set (#288, frozen)
Live hostile layouts
The commands are every subcommand with hostile arguments: the PR's REFUSED list plus The layouts:
Results:
Mutants
download_from_hf (fake
|
70db097 to
ebda932
Compare
|
VERDICT: FAIL Reviewed SHA: a88f1f5. It sits directly on origin/dev 4de5cc0, and Safety. I did not run verify.sh outside the PR's own fake-root Blocking findingOne fixture-copy site still fails on read-only fixtures (acceptance item 4 is claimed but not done). The site is Repro: lock a reflink copy of the fixtures (
The head fixes the 4 Commands and results at head (all exit 0)
Census audit (verify.sh)My own census of the redirections, cp, mv, ln, mkdir, rm, chmod, find -delete, tee, touch, here-docs, process substitution and traps in I added fake-root probes through the PR's
Notes (not blocking)
|
|
VERDICT: FAIL Reviewed SHA: a88f1f5, directly on origin/dev 4de5cc0 ( Safety. Every execution of verify.sh (head, r1, r2, dev, 57 mutants) and every test that drives it ran inside Finding (blocking)A fixture-copy site still fails on read-only fixtures.
Acceptance set (#288, frozen)
Live replays and hostile layouts (head)
Mutants
CI gates at head (through
|
|
VERDICT: FAIL Reviewed SHA: 49a7c2f (round 4). The base in the spawn brief was origin/dev 4de5cc0. During this review origin/dev moved to f93c8bd (#304). Safety. Every run of verify.sh, its tests, its mutants and the data suite used Blocking findingOnce merged into current dev, a fixture-copy site fails on locked fixtures. Acceptance item 4 is false for the result of the merge.
Acceptance set (#288), at 49a7c2f
My own grep of the head finds no unconverted site.
Round-4 changes, checked
GatesAt 49a7c2f, through
On the head merged with f93c8bd (scratch clone, same env):
Notes (non-blocking)
|
copytree keeps a read-only fixture's modes, so a test that writes into its copy fails with PermissionError once fixtures are locked. writable_copy dereferences symlinks and adds user write to the copy only.
… its own copies Run a copy of the harness in scratch repos (data in the repo, data linked, and a linked worktree) with a stub env. Cover the round-2 escapes: a symlinked fixture given to run, a symlinked work/ given to clean and gc, glob characters given to fetch, and a fixture root that is a symlink.
…th resolver fixture_path turns an argument into the real path of a fixture inside data/ or refuses it: plain names only, so no glob reaches the Hugging Face pattern, and the resolved path must be a fixture path. fetch, run and doctor act only on that path. run copies with cp -RL, so its copy holds no symlink. Every chmod runs through find -P ! -type l, which never follows a symlink. fetch needs the env only to download.
…and drop their comments
…hostile data roots Unenterable data, data at the top of the tree, data linked to a copy, a leading // spelling, a verify-runs link out of data/, hard links to outside files, and unlockable roots in doctor. A sweep runs every command against a sacrificial tree and compares it before and after. download_from_hf fetches only the archive named exactly. Refs #288
real_path fails on any failed cd and drops a leading //. Every command that touches data/ first requires data to resolve to <main>/data and verify-runs to be a directory in it, compares paths by component, and chmods only through chmod_below_data, which skips symlinks and files with a second hard link. doctor lists roots fetch cannot lock. download_from_hf fetches exactly <relpath>.zip and no longer deletes <output>/.cache; fetch points its HF cache into data/verify-runs. Refs #288
…ed HF cache download_from_hf deleted the dataset's folder from the active Hugging Face cache after every download. snapshot_download with local_dir set to a staging dir beside the target touches no shared cache, and the archive path is spelled once. Refs #288
…ctor list printer scrub requires a resolved run dir below verify-runs before rm -rf, the lint now covers rm and mv too, fetch no longer redirects HF_HUB_CACHE, and the tests share one env-and-driver setup. Refs #288
…tor exits 2 on a wrong data root Plants a symlink, a real dir, and real dirs with symlinked work, .pid or evidence at the next run names, and snapshots the whole fake tree. The sweep test now gives the fake worktree a pyprocar/ dir, so worktree-setup reaches its mkdir and ln lines.
run named its dir <timestamp>-<name> and wrote into it with mkdir -p, a .pid redirect and cp -RL without checking it was new, so an entry planted at that name (a symlink, or a run dir whose work, .pid or evidence links out of data/) made run overwrite and then delete files outside data/ before chmod_below_data refused. A plain mkdir now creates the dir and fails on any existing entry, the dir must resolve strictly below verify-runs, its subdirs use plain mkdir, and noclobber guards every redirect. This also stops two same-second runs with one name from sharing a dir. doctor exits 2, like the other data commands, when the data root is wrong. The lint now lists every line of verify.sh that writes, links, moves or deletes, not only chmod and rm.
…he run-dir rule and how to refresh a fixture
…t one stamped run in the test A date stub fixes the run name, so the planted-run test plants one entry instead of guessing the next 31 seconds, and the post-driver writes into evidence keep their old behaviour.
…us lines under 100 columns CI runs in an ASCII locale, where mkdir's quoted error message failed to decode. Two long verify.sh lines are split so the census lint pins them without implicit string concatenation.
…locked PROCAR can be edited
…hrough pixi run --locked in the main checkout
…i --locked, and stop doctor's git status rewriting the index
… edits, install, tar, unzip and a command held in a variable
…xture by its directories only
…sage, and mark the census self-test as guarding
…able_copy The check reads tests/ and flags shutil.copy, copy2, copyfile, copytree, copymode, copystat, os.link, Path.hardlink_to, and cp or rsync through subprocess or os.system, when an argument holds a path built from DATA_DIR or the repo root joined with data. It follows names, attributes, defaults, call-site arguments, return values, pytest fixtures and parametrize values, across modules. It flags #304's Elk test and the auto bandsplot fixture. The module-name helpers move from test_user_warnings into tests/utils so both checks share them.
…ixtures stay usable #304's Elk test copied a 0555 bands dir with copytree, then copied ELMIREP.OUT into it, which raised PermissionError on locked fixtures. writable_copy now also copies a single file.
…ure path, so its exemption is exercised
…BYTECODE=1 py ran pixi run without it, so doctor, a download and a run driver rewrote stale __pycache__ files in the main checkout (measured in an overlay sandbox). The env stub now logs the variable, and the harness clears it so the caller's env cannot satisfy the test.
Dropping GIT_OPTIONAL_LOCKS=0 from doctor's git status survived the suite. The test makes the index stale, runs doctor, and checks the index bytes; a plain git status afterwards must rewrite it, so the setup is proven stale.
…"$w", and ignores [[ ]] and arithmetic comparisons A > inside [[ ]], (( )) or $(( )) is a comparison, not a redirect.
…s, and add the fixture-copy rule The rule table names tests/test_fixture_copies.py as the enforcer of copying fixtures only through tests.utils.writable_copy.
… and map call arguments to positional parameters only
… untyped for from_code, and wrap two long self-test lines
49a7c2f to
5222718
Compare
…ery python verify.sh starts
…hardlink first, fixture registration in the bind loop) and document writable_copy's file form
…ports line under 100 columns
|
VERDICT: PASS+NOTES Reviewed SHA: bf6b7a1 (round 5). origin/dev is still 499b1bc, and it is an ancestor of the head, so the head is the merged result. Nothing needed merging. Safety. Every command that ran verify.sh, its tests, its mutants or the data suite ran inside Acceptance set (#288)
Round-5 claims
Gates at bf6b7a1 (= merged), in the overlay sandbox through
|
Why
Fixtures in
data/are writable, so only the test audit hook stops a stray write, and agent briefs repeat the ban by hand (#288). This PR makesverify.sh fetchlock each fixture read-only, and it keeps every write, chmod and delete thatverify.shruns inside the main checkout'sdata/.Round 1 of this PR failed review on that second promise. A verifier pointed a scratch
datasymlink at/and ranfetch data/homeandfetch data/tmp. Two bugs let the round-1 code runchmod a-won about 2 million of the operator's files, since restored:real_pathignored a failedcd -P. Whendataexists but cannot be entered,real_path dataprinted an empty string, so"$data"/?*became/?*andfixture_path data/homereturned/home.dataresolves to/, bash printscd -P data/tmp && pwdas//tmp, and"$data"/?*became//?*, which matches.Round 2 failed on a write rather than a chmod.
runnamed its dir$RUNS/<timestamp>-<name>and wrote into it withmkdir -p,echo $$ >.pidandcp -RLwithout checking that the dir was new. An entry planted at that name maderunwrite outsidedata/beforechmod_below_datarefused (exit 2). The entry could be a symlink, or a real run dir whosework,.pidorevidencelinks out. The EXIT trap then deleted the outside.pid. origin/dev has the same defect, and thererunexits 0.What changed
The data root is now one fixed place, checked before any command touches
data/:require_data(verify.sh) resolvesdataand accepts it only when it equals<main checkout>/data, withdata/verify-runsa real directory in it. In the main checkout that means the directory itself; in a linked worktree, the symlinkworktree-setupmakes.doctor,fetch,run,cleanandgcexit 2 otherwise. An empty root,/,//, an unenterable directory or a copy elsewhere can never equal it.real_pathreturns non-zero on any failedcd, andphysical_dirstrips a leading//.is_belowcompares canonical paths component by component, and needs the child strictly below the parent.fixture_pathuses it against the data root,cleanagainstverify-runs.chmod_below_data, and everyrm -rfthroughscrub. Both first callrequire_resolved_below, which refuses a path that is not already resolved and strictly below the data root (scrub: belowverify-runs). The chmod never follows a symlink and skips files with a second hard link (D).fetchnames each skipped file on stderr, anddoctorkeeps counting it as writable, so the operator sees it.download_from_hffetches exactly<relpath>.zipwithsnapshot_download(local_dir=<staging dir beside the target>)and raisesFileNotFoundErrorwhen the dataset has no such archive, sofetch data/cdownloads nothing (C). This also removes two deletes outsidedata/that the helper ran:rmtreeof the dataset's folder in the active Hugging Face cache, andrmtree <output_path>/.cache, which forfetchwas the repo root's.cache.clean,runandgcuse the resolvedverify-runs, so averify-runssymlink out ofdata/is refused (E).doctorlists every root thatfetchrefuses to lock (a dot name, a space, a symlink out ofdata/) underunlockable fixture roots(F).runcreates its run dir with a plainmkdir "$run", which fails on any existing entry, including a dangling symlink, and then exits 2. The dir must resolve strictly belowverify-runs(require_resolved_below). Its subdirs use plainmkdir,.pidis written with noclobber, and the trap that removes.pidis set only after this process made the dir. Every later write goes into that new dir. This also stops two runs with one name in the same second from sharing a dir: the second is refused and can be rerun (round 3, finding A).doctorexits 2 like the other data commands when the data root is wrong. Before, it printed the refusal and exited 0.data/before and after. A census lint lists every line ofverify.shthat writes, links, moves, deletes or changes modes, and fails when one is added or changed. It matcheschmod,chown,chgrp,chattr,setfacl,install(any form),rm,rmdir,unlink,mv,shred,cp,ln,mkdir,touch,tee,truncate,dd,rsync,mktemp,mkfifo,mknod,tar,unzip,sed -i,perl -i,find -delete, the download call, a command held in a variable ($w f,"$@","${cmd[@]}",if "$w"), and every redirect to a file except/dev/null:>,>>,>|,<>,N>,&>,>&fileandexec N>(G, widened in round 4).test_the_census_flags_each_way_a_line_can_writeappends each form toverify.shand checks that the census grows by that line, and that2>/dev/null,>&2,2>&1,exec 3>&-and a>inside quotes do not count. Python writes insidepy -ccode and git writes are outside the lint; the sweep test is the behavioural backstop.verify.shwrites: only below the data root, plus the writes SKILL.md lists outside it (round 4, finding B).doctor, a download infetchand arundriver importpyprocar, which appends to<repo>/pyprocar.log. A download writes the dataset'srefs/mainin the shared cache even withlocal_dir(_snapshot_download.py:369-376at 1.2.3), andhf_xetwrites a log to$HF_HOME/xet/logs/and createschunk-cache/andstaging/under$HF_HOME/xet/<endpoint>/. In the main checkoutpixi run --lockedcan install or update the env but never rewritespixi.lock.worktree-setupandexecwrite their own worktree's_version.py,.tmpanddatalink, andworktree-setup'sgit diffcan refresh that worktree's index. SKILL.md says the same, names the leading.and-that plain names exclude, and shows how the operator unlocks a fixture to refresh it. SKILL.md replaces the rawchmod -R u+wbeforeworktree removewithfind -P .tmp -type d -exec chmod u+w {} +on named copies, neverdata(H).tests.utils.writable_copy, which now also copies a single file, andtests/test_fixture_copies.pyenforces it (see Round 5).verify.shexportsPYTHONDONTWRITEBYTECODE=1. SKILL.md and CLAUDE.md list matplotlib's config dir and font cache among the writes outsidedata/.fetchruns the download withHF_XET_LOG_DIR_DISABLE_CLEANUP=1, because hf_xet's log cleanup otherwise deletes older logs in$HF_HOME/xet/logs/. In the main checkoutpyrunspixi run -q --locked -e default, asexecalready did, so it never re-locks.doctorrunsgit statuswithGIT_OPTIONAL_LOCKS=0, so it no longer rewrites the git index (B).Census
Every line of
verify.shthat writes, overwrites, links, moves, deletes or changes modes.test_verify_sh_writes_only_on_the_lines_its_census_reviewedlists the same lines. "Resolved" meansreal_pathgives the path back unchanged: no symlink in it, no.or...PATH=... "$@"shared_env(exec, andpyin a worktree)execis the CI-gate runner and writes what that tool writes[ -x "$SHARED_ENV/python" ] || { echo "...run 'pixi install -e dev'..." }pixi install; it prints and exits 2find -P "$2" ... -exec chmod "$1" {} +chmod_below_data(fetch, run)$2and belowrequire_resolved_below "$2" "$DATA"first;find -Pnever follows a link; skips links and files with a second hard linkrm -rf "$1/work" "$1/.start" "$1/.pid"scrub(clean, gc)require_resolved_below "$1" "$RUNS"first;rm -rfremoves a symlink entry without following itpyprocar.download_from_hf(...)data/examples/<cat>/<name>fixture_pathputs it strictly below the resolved data root; stages in a new temp dir beside it; the documented exceptions are therefs/mainfile, hf_xet's log and caches (cleanup off) andpyprocar.logmkdir -p "$RUNS"<data>/verify-runsrequire_datachecked that it resolves to exactly$DATA/verify-runsmkdir "$run"$RUNS/<stamp>-<name>mkdirfails on any existing entry;<name>is one plain namemkdir "$run/evidence" "$run/work" "$run/work/tmp"require_resolved_below "$run" "$RUNS"(set -C; echo $$ >"$run/.pid")$run/.pidtrap 'rm -f "$run/.pid"' EXIT$run/.pid$run;rm -fremoves a link without following itcp -RL ... "$fixture" "$run/work/calc"$run/work/calccalcdoes not exist yet, so every destination file is new; the source is a resolved fixturecp "$driver" "$run/evidence/driver.py"touch "$run/.start"py ... >"$run/evidence/run.log"echo "$code" >"$run/evidence/exit_code"verify.shcreates this file earlier(cd calc && find ...) >"$run/evidence/side_effects.txt"{ echo "missing $MAIN/pyprocar/_version.py; run 'pixi install -e dev'..." }pixi install; it prints and exits 2cp "$MAIN/pyprocar/_version.py" pyprocar/_version.py_version.py(pre-existing, see Scope)mkdir -p "$MAIN/data" .tmpdata/, this worktree's.tmpmkdir -pcreates nothing when the path existsln -sfn "$MAIN/data" datadatalinkdatais a real directory;-nreplaces the link itselfmkdir -p .tmp.tmpThe library side of
fetch(download_test_data):mkdir(parents=True, exist_ok=True)of the category dir,TemporaryDirectory(dir=<category dir>),replaceof the archive into the category dir, extraction into the new fixture dir, andunlinkof the archive. All of these are inside the target's category dir. The library no longer runsrmtree.Scope
data/stays writable until the operator locks it once after merge. Run the lock from a checkout at mergeddev(a linked worktree afterverify.sh worktree-setup, or the main checkout once it is at mergeddev), because the main checkout today is at an older commit whosefetchlocks nothing. Run it from that checkout's root, because the shell expandsdata/examples/*/*from the working directory:.claude/skills/verify-pyprocar/scripts/verify.sh fetch data/examples/*/* data/codes data/io data/issues.verify.sh doctorthen printswritable fixture paths: 0. To refresh one fixture later, unlock its directories from the main checkout's root withfind -P data/examples/<category>/<name> -type d -exec chmod u+w {} +, remove it, and fetch it again.rm -rfneeds write permission on directories only, so this never unlocks a file whose second hard link may be outsidedata/(round 4, finding D). SKILL.md shows the same three commands.download_test_data(force=True)on a locked fixture raisesPermissionErroruntil it is unlocked.data/must be a real directory now. Adatasymlink in the main checkout (say, to another disk) is refused, because it cannot equal<main>/data. SKILL.md says so.tests/scripts/download_test_data.pycallsdownload_test_data("data")andscripts/download.pyshows a category prefix in its epilog. Both now raiseFileNotFoundError. Both already crashed ondev: the first on the missingdata.zipafter copying every archive, the second onstr / strbecause it passesoutput_pathas a string. They are left for a follow-up.cp --reflink=autoinrunis GNU-only, as before. CI runs on Ubuntu only.worktree-setup'scpwrites through a symlinked worktreepyprocar/_version.py; a newline in the main checkout's path breaksrun'sduline (exit 1);doctortruncates an unlockable name that holds a newline; a library caller who passes glob characters todownload_from_hfdownloads every matching archive into staging and then raises.Blast Radius
Skill scripts,
download_from_hf, two test helpers, the test modules that copy fixtures, and a new static check ontests/.download_from_hfnow raisesFileNotFoundErrorfor a relpath that is not exactly one archive in the dataset; before, a category path such asdata/examples/bandsdownloaded every archive in it and then crashed on the missingbands.zip. After the operator locksdata/, code that writes into a fixture fails withPermissionErrorinstead of changing shared data.Verification
Round 5 (head bf6b7a1 on dev 499b1bc)
Safety as in rounds 3 and 4. Every run of
verify.sh, its tests and its mutants usedbwrap --ro-bind / / --bind <fake root> --tmpfs /tmp --unshare-net, with the fake root under the worktree's.tmp. Gates and the lint ran in the overlay sandbox, where worktree writes go to an invisible tmpfs. The shareddata/was only read. It still has 2230 writable entries undercodes,examples,ioandissues.Finding (blocking): #304's Elk test on locked fixtures. The branch is rebased onto dev 499b1bc (#304 and #305). In
test_real_elk_6_3_bands_beside_a_task_10_elmirep_keep_their_l_m_names(tests/pyprocar/io/elk/test_parser.py),copytreekept the 0555 dir mode, so the nextshutil.copyofELMIREP.OUTraisedPermissionError. Both copies now go throughwritable_copy, which also copies a single file now (shutil.copyfile, so the new file gets the default mode).test_auto_bandsplot.py'sauto_calcfixture also useswritable_copy. It passed when locked, because it writes nothing into the copy, but the new rule allows no raw fixture copy.The lesson as a check:
tests/test_fixture_copies.py. This is the second merged PR that added a raw fixture copy, and CI has nodata/, so the check reads the source. It fails onshutil.copy,copy2,copyfile,copytree,copymode,copystat,os.link,Path.hardlink_to, orcporrsyncthroughsubprocessoros.system, anywhere intests/outsidewritable_copy, when an argument holds a fixture path. A fixture path is built fromDATA_DIR, or from__file__orROOT_DIRjoined with"data". The check follows it through names (by scope), attributes, defaults, call arguments, return values, pytest fixtures (conftest included),parametrizevalues and imports across modules. The result ofwritable_copyis not a fixture path. Each message ends withcopy it with tests.utils.writable_copy. The CLAUDE.md rule table has the rule and the enforcer.tests/: 0 hits. At the first lint commit (2beef1c, before the fix): 3 hits,elk/test_parser.py:862(copytree),:867(copy) andtest_auto_bandsplot.py:52.tests/utils:f93c8bdf:tests/pyprocar/io/elk/test_parser.pygives 2 hits (lines 862 and 867),60fc0c02:tests/pyprocar/scripts/test_filtered_procar.pygives 1 hit (line 43,shutil.copy(src / name, ...)in_copy_calc, reached through thesrc=CALCdefault). Both files at the head give 0.rmtree, a copy out of awritable_copyresult, a"data"dir under a temp path, a shadowing parameter, andwritable_copy's own body. It also holds verbatim excerpts of both historical sites. Fourteen mutants of the check are all killed: no helper exemption,writable_copynot a sanitizer, no repo-data seed, and no defaults, call arguments, returns, cross-module exports, fixtures, attributes, shell,hardlink_to,parametrizeor loops, and scope lookup that ignores shadowing."data"path component is not a seed, becausetest_verify_fixtures.pybuilds sacrificialdata/trees undertmp_path. A seed of that kind flagged three of its lines.Locked run of the two sites (private copy locked by this PR's
fetch,-kon the Elk andauto_calctests): at 2beef1c, 1 failed (PermissionError: ... bands/ELMIREP.OUT), 2 passed. At 5222718, 3 passed.Full
-m data -n 4on locked fixtures, at bf6b7a1. This PR'sfetchlocked a private reflink copy ofdata/{codes,examples,io,issues}in a fake main checkout. Only the 9 container dirs stayed writable. A linked worktree at the head, whosedatalinks to that copy, ran the suite in the overlay sandbox: 411 passed, 1 skipped, 0 failed.Round-4 notes, fixed.
__pycache__: measured in a sandbox with an overlay on the main checkout. After atouchofpyprocar/core/ebs.py,pixi run -q --locked -e default python -c 'import pyprocar.core.ebs'rewroteebs.cpython-312.pyc. WithPYTHONDONTWRITEBYTECODE=1it left the file alone.verify.shnow exports the variable once, so every python it starts writes no bytecode in either checkout. The env stub logs it, and the harness clears it from the caller's env soverify.sh execcannot satisfy the test. Removing the export failstest_fetch_downloads_a_missing_fixture_by_its_literal_name_then_locks_itin both layouts.doctorin the sandbox with an emptyHOMEcreated~/.config/matplotliband~/.cache/matplotlib/fontlist-v390.json, and nothing else. SKILL.md and the CLAUDE.md row list both.GIT_OPTIONAL_LOCKS=0:test_doctor_leaves_a_stale_git_index_alonemakes the index stat-dirty, runsdoctor, and checks that the index bytes are unchanged. A plaingit statusafterwards must change them, which proves the setup is stale. Dropping the variable fails the test in both layouts (it passed the whole module in round 4)."${cmd[@]}" "$f",X=1 "${@}",if "$w"; thenandwhile "$w"; do. A>inside[[ ]],(( ))or$(( ))is not a redirect, and a redirect after[[ ]]still counts. With round 4's regexes, the new harmless line[[ "$a" > "$b" ]] && echo $((a > b)); (( a > b ))is flagged, so every case fails. Without that line, exactly the 4 new forms fail.Gates at bf6b7a1, all exit 0:
-n 8: 2238 passed, 4 skipped.ruff_new_violations.pycheck and format against origin/dev: ok, 15 files.red_green.py origin/dev: ok.test_doctor_leaves_a_stale_git_index_aloneand the download test fail on base. The two fixture-copy tests are markedguards_existing_behaviour, because red-green's base run takestests/from the branch.Round 4 (head 49a7c2f on dev 4de5cc0)
Safety as in round 3. Every run of
verify.sh, its tests and its mutants usedbwrap --ro-bind / / --bind <fake root> --tmpfs /tmp --unshare-netwith the fake root under the worktree's.tmp. Gates used the overlay sandbox. The shareddata/was only read: it still has 2230 writable entries undercodes,examples,ioandissues.Finding A (blocking):
test_filtered_procar.pyon locked fixtures._copy_calccopied withshutil.copy, which keeps the 0444 mode, and the test then writescalc/PROCAR. It now useswritable_copyfromtests.utilswith anignorethat keeps the four VASP files. A repo-wide grep forshutil.copy,copy2,copytree,copyfile,cp,os.linkandwrite_bytes(read_bytes())finds no other fixture-copy site in the tests.write_bytesmakes a new file with the default mode,tests/pyprocar/pyposcar/run.pyis a script that pytest does not collect, anddata_utils.copy_fileshas no caller.The full data suite ran on a private reflink copy of
data/{codes,examples,io,issues}. This PR'sfetchlocked the copy in a fake main checkout (fetch data/examples/*/* data/codes data/io data/issues). That left only the 9 container dirs writable. A linked worktree whosedatalinks to that copy then ranpytest tests -m data -n 4in an overlay sandbox:-m data,-n 4, locked copytest_selecting_one_orbital_by_name_raises_whichever_name_the_header_uses[x2-y2-dx2, dx2-x2-y2, dx2-dx2, x2-y2-x2-y2],PermissionError: [Errno 13] ... calc/PROCAR), 399 passed, 1 skippedFinding B: writes outside
data/. These now appear in SKILL.md and the CLAUDE.md row, as listed under What changed. Each change was checked live:data/examples/bands/atomic_levelsran in a network sandbox withHF_HOMEin the fake root and an old planted logxet_20250101T000000000-0400_999999.log. Without the variable, hf_xet deleted the planted log. WithHF_XET_LOG_DIR_DISABLE_CLEANUP=1, it kept it. Both runs createdhub/.../refs,xet/logs/<new log>, andxet/<endpoint>/chunk-cacheandstaging.git statusrewrote.git/index, andGIT_OPTIONAL_LOCKS=0 git statusleft it alone.git diffrewrote the index even with the variable, so SKILL.md names it forworktree-setup.test_fetch_downloads_a_missing_fixture_by_its_literal_name_then_locks_itasserts that the stub env seesHF_XET_LOG_DIR_DISABLE_CLEANUP=1, and that in the main checkout it getspixi run -q --locked -e. The test fails on the round-3 script (cleanup=unset) and passes at head.Finding C: census gaps. See What changed. The new self-test covers 17 forms. Eleven mutants that insert one of the new forms into
fetchare all killed:2>,1>,&>,>&file,exec 3>,sed -i,perl -pi, plaininstall,tar -x,unzip, andw=touch; $w.Finding D: unlock. In the fake-root sandbox, a fixture held a hard link to an outside file with mode 0444, and
fetchlocked it. The round-3 unlock (! -type l) changed the outside file to-rw-r--r--. The new unlock (-type d) left it at-r--r--r--. In both casesrm -rfexited 0 and removed the fixture.Finding E: new tests.
test_gc_keeps_the_work_of_a_run_in_progress_and_removes_a_finished_oneplants two old runs. One has a live pid (pytest's own) and one a pid that has exited. The test kills thelive()-always-false mutant. It passes on dev's script, so it is markedguards_existing_behaviour.test_download_from_hf_stages_beside_the_fixture_not_in_the_system_tmppointstempfile.tempdirat a missing dir and asserts that the hub got a staging dir in<out>/data. It kills theTemporaryDirectory()-without-dir=mutant.Mutants: 59 in all, 52 killed. The 7 survivors are the same 7 as in round 3: the
require_resolved_belowandreal_path/physical_dir/is_belowdefence-in-depth checks. Only a concurrent swap insidedata/reaches them, and the root has scoped that out.Gates at 49a7c2f, all exit 0:
-n 8: 2178 passed, 4 skipped.ruff_new_violations.pycheck and format against origin/dev: ok, 10 files.red_green.py origin/dev: ok.-m data -n 4on the shareddata/: 403 passed, 1 skipped.Round 3 (head a88f1f5 on dev 4de5cc0)
Every execution of
verify.sh, its tests and its mutants ran insidebwrap --ro-bind / / --bind <fake root> <fake root> --tmpfs /tmp --unshare-net. The fake root is one directory under the worktree's.tmp. Everything else, including/home, the worktree and the shareddata/, was read-only. A probe inside that sandbox could not touch or chmod anything under/home/lllang. Each test's "outside" targets are directories inside its own fake root. No link points at/or at any path outside that root. The CI-equivalent gates ran in a second sandbox, with the filesystem read-only and the worktree an overlay whose writes go to an invisible tmpfs. That sandbox was needed because the library writespyprocar.loginto the working directory.Finding A, replayed with the verifier's smallest layout (
repro.sh): a fake main checkout withdata/examples/bands/x/{PROCAR,sub/OUTCAR}, and symlinks atdata/verify-runs/<stamp>-probefor the next 12 seconds, each pointing to anoutside/<stamp>/that holds.pidandwork/calc/x/PROCAR. Thenverify.sh run probe data/examples/bands/x driver.py:New tests:
test_run_refuses_a_run_dir_that_already_exists_and_changes_nothing, 5 layouts x 2 checkouts: a symlinked run dir, a real run dir, and a real run dir whosework,.pidorevidencelinks out. Adatestub fixes the run name, and the whole fake tree (mode, type, link target, sha256) is compared before and afterwork/calc/x/PROCARoverwritten, or exit 0_refuses_everythingnow expectsdoctorto exit 2 (8 cases)test_no_command_changes_anything_outside_datagives the fake worktree apyprocar/dir, soworktree-setupexits 0 and runs itscp,mkdir -pandln -sfnlines. The test asserts the copied version, thedatalink target and.tmptest_verify_sh_writes_only_on_the_lines_its_census_reviewedThe whole module: round-2 head 19 failed, 126 passed; origin/dev 105 failed, 40 passed; head 145 passed, 2 skipped. It also passes with
LC_ALL=C, the locale CI runs in.Mutants (47 single, written into the fake root and passed through
VERIFY_SH_UNDER_TEST): 40 killed. The new ones:mkdir -p "$run"(9 tests), the round-2 run block restored (11), the trap set before the run dir exists (5), subdirs made withmkdir -p(1), noclobber dropped from the.pidwrite (1, census lint only), doctor exiting 0 on a wrong root (8). Also killed: acp, a>redirect,ln -sf,find -deleteandteeadded infetch(1 to 5 each).chmod_below_datawithout its guard" still survives alone. On the fixed head no caller can reachchmod_below_datawith an unresolved path, becauserunnow refuses the planted layout atmkdir "$run", before the copy. Paired with the round-2 run block it dies (11 tests), and paired withmkdir -p "$run"it dies (9 tests). That pairing is the verifier's scenario.cdmutants, twois_belowmutants,scrubwithout its guard (all as in round 2), and the newrequire_resolved_below "$run" "$RUNS". A plainmkdirthat succeeds always makes a real directory in the resolvedverify-runs, so this check fires only ifverify-runsis swapped during the run.Gates at a88f1f5, each through
verify.sh execin the overlay sandbox:ruff_new_violations.pycheck and format against origin/dev: ok (9 files).red_green.py origin/dev: ok, 29 added or changed tests.-m datafortest_one_call_functions.py,test_unfold.pyandtest_download_examples.py: 57 passed. The shareddata/was read-only in the sandbox.Rounds 1 and 2
All verify.sh runs below happen inside pytest's fake roots under the worktree's
.tmp. Each fake root holds its own git repo, anoutside/tree (home/,tmp/runs/, an unenterableunenterable/) and a read-onlyvictim/, and every symlink or hard link a test makes points into that tree. No test or mutant points at a real system path. The unenterable tests first assert that/pyprocar-288-absent,/codes,/examplesand/verify-runsdo not exist, because round-1 code would have mapped those names to/.Findings, each with its test (both layouts, data in the repo and a linked worktree):
test_an_unenterable_data_root_refuses_every_command(data dir mode 000; data linked to a mode-000 dir)fetchexit 1 after a download attempttest_a_data_root_at_the_top_of_the_tree_refuses_every_command(data linked to the fake root's top, which stands in for/)fetch data/outsidelocksoutside/test_a_worktree_data_link_spelled_with_a_leading_double_slash_still_locks//normalisation is removedtest_a_data_link_to_a_copy_outside_the_main_checkout_is_refusedtest_download_from_hf_refuses_a_prefix_without_downloading,..._fetches_only_the_archive_named_exactlydata/cdownloadscodes.zipandcodes-extra.zip;data/codesalso pullscodes-extra.zip, deletesout/.cacheand deletes the dataset folder in the shared cachetest_fetch_leaves_a_hard_link_to_a_file_outside_data_writableoutside/home/flockedtest_a_verify_runs_link_out_of_data_is_never_followedcleandeletesoutside/tmp/runs/x/workclean,run,gcexit 2test_doctor_lists_the_fixture_roots_that_fetch_refuses_to_lock.hidden,escape,dangling,loop,runs,self,two wordstest_no_command_changes_anything_outside_data(every subcommand, every hostile argument) andtest_verify_sh_changes_modes_only_in_chmod_below_dataAgainst round-1 head
6587c87d, 17 tests oftests/test_verify_fixtures.pyfail and 118 pass; against origin/dev'sverify.sh, 95 fail; against the final head, 135 pass and 2 skip (the//test applies to a worktree only) (.../against.py <rev>writes that revision'sverify.shto.tmpand points the tests at it throughVERIFY_SH_UNDER_TEST, so no script is rewritten in place).red_green.py origin/dev: ok.Mutants, each written to
.tmp/mutants/<n>/verify.shand passed throughVERIFY_SH_UNDER_TEST://kept, data root not checked,is_belowadmitting the root itself,verify-runslink followed, clean through a symlinkedwork, chmod following hard links, doctor hiding unlockable roots, a rawchmod -R, anrmand aninstall -madded beside a guarded call infetch.real_pathignoring a failedcd,physical_dirignoring a failedcd,is_belowby glob prefix,is_belowadmitting an empty parent, andchmod_below_dataorscrubwithoutrequire_resolved_below. A failedcdyields a path that cannot equal<main>/dataand is never strictly below a non-empty canonical root, and a glob prefix equals the component test on canonical paths. Paired with the layer they back up, the guards show:clean any pathalone fails 2 tests, withscrub's guard removed too it fails 8;no check of the real pathfails 20, withchmod_below_data's guard removed too 24.Real tool:
download_from_hfat huggingface_hub 1.2.3 into a scratch root under the worktree's.tmp, withHF_HUB_CACHEpointed at an empty scratch dir.data/examples/bands/atomic(a prefix) raiseddata/examples/bands/atomic.zip is not in the lllangWV/pyprocar_test_data datasetafter fetching 0 files.data/examples/bands/atomic_levelsgave 15 files, no zip and no staging dir left. A pre-existingout/.cache/keepsurvived, and the scratch HF cache held onlyrefs/main.Gates at 70db097 on dev 5d6ae01, from the worktree with
verify.sh exec: non-data pytest (2092 passed, 4 skipped), basedpyright (0 errors, 0 warnings),ruff_new_violations.pycheck and format (9 changed files, ok), the F821/NPY201 check,red_green.py origin/dev(ok; every added test fails on base except the threewritable_copytests, which carry the guard mark), and the-m datamodules this PR edits (57 passed, read-only use of the shareddata/).Refs #288 (closes it once the operator has locked the shared
data/)