Skip to content

Add an opt-in persistent pnpm store for self-hosted runners - #14

Merged
Harold Hunt (huntharo) merged 3 commits into
mainfrom
feat/persistent-pnpm-store
Oct 28, 2026
Merged

Harold Hunt (huntharo) merged 3 commits into
mainfrom
feat/persistent-pnpm-store

Conversation

@huntharo

Copy link
Copy Markdown
Contributor

Summary

Adds an opt-in persistent pnpm store for self-hosted runners whose disk survives between jobs (e.g. the PwrSuiteLab macOS Tart guests). Checkout's git clean -ffdx deletes the in-workspace .pnpm-store every job, so each job restores the same ~750 MB archive only for pnpm install to finish in ~4.5 s. This absorbs the behavior of PwrAgent's scripts/ci/macos-e2e-guest-store.sh (pwrdrvr/PwrAgent#2892), where warm lanes installed in 5–16 s instead of 38–72 s.

Default behavior is unchanged for callers that do not pass the new inputs.

Design decisions

Point Decision
Inputs persistent-store-dir (empty = off; leading ~/ expanded because with: can't expand $HOME) and persistent-store-max-age-days (default 14). Corepack and Electron caches follow the same root; no extra inputs. A caller-set COREPACK_HOME is still preserved.
Layout <root>/pnpm-store, <root>/corepack, <root>/electron, <root>/npm (same as the PwrAgent script), plus the marker <root>/.configure-nodejs-seeded. Nothing else under the root is touched.
Path guard Default mode keeps the workspace-boundary assertion. Persistent mode requires an absolute root that is not a filesystem root, not $HOME or an ancestor of it, and not the workspace, inside it, or an ancestor of it (an ancestor could name the workspace as a managed entry). Checked on both the given path and its real path. Every delete goes through one helper that only removes named, direct children of the root and refuses symlinks that escape it.
Warm detection Marker holds epoch seconds and is written last (temp file + rename). Missing, unreadable, future-dated, or expired means cold, and the store, Corepack home, and Electron caches are dropped, marker first, so an interrupted drop is never trusted.
Cold path Runs exactly like default mode (restore into the workspace, install, save on a miss), so the cache entry stays shared with default-mode jobs. Then moves the store/caches under the root (rename via a temporary name; falls back to copy on EXDEV; retries transient Windows locks), writes the marker, and re-exports PNPM_CONFIG_STORE_DIR / npm_config_store_dir / Electron vars to the moved paths. I checked that pnpm 10 re-links cleanly in CI if a later step re-runs pnpm install after the store moved.
Warm path Skips actions/cache/restore, the Corepack actions/cache, and the dependency save. Runs setup-node, Corepack activation with the existing explicit pnpm --version vs packageManager check, then pnpm install --frozen-lockfile --store-dir <root>/pnpm-store. The Node-major mismatch guard still runs.
Failure A failed warm install removes the marker before rethrowing, so the next job re-seeds.
Outputs New persistent-store-warm (true/false). pnpm-store-path and install-command report the persistent paths when used. The job summary notes warm/cold and the seed time.
Concurrency Documented rather than locked: pnpm's store is safe for concurrent installs, but seeding/expiry are not coordinated across runners sharing a disk. Recommend one root per runner (e.g. .../${{ runner.name }}) and per repository.
Combinations Rejected: lookup-only: true, dependency-cache: node-modules, npm/Yarn, and pnpm without an exact packageManager pin.

Tests

  • test/persistent-store.test.mjs (new, 21 tests): path guard (relative, /, ~, home and ancestors, workspace / inside / ancestor, symlinked roots), warm / missing / unreadable / future / expired / boundary markers, drop removes only managed entries, seeding moves everything and writes the marker last (asserted on every rename), failed seed leaves no marker, EXDEV copy fallback, Windows retry, invalidation, --store-dir rewrite, Corepack home selection.
  • test/action-wiring.test.mjs: default mode leaves every pre-existing step condition unchanged; warm skips restore / Corepack cache / save; cold still saves and seeds; executes the inline check, prepare, and install scripts (warm install failure drops the marker; cold/default failures do not).
  • CI: new Persistent pnpm store job on ubuntu/macos/windows driving the real action: cold seed from a miss → simulated checkout clean with Corepack offline → warm run (no restore, no save) → broken lockfile fails the warm install and drops the marker → re-seed from the saved cache → expired marker re-seeds → root inside the workspace is rejected.

npm test: 135 passing locally.

Release plan

Minor bump: v1.7.0. After merge, tag the merge commit: git tag -a v1.7.0 <merge-sha> -m "v1.7.0" && git push origin v1.7.0. The release workflow moves v1.

Follow-up (separate PR in PwrAgent)

Switch desktop-e2e-macos to persistent-store-dir: ~/Library/Caches/pwragent-ci and delete scripts/ci/macos-e2e-guest-store.sh and its test. The directory layout matches, but the marker name differs (.configure-nodejs-seeded vs pnpm-store/.pwragent-seeded), so each guest re-seeds once on its first job after the switch unless the PR also renames the old marker.

Persistent self-hosted runners keep their disk between jobs, but checkout's
`git clean -ffdx` deletes the in-workspace pnpm store every job, so each job
restores the same dependency archive only for `pnpm install` to finish in
seconds.

`persistent-store-dir` keeps the pnpm store, Corepack home, and Electron
caches under a root outside the workspace. A seed marker (epoch seconds,
written last) makes the store warm; a missing, unreadable, future-dated, or
expired marker drops it. Warm jobs skip the dependency cache restore and save
and install with `--store-dir` at the persistent store. Cold jobs run exactly
as in default mode, so cache entries stay shared, then move the store and
caches under the root. A failed warm install removes the marker so the next
job re-seeds.

The root must be absolute and cannot overlap the workspace, the home
directory, or a filesystem root; every delete is limited to named entries
directly under it. Persistent mode is pnpm-only, requires an exact pin, and
rejects lookup-only and dependency-cache: node-modules. Default behavior is
unchanged. New output: persistent-store-warm.
Harold Hunt (huntharo) added a commit to pwrdrvr/PwrAgent that referenced this pull request Oct 26, 2026
Try pwrdrvr/configure-nodejs#14 by branch reference: persistent-store-dir
keeps the pnpm store, Corepack home, and Electron caches on the guest and
skips the dependency cache restore when the store is warm. It replaces
scripts/ci/macos-e2e-guest-store.sh and its test, which are deleted.

The root stays ~/Library/Caches/pwragent-ci, beside desktop-out/. The
action's marker name differs from the script's, so each guest re-seeds
once. Pin the v1.7.0 release SHA before merging.
… reuse helpers

- Treat a marker without its pnpm-store directory as cold, so the job
  re-seeds from the cache archive instead of downloading from the registry.
- Start the floating setup-node timing after the persistent store check, so
  dropping an expired store is not reported as setup-node time.
- Use a fixed temporary marker name that drops clean up.
- Warn when the root is under RUNNER_TEMP, which never survives a job.
- Reuse isPathWithin, findNearestExistingPath, and a shared exact pnpm pin
  pattern instead of local copies; move status messages into the module.
Harold Hunt (huntharo) added a commit to pwrdrvr/PwrAgent that referenced this pull request Oct 26, 2026
Try pwrdrvr/configure-nodejs#14 by branch reference: persistent-store-dir
keeps the pnpm store, Corepack home, and Electron caches on the guest and
skips the dependency cache restore when the store is warm. It replaces
scripts/ci/macos-e2e-guest-store.sh and its test, which are deleted.

The root stays ~/Library/Caches/pwragent-ci, beside desktop-out/. The
action's marker name differs from the script's, so each guest re-seeds
once. Pin the v1.7.0 release SHA before merging.
@huntharo
Harold Hunt (huntharo) merged commit 9bb6e78 into main Oct 28, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant