Skip to content

test(sdp): fail the CI check scripts' walks closed, and drive them for real - #2923

Open
cutbow7 wants to merge 3 commits into
mainfrom
fix/fabricated-tests-sweep-1795118419268185070-522958
Open

cutbow7 wants to merge 3 commits into
mainfrom
fix/fabricated-tests-sweep-1795118419268185070-522958

Conversation

@cutbow7

@cutbow7 cutbow7 commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

check-secretref-serialization, check-pinned-dependency-versions and check-module-boundaries are the CI enforcement behind the SecretRef redaction, exact-version dependency and module boundary rules, but a walk that came back empty passed as clean:

  • a missing scan root was skipped (collectSourceFiles swallowed the stat/readdir error), so a renamed apps/ or packages/ shrank the SecretRef sweep to nothing and it still printed "passed";
  • an empty git ls-files listing passed, so a pathspec drift turned the manifest sweep into a no-op, and a renamed pnpm-workspace.yaml silently dropped the catalog exact-pin rule (the AGENTS.md hard rule for Solana Earn consumption);
  • a declared module whose src/ was renamed away contributed zero imports (listFiles returned []), so its boundary rules silently stopped running.

Each walk now refuses to pass what it cannot see, bigint-buffer declares its root-level CommonJS sources explicitly (sourceDirectory: null) instead of inheriting a src/ that does not exist, and the sibling tests drive the real walks, not only the pure analysis functions on synthetic strings: a fixture repo violation end to end, each fail-closed alarm, a missing catalog as a violation, and the real workspace walk floored (22k+ imports) so a future discovery bug fails the test instead of passing vacuously.

@vercel

vercel Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
sdp-docs Ready Ready Preview Nov 21, 2026 10:49pm UTC
sdp-web Ready Ready Preview Nov 21, 2026 10:49pm UTC
2 Skipped Deployments
Project Deployment Actions Updated
sdp-docs-beta Ignored Ignored Preview Nov 21, 2026 10:49pm UTC
sdp-web-beta Ignored Ignored Preview Nov 21, 2026 10:49pm UTC

Request Review

check-secretref-serialization, check-pinned-dependency-versions and
check-module-boundaries are the CI enforcement behind the SecretRef
redaction, exact-version dependency and module boundary rules, but a
walk that came back empty passed as clean:

- a missing scan root was skipped (collectSourceFiles swallowed the
  stat/readdir error), so a renamed apps/ or packages/ shrank the
  SecretRef sweep to nothing and it still printed "passed";
- an empty git ls-files listing passed, so a pathspec drift turned the
  manifest sweep into a no-op, and a renamed pnpm-workspace.yaml
  silently dropped the catalog exact-pin rule (the AGENTS.md hard rule
  for Solana Earn consumption);
- a declared module whose src/ was renamed away contributed zero
  imports (listFiles returned []), so its boundary rules silently
  stopped running.

Each walk now refuses to pass what it cannot see, bigint-buffer
declares its root-level CommonJS sources explicitly
(sourceDirectory: null) instead of inheriting a src/ that does not
exist, and the sibling tests drive the real walks, not only the pure
analysis functions on synthetic strings: a fixture repo violation end
to end, each fail-closed alarm, a missing catalog as a violation, and
the real workspace walk floored (22k+ imports) so a future discovery
bug fails the test instead of passing vacuously.
@greptile-apps

greptile-apps Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; the latest fixture closes the previous test gap.

Summary

The PR makes three CI checks fail when required scan inputs disappear and adds tests that exercise their file walks.

  • SecretRef scans now stop when a repository root is missing.
  • Dependency pin checks now require manifests and the workspace catalog.
  • Module boundary checks stop when source files disappear.

Reviews (3) · Last reviewed commit: "test: drive the module-boundary walk end..." · Reviewed by Greptile

Comment thread scripts/check-module-boundaries.mjs Resolved

@multipletwigs multipletwigs left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs changes. This PR makes the three CI check scripts (SecretRef, pinned dependencies, module boundaries) fail when their file walk finds nothing, and adds tests that run the real walks.

1. Blocking: the boundary check no longer covers imports into bigint-buffer
For a relative import, the check works out which package the path lands in from that package's src/ folder (scripts/check-module-boundaries.mjs:430-436). This PR records bigint-buffer as having no src/, so that lookup now skips it. As a result, a package that imports ../../bigint-buffer/node.js without declaring the dependency passes on this branch and fails on main. I ran both.

Potential fix: take the package root from path.dirname(module.manifestPath) instead. Every module already has it (:388). This also removes the module.sourceDirectory && guard and the ?.length ?? 0 sort that Greptile flagged. The test fixtures at scripts/check-module-boundaries.test.mjs:44-53 would need a manifestPath.

2. The "no src/" opt-out is more flexible than it needs to be
sourceDirectory: null in the metadata becomes undefined on the module, and ?? "src" allows a custom folder name that no module uses (:108, :389-392, :621).

Potential fix: once 1 is fixed, use a plain flag such as walkSource: false and always walk src. sourceDirectory then stays a string, and the appSourceRoots filter can go.

3. One SecretRef test repeats a CI step
"the real repo's scan roots are readable" (scripts/check-secretref-serialization.test.mjs:227) runs the full check over the repo. The same CI job already does that with pnpm check:secretref-serialization (.github/workflows/ci.yml:526,532).

Potential fix: delete the test.

4. A test comment claims more than the test checks
The end-to-end SecretRef test says it covers the test-file exemption (scripts/check-secretref-serialization.test.mjs:195), but none of its fixture files is a test file.

Potential fix: add a *.test.ts fixture with the same leak and assert that it isn't reported, or drop that part of the comment.

5. The "walk found enough files" check only runs in a test
The pinned-dependency script refuses an empty walk inside the script itself (scripts/check-pinned-dependency-versions.mjs:135-137). The boundary script only has a > 10_000 floor in a test, which needs readModules and collectSourceImports exported just for that test.

Potential fix: have checkModuleBoundaries throw when a module's walk finds zero files, then drop the floor test and both exports. The doc comment on listFiles (:313) also says it is exported for the floor, but the test that uses it is the missing-folder one.

6. Optional: three ways to report a missing folder
readdirSync already throws ENOENT with the path, so neither the existsSync check in listFiles (:315-320) nor the catch-and-rethrow in collectSourceFiles (scripts/check-secretref-serialization.mjs:286-291) is needed.

Potential fix: delete both and let readdirSync throw.

Checked and fine

  • Failure points: none. These are read-only CI scripts with no durable step.
  • The pinned and SecretRef checks pass on this branch against the real repo. CI is green apart from the E2E jobs still running.

…aner walk guards

Review of the fail-closed walk change:

- resolve a relative import's owning package from the manifest's
  directory instead of the module's src/, restoring boundary coverage
  for imports into bigint-buffer (which has no src/ to anchor the
  lookup); the lookup is pinned by a test
- replace the sourceDirectory: null opt-out with a walkSource: false
  flag and always derive src/, so sourceDirectory stays a string and
  the appSourceRoots filter goes away
- fail the boundary check when a walked module finds zero source files
  inside the script itself, and drop the >10k floor test plus the
  readModules/collectSourceImports exports it needed
- let readdirSync's ENOENT report missing directories (the existsSync
  guard in listFiles and the catch-and-rethrow in the SecretRef walk
  were redundant), updating the two fail-closed tests to match
- drop the SecretRef test that re-ran the full check over the real
  repo (CI's check:secretref-serialization step already does), and
  drive the test-file exemption in the end-to-end fixture instead of
  just claiming it
The fail-closed walk had no test proving collectSourceImports actually
finds imports: an extractor that returned nothing would pass every test
and the CI check without ever checking a source import. Drive
checkModuleBoundaries over a repo-shaped fixture (manifests mirrored
from the real workspace, one undeclared relative import injected) and
assert the known violation is reported — no script exports needed.

This branch was successfully deployed

2 active deployments
Preview – sdp-web — e05c124b Deployed Nov 21, 2026 by vercel[bot]
Preview – sdp-docs — e05c124b Deployed Nov 21, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants