fix: keep runtime-selected printf reconstruction incomplete - #514
mohgupta-ship-it wants to merge 34 commits into
Conversation
Prepared by Codex for Mohit Gupta. Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Allow missing Git metadata only while cloning is active, then require a strict final measurement. Preserve permission failures, checkout errors, and all ingest limits. Cover disappearing files and directories, final budget enforcement, and fail-closed controls. Prepared by Codex for Mohit Gupta. Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Build the integration-test graph after configuring mock availability so semantic transports are exercised without provider credentials. Prepared by Codex for Mohit Gupta. Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Run CLI and MCP gates through static-only and semantic-enabled workflows using deterministic model responses. Prepared by Codex on behalf of Mohit Gupta. Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
yashrajp22
left a comment
There was a problem hiding this comment.
I checked these seven cases against this commit and the base. The PowerShell issue is new; the other six are existing gaps that still show up here.
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Reviewed current head 11f15ab947d8db05c6fb3d8f371eddf946519a28 — REQUEST_CHANGES.
The seven unresolved, non-outdated review threads are reproducible gaps in the fail-closed contract and remain unaddressed on the merge-only head:
tests/nodes/analyzers/test_security_reconstruction.py:1721: a runtime-selected destructive command inside a triple-backtick Markdown fence is skipped and can yield complete/SAFE. Route fenced code bodies through the incomplete-analysis check and add this regression.src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py:729: assignment/exec/separator prefixes can make the parser stop before$CMD(for example$(MODE=x $CMD %s r m) -rf /). Continue to the runtime command or fail closed.src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py:1203: brace-expanded absolute targets such as{/,/tmp}bypass the root-path check. Expand/prove the bounded brace alternatives or retain partial status.tests/nodes/test_security_end_to_end.py:1021: declared-marker reconstruction scans findings but does not propagate parse-exhaustion status, allowing a reconstructed runtime command to become complete/SAFE. Carry the exhaustion result into the ledger.src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py:1180: an empty quoted prefix ("""$($CMD)" -rf /) causes tokenizer uncertainty that is discarded. Preserve the tokenizer's limited flag.src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py:1186: a destructive target just beyond the 8,192-character lookahead is treated as clean. Reaching the lookahead boundary must produce partial analysis.src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py:742: ordinary PowerShell replacement expressions containing%TEMP%,%s, orprintfare misclassified as printf reconstruction and blocked. Exclude PowerShell value expressions while retaining unresolved shell-command cases.
The head changed during review from d95663d9766ae214a6b6668dac8bda62b41f5c1e to this merge commit; the current threads are still unresolved. The required fixes, unresolved threads, absent checks on the new head, and mergeStateStatus=BLOCKED all block merging.
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-reviewed current head 2c6a19c99ed8f46cc0ad5e7aa532c7b50b5abe6d.
The Markdown-fence finding from the previous review is resolved by the synchronized #516 changes: fence delimiters are now masked while fenced command bodies still reach the bounded parse-exhaustion check, with Markdown regression coverage.
The other six prior blockers remain on this head: assignment, exec, or separator prefixes can stop before the runtime command; brace-expanded absolute targets are not expanded by the root-path check; declared-marker projections still do not propagate parse exhaustion; tokenizer uncertainty from an empty quoted prefix is discarded; reaching the 8,192-character lookahead boundary is treated as clean; and ordinary PowerShell replacement strings containing %TEMP%, %s, or printf still trigger the printf-reconstruction heuristic. The existing inline threads contain the exact reproductions and requested fixes, so I have not duplicated them.
No hosted checks are reported on this head, and GitHub reports mergeStateStatus=BLOCKED.
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-reviewed current head b4971cafc36d895ae265d80d576175c4f7a47d56. The commits since the marked review at 2c6a19c99ed8f46cc0ad5e7aa532c7b50b5abe6d only merge #522's configurable static-analysis allowance. That synchronization changes budget configuration, documentation, and tests but does not alter the six remaining parser/completeness blockers.
Assignment, exec, or separator prefixes can still stop before the runtime command; brace-expanded absolute targets remain unexpanded by the root-path check; declared-marker projections still fail to propagate parse exhaustion; tokenizer uncertainty from an empty quoted prefix is discarded; reaching the 8,192-character lookahead boundary is treated as clean; and ordinary PowerShell replacement strings containing %TEMP%, %s, or printf still trigger the printf-reconstruction heuristic. The existing inline threads contain the exact cases and requested fixes, so I have not duplicated them. The Markdown-fence finding remains resolved by the earlier #516 synchronization.
No hosted checks are reported, and GitHub reports mergeStateStatus=BLOCKED.
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-reviewed current head 0ad0b8e65a002152b8cfbd3054ab3c9b1c2e0464 after the marked review at b4971cafc36d895ae265d80d576175c4f7a47d56. All six PR-owned production and test blobs are byte-for-byte unchanged.
The Markdown-fence bypass remains resolved by the synchronized #516 work. The other six blockers remain: assignment/exec/separator prefixes can stop before the runtime command; brace-expanded absolute paths evade root-target recognition; declared-marker scans do not propagate parse exhaustion; empty quoted prefixes discard tokenizer uncertainty; reaching the bounded lookahead is treated as clean; and ordinary PowerShell replacement expressions are still mistaken for printf reconstruction. The existing threads contain the exact reproductions and expected corrections, so I have not duplicated them.
No required checks are reported on this head, active change requests and unresolved threads remain, and GitHub reports mergeStateStatus=BLOCKED.
yashrajp22
left a comment
There was a problem hiding this comment.
Please fix the time-budget issue below before merging.
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-reviewed current head 4b2584c721179b70443aa1f12739cec1a1cd5f4a. The head change only synchronizes unrelated main work; the PR-owned analyzer and regression files are unchanged. The eight previously reported fail-open and runtime-bound cases are now represented by exact regressions and their threads are resolved. A new regression in the current implementation blocks approval.
At src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py:3011-3024, the additive raw-Markdown pass sends the entire unmasked document through _has_shell_command_word_exhaustion. That discards established inline-code, JSON-string, table-cell, and paragraph ownership and marks benign documentation such as $(hostname).example partial. The latest tested implementation failed 172 tests across those contracts and CLI/MCP gates; three parser stress cases also exceeded their 12-second bound. Restrict the raw fallback to proven runtime-selected destructive or backtick evidence (or preserve ownership metadata), restore the completeness contracts, and obtain a green exact-head rerun.
The current head has no attached checks. The correctness/performance regression, missing exact-head validation, active change request, and GitHub BLOCKED state prevent merge.
Runtime-selected executable and wrapper names can fall out of bounded printf reconstruction and incorrectly produce a complete/SAFE result. Recognize ambiguous invocations from printf-style operands or destructive outer arguments and report partial analysis so strict CLI and MCP installation gates reject unresolved work.
Preserve ordinary parameter documentation, PowerShell expressions, and the reference-accounting fixes from #507. Operand lookahead remains bounded. Clone monitoring tolerates disappearing Git metadata only while the clone is active and still performs a strict final inspection.
Regression tests distinguish literal shell backticks from Markdown delimiters, including longer code spans that contain actual shell substitutions. Paired CLI/MCP tests exercise static-only and semantic-enabled workflows with deterministic model responses and verify that successful LLM analysis does not erase incomplete static coverage. The mocked LLM-failure integration test builds its graph after configuring provider availability.
Validation:
make test-cipassed 4,179 tests, 14 skipped, 38 deselected, 4 expected failures; 89% coverage.The full suite was rerun under controlled load after concurrent suites caused two large-file timing failures; both cases passed without changing their assertions or resource limits. Joint validation with the documentation and quote-scan fixes is tracked separately.
Prepared by Codex on behalf of Mohit Gupta.