[Pedro Henrique Penna (ppenna)]

Fixes #1341

What and why

On kernels before Linux 6.16, nvx::guest::exec killed, by its ID, the process group of a program whose status another collector had taken while a descendant kept the group: the kill by ID that an execution must never send, since the ID may name another group by then. Two library tests of the nvx crate caught it in every run on a Linux 6.6 host, and a third failed there because it checked what only Linux 6.8 can tell without checking for Linux 6.8. NVX's own guests run Linux 6.18, so NVX isn't affected, but an adapter that runs nvx::guest::exec on an older guest kernel is.

Root cause

Execution::new and Execution::adopt open the program's pidfd by its ID through sys::open_pidfd, and if that fails with anything but ESRCH, they end the program, and with Options::process_group its process group, by their IDs. pidfd_open answers ESRCH for an ID that nothing holds, but before Linux 6.16 it answers EINVAL for an ID that no thread-group leader has while something still holds it: the process group or the session that a reaped process led, or a thread that leads no process. torvalds/linux@8cf4b738f6d8, first in Linux 6.16, made those answers ESRCH and ENOENT. Python's os.pidfd_open answers, on the test VMs:

Kernel	Reaped leader, group alive	Reaped leader, group gone	Thread that leads no process
6.6.159.1-1.azl3, nvx-kvm-test	EINVAL	ESRCH	EINVAL
6.18.34.mshv3-1.azl3, nvx-mshv-test	ESRCH	ESRCH	ENOENT

Child::with_pidfd, which documents ESRCH for an ID that names no process, returned EINVAL there too.

Changes

1. nvx-rs: report an ID that no process has as ESRCH on every kernel. sys::open_pidfd reports EINVAL and ENOENT as ESRCH: for a positive ID and no flags, they mean nothing else. Its documentation and Child::with_pidfd's say so. One test hook in it, between the call and the mapping, lets a test have pidfd_open answer the test's thread as a kernel before Linux 6.16 does, with EINVAL for ENOENT, and for ESRCH while a process group has the ID, as BEFORE_LINUX_6_9 simulates a kernel before Linux 6.9, so that hosts whose kernels answer as CI's do test the older answers too. The new an_id_that_no_process_has_opens_no_pidfd_on_any_kernel checks that open_pidfd fails with ESRCH, with the kernel's own answers and with the simulated ones, for a reaped leader whose group a member keeps and for a thread that leads no process.
2. nvx-rs: expect EOPNOTSUPP for a runtime bind's root before Linux 6.8. Only the unique mount IDs of statx, which Linux 6.8 added, tell a mount's root, so on an earlier kernel Entry::bind_runtime's check of the root fails first, with EOPNOTSUPP, as sys::file_status documents. runtime_binds_need_a_mount_root_unless_there_are_none now checks for unique mount IDs as the activation tests do, and expects InvalidInput with them and EOPNOTSUPP without them.

Acceptance criteria of #1341

• NVX_BUNDLE=skip cargo test --locked --all-features passes on a Linux 6.6 host and on one with Linux 6.16 or later: met on nvx-kvm-test and nvx-mshv-test.
• A test checks on every kernel that Execution::new and Execution::adopt end nothing by ID for a program whose status another collector took, with the pidfd_open answers of a kernel before Linux 6.16: met by the two exec tests' simulated runs, which catch the kill by ID without the mapping on the 6.18 host, as the kernel's own answers do on the 6.6 host.
• The coverage measurement of ci: report the coverage of the nvx crate's tests on Linux and Windows #1323 passes on nvx-kvm-test: met. scripts/nvx_rs_coverage.py exits 0 there, in 116 s, with no reruns.

Found, not fixed here

• The host has the same kernel change on its side. nvx::host::sys::PidFd::open documents EINVAL for the ID of a thread that leads no process, and open_process in nvx-rs/src/host/platform/linux/process.rs takes ESRCH and EINVAL for an exited process, but from Linux 6.16 such an ID is ENOENT, which open_process returns as an error instead. That is host code, outside nvx-rs: three guest tests fail on Linux 6.6 hosts; before Linux 6.16, exec ends a reaped program's process group by ID #1341, so it is left for a separate change.

Validation

Locally, on Windows, at the head: rustfmt (cargo fmt --all --check), check_test_layout.py, which fails without the new allowlist entry, check_test_changes.py --base origin/dev, test_test_layout.py and test_test_changes.py (80 tests), and nvx.py verify. Windows builds never compile nvx-rs/src/guest, so the Linux hosts ran the Rust gates.

The hosts ran c741ba71c, the head before its commit messages gained these results; the head, 06c59acf2, and its first commit have the same trees as c741ba71c and its first commit, 14eabcdf.

Profile	Backend	Host type	Revision	Run	Result
nvx-kvm-test, Linux 6.6.159.1-1.azl3	kvm	virtual-machine	8394746fd (dev)	The issue's command, three times	The three tests failed in each run, as #1341 reports
nvx-kvm-test	kvm	virtual-machine	8394746fd	NVX_BUNDLE=skip cargo test --locked --all-features --no-fail-fast	Failed: 1,878 library tests passed and those three failed; every other target passed
nvx-kvm-test	kvm	virtual-machine	8394746fd	python3 scripts/nvx_rs_coverage.py	Exit 1 in 157 s: the three tests failed again when it reran them
nvx-mshv-test, Linux 6.18.34.mshv3-1.azl3	mshv	virtual-machine	8394746fd	The issue's command, three times, and the whole suite	Passed
nvx-kvm-test	kvm	virtual-machine	The head without the mapping	The new test and the two exec tests	Failed: open_pidfd answered EINVAL for both IDs, and both exec tests killed their descendant with the kernel's own answers
nvx-mshv-test	mshv	virtual-machine	The head without the mapping	The new test and the two exec tests	Failed: [ESRCH, ENOENT] from the kernel and [EINVAL, EINVAL] simulated, and both exec tests killed their descendant with the simulated answers
Both	kvm, mshv	virtual-machine	Head	The issue's three tests and the new one, three times	Passed
Both	kvm, mshv	virtual-machine	Head	NVX_BUNDLE=skip cargo test --locked --all-features	Passed: 1,882 library tests, 19 ignored, and every other target
Both	kvm, mshv	virtual-machine	Each commit	The library tests	Linux 6.18: both commits pass; Linux 6.6: the first fails only the runtime binds' test, which the second fixes
Both	kvm, mshv	virtual-machine	Head	check-nvx-rs's Linux steps: rustfmt, the five Clippy runs, the default and no-default builds and tests, rustdoc, the MSRV check with 1.89, the guest binaries' check, build, and tests, and the runtime adapter's protocol-only, host-only, and guest-only configurations	Passed
nvx-kvm-test	kvm	virtual-machine	Head	check-nvx-rs's macOS check, public API check, and artifact bundling	Passed
nvx-kvm-test	kvm	virtual-machine	Head	python3 scripts/nvx_rs_coverage.py, with cargo-llvm-cov 0.9.1 and Rust 1.97.1's llvm-tools in temporary Rust homes	Exit 0 in 116 s, no reruns; production 84.5% of 45,168 lines, as before

Left to CI: check-nvx-rs on Windows, validate-nvx on Linux and Windows, check-quality, and the microVM scenarios on KVM, MSHV, and WHP. NVX's guests run Linux 6.18, where the change alters only the answer for a thread's ID, which no scenario opens a pidfd for, so the hosts reran no microVM scenario.

Changes to existing tests

check_test_changes.py --base origin/dev lists 4, hence the tests-modified label:

• a_start_after_another_collector_took_the_status_ends_nothing_by_id and adopting_a_program_whose_status_another_collector_took_ends_nothing_by_id now run with the kernel's own pidfd_open answers and with those of a kernel before Linux 6.16, and check that the descendant still runs before they check the error. They check what they did, ESRCH and a descendant that nothing ended; the simulated run makes them check the older answers on every kernel, and the new order makes a kill by ID fail them as such rather than at the error, which nvx-rs: three guest tests fail on Linux 6.6 hosts; before Linux 6.16, exec ends a reaped program's process group by ID #1341 notes. Without the mapping, both fail on both hosts.
• runtime_binds_need_a_mount_root_unless_there_are_none keeps its InvalidInput check where the kernel has unique mount IDs, and expects file_status's EOPNOTSUPP where it doesn't. Its other checks don't change.
• scripts/test-layout-allowlist.json allows one test hook in nvx-rs/src/guest/sys/process.rs. To test the mapping, the simulation has to sit between the pidfd_open call and the mapping, inside open_pidfd, so it cannot move to the tests module. check_test_layout.py fails without the entry, counting the hook.

[github-actions]

This pull request changes existing tests

It modifies or removes 4 existing test asset(s), listed below. Review each with care: a change to a test can hide a regression that the test caught. See doc/contribute.md#changes-to-existing-tests.

Test code

Change	What	Before	After
modified test	fn runtime_binds_need_a_mount_root_unless_there_are_none in guest::entry::tests	nvx-rs/src/guest/entry/tests.rs:551	nvx-rs/src/guest/entry/tests.rs:551
modified test	fn a_start_after_another_collector_took_the_status_ends_nothing_by_id in guest::exec::tests	nvx-rs/src/guest/exec/tests.rs:769	nvx-rs/src/guest/exec/tests.rs:769
modified test	fn adopting_a_program_whose_status_another_collector_took_ends_nothing_by_id in guest::exec::tests	nvx-rs/src/guest/exec/tests.rs:1332	nvx-rs/src/guest/exec/tests.rs:1340

Test policy

Change	What	Before	After
loosened allowlist entry	nvx-rs/src/guest/sys/process.rs (the seams allow 1 test hooks)		scripts/test-layout-allowlist.json

Relocated unchanged: 0; added: 5. Compared the merge base 8394746fd864 with 06c59acf2678.

[Copilot]

🟢 Approval recommended

No active review issues remain for the guest process-handling compatibility changes.

No open issues 🎉

What changed in this PR

Fixes #1341 by making guest process handling consistent across Linux kernels, preventing exec from killing a process group by ID after another collector has reaped its leader.

Changes:

• Normalize pidfd_open errors for IDs that name no process to ESRCH, and document that behavior.
• Add simulated pre-6.16 kernel responses and regression coverage for exec start and adoption.
• Adjust the runtime-bind test to expect EOPNOTSUPP when unique mount IDs are unavailable.

File	Description
scripts/​test-layout-allowlist.json	Allows the kernel-response simulation hook in production code.
nvx-rs/​src/​guest/​sys/​tests.rs	Exports the simulation control for exec tests.
nvx-rs/​src/​guest/​sys/​process/​tests.rs	Adds older-kernel simulation and pidfd error regression coverage.
nvx-rs/​src/​guest/​sys/​process.rs	Maps EINVAL and ENOENT to ESRCH when opening a pidfd.
nvx-rs/​src/​guest/​process.rs	Documents consistent Child::with_pidfd error behavior.
nvx-rs/​src/​guest/​exec/​tests.rs	Checks descendant survival with native and simulated kernel responses.
nvx-rs/​src/​guest/​entry/​tests.rs	Makes runtime-bind error expectations depend on unique mount ID support.

_{Review effort: Balanced}