Repository navigation
nvx-rs: boot templates with an adapter's kernel tuning parameters - #1422
Merged
Pedro Henrique Penna (ppenna) merged 5 commits intoNov 9, 2026
Merged
Conversation
The runtime adapter's template tests took `quiet` for a kernel parameter that the platform tier's allowlist refuses. The next pin of OpenVMM admits `quiet` on a platform snapshot's command line, with the other kernel tuning parameters of #995, and so does the guest agent, which checks a template's command line with the excerpt of OpenVMM's allowlist. the_platform_tier_s_allowlist_is_openvmm_s, the_platform_tier_s_allowlist_takes_tokens_as_openvmm_splits_them, and template_mode_refuses_a_token_outside_the_allowlist take `nokaslr` instead, which the allowlist refuses before and after that pin, and check the same refusals. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pin microsoft/openvmm#5001's head, b4028c1aa775d8aafba1a170857acc032c7c86a1, "openvmm_helpers: admit kernel tuning on platform command lines", one commit on feature/microvm/dev f41a1ecfc. Its platform snapshots admit a closed set of guest kernel tuning parameters on their kernel command lines, at publication and at every restore, each parameter at most once: `mitigations=off`, `mitigations=auto`, and `mitigations=auto,nosmt`; `no_timer_check`; and `quiet`, and `loglevel=0` to `loglevel=7` (#995). It refuses every other token, as before. The excerpt of OpenVMM's platform allowlist that nvx-rs's template tests and the runtime adapter compile, nvx-rs/tests/vectors/openvmm/platform_command_line_token_is_invariant.rs, takes the function's new lines unchanged, with the constant of the tuning parameters before it, and the vectors' README records the commit. The guest agent of a runtime adapter's template, which checks its kernel command line with the excerpt, so admits the tuning parameters too. nvx-rs's openvmms_platform_allowlist_admits_its_tokens_and_refuses_tenant_data took `quiet` for a token outside the allowlist, which the new excerpt admits. It now admits the tuning parameters, and refuses other values and spellings of them, such as `mitigations=on`, `loglevel=8`, and `no-timer-check`, and other parameters, such as `nokaslr`. `cargo test --test openvmm_vectors -- --ignored` and `--test openvmm_cpu_profiles -- --ignored` pass at this pin. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A template's kernel booted with NVX's own command line alone, `nvx_sandbox=1` and, for a template whose processors are not all online at boot, `maxcpus=`. An adapter whose cold sandboxes boot with kernel tuning parameters, such as `mitigations=off`, could not give them to its templates, so its clones ran another kernel configuration than its cold sandboxes: Linux applies `mitigations=` once, as it boots, and a clone has no other way to get it (#995). KERNEL_TUNING_PARAMETERS names the parameters that OpenVMM now admits on a platform capture's command line: `mitigations=off`, `mitigations=auto`, and `mitigations=auto,nosmt`; `no_timer_check`; and `quiet`, and `loglevel=0` to `loglevel=7`. CaptureClass gains kernel_tuning, the parameters with which the template's kernel boots, in their order, which CaptureClass::check refuses unless each is one of them and each parameter appears at most once, so that a class that no capture takes is refused before any VM boots. TemplatePlan::with_kernel_tuning renders them after the plan's own tokens, and a Store's captures boot with their class's. TemplateKey::new also checks that the tuning parameters of the manifest's command line are the class's, in their order, as it checks the rest of the machine that the class requested. The key holds the command line already, so its encoding is unchanged, and templates of classes that differ in their tuning alone have keys of their own. NVX's fake OpenVMM accepts the parameters after a capture's own tokens, as OpenVMM does, each at most once. The class's canonical encoding holds the parameters last, as a list of texts, under version 2. A class's identity names the slot of its captures and its generations in a store's index, so a creation finds no generation that a class of version 1 captured, and captures one again; those generations still serve the clones that select them by their keys, which are unchanged. tests/vectors/capture-class-v2.txt holds the class vectors of version 2, derived from the bytes of version 1's vectors and the definition, and one with tuning parameters. template-key-v1.txt keeps the keys' vectors alone, since nothing encodes a class of version 1 any more. Tests: the plan renders the parameters after its own tokens, in their order, and replaces them; the class check and the plan refuse a parameter outside the set, another value or spelling of one, and a repeated parameter; classes that differ in their tuning alone, order included, have identities of their own, and encode the parameters last; TemplateKey::new refuses a capture whose command line holds other tuning than its class's; a store's capture passes its class's parameters in OpenVMM's --cmdline, and refuses a snapshot whose command line lacks them; nvx-rs's set is OpenVMM's, as the excerpt of its allowlist defines it; the class vectors encode; and the fake OpenVMM accepts a capture's tuning parameters, with or without a limit of the processors online at boot, refuses others and repeated ones, and holds nvx-rs's set. With the check, the encoding, the plan, the key's check, and the capture's tuning neutralized, the seven tests of that logic fail. Existing tests change where the new check or encoding reaches them: the key test's change of its command line adds `quiet` to its class too, the class vectors move to capture-class-v2.txt, and the helpers that build classes name no tuning. The runtime adapter's class names none either. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The runtime adapter, which builds on nvx's public API as an independent adapter does, could not ask for the kernel tuning parameters that NVX's capture classes now take (#995). create and refresh take --kernel-tuning PARAMETER, once for each parameter, in order, which capture::Request carries into the class's kernel_tuning, so that the template's kernel, and every clone of it, runs with them. NVX's class check refuses a parameter that is not one of its KERNEL_TUNING_PARAMETERS, and a parameter given twice, before the host program creates the store. The guest agent's check of a template's command line, which compiles the excerpt of OpenVMM's platform allowlist, admits the parameters since the pin of OpenVMM that admits them, and is_platform_token's documentation and the README's template mode now list them. Tests: template_capture's a_capture_takes_its_kernel_tuning_from_the_request checks that the class takes the request's parameters in their order, which name a class of their own, order included, while nothing else of the capture changes, and that NVX's class refuses other parameters; vm_template's kernel_tuning_that_no_capture_takes_is_refused_before_the_store checks that `nokaslr`, `mitigations=on`, and `mitigations=` with two values fail with status 1 before the store exists; and vm_clone's a_template_s_clones_run_with_its_kernel_tuning_parameters captures a template with `mitigations=off`, whose key's command line holds it, and starts a clone whose /proc/cmdline is that command line. The template module's a_template_s_command_line_may_hold_kernel_tuning_parameters checks that a template's VM boots with the parameters on its command line, and the_platform_tier_s_allowlist_is_openvmm_s and template_mode_refuses_a_token_outside_the_allowlist now admit them and refuse other values and spellings of them, such as `mitigations=on`, `loglevel=8`, and `no-timer-check`. scenarios.txt and the README's tables name the new tests, and template_capture's request helper names no tuning. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…oadmap Maps NVX's new pin, b4028c1aa, the head of microsoft/openvmm#5001, at position 235, after position 234, f41a1ecfc, the head of feature/microvm/dev, which microsoft/openvmm#4997 merged. The pin is 104 commits past the mapped tip, and positions 132-235 stay unassigned. Position 235 makes the platform tier's check of a snapshot's command line, which PR19's position 093 adds and PR20's position 097, PR23's position 116, and positions 147 and 168 change, admit a closed set of kernel tuning parameters in its allowlist, and refuse a command line that repeats one of them, after position 097's check of the boot-online processor count. It adds tests beside position 093's, and extends the --snapshot-tier entry that position 093 adds to the CLI reference of PR15's position 064. Replaying it on upstream main conflicts in both of its files, since upstream main lacks the microVM snapshot module of PR17's position 074, the tiers of position 093, and the microVM CLI of position 064, so it follows PR15, PR17, and PR19 upstream. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
This pull request changes existing testsIt modifies or removes 17 existing test asset(s), listed below. Review each with care: a change to a test can hide a regression that the test caught. See doc/contribute.md#changes-to-existing-tests. Test codeTest data
Test doubles
Relocated unchanged: 0; added: 22. Compared the merge base |
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The OpenVMM pin depends on still-open PR #5001 with CI running, requiring maintainer confirmation that exact-pin CI passes and the revision lands on feature/microvm/dev before merging.
No open issues 🎉
What changed in this PR
Enables NVX templates to boot with an adapter’s supported kernel tuning parameters, addressing #995 so restored clones can use the same boot-time kernel configuration as cold sandboxes.
Changes:
- Adds ordered, validated kernel tuning to capture classes, template plans, and capture verification, with version 2 class encoding.
- Exposes repeatable
--kernel-tuningoptions in the runtime adapter and adds validation and clone-inheritance tests. - Updates OpenVMM allowlist excerpts, conformance vectors, public API inventories, documentation, and pin roadmap mapping.
| File | Description |
|---|---|
tests/runtime-adapter/tests/vm_template.rs |
Tests invalid tuning rejection before store creation. |
tests/runtime-adapter/tests/vm_clone.rs |
Tests that clones retain template kernel tuning. |
tests/runtime-adapter/tests/template_capture.rs |
Tests request propagation, identities, and validation. |
tests/runtime-adapter/src/template/tests.rs |
Tests expanded allowlist and unsupported tokens. |
tests/runtime-adapter/src/template.rs |
Documents supported tuning and duplicate validation. |
tests/runtime-adapter/src/capture.rs |
Carries requested tuning into capture classes. |
tests/runtime-adapter/src/bin/runtime-adapter-host.rs |
Adds repeatable kernel tuning CLI options. |
tests/runtime-adapter/scenarios.txt |
Registers new tuning scenarios. |
tests/runtime-adapter/README.md |
Documents tuning options, behavior, and scenarios. |
nvx-rs/tests/vectors/template-key-v1.txt |
Removes class vectors while retaining key vectors. |
nvx-rs/tests/vectors/README.md |
Documents class encoding v2 and excerpt provenance. |
nvx-rs/tests/vectors/openvmm/platform_command_line_token_is_invariant.rs |
Copies OpenVMM’s expanded tuning allowlist. |
nvx-rs/tests/vectors/capture-class-v2.txt |
Adds class encoding v2 conformance vectors. |
nvx-rs/tests/template_capture.rs |
Initializes tuning in capture test fixtures. |
nvx-rs/src/testing/fake_openvmm/tests.rs |
Tests fake capture tuning validation. |
nvx-rs/src/testing/fake_openvmm.rs |
Accepts supported, nonduplicate capture tuning. |
nvx-rs/src/host/template/tests.rs |
Initializes tuning in shared class fixtures. |
nvx-rs/src/host/template/store/capture/tests.rs |
Tests capture arguments and tuning mismatch rejection. |
nvx-rs/src/host/template/store/capture.rs |
Applies class tuning to capture plans. |
nvx-rs/src/host/template/plan/tests.rs |
Tests tuning order, replacement, and validation. |
nvx-rs/src/host/template/plan.rs |
Adds the kernel tuning builder method. |
nvx-rs/src/host/template/mod.rs |
Exports tuning parameters and updates encoding documentation. |
nvx-rs/src/host/template/key/tests.rs |
Tests tuning identities, matching, and conformance vectors. |
nvx-rs/src/host/template/key.rs |
Validates and encodes tuning; checks manifest agreement. |
nvx-rs/README.md |
Documents template tuning and class identity changes. |
nvx-rs/public-api/windows.txt |
Records new Windows-visible public API items. |
nvx-rs/public-api/linux.txt |
Records new Linux-visible public API items. |
doc/openvmm-upstream-roadmap.md |
Maps the kernel tuning pin at position 235. |
Review effort: Balanced
This was referenced Nov 9, 2026
Pedro Henrique Penna (ppenna)
deleted the
agents/issue-995-kernel-tuning
branch
November 9, 2026 21:40
This was referenced Nov 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #995
What and why
A template's kernel booted with NVX's own command line alone,
nvx_sandbox=1and, for a template whose processors are not all online at boot,
maxcpus=.OpenVMM checks a platform snapshot's manifest when it publishes a capture and
again at every restore, and refused every other token of its command line as
"tenant or unsupported configuration". So an adapter whose cold sandboxes boot
with kernel tuning parameters, such as
mitigations=off, could not give them toits templates, and its clones ran another kernel configuration than its cold
sandboxes: Linux applies
mitigations=once, as it boots, so a clone has noother way to get it.
b4028c1aa775d8aafba1a170857acc032c7c86a1,one commit on
feature/microvm/devf41a1ecfc): a platform snapshot'scommand line admits a closed set of guest kernel tuning parameters, at
publication and at every restore, each parameter at most once:
mitigations=off,mitigations=auto, andmitigations=auto,nosmt;no_timer_check; andquiet, andloglevel=0tologlevel=7. Every othertoken stays refused, the time ABI's clock parameters included. The Guide's
--snapshot-tierentry lists what a platform capture's command line may hold.openvmm: pin kernel tuning on platform command lines): moves thegitlink, copies the new lines of OpenVMM's allowlist, with the constant of the
tuning parameters, into
nvx-rs/tests/vectors/openvmm/, records the commit inthe vectors' README, and updates nvx-rs's allowlist test, which took
quietfor a refused token. The commit before it makes the runtime adapter's template
tests take
nokaslrfor that token instead, since the guest agent compilesthe same excerpt, so that the gitlink commit holds only the gitlink and
coordinated
nvx-rs/changes, asopenvmm_pin.py check-branch --topicrequires.
KERNEL_TUNING_PARAMETERS;CaptureClass::kernel_tuning, whichCaptureClass::checkrefuses unless each is in the set and each parameterappears once, before any VM boots;
TemplatePlan::with_kernel_tuning, whichrenders them after the plan's own tokens, in their order; a
Store's capturesboot with their class's;
TemplateKey::newrefuses a capture whose commandline holds other tuning than its class's. The class's canonical encoding holds
the parameters last under
CaptureClass::VERSION2, with conformance vectorsin
nvx-rs/tests/vectors/capture-class-v2.txt; the key's encoding isunchanged, since the key holds the command line. NVX's fake OpenVMM accepts
the parameters after a capture's own tokens, as OpenVMM does.
createandrefreshtake--kernel-tuning PARAMETER,once per parameter, in order. Its guest agent's check of a template's command
line, which compiles the excerpt, admits the parameters; its documentation and
tests now say so.
doc: map the kernel tuning on platform command lines in the OpenVMM roadmap): maps the pin at position 235, after position 234,f41a1ecfc, the head offeature/microvm/dev; the pin is 104 commits pastthe mapped tip, and positions 132-235 stay unassigned.
Snapshot formats: OpenVMM's manifest is unchanged, so every existing snapshot
stays restorable. The class encoding's new version changes every class's
identity, so
Store::create_templatefinds no generation that a class ofversion 1 captured, and captures one again; those generations still serve the
clones that select them by their keys, which are unchanged.
Design decisions:
Vec<String>in their order, whichCaptureClass::checkchecks, as the issue proposes, rather than a type thatis valid by construction. The order is part of the class, since the kernel
applies the last of
quietandloglevel=.TemplateKey::newalso checks the manifest's tuning against the class's, asit checks the rest of the machine that the class requested, so a capture that
did not boot with them makes no key of its class.
TemplatePlan::with_kernel_tuningis a builder method, which leavesTemplatePlan::new's signature as it was.template-key-v1.txt: nothing encodes ordecodes a class of version 1 any more, and the version 2 vectors encode the
same classes, and one with tuning.
--kernel-tuningvalues to NVX'sclass, which refuses them before the store is created.
as the command line's do.
Commits:
runtime-adapter: take nokaslr for a token outside the platform allowlist(preparation for the pin, valid before and after it).
openvmm: pin kernel tuning on platform command lines.nvx-rs: boot templates with their class's kernel tuning parameters.runtime-adapter: capture templates with kernel tuning parameters.doc: map the kernel tuning on platform command lines in the OpenVMM roadmap.Landing order: land microsoft/openvmm#5001 first, by fast-forwarding
feature/microvm/devtob4028c1aa775d8aafba1a170857acc032c7c86a1exactly(OpenVMM's merge button squashes and would leave the pin off the branch), then
this pull request. Its only prerequisite, microsoft/openvmm#4997 at
f41a1ecfc, is onfeature/microvm/dev, and #1420 pinned it ondev.Validation
The tree validated below is the branch's at
6bc5cbc34, which differs from thishead only in
doc/openvmm-upstream-roadmap.md: #1420's roadmap commit, ontowhose merge the branch was rebased, and this branch's roadmap commit.
Local (Windows):
python scripts\local_gates.py --base origin/dev --only test-changes markdown-links test-layout openvmm-vectors openvmm-cpu-profiles nvx-rs-format nvx-rs-lint nvx-rs-test nvx-rs-bundling nvx-rs-doc nvx-rs-msrv nvx-rs-adapter-protocol nvx-rs-adapter-host --run --keep-going:all 13 passed; at this head,
test-changes,markdown-links, andtest-layoutpassed again.nvx-cli, before the rebase onto openvmm: refuse an unknown paired-scratch restore mode #1418-openvmm: don't publish a snapshot over an empty directory #1420, at the same change: 2188 tests,of which only the 3 subtests of
test_host_loopback_allow_script_checks_publication_and_trafficfailed, eachtiming out after 10 s in Git for Windows'
sh, which this machine'shost-loopback script tests always do (they pass in CI and on Linux); its other
steps passed. The change touches no script, so its rerun is left to CI.
python scripts\openvmm_pin.py check-branch --topic --target origin/dev,check-roadmap --worktree,python scripts\check_markdown_links.py,python scripts\rebase_check.py verify, andpython scripts\nvx.py verify:passed.
openvmm_helpers:cargo check,cargo clippy --all-targets -- -D warnings,cargo doc --no-depswith-D warnings,cargo nextest run --profile agent(126 passed), andcargo xtask fmt --fix(clean) on Windows; the same clippy, doc, and nextest(132 passed) on Linux. With the production change reverted, both new OpenVMM
tests fail on Windows and Linux; with the new nvx-rs logic neutralized, the
seven nvx-rs tests of the class check, encoding, plan, key check, and capture
fail.
Linux (
nvx-kvm-test): thecheck-nvx-rssteps passed:scripts/public_api.py,cargo fmt --all --check, clippy with all features, withtestingalone, andwith
cli,cargo testwith all features and by default,cargo doc, themacOS check, the 1.89 check, the ignored
openvmm_vectorsandopenvmm_cpu_profilestests, and the runtime adapter's fmt, guest clippy, test,and doc for
x86_64-unknown-linux-musl, host clippy, test, and doc, and 1.89checks.
Remote runs, with the runtime adapter's new tests
vm_clone a_template_s_clones_run_with_its_kernel_tuning_parameters(capture atemplate with
mitigations=off, start a clone, and compare its/proc/cmdline)and
vm_template kernel_tuning_that_no_capture_takes_is_refused_before_the_store:98e5b3ca5)b4028c1aabuilt from the pin)vm_clonefails: OpenVMM refuses the capture, "platform snapshot command line contains tenant or unsupported configuration"nvx.py test-runtime-adapter --backend kvm: the 161 scenarios of 11 targets passed;nvx.py test-microvm --backend kvm --scenario snapshot-tiers --scenario platform-restore-processors --scenario platform-aggregate-export --processors 1 2: passedtest-runtime-adapter --backend mshv: the 161 scenarios passedtest-runtime-adapter --backend whp: the 161 scenarios passedThe before runs used the change's first revision, whose OpenVMM commit
8efe38c7fhas the same patch asb4028c1aa(git range-diffreports themequal); the OpenVMM allowlist that refused the capture is unchanged between
98e5b3ca5andf41a1ecfc.Left to CI: the
nvx-clisuites, the microVM scenarios on every backend, theOpenVMM suites and OpenVMM's own CI of the pin (microsoft/openvmm#5001's run is
in progress), guest determinism, and the benchmarks.
Changes to existing tests (
tests-modified), aspython scripts/check_test_changes.py --base origin/devlists them (17):openvmms_platform_allowlist_admits_its_tokens_and_refuses_tenant_data(nvx-rs),
the_platform_tier_s_allowlist_is_openvmm_s,the_platform_tier_s_allowlist_takes_tokens_as_openvmm_splits_them, andtemplate_mode_refuses_a_token_outside_the_allowlist(runtime adapter): theytook
quietfor a token outside OpenVMM's platform allowlist, which nowadmits it. They take
nokaslrinstead, admit the tuning parameters, andrefuse other values and spellings of them.
platform_command_line_token_is_invariant.rs(test data): the excerpt ofOpenVMM's allowlist, copied unchanged from the pinned revision, as
openvmm_vectorsrequires.each_part_of_the_key_changes_it: its change of the manifest's command lineadds
quiet, which is now a tuning parameter, so its class requests it too,or
TemplateKey::new's new check would refuse the capture.the_conformance_vectors_decode_and_encode,KEY_VECTORS(wasVECTORS),vectors,vector_class, andtemplate-key-v1.txt: the class vectors ofversion 1 moved to
capture-class-v2.txtunder the new class encoding; thekey vectors are unchanged.
classhelpers ofhost::template::testsand nvx-rs'stemplate_capture, and the runtime adapter'stemplate_capturetemplatehelper: they name no tuning, for the new field.
CAPTURE_COMMAND_LINEandparse: the fake accepts thetuning parameters after a capture's own tokens, as OpenVMM does.
tests/runtime-adapter/scenarios.txt: names the three new runtime adaptertests.