Repository navigation
Conversation
Cold boots attached eth0 without guest_mac, so virtio_net picked a random MAC. Guest images running systemd >= 242 with udevd rewrite such a MAC via 99-default.link (MACAddressPolicy=persistent) without a gratuitous ARP, and the host tap0 neighbour entry kept the boot MAC until it aged out; proxied connects hit the 5s timeout and returned 502 for up to ~40s. A configured MAC is exposed via VIRTIO_NET_F_MAC and reported as NET_ADDR_PERM, which udev leaves alone. Restore paths load the NIC from vm_state.bin and are unchanged. Closes kvcache-ai#407 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
|
✅ OpenCodeReview: Review complete: 0 finding(s) across 1 selected item(s). |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
冷启动(
start_fresh)时给eth0配置一个固定的、本地管理的单播 MAC02:fc:00:00:00:05(Firecrackerguest_mac),不再让 guest 内核随机生成 MAC。同时在docs/src/internals/networking.md的地址规划里补充说明。Why
详见 #407,简述如下:
start_fresh添加eth0时guest_mac为None,Firecracker 不提供VIRTIO_NET_F_MAC,guest 的 virtio_net 随机分配 MAC(addr_assign_type = NET_ADDR_RANDOM)。pivot-init交给 systemd(>= 242)后,systemd-udevd按自带的99-default.link(MACAddressPolicy=persistent)把随机 MAC 改写成 persistent MAC,且不发 gratuitous ARP。tap0上169.254.0.21的 neighbour 表项仍指向启动时的 MAC,host 发往 guest 的帧被丢弃。proxy 的连接一直等到 5s connect timeout 后返回502,直到表项老化、广播重新解析为止(复现中约 43s)。这段时间里沙箱已经创建成功,envd 也已经应答过请求。配置了 MAC 之后,guest 报告
NET_ADDR_PERM,udev 的 persistent 策略不会再改写它。Related issue
Closes #407
Scope and non-goals
start_fresh里唯一一处add_network_interface调用,新增一个常量。vm_state.bin加载网卡(只覆盖host_dev_name),guest 保留快照里已有的 MAC。tap0的 neighbour 参数,不往用户 rootfs 注入.link文件,也不加静态 neighbour 表项(理由见 bug(network): guest udev rewrites the random eth0 MAC on cold boot, stale host neighbour makes the proxy 502 for ~40s #407 的 Alternatives considered)。Design and behavior changes
tap0,guest IP169.254.0.21本来就在所有 slot 间相同。这个 MAC 只出现在 slot 内部的 TAP 私有链路上。host_interaction_ip经veth-<idx>访问 guest,解析的是 veth 对端的 MAC(每个 slot 不同),与 guest MAC 无关。02前缀表示 locally administered 的单播地址,不会和厂商 OUI 冲突。99-default.link的NamePolicy=keep ...仍保留eth0。Compatibility and operations
Validation
make fmtmake clippymake test-unit(只跑了其中的cargo test -p agentenv --lib,见下)make -C services test(required whenservices/changes)maketargetCommands and results:
线上验证:在一个下游部署里,把一台节点换成包含此改动的构建做 canary,环境与 #407 相同(KVM,Firecracker
1.15.1-patch-v1,guest kernelvmlinux-6.1.175)。使用 #407 中的 Image A(Ubuntu 24.04,systemd 255,avahi-daemon),方法与 #407 一致:在 slot netns 里运行ip -timestamp monitor neigh,每 250 ms 请求一次/proxy/health。204之后 154×502(每次 5.00 s,约 43 s)204,最大 5 mstap0neighbourSTALE→FAILED→ 新 MAC02:fc:00:00:00:05addr_assign_typeNET_ADDR_SET)NET_ADDR_PERM)Assigned random MACguest 中
udevadm info仍显示ID_NET_LINK_FILE=/usr/lib/systemd/network/99-default.link,即 udev 照常处理了这块网卡,只是不再改写 MAC。8 个沙箱并发运行时,还做了串流检查:通过每个沙箱的 envd 往 guest 串口写入带自身 sandbox ID 的标记,再到 host 上对应 sandbox 的 serial 日志里核对。8/8 都只出现在自己的日志里;8 个 guest 的
boot_id各不相同。Skipped checks and reasons:
make test-unit中需要 capability runner 的--ignored测试,以及 Rust 集成测试:本地没有运行(需要 root / KVM 环境)。改动只是一个网卡配置参数,已用上面的 canary 做了端到端验证。sandbox.rs还原为 main 8428a32 的版本后,在同一台机器上单独重跑,两者以同样的方式失败(child did not start/timed out waiting for .../passwd.lock)。它们依赖子进程与 tempdir 上的时序,是本地开发环境的问题,以 CI 结果为准。services/,也没有生成代码;不涉及性能。Risks and reviewer notes
src/sandbox/firecracker/sandbox.rs中start_fresh的add_network_interface调用。Checklist