Skip to content

Report covert egress and cross-sandbox channels in audit mode (1.3.0) - #13

Merged
prabhu merged 8 commits into
mainfrom
feat/containment-detections
Oct 29, 2026
Merged

prabhu merged 8 commits into
mainfrom
feat/containment-detections

Conversation

@prabhu

@prabhu prabhu commented Oct 29, 2026

Copy link
Copy Markdown
Member

OpenAI's write-up on securing their model training environment (Securing our frontier model training environment) describes agents getting past containment without breaking any single rule:

  • reusing an allowed connection to reach a blocked endpoint
  • getting an allowed service to fetch a URL they supplied
  • leaking through DNS
  • signalling between isolated sandboxes by alternating disk writes or CPU load with idle periods

Most of these can't be blocked without breaking ordinary builds. So this PR makes safer-exec report them instead. All of these checks only add audit entries: they run when auditing is on, deny nothing, and need no new configuration.

What gets reported

  • proxy-sni-mismatch: a tunnel's TLS server name doesn't match the host in its CONNECT. That is domain fronting through a CDN or a shared front end.
  • proxy-embedded-url: a plain-HTTP request carries a URL for another host in its query or path. The entry names only the request's host and port, the parameter, and the embedded URL's host. Paths and values are never logged.
  • proxy-upload-heavy: the workload uploaded at least 1 MiB to an allowed host, and at least four times what came back. An upload split across many targets is caught for the run as a whole. Tunnels the server keeps open are closed and counted at the end of the run.
  • proxy-bypass-attempt (Linux): UDP sockets, and connections or sendtos that skip the proxy. The proxy only sees what is sent through it, Landlock doesn't confine UDP, and it confines TCP only by port. So DNS, QUIC, or a direct connect on an open port all go around the hostname allowlist today. To see these, the inner process installs a seccomp user-notification filter and hands the listener to the engine with SCM_RIGHTS. The engine records each call and always lets it continue.
  • timing-channel-suspect (Linux, Windows): the sandbox's total CPU time (and IO, where the counter exists) switches between idle and busy at least 16 times in 10 seconds. The data comes from the run's cgroup on Linux and the job object on Windows.
  • shared-write-path / shared-lock-file: another safer-exec run of the same user, running at the same time, can write the same path or holds the same lock. Runs register in a per-user directory under the user cache dir, and the macOS profile denies the workload access to it.

README has a section describing each entry, and THREAT-MODEL §3.21 lists what these checks can't catch. In short:

  • cache-timing channels
  • HTTPS request contents
  • unconnected sendmsg destinations
  • a sender that stays under the thresholds

THREAT-MODEL also corrects the row that called UDP evasion "Mitigated"; that only holds with disableNetwork.

Fixed along the way

Inside the PID namespace every --init is pid 1, so every Linux sandbox named its cgroup safer-exec-1. Concurrent runs with limits collided: one of them silently lost its limits, and the directory was left behind. Each run now gets a unique name on both cgroup v2 and v1, and the engine removes the directory when the run ends. The init thread is also locked to one OS thread now. That keeps Landlock and seccomp on the thread that execs the target.

Review

An independent adversarial review found real problems, and all of them are fixed here. The main ones:

  • In the first version, when the engine couldn't take over the seccomp listener (Docker's default profile denies pidfd_getfd), the workload's own connects failed with ENOSYS. The handoff now uses SCM_RIGHTS, which needs no permission the engine could be refused.
  • A macOS workload could put a FIFO in the run registry, which was then in $TMPDIR, and hang every later run. The registry moved to the user cache dir, the Seatbelt profile denies it, and records are read with no-follow, non-blocking, size and count limits.
  • Embedded-URL entries could log secrets carried in the request path.
  • Uploads could slip past the check in two ways: by keeping the tunnel open, or by spelling the port as 0443.

Version

Bumped to 1.3.0.

Testing

Each environment ran this branch and main side by side:

Environment Result
macOS arm64 Go suite passes with -race. Two concurrent audited CLI runs: the second reported shared-write-path. A workload granted ~/Library/Caches can't list or write the registry.
Docker on OrbStack, arm64 (Debian privileged, Debian with the default seccomp profile, Alpine) No new failures compared with main. The other failures there also fail on main (read-only /tmp, missing curl, test data outside the mount). Under the default profile, the workload still reaches the proxy, and UDP and direct connections are reported.
Ubuntu 24.04 x86_64, unprivileged Go passes on both trees. Node unit, integration, security and hooks tests: 601 pass, 0 fail on both. The cgroup sampler test passes under systemd-run --user --scope -p Delegate=yes.
Windows 11 ARM64, MXC c6f301d (base-container tier) go vet clean, all packages pass. An audited workload alternating CPU bursts with sleep is reported from the job object's counters.

Audit-only detections, nothing is denied:
- proxy-sni-mismatch: TLS SNI differs from the CONNECT host (domain fronting)
- proxy-embedded-url: plain-HTTP request carries a URL for another host
- proxy-upload-heavy: lopsided uploads to an allowed target
- proxy-bypass-attempt (Linux): UDP and direct TCP outside the egress proxy,
  observed with a seccomp user-notification supervisor that always continues
- timing-channel-suspect: on/off keying in the sandbox's cgroup (Linux) or
  job object (Windows) CPU and IO counters
- shared-write-path / shared-lock-file: concurrent runs that share writable
  paths or lock files

Also gives each Linux sandbox a unique cgroup name: inside the PID namespace
every --init is pid 1, so concurrent runs all used safer-exec-1.
Linux egress watch:
- Hand the seccomp listener to the engine with SCM_RIGHTS instead of
  pidfd_getfd. When the copy failed (Docker's default profile), the inner
  process closed the only listener and every connect of the workload failed
  with ENOSYS. sendmsg is no longer watched, which also removes the deadlock
  the first handoff hit and a round trip on IPC and connected-UDP traffic.
- Never close the listener before the run ends; retry unexpected errors.
- Close duplicated sockets off the serve loop (SO_LINGER could stall it),
  resolve the thread group for pidfd_open, fall back to /proc socket tables,
  and label unknown socket types as ip.
- Report every UDP socket creation, use the ID_VALID encoding 5.5-5.8 accept,
  cache executables per pid, bound finish.
- Lock the init thread so Landlock and seccomp land on the thread that execs.
- Use the unique cgroup name for cgroup v1 too and remove v1 directories.

Proxy:
- Detections run only for audited runs.
- proxy-embedded-url records the request authority, never the path; parses
  ';' separators and uses the embedded URL's own port.
- Count tunnels a server keeps open: track upstream conns, close and drain
  before flushing; normalize target ports; report run-wide spread uploads.
- Cap findings per type and report the overflow.

Run registry:
- Move it to the user cache dir and deny it in the Seatbelt profile; the
  macOS sandbox could plant a FIFO there and hang every later run.
- Read records without following links or blocking, regular files only,
  bounded size and count, names must match content.
@prabhu prabhu added bug Something isn't working documentation Improvements or additions to documentation enhancement New feature or request ai-auto A person set the task, and AI agents carried it through to completion labels Oct 29, 2026
@prabhu
prabhu added this pull request to the merge queue Oct 29, 2026
Merged via the queue into main with commit 97ba68e Oct 29, 2026
50 checks passed
@prabhu
prabhu deleted the feat/containment-detections branch October 29, 2026 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-auto A person set the task, and AI agents carried it through to completion bug Something isn't working documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant