Skip to content

Allow disabling the cpu controller for task cgroups - #13668

Open
bduffany wants to merge 3 commits into
masterfrom
task-cgroup-cpu-controller
Open

bduffany wants to merge 3 commits into
masterfrom
task-cgroup-cpu-controller

Conversation

@bduffany

@bduffany bduffany commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

Allow disabling the task cgroup CPU controller, which disables tasksize-based CPU weighting. This requires disabling it both during child cgroup setup, as well as disabling crun's cgroup manager, which tries to enable the CPU controller on ancestor cgroups.

Unfortunately, disabling crun's cgroup manager requires us to manage a few details that crun's manager otherwise manages for us, but I think it's OK. The implementation tries to match crun's behavior as closely as possible.

@bduffany
bduffany requested a review from vanja-p October 5, 2026 15:17
@bduffany
bduffany marked this pull request as ready for review October 5, 2026 15:17

@buildbuddy-io buildbuddy-io Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adds a flag that turns off the cpu controller for task cgroups, and replaces parts of crun's cgroup manager (signal, pause, kill) with hand-rolled cgroup operations. Most of the issues found are places where these replacements behave differently from crun.

Comment thread enterprise/server/remote_execution/containers/ociruntime/ociruntime.go Outdated
Comment thread enterprise/server/remote_execution/containers/ociruntime/ociruntime.go Outdated
Comment thread enterprise/server/remote_execution/cgroup/cgroup.go Outdated
Comment thread enterprise/server/cmd/executor/executor_linux.go
Comment thread enterprise/server/cmd/executor/executor_linux.go
Comment thread enterprise/server/remote_execution/containers/ociruntime/ociruntime_test.go Outdated
@bduffany
bduffany force-pushed the task-cgroup-cpu-controller branch from ea39d27 to d2b7065 Compare October 5, 2026 15:43
@bduffany
bduffany marked this pull request as draft October 5, 2026 18:43
@bduffany
bduffany removed the request for review from vanja-p October 5, 2026 18:43
Base automatically changed from executor-cgroup-cpu-weight to master October 5, 2026 19:38
@bduffany
bduffany force-pushed the task-cgroup-cpu-controller branch 3 times, most recently from 7c16d20 to 6aca341 Compare October 5, 2026 20:23
@bduffany
bduffany force-pushed the task-cgroup-cpu-controller branch from 6aca341 to bc67247 Compare October 5, 2026 20:23
@bduffany
bduffany marked this pull request as ready for review October 5, 2026 20:28
@bduffany
bduffany requested a review from vanja-p October 5, 2026 20:36
memoryBytes: args.Task.GetSchedulingMetadata().GetTaskSize().GetEstimatedMemoryBytes(),
useOCIFetcher: args.Props.UseOCIFetcher,

runtimePIDs: map[int]struct{}{},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: you can use server/util/set here


c.cgroupMu.Lock()
defer c.cgroupMu.Unlock()
err := cgroup.SignalAll(c.cgroupPath(), sig, func(pid int) bool {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if runtimePIDs was a set, then this could be:

err := cgroup.SignalAll(c.cgroupPath(), sig, c.runtimePIDs.Contains)

// Without a cgroup manager, crun leaves container processes in the cgroup
// that crun runs in. So start the commands that create container
// processes directly in the task's cgroup.
startInCgroup := !crunManagesCgroups() && (args[0] == "run" || args[0] == "create" || args[0] == "exec")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you document the importance of (args[0] == "run" || args[0] == "create" || args[0] == "exec")?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants