Repository navigation
feat(cloudflare): support native Durable Object containers - #1905
danieljvdm wants to merge 15 commits into
Conversation
|
Install the packages built from this commit: alchemypnpm install https://pkg.alchemy.run/alchemy/pr:1905:c34ad29@alchemy.run (6)pnpm install https://pkg.alchemy.run/@alchemy.run/better-auth/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@alchemy.run/cloudflare-runtime/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@alchemy.run/frontend-frameworks/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@alchemy.run/node-utils/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@alchemy.run/floci/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@alchemy.run/pkg/pr:1905:c34ad29@distilled.cloud (16)pnpm install https://pkg.alchemy.run/@distilled.cloud/core/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@distilled.cloud/acme/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@distilled.cloud/aws/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@distilled.cloud/axiom/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@distilled.cloud/cloudflare/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@distilled.cloud/doppler/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@distilled.cloud/fly-io/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@distilled.cloud/github/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@distilled.cloud/hetzner/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@distilled.cloud/infisical/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@distilled.cloud/neon/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@distilled.cloud/prisma/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@distilled.cloud/planetscale/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@distilled.cloud/railway/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@distilled.cloud/stripe/pr:1905:c34ad29pnpm install https://pkg.alchemy.run/@distilled.cloud/zerossl/pr:1905:c34ad29Published |
|
Three notes from checking this against two live
I'm happy to send a follow-up PR for 1 and 2 on top of whichever implementation lands. |
|
Closed #1904 in favour of this one. A data point from deploying the #1904 preview with a large image, in case it helps here. Image preparation on large images. Our container image is a nix + devenv image (~1.8 GB). Under Order of operations. Preparing named images before the Worker uploads, as this PR does, matters. On #1904 the Worker uploaded first, so the timeout left a stage whose Durable Object code expected
|
Merges the Effect-native sandbox: Scorer and Worktree Durable Objects on Durable Object-managed containers (machine.ts, containers.ts), Egress as the Worker's default export, warmed snapshots per base. Kept from main and dogfood: the tree tends its agents (PR #5's Api-started agents and src/api/agents.ts are dropped), the [fetch] phase, streamed scoring progress, SECRETSPEC_REASON, the release-assets host, judged reports' touched paths. The Rust snapshot logic is ported to the TS tree: core's snapshotPlans, SnapshotNews in the streamed outcome line (headers on a plain answer), snapshot:<base> in TreeObject, and Started carries the base's snapshot to the agent's Worktree. Legacy removed: the botany-vocabulary layer (legacy.ts, its fixture and test) and scores without confidence from before judges. Does not typecheck yet: Containers.bind and schedulingPolicy come from alchemy-run/alchemy#1905, unreleased; package.json is on alchemy 2.0.0-beta.80. How #1905 is pulled in is undecided. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
a1967bc to
5eeae6f
Compare
4f6eb94 to
28850e8
Compare
28850e8 to
bc7acba
Compare
11ecfda to
b2bdc96
Compare
Validate native image names and counts, preserve named-image records, and use the concrete Cloudflare Fetcher contract. Scope Docker initialization and generate local Docker tags independently of user image names. Extend lifecycle coverage for image additions and removals, verify uploaded Worker metadata, and bound runtime requests. Update distilled to the reviewed container SDK fixes. Validated with 50 focused Alchemy/runtime tests, local and live Cloudflare lifecycles, type checks, provider lint, JSDoc checks, documentation generation, and formatting.
…oute docker exec Layer the native container pieces onto the shared loopback proxy: pull images that workerd starts without preparation, and send docker exec upgrade streams straight to Docker through the router.
Label every container created through the Docker proxy with the runtime's id and remove them all when the Docker layer closes. Native containers can start from snapshot or managed images the runtime never prepared, which the per-image cleanup could not find, so each run leaked the containers and their proxy sidecars.
…ve coverage - Delete ContainerClient.test.ts (fake cf.Container, constant restatements, direct settings-helper calls); keep its instanceType type check - Keep only the deadline and error-status cases of image preparation - Exercise exec interrupt, streamed stdout and monitor() failures through the native fixture, locally and live, and assert local dev: identities - Add live cases for SSH/authorized keys/observability and for rejecting scheduling-policy switches; re-enable local image replacement - Use real Docker for pull output notes and on-demand pull retry instead of stubbed spawners - Note that the docker exec router and half-close socket wrapper are only needed on Bun 1.3.x
…her in Containers.bind - Pass `binding.raw` to interceptOutboundHttp, interceptAllOutboundHttp and interceptOutboundHttps on the Containers.bind client, matching alchemy-run#2062 - Keep alchemy-run#2062's fix in the Containers.layer handle after the refactor - Fail Containers.layer monitor() with ContainerError instead of a defect, so StartContainer's catchTag("ContainerError") handles crashes - Cover interception through Containers.bind in the native fixture locally; live interception closes the connection with no response (TODO)
9b3ef6b to
1b7a3c1
Compare
Cloudflare routes intercepted container traffic only to a Worker entrypoint or service binding (ctx.exports); Durable Object stubs work in local workerd but not live. Register the Worker's default entrypoint so the intercept assertion runs live as well.
Adds Cloudflare's Durable Object-managed containers (
schedulingPolicy: "durable_object"): one container application exposes several named images, and each Durable Object picks the image, instance size, entrypoint and env when it starts its container. Works inalchemy deployand inalchemy dev(local Docker).Declaring images
Each image is a registry reference or a Docker build (up to 100 named images). Images are prepared and pushed before the Worker uploads, so a new Worker version never references an image that doesn't exist yet.
Choosing an image at runtime
Containers.bindattaches the container to a Durable Object and returns an Effect client. The image names are typed from the declaration, soimages.shellandimages.nodearestring. Starting is explicit, so the container only runs when a request needs it.startalso accepts managed images (image: "cloudflare/debian-trixie") that aren't declared inimages.Exec, stdin and snapshots
execreturns a process owned by the calling scope: closing the scope kills it.stdinis aSink, so input streams in.Snapshots capture the writable filesystem and restore it into a fresh container:
The client also exposes
inspect,monitor,signal,getTcpPort,setInactivityTimeoutand outbound HTTP interception. All runtime methods requireRuntimeContext.Async Workers
The same declaration binds on a plain Worker's
envwith aclassName. The Durable Object then usesthis.ctx.container(images,start,exec,snapshotContainer) directly.Application settings
wranglerSsh,authorizedKeysandobservabilityare managed on the application; removing one clears it.durable_objectis rejected with a clear error instead of being applied in place.Local development
docker execstreams) and snapshots run against local Docker underalchemy dev.Depends on distilled#685 (merged) and builds on #2028 and #2059.
Known limitations
ctx.container.imagesstale even though the Worker's metadata is correct. Those live regressions are marked TODO; local coverage stays enabled.I ran the container tests locally, both under
alchemy dev(workerd + Docker) and live against Cloudflare. All passed:DurableObjectContainer6 (2 todo), image preparation 3,Docker27, andcloudflare-runtimeDocker +NativeContainer25.