Loom

Loom runs large Mixture-of-Experts models (GLM, DeepSeek, Qwen, Kimi) on groups of ordinary machines. The idea is simple. In a model like GLM 5.2, each token touches only about 11 GB of the ~370 GB of expert weights, so there is no need for any one machine to hold the whole model. Compute stays local to each node. What moves between nodes is weights: content-addressed byte ranges of a GGUF file, found through gossip, repaired on churn, and verified against a Merkle manifest before use.

Loom is a single static Zig binary with no runtime dependencies. It has SIMD CPU kernels, a Metal backend on macOS, a Vulkan backend on Linux and Windows, and serves an OpenAI-compatible API.

Design rationale is in the whitepaper, current status in the roadmap, and the wire protocol in the p2p spec. Single-node expert streaming (v0) is done. Distributed weight sharing (v1) works across real machines. Untrusted peers (v2) is a design only.

Quickstart

Install on macOS or Linux (detects your platform, verifies checksums, installs to /usr/local/bin; on Windows download the release zip instead):

curl -fsSL https://raw.githubusercontent.com/ch4r10t33r/loom/main/install.sh | sh

Join the devnet. This installs loom if it is missing, then starts a node that downloads its share of the model's GGUF shards to local disk (about a fifth of the ~11 GB Qwen3-30B-A3B checkpoint by default) and serves an OpenAI API on :8772. The initial sync pulls from the upstream Hugging Face repo over parallel HTTP range requests — every shard digest-verified against the network's Merkle manifest, so the mirror donates bandwidth but cannot alter what your node runs — with the p2p mesh serving whatever the mirror cannot (set LOOM_HTTP_MIRROR="" to sync from peers only):

curl -fsSL https://raw.githubusercontent.com/ch4r10t33r/loom/main/scripts/join-devnet.sh | sh

Or take a 60-second local tour with no network involved:

# zero-config node: generates a tiny model, serves RPC :8770 / P2P :8771
loom node
printf '{"prompt":"hello","max_tokens":16}\n' | nc -w 3 127.0.0.1 8770

# run a real GGUF model (any small GGUF works)
curl -LO https://huggingface.co/ggml-org/models/resolve/main/tinyllamas/stories15M-q4_0.gguf
loom gguf run stories15M-q4_0.gguf --prompt "Once upon a time"

A chat UI is compiled into the binary: run loom node and open http://127.0.0.1:8555 (details).

Nodes can also share retrieval context. With --rag, a text chunk ingested on any node (POST /v1/rag/chunks) gossips to every peer on the network, and each node searches the shared store before generating, prepending what it finds to the prompt. Ingest a fact once, and the whole swarm answers with it.

Requirements

• CPU: any 64-bit x86-64 or arm64. The kernels are portable SIMD. A GPU is optional and used automatically where present: Metal on Apple silicon, Vulkan (-Dgpu=vulkan builds) on Linux and Windows.
• RAM: 8 GB minimum, 16 GB comfortable. --ram-gb (default 4) caps the weight cache, not the whole process.
• Disk: joining a network downloads a shard of the model's GGUF file to the local machine (under the node's home directory) and serves it back to peers. Space scales with --hold-fraction: your fraction of the routed experts, plus the resident chunks every node holds (about 10% of the model). For the devnet's ~11 GB Qwen3-30B-A3B at the default hold-fraction of 0.7, budget about 9 GB free.
• Network: any. A faster link shortens cold-miss expert fetches and the initial sync, nothing else. Peers behind NAT work in dial-out-only mode.

Build from source

Loom targets Zig 0.16.0, pinned in build.zig.zon. anyzig resolves the version automatically.

zig build                        # debug binary -> zig-out/bin/loom
zig build -Doptimize=ReleaseSafe # what releases ship: fast, bounds checks on
zig build test                   # unit tests
zig build -Dgpu=vulkan           # GPU backend on Linux/Windows (Metal is automatic on macOS)
zig build -Dtarget=x86_64-linux-musl -Doptimize=ReleaseSafe -p out   # cross-compile

Releases are built ReleaseSafe rather than ReleaseFast. Loom is a daemon that writes peer-supplied weight shards into buffers, and the roughly 11% speed difference between the two modes is the difference between catching an out-of-bounds write and executing it. Releases are cut by pushing a tag; the tag names the version (no bump commit needed) — see .github/workflows/release.yml. Docker images (linux/amd64 and arm64) publish to ghcr.io/ch4r10t33r/loom:latest (docker guide).

Commands

Command	Purpose
loom node	the daemon: load a model, serve inference (native RPC + OpenAI API), share/sync/repair GGUF weight ranges over P2P
loom gguf	GGUF tools: inspect any GGUF, shard by expert, run deepseek2/llama/qwen/glm models locally or from a partial store
loom light	light node: no weights or engine; delegates RPC/OpenAI to full nodes, metered
loom run	one-shot local inference against a loom checkpoint (no servers)
loom gen / loom info	create / inspect + verify loom-format checkpoints
loom bench / loom iobench	kernel invariants / disk profiler for the engine's random-read pattern
loomtrain	the training executable (Python, train/): pre-gate heads, Recover-LoRA adapters, router retraining — bolt-on artifacts the engine loads beside a frozen model

loom node --network devnet|testnet|mainnet selects a named network. Each network serves exactly one model and rejects peers from other networks, the same way a chain id works. Every command and flag is listed in the CLI reference; worked examples with real output are in the command tour.

Documentation

• Alpha testing guide — join the devnet, what to try, what telemetry is collected
• Command tour — every command, worked examples, protocols (RPC, OpenAI, P2P wire)
• CLI reference — every flag, defaults, env overrides
• Install details — manual install, checksum verification, Gatekeeper
• Training getting started — loomtrain install (Docker/pip/Colab), first run, artifact-to-flag map
• Multi-node walkthrough — shard a real MoE model across a swarm on one host
• Two-machine test — a real cross-machine swarm with measured tok/s
• Named networks — devnet / testnet / mainnet, models and policies
• Performance — CPU SIMD numbers, batching, benchmarking rules
• GPU backends — Metal and Vulkan design and status
• Chat UI & node console
• Docker
• GLM 5.2 target deployment plan · readiness audit
• Repository layout & source map
• Roadmap · p2p spec · whitepaper

License

Apache 2.0. Redistributions must retain the copyright and attribution notices in NOTICE.