Serving a 3-Trillion-Parameter Model Takes 59 Wafers, and Cerebras Has Published the Arithmetic

Most arguments about serving giant models are arguments about memory. Cerebras published a walkthrough on October 22 that makes the tradeoff explicit, using Moonshot's Kimi K3, roughly 3 trillion parameters across 93 layers, as the worked example.

The arithmetic starts with weight precision. At 4-bit quantization, comparable to the Hugging Face implementation, a 3-trillion-parameter model needs about 1.5 terabytes just for weights, which works out to roughly 16 gigabytes per layer. Each WSE-3T wafer carries 44 gigabytes of on-chip SRAM, so the question becomes how many layers fit on one wafer once the cache is accounted for.

Kimi K3's architecture makes the split unusually easy. Sixty-nine of its layers use linear attention and keep only a fixed-size state per request, so two 16-gigabyte layers fit on a wafer with room to spare: 35 wafers for that group. The other 24 layers are dense and need KV cache space, so Cerebras places one per wafer and leaves the remaining 28 gigabytes for an 8-bit cache. Thirty-five plus 24 gives 59 wafers.

What moves between wafers is activations, not weights. Cerebras says the weights stay resident on their assigned stages and token paths are passed over RDMA over Ethernet at roughly 2 microseconds per hop, which lets layers stay whole instead of being split. Each wafer becomes a pipeline stage, and because stages handle different requests at the same time, requests stream through rather than waiting for a batch to fill.

The concurrency figure follows from the same assumptions. A 256,000-token average request needs about 147 megabytes of cache, and a dense-layer wafer has about 28 gigabytes to give, which produces an estimate of roughly 190 requests in flight. Smaller micro-batches make each individual request faster; larger ones raise aggregate throughput. Cerebras frames the useful efficiency metric as tokens per second per megawatt at a fixed output speed, rather than tokens per second alone.

The performance claims in the post are the vendor's own: 1,000 to 2,000 or more tokens per second on multi-trillion-parameter models, and Kimi K3 benchmark figures of 88.3 percent on Terminal-Bench 2.1 against 88.8 percent for GPT-5.6 Sol, and 91.2 percent against 90.4 percent on BrowseComp. Those comparisons are selected by the company, describe model quality rather than serving throughput, and have not been independently reproduced here.

The illustrative nature of the exercise is the point worth keeping. The 59-wafer figure rests on stated assumptions, namely 4-bit weights, an 8-bit cache and 256,000-token average requests, and Cerebras presents the layer placement as a configuration rather than a production benchmark. Change the request size and the concurrency number moves; change the precision and the wafer count moves.

Still, the disclosure is useful because it prices the alternative. Open weights at three trillion parameters are only as open as the hardware that can hold them, and putting a terabyte and a half of weights into SRAM across 59 wafers is one answer to a question every inference provider is now asked: what does it cost to serve the model you just released.