Cerebras Deploys Trillion-Parameter AI Models

The post uses Kimi K3, an open-weight model with about 3 trillion parameters, as a worked example. With 4-bit weight quantization, each of Kimi K3's 93 layers needs about 16GB. Dense layers must store a KV cache holding keys and values for every token in every active request, so they need extra room on their wafer. Linear-attention layers instead keep a small, fixed-size state