An independent analysis of the newly released Kimi K3 model describes it as a much larger version of the earlier Kimi Linear design, grown from 48 billion to 2.8 trillion parameters, making it the biggest openly released model so far. Most of the changes aim at cheaper, faster inference, and the model handles images natively. It is also the first frontier-scale model to drop rotary position encoding entirely, according to the author's knowledge.
What changed
The same design was previously shipped only at 48 billion parameters in Kimi Linear, and frontier models generally kept rotary position encoding in at least some layers.
What it unlocks
Studying and running the largest publicly released open-weight model to date, with built-in handling of images as well as text.
- scaled from 48B to 2.8T parameters
- attention residuals add about 4% training cost and 2% inference cost
Sources