In a 47-page technical report, Kimi K3, Moonshot AI demonstrates that it is setting industry challenges rather than merely following existing solutions. The main thesis of the report is the existence of two key axes of artificial intelligence scaling: computational resources before deployment for larger models and computational resources after deployment for inference.
The K3 report shows that open source excels in the second aspect but faces stagnation in the first, as most models reach the level of 1 trillion. K3 advances both axes, achieving a total volume of 2.78 trillion with 104.2 billion active parameters per token, more than double the figures of DeepSeek V4 Pro.
Although K3 utilizes some innovations from DeepSeek, it also establishes its own challenges, such as investigating the DSA core case and mathematically proven perfect expert equilibrium, which improve upon the V3 version. The report signals that the chosen direction was correct, but better execution is now required. Three architectural innovations concern sequence measurement, depth, and model width scaling.
For sequence handling, K3 replaces 75% of attention layers with linear KDA attention, which supports a fixed-size state buffer for linearly scalable computations. One MLA layer is maintained for every three KDA layers to ensure a global receptive field. A significant improvement is the addition of a lower bound to the KDA decay rate, allowing all computations to run on fast GPU tensor cores instead of requiring a special path. Furthermore, KDA completely eliminates positional encoding, enabling native processing of 100K tokens without the need for engineering interpolation.
Regarding depth, the AttnRes mechanism provides each layer with extraction parameters, allowing direct access to the output of any previous layer among the 93 layers organized into 9 compressed blocks. This solves the feature dilution problem created by standard residual connections during scaling. For width, Stable LatentMoE is used with 896 experts and 16 active per token. It applies quantile-based load balancing, which calculates scheduling preferences derived from the distribution in a single pass. It is mathematically proven that this ensures perfect equilibrium without unstable auxiliary penalties.
The training infrastructure demonstrates the scale of the project. The MoonEP inter-process communication library, optimized for expert parallelism, is mathematically proven to require only a minimal number of redundant experts for guaranteed balanced distribution under any load, with training never being interrupted. The reinforcement learning sandbox, based on lightweight virtual machines, created 51.2 million instances during training and evaluation, with a state snapshot time of 133 ms and a recovery time of 49 ms. After training, nine self-developed teachers in general, agentic, and software domains, each with three levels of reasoning complexity, are distilled into a single model.
The results show that the approach considering both axes yields tangible results: the BrowseComp score is 91.2% at half the cost of GPT-5.6 Sol, and the Kimi Code Bench score is 4 points lower than Claude Fable 5, at 38% of its cost.
The K3 report prompts DeepSeek to react to architectural changes, not just pricing. The document serves a dual function: it is documentation and simultaneously a provocation, positioning Moonshot AI as an architect of the field rather than a follower. The debate between K3 and V4 defines the next stage of development.
Moonshot AI has officially released the open-source Kimi K3 model. This model features 2.8 trillion MoE parameters, incorporates 896 routed experts, supports 16 active experts per token, and boasts a context window of 1 million tokens. Additionally, the infrastructure tools MoonEP, FlashKDA, and AgentEnv have been released.
The launch of Kimi K3 occurred on July 27th. The model has become the largest in the world among open-weight models. Compared to the previous generation K2.5, the parameter scale of K3 has increased by approximately three times, and computational efficiency has risen by 2.5 times due to the use of Kimi Delta Attention, Attention Residuals, and MoonEP technologies, enabling operation under limited computational resources.
The technical report reveals several architectural novelties. The hybrid KDA attention structure combined with Gated MLA merges linear attention for efficient long-context processing and gated multi-head latent attention for high-quality local representation. The Stable LatentMoE routing mechanism addresses the expert collapse problem often encountered when training large MoE models by stabilizing the router's gradient dynamics. Furthermore, the MoonViT-V2 vision encoder training scheme ensures native multimodal interpretation without the need for separate vision and language adapters. The model was trained and tested on universal reasoning, agent, and coding tasks, demonstrating K3's broad capabilities beyond purely linguistic tasks.
In addition to the model itself, three infrastructure tools were opened up. MoonEP is a high-performance communication library designed for large-scale MoE training, which eliminates the all-to-all communication bottleneck that limits MoE scalability. FlashKDA provides a high-performance computation kernel for Kimi Delta Attention; benchmarks show it achieves a prefill speed improvement from 1.72x to 2.22x on the H20 GPU compared to the baseline flash-linear-attention. AgentEnv is an agent sandbox system developed using KVCache.ai, supporting fast snapshot creation, restoration, and forking for large-scale agent training environments. These infrastructure releases lower the barrier for reproducing and further developing K3-class models.
The open-source release has significant implications for the ecosystem. K3 meets or surpasses closed state-of-the-art models such as GPT-5.6 and Claude Fable 5 across several specific tests, particularly in frontend code generation. Moreover, the cost of using K3 via API is 40% lower than Claude and 70% lower than Fable. The open-weight license allows for local deployment and the development of proprietary applications without API dependency. However, due to its 2.8 trillion parameter scale, even local implementation requires substantial hardware—at least 10 GB300 class GPUs just to load the model, making inference infrastructure costs the primary factor in deployment, rather than API fees.
This release triggered a chain reaction in the ecosystem. Alibaba Cloud's Qianwen AI platform announced K3 availability. Huawei announced the adaptation of Ascend 0-day for K3, marking the first instance of launching a 3-trillion-level model on domestic hardware. The open-source community gained access to model weights that previously required hundreds of millions in investment to reproduce. Moonshot AI stated that the open-source initiative aims to advance the open-weight model ecosystem, lowering barriers to development and deployment for the broader AI community. However, shortly thereafter, it was announced that K3 suspended new user subscriptions due to computational capacity constraints, highlighting the contradiction between open-source ideals and the reality of serving a 2.8 trillion parameter model at scale.
