Kimi K3 - A 2.8 trillion parameter open-source model released by Dark Side of the Moon

What is Kimi K3?

Kimi K3 is a 2.8 trillion parameter open-source model developed by Dark Side of the Moon. It is built on a KDA hybrid linear attention mechanism and attention residual technology, natively supports visual understanding, and has a context window of 1 million tokens. As the world's first open-source model with a parameter level of 3 trillion, Kimi K3 is designed for cutting-edge intelligent scenarios such as long-range programming, knowledge work, and reasoning, and has achieved leading-edge levels in programming, agent, and knowledge work evaluations.

Kimi K3's main functions

• Long-term programmingIt supports understanding and processing large codebases, continuously completing long-term engineering tasks, and coordinating the use of terminal tools.
• Visual reasoningCombining software engineering and visual reasoning, this approach leverages screenshots and visual feedback to optimize game development, front-end development, and CAD.
• Kernel optimizationIndependently analyze, rewrite, and verify GPU kernel optimizations within the GPU sandbox, iterating continuously for 24 hours.
• Compiler development: Build the Triton GPU compiler (MiniTriton) from scratch to achieve full optimization to the PTX code generation pipeline.
• Interactive CreationIt integrates 3D reasoning, programming, and vision capabilities to transform concepts into playable interactive experiences.
• Chip designBased on open-source EDA tools, we can independently complete chip construction, optimization and verification, and realize chip design that uses the model to serve itself.
• Scientific ProgrammingIt bridges the gap between scientific literature and executable code, enabling the autonomous implementation, verification, and analysis of complex computational research processes.

Technical Principles of Kimi K3

• KDA Hybrid Linear Attention MechanismKimi K3 is built on Kimi Delta Attention (KDA), a hybrid linear attention mechanism that improves long context processing capabilities by allowing information to flow more smoothly in longer sequences and deeper models.
• Attention ResidualsThe model introduces Attention Residuals technology to make information transmission in deep networks more stable and alleviate the gradient decay problem that may occur as the model depth increases.
• Stable LatentMoE Sparse ArchitectureIt adopts the Mixture of Experts (MoE) architecture and combines it with the Stable LatentMoE framework, which efficiently activates only 16 out of 896 experts, greatly improving computational efficiency and parameter utilization.
• Native vision and long context supportThe model natively supports visual understanding, has a context window with 1 million tokens, and can handle extremely long texts and multimodal inputs simultaneously.

How to use Kimi K3

• Official entranceAccess Kimi, the latest version of the Kimi mobile app, or the Kimi Work desktop client to directly invoke the Kimi K3 model for dialogue.
• ProgrammingUse K3 through the Kimi Code client, optimized for development scenarios such as long-range programming, code library understanding, and terminal tool coordination.
• API Access: Call the Kimi API to integrate K3 into your own applications or workflows, and support automation tasks and interface with third-party systems.
• Thinking intensity adjustmentThe current default setting is max (extreme) thinking mode. Future updates will support low and high intensities, which can be switched flexibly according to task complexity.

Kimi K3's core advantages

• Leading in scaleThe world's first open-source model with a scale of 3 trillion has maintained the upper limit of open-source model size for 9 out of the past 12 months.
• Long-range autonomousIt performs stably during long-term engineering tasks, and supports 24 hours of autonomous kernel optimization and 48 hours of chip design agent operation.
• Cost efficiencyIn benchmarks such as Kimi Code Bench and BrowseComp, it achieves near-top-tier closed-source performance at a lower cost.
• Visual closed loopSupports seamless iteration between code and real-time screenshots, achieving true visual feedback optimization (vision in the loop).
• End-to-end researchIt can independently read literature, implement numerical processes, discover inconsistencies in published formulas, and generate interactive dashboards.

Kimi K3 project address

The complete model weights will be released before July 27, 2026.

Kimi K3 Comparison with Similar Products

Application scenarios of Kimi K3

• Large-scale software engineeringThe model can independently analyze and refactor large codebases, continuously completing complex development tasks that would otherwise take hours or even days.
• GPU kernel and compiler optimizationAutomatically optimize GPU kernels for high-performance computing scenarios or build domain-specific compilers from scratch.
• Interactive 3D content developmentCombine visual feedback for rapid iteration, transforming concepts into playable 3D games, simulators, or visualizations.
• Scientific computing and data analysisThe system allows users to independently read literature, reproduce numerical processes, evaluate a large number of state equations, and generate interactive results dashboards.
• Hardware chip designBased on the open-source EDA toolchain, we independently complete the chip architecture design, optimization and verification, and realize chip design that uses the model to serve itself.