Model1 - A new model version updated in the DeepSeek codebase.

What is Model 1?

Model1 is a mysterious model revealed in an update to DeepSeek's FlashMLA codebase, possibly the internal codename or the first engineering version of its next-generation flagship model, DeepSeek-V4. The model features several technical innovations, including a return to the standard 512-dimensional architecture, full compatibility with NVIDIA's Blackwell architecture (SM100), and the introduction of new mechanisms such as Token-level Sparse MLA and VVPA, optimizing performance, improving long text processing capabilities, and enhancing hardware compatibility. Currently, Model1 is still under development, and specific details and capabilities await official confirmation from DeepSeek.

Main functions of Model 1

• High-performance computingFully compatible with NVIDIA's Blackwell architecture (SM100), achieving 350 TFlops of sparse operators on the B200 GPU, significantly improving computational efficiency.
• Long text processingThe system introduces a hybrid precision scheme of Token-level sparse MLA and FP8 KV Cache to reduce memory pressure and improve the speed of long text inference through sparse inference.
• Location awarenessThe model employs a VVPA (numerical vector position awareness) mechanism to address the issue of positional information attenuation in long text scenarios using traditional MLA, thereby enhancing the model's ability to understand and generate long texts.
• Distributed storage optimization: Introducing the Engram mechanism for distributed storage or key-value compression, in order to meet high throughput requirements, optimizes the storage and computation efficiency of the model.

The technical principles of Model 1

• Architectural Regression 512-Dimensional StandardThe core architecture of Model 1 has reverted to 512 dimensions, unlike the 576-dimensional architecture of DeepSeek-V3. This revert to a 512-dimensional architecture is intended to better align with the computing power of next-generation hardware (such as the Blackwell architecture) or to optimize the latent compression ratio.
• Sparse and Mixed Precision InferenceModel 1 introduces a token-level sparse MLA (multi-head self-attention mechanism), using FP8 to store the KV cache during inference and bfloat16 to ensure precision in matrix multiplication calculations. This sparsity and mixed precision scheme significantly reduces memory usage and improves inference speed.
• Location-aware mechanism (VVPA)To address the issue of positional information attenuation in long texts, Model 1 introduces the VVPA mechanism to enhance the model's ability to perceive positional information and improve the processing performance of long texts.
• Hardware adaptation and optimizationModel 1 is heavily optimized for NVIDIA’s Blackwell architecture (SM100), including dedicated CUDA instruction set optimizations and support for CUDA 12.9, ensuring optimal performance on next-generation hardware.

Project address for Model1

• GitHub repositoryhttps://github.com/deepseek-ai/FlashMLA

Application scenarios of Model 1

• Natural Language Processing (NLP)Model1 can be used for high-quality text generation, machine translation, and sentiment analysis. With its long text processing capabilities, it can generate coherent and logical content, making it suitable for tasks such as news reporting, creative copywriting, and multilingual translation.
• Intelligent Customer ServiceModel 1 supports intelligent question answering and multi-turn dialogue, which can quickly understand user intent and provide personalized solutions, suitable for real-time customer support and answers to complex questions.
• Content creationModel 1 can assist in creative writing and video script generation, providing writers and screenwriters with inspiration and structured content support, and improving creative efficiency.
• EducationModel 1 can generate personalized learning suggestions, practice questions, and explanations, providing intelligent tutoring for students while also assisting in language learning and grammar correction.
• HealthcareModel 1 can be used to analyze medical reports and medical records to assist doctors in making diagnoses and developing treatment plans, while also providing patients with health consultations and preliminary diagnostic suggestions.