project
NVIDIA Nemotron Nano 2 - NVIDIA's High-Efficiency Inference Model
The NVIDIA Nemotron Nano 2 is a high-performance inference model from NVIDIA, with 9 bytes of parameters. The model is based on a hybrid Mamba-Transformer architecture, pre-trained on 20 trillion tokens, and supports a 128k context length.
What is NVIDIA Nemotron Nano 2?
The NVIDIA Nemotron Nano 2 is a high-performance inference model from NVIDIA, with 9 bytes of parameters. Based on a hybrid Mamba-Transformer architecture, it is pre-trained on 20 trillion tokens and supports a 128k context length. Compared to Qwen3-8B, it offers a 6x speedup inference while maintaining comparable or higher accuracy. The model features think budget control, allowing users to specify the number of inference tokens. NVIDIA has open-sourced the base model and most of the pre-trained datasets to support further research and applications by developers.
Key features of NVIDIA Nemotron Nano 2
- High throughputThe NVIDIA Nemotron Nano 2 performs exceptionally well in complex inference tasks, delivering up to 6 times the throughput of the Qwen3-8B.
- Long context supportIt supports context lengths of up to 128k and can perform inference on a single NVIDIA A10G GPU, making it suitable for handling long texts and complex tasks.
- Reasoning process supportThe model generates a reasoning trace before producing the final answer, and the user can specify the model's "thinking" budget.
- Flexible output modesUsers can choose to skip intermediate reasoning steps and directly obtain the final answer.
- Multilingual abilityThe pre-trained dataset contains data in multiple languages, supporting powerful multilingual reasoning capabilities.
- Multi-field coverageIt covers data from multiple fields such as mathematics, coding, academia, and STEM, and is suitable for various application scenarios.
The technical principles of NVIDIA Nemotron Nano 2
- Hybrid Mamba-Transformer architectureReplacing most of the self-attention layers in a traditional Transformer with Mamba-2 layers significantly improves inference speed, especially when generating long inference chains. The Transformer layers retain some self-attention layers to maintain the model's flexibility and accuracy.
- Pre-training processPre-trained on 20 trillion tokens, with FP8 accuracy and Warmup-Stable-Decay learning rate scheduling. Through continuous pre-training of a long context expansion stage, the model can handle context lengths of 128k without degrading performance on other benchmarks.
- Post-training optimizationSupervised fine-tuning (SFT) involves supervising and fine-tuning the model to improve its performance on specific tasks. This includes: improving the model's obedience to instructions through policy optimization; optimizing the model's preferences to better align with human preferences; and using reinforcement learning based on human feedback to enhance the model's conversational and instruction-following abilities.
- Model compressionBased on pruning and knowledge distillation techniques, the base model with 12B parameters is compressed to 9B parameters while maintaining model performance. The optimized model supports contextual reasoning with 128k tokens on a single NVIDIA A10G GPU, significantly reducing inference costs.
- Reasoning Budget ControlBased on truncated training, the model can reason according to the user-specified "thinking" budget, avoiding unnecessary computation. Users can flexibly control the model's reasoning process, choosing whether to display the reasoning process or directly obtain the final answer.
NVIDIA Nemotron Nano 2 project address
- Project official website: https://research.nvidia.com/labs/adlr/NVIDIA-Nemotron-Nano-2/
- HuggingFace model library: https://huggingface.co/collections/nvidia/nvidia-nemotron-689f6d6e6ead8e77dd641615
- Technical Papers: https://research.nvidia.com/labs/adlr/files/NVIDIA-Nemotron-Nano-2-Technical-Report.pdf
- Experience the demo online: https://build.nvidia.com/nvidia/nvidia-nemotron-nano-9b-v2
Application scenarios of NVIDIA Nemotron Nano 2
- EducationIn the field of education, it helps students solve complex mathematical and scientific problems. It explains complex mathematical formulas or physical laws through step-by-step reasoning, helping students better understand and master knowledge.
- academic researchResearchers conduct academic research, generating detailed reasoning processes and analysis reports to assist in paper writing and experimental design.
- Software developmentIt generates high-quality code snippets for developers, helping them to quickly develop and optimize code.
- Programming EducationIn programming education, models provide code examples and explanations to help beginners better understand programming languages and algorithms.
- Customer ServiceIn the field of customer service, it serves as a multilingual chatbot, providing efficient and accurate customer support.