OpenThinker-32B - An inference model jointly open-sourced by Stanford, UC Berkeley, and other institutions.
OpenThinker-32B is an open-source inference model jointly developed by Stanford University, UC Berkeley, the University of Washington, and other institutions. It boasts 32.8 billion parameters and supports a context length of 16,000 tokens. The model uses only 114k of data...
What is OpenThinker-32B?
OpenThinker-32B is an open-source inference model jointly developed by Stanford University, UC Berkeley, the University of Washington, and other institutions. It boasts 32.8 billion parameters and supports context lengths of up to 16,000 tokens. Trained using only 114k data samples, the model performs exceptionally well on multiple benchmarks, outperforming DeepSeek-R1-32B. Key to its success lies in its large-scale data processing, rigorous validation, and model scaling. OpenThinker-32B is fully open-source, including model weights, datasets, data generation code, and training code, providing a valuable resource for the AI community.
Main functions of OpenThinker-32B
- High-efficiency reasoning abilityThe OpenThinker-32B has 32.8 billion parameters, supports a context length of 16,000 tokens, and can handle complex inference tasks.
- High data utilization efficiencyThe model was trained using only 114k data samples, compared to 800k data samples used by DeepSeek-R1-Distill. This demonstrates that OpenThinker-32B has a significant improvement in data utilization efficiency.
- Open source and scalabilityThe OpenThinker-32B model weights, dataset, data generation code, and training code are all open source. Researchers and developers can easily reproduce the model and optimize and extend it.
- Data validation and quality controlThe research team collected reasoning processes and solution attempts for 173,000 questions using the DeepSeek-R1 model, and conducted rigorous validation to ensure data quality and diversity. This high-quality data curation method enabled the model to achieve excellent performance even with a relatively small amount of data.
- Multi-domain applicationsOpenThinker-32B can be applied to multiple fields, including but not limited to code generation, mathematical problem solving, and scientific reasoning.
Technical Principles of OpenThinker-32B
- Data curation and validationOpenThinker-32B is trained using the rigorously selected and validated OpenThoughts-114k dataset.
- Model Architecture and TrainingOpenThinker-32B is based on the Qwen2.5-32B-Instruct model, employs a 64-layer Transformer architecture, and supports a context length of 16k. During training, the team used LLaMa-Factory for three rounds of fine-tuning.
OpenThinker-32B project address
- Project official website:https://www.open-thoughts.ai/blog/scale
- HuggingFace model library:https://huggingface.co/open-thoughts/OpenThinker-32B
Application scenarios of OpenThinker-32B
- Mathematical and scientific problem solvingThe OpenThinker-32B excels in mathematical and scientific reasoning, handling complex mathematical problems, puzzles, and reasoning tasks in scientific fields.
- Code generation and optimizationIt can handle code issues, generate high-quality code solutions, and verify their correctness through test cases. OpenThinker-32B can serve as an intelligent assistant for developers, helping to generate and optimize code and improve development efficiency.
- Cross-domain reasoningThe training data for OpenThinker-32B covers multiple fields, including code, puzzles, science, and mathematics. It can handle various types of reasoning tasks and is suitable for complex scenarios that require the comprehensive application of different knowledge.
- Reinforcement learning researchResearchers can use the OpenThinker-32B model weights, datasets, and training code for reinforcement learning research.