Seed-Thinking-v1.5 - ByteDance's latest thinking model

What is Seed-Thinking-v1.5?

Seed-Thinking-v1.5 is an intelligent reasoning model launched by ByteDance. It adopts a hybrid expert (MoE) architecture with a total of 200B parameters, activating 20B parameters per session. The model performs exceptionally well in multiple benchmark tests, achieving a score of 86.7 in AIME 2024, 55.0 in the Codeforces pass@8 metric, and 77.3 in the GPQA test. In non-reasoning tasks, it outperforms DeepSeek R1 by 8%, demonstrating broad applicability. The development of Seed-Thinking-v1.5 involved carefully curated training data, an advanced reinforcement learning framework, a two-layer reward system, and efficient infrastructure. The model will be available for user testing via the Volcano Engine open interface on April 17th.

Main features of Seed-Thinking-v1.5

• Exceptional reasoning abilitySeed-Thinking-v1.5 has performed exceptionally well in multiple authoritative benchmark tests, including an AIME 2024 score of 86.7, a pass@8 rating of 55.0% in Codeforces, and a GPQA score of 77.3. This demonstrates its strong reasoning capabilities in STEM (science, technology, engineering, and mathematics) fields and programming.
• Broad generalization abilityThe model also performs well in non-inference tasks, with a win rate 8% higher than DeepSeek R1, demonstrating its advantage in handling complex user scenarios.
• efficient infrastructureTo support large-scale training, Seed-Thinking-v1.5 adopts the HybridFlow programming model and Streaming Inference System (SRS), which optimizes training efficiency through a three-layer parallel architecture (tensor/expert/sequence parallelism).

Technical Principles of Seed-Thinking-v1.5

• Hybrid Expert Model (MoE) ArchitectureSeed-Thinking-v1.5 employs a Mixture-of-Experts (MoE) architecture with a total of 200 parameters, activating 20 parameters at a time. This significantly reduces computational resource consumption and improves model efficiency while maintaining high performance.
• Reinforcement learning algorithms
  • VAPO and DAPO frameworksTo address the instability issue in reinforcement learning training, the research team proposed two major frameworks: VAPO (Actor-Critic) and DAPO (Policy Gradient). These two methods provide robust training trajectories and effectively optimize inference models.
  • Reward ModelingThe team designed two reward modeling schemes: Seed-Verifier and Seed-Thinking-Verifier. Seed-Thinking-Verifier addresses issues such as reward deception, prediction uncertainty, and failure to handle boundary cases through a detailed reasoning process.
• Data processing and augmentation
  • Verifiable issuesIt includes mathematical, programming, and logic puzzles, and through rigorous screening and enhancement (such as converting multiple-choice questions into fill-in-the-blank questions), it ensures that the model learns real reasoning abilities.
  • Unverifiable problemFor example, in creative writing and dialogue, low-variance samples are dynamically filtered to prevent the model from getting stuck in local optimization.
  • Data augmentation strategyFor example, models can be used to generate candidate answers, and manual verification can be used to correct incorrect reference answers, thereby improving data reliability.
• Distributed training infrastructure
  • Hybrid parallel architectureIt combines tensor parallelism (TP), expert parallelism (EP), and context parallelism (CP) to support efficient large-scale training.
  • Streaming generation system (SRS)By using asynchronous processing and dynamic resource scheduling, the efficiency of long text generation is improved by 3 times, solving the "tailing problem" of traditional synchronous frameworks.
  • Automatic tuning systemThe system dynamically selects the optimal computing configuration based on real-time load, balancing memory and computing resources.

Project address for Seed-Thinking-v1.5

• Github repository:https://github.com/ByteDance-Seed/Seed-Thinking-v1.5

Performance of Seed-Thinking-v1.5

• Mathematical reasoningIt scored 86.7 points in the AIME 2024 test, on par with OpenAI's o3-mini-high.
• Programming taskIn Codeforces benchmarks, the pass@8 metric reached 55.0%, outperforming DeepSeek-R1.
• Science Q&AIt achieved an accuracy of 77.3% in the GPQA test, approaching the level of top-tier models.
• Non-reasoning tasksIn non-reasoning tasks, the win rate is 8% higher than DeepSeek R1, demonstrating strong generalization ability.

Application Scenarios of Seed-Thinking-v1.5

• Science Q&ASeed-Thinking-v1.5 also demonstrates significant performance in scientific question answering. The model can understand and answer complex questions involving scientific concepts and principles, making it suitable for both education and research.
• Creative WritingThe model can generate cross-temporal dialogues, such as simulating the inner monologues of historical figures or narratives that blend terminology from different fields. It has potential applications in content creation, advertising, scriptwriting, and other fields.
• Logical reasoningSeed-Thinking-v1.5 has significant advantages in handling problems that require logical analysis and reasoning, and is suitable for scenarios that require logical judgment and analysis, such as legal analysis and market strategy planning.
• Educational SupportSeed-Thinking-v1.5's reasoning capabilities can help students solve math and science problems, provide feedback for programming exercises, and support language learning.