Absolute Zero - A language model reasoning training method developed by Tsinghua University and other institutions.
Absolute Zero is a novel language model inference training method developed by the Tsinghua University LeapLab team in collaboration with the Beijing General Artificial Intelligence Research Institute (NLCo) and Pennsylvania State University. Absolute Zero is based on a model...
What is Absolute Zero?
Absolute Zero is a collaborative project between Tsinghua University's LeapLab team, the Beijing General Artificial Intelligence Research Institute (NLCo) laboratory, and Pennsylvania State University.Absolute Zero introduces a novel language model inference training method. It allows models to autonomously propose and solve tasks, achieving self-evolutionary learning without relying on human-labeled data or predefined tasks. Models receive learnable rewards for proposing tasks and solution rewards for solving them, continuously optimizing their inference capabilities through interaction with the environment. The core of Absolute Zero lies in shifting inference models from human-supervised to environment-supervised learning, enabling models to generate verifiable tasks based on feedback from real-world environments to improve performance.
The main functions of Absolute Zero
- Task self-generationThe model autonomously proposes learnable tasks that are neither too simple nor too complex, providing effective learning signals for the model.
- Task self-solutionThe model acts as a solver, attempting to solve the tasks it proposes or generates, and verifying the correctness of the solution based on environmental feedback.
- Improved reasoning abilityAs tasks are continuously proposed and solved, the model's reasoning ability (such as induction, deduction, abduction, etc.) is continuously enhanced.
- Cross-domain generalizationThe model leverages its general reasoning abilities acquired through self-learning to transfer to new tasks and domains.
- Zero Data TrainingIt is a task that does not rely on manually labeled data or human design, but learns based on interaction and feedback with the environment.
The technical principles of Absolute Zero
- Dual role modelThe model simultaneously acts as both a task proposer and a task solver. The proposer generates the task, and the solver attempts to solve it; both share model parameters and optimize synchronously.
- Environmental feedback mechanismThe model interacts with the environment (such as a code executor), which provides feedback on the solvability of the task. The proposed task receives a learnability reward (based on the task's difficulty and the model's success rate in solving it), while the solved task receives a solution reward (based on the correctness of the solution).
- Reinforcement learning optimization: Optimize model parameters using reinforcement learning algorithms (such as TRR++), and combine learnability rewards and solution rewards to achieve self-evolutionary learning under multiple tasks.
- Inference mode supportIt supports three basic reasoning modes: deduction, abduction, and induction. Each reasoning mode corresponds to a different task type, and the model improves specific reasoning capabilities by solving these tasks.
- Self-game closed loopThe model continuously proposes and solves new tasks, updating its strategy based on environmental feedback, forming a closed-loop self-game learning process. This closed-loop mechanism ensures the model continuously optimizes its capabilities without requiring external data support.
Absolute Zero's project address
- Project official website:https://andrewzh112.github.io/absolute-zero-reasoner/
- GitHub repository:https://github.com/LeapLabTHU/Absolute-Zero-Reasoner
- HuggingFace model library:https://huggingface.co/collections/andrewzh/absolute-zero-reasoner
- arXiv technical paper:https://www.arxiv.org/pdf/2505.03335
Application scenarios of Absolute Zero
- Artificial General Intelligence (AGI)To promote the autonomous learning and evolution of models, gradually approaching the level of human intelligence.
- Code generationIt automatically generates efficient code, solves complex programming problems, and improves development efficiency.
- Mathematical reasoningTo enhance the generalization ability of models on mathematical problems and to support mathematics education and research.
- Natural Language Processing (NLP)It enhances language comprehension and generation capabilities through self-learning, and optimizes text generation and question-answering systems.
- Safety and EthicsTo study the behavioral patterns of AI in its autonomous evolution, ensuring system safety and ethical compliance.