EAPO - A new reinforcement learning framework launched by Alitongyi

What is EAPO?

EAPO is a long-text reasoning reinforcement learning framework launched by Alibaba Tongyi Lab. By introducing an "evidence reward" mechanism, it shifts the supervision signal from the final answer to the evidence extraction process. The framework has been accepted by ACL 2026 and has enabled a 30B model to outperform closed-source large models such as GPT-OSS and Claude-Sonnet-4 with a capacity of 120B in eight authoritative long-text benchmark tests.

EAPO's main functions

• Structured Evidence ReasoningThe forced model executes a four-step workflow: task analysis, evidence extraction, reasoning execution, and answer generation. Each step is separated by a special token, allowing the intermediate evidence status to be directly monitored.
• Multi-granularity process reward: Construct a composite reward signal, including format compliance reward, group relative evidence quality reward, and result accuracy reward, shifting from sparse result supervision to intensive process supervision.
• Group relative evidence assessmentMultiple evidence trajectories are sampled for the same issue, and a reward model uniformly evaluates them and gives an integer quality score of 1-5. The scores are then normalized within the group to generate relative rewards, guiding the model to prioritize the extraction of high-quality evidence.
• Reward-Strategy Co-evolutionThe design incorporates an adaptive closed-loop mechanism that filters high-confidence, consistent, and high-quality evidence chains generated by the strategy model and feeds them back into the reward model for rejection fine-tuning, allowing the evaluation criteria to dynamically evolve as the model's capabilities improve.
• Long text reasoning enhancementSignificantly improves performance on eight authoritative long text benchmarks, including SEAL and LongBench-V1/V2, enabling the 30B model to outperform closed-source large models such as 120B GPT-OSS and Claude-Sonnet-4 in long text inference.

EAPO's technical principles

• Evidence-Augmented Reasoning (EAR) paradigmThis approach establishes a structured reasoning paradigm, forcing the model to extract relevant evidence fragments word-for-word from the original text before generating an answer. By using special tokens, the process is broken down into four steps: "task analysis → evidence extraction → reasoning execution → answer generation." This exposes the intermediate evidence state and allows for direct supervision, fundamentally solving the illusion problem of traditional models where "guessing the right answer but quoting the wrong one."
• Group-Relative Evidence RewardEAPO shifts the optimization objective of reinforcement learning from "correct outcome" to "correct evidence." During training, the model samples multiple different evidence trajectories for the same question. The reward model evaluates these evidence sets simultaneously, assigning a utility score of 1-5, normalized to a relative reward of [0,1] within the sampled group. This intensive process supervision teaches the model that "finding the right evidence" is more important than "guessing the right answer," effectively suppressing parameterization shortcuts.
• Adaptive Reward-Policy Co-EvolutionAs the policy model's capabilities improve, the fixed-reward model gradually becomes unable to distinguish subtle differences in the quality of evidence. EAPO designs a self-reinforcing closed loop: through Outcome-Consistent Rejection Fine-Tuning, it filters out high-confidence rollout data of "high evidence scores and correct answers" or "low scores and incorrect answers," using these instances to supervise and fine-tune the reward model. The stronger the policy model, the better the generated training data; the more accurate the reward model's evaluation, the more it can in turn guide the policy model to extract more precise evidence, achieving dynamic synchronous evolution.
• GRPO-based composite reward mechanismEAPO uses Group Relative Policy Optimization as its base algorithm, constructing a multi-granularity composite reward: a format compliance reward (α=0.1) ensures the output conforms to the EAR structure; a group relative evidence quality reward (β=0.3) provides process supervision; and a result accuracy reward (γ=0.6) verifies the final answer. The weighted combination of these three rewards transforms sparse result signals into dense process-oriented guidance.

Key information and usage requirements of EAPO

• R&D teamThe paper, authored by Xin Guan, Zijian Li, Shen Huang, and others from Alibaba Tongyi Lab, has been accepted by ACL 2026.
• Base modelTraining is based on three models: Qwen3-14B (Dense architecture), Qwen3-30B-A3B-Instruct (MoE architecture), and Qwen3-30B-A3B-Thinking.
• Context lengthThe training and evaluation samples are uniformly limited to 128K tokens.
• Training data4,664 composite samples, covering 32K-128K context-filled multi-hop QA (MuSiQue) and Wikipedia hybrid QA (structured + unstructured).
• Reward ModelInitialized based on Qwen3-30B-A3B-Thinking, updated every 20 RL steps.
• Reward WeightThe format follows α=0.1, the quality of evidence β=0.3, and the accuracy of the result γ=0.6.
• Core AlgorithmBased on GRPO (Group Relative Policy Optimization), it introduces a group relative evidence reward and co-evolution mechanism.
• Model BasicsIt needs to be trained on the Qwen3 series model that supports long text, and 30B-A3B-Thinking is recommended for the best results.

EAPO's core advantages

• Process supervision innovationBreaking through the sparse reward bottleneck of long text RL, it achieves evidence-level dense process supervision for the first time, forcing the model to execute a four-step workflow of "analysis → evidence extraction → reasoning → response", making each step of reasoning traceable.
• Performance BreakthroughThe model trained based on Qwen3-30B-Thinking achieved an average score of 63.1%, surpassing large closed-source models such as GPT-OSS (120B) and Claude-Sonnet-4, demonstrating that a small model can outperform a large model.
• Error double dropThe evidence error rate decreased from 17.7% to 13.5%, and the reasoning error rate decreased from 20.7% to 15.4%, achieving a simultaneous decrease in both types of errors without explicit supervision of the reasoning process.
• Leading in training efficiencyCompared to the GRPO baseline, which relies solely on outcome rewards, EAPO converges significantly faster and has a higher accuracy ceiling, consistently leading in evidence quality score throughout the training process.
• Evolution of evaluation criteriaThe reward model and the policy model continuously iterate and co-evolve through high-confidence rollout data, avoiding fixed evaluation criteria as the model's capabilities improve.

EAPO project address

• Technical Papers: https://arxiv.org/pdf/2601.10306

Comparison of EAPO's similar competing products

EAPO application scenarios

• AI Search and Question AnsweringTo address the core pain point of AI search engines that "search correctly but answer incorrectly," the system forces the model to accurately locate and cite supporting evidence from massive search results, thus eliminating illusion-based responses.
• Professional domain document analysisIt applies to legal, financial, medical, and other scenarios that require strict factual basis, ensuring that every report and every analysis has a clear original source and is supported by a chain of evidence.
• Scientific Literature ReviewIt supports cross-validation and comprehensive reasoning across multiple papers, automatically extracts key experimental data and cites it accurately, ensuring that all review conclusions are supported by literature.
• Enterprise Knowledge Base Q&AIt accurately locates the basis for decision-making in extremely long internal documents, contracts, and manuals, helping employees quickly obtain business answers with clear sources.
• Educational tutoring and automatic gradingWhen used for problem-solving guidance, it requires that each step of reasoning be marked with original text evidence, and during automatic grading, it verifies whether the student's answer is based on valid information in the question material.