QwenLong-L1-32B - Alibaba's open-source long text inference model based on Qwen-Doc

What is QwenLong-L1-32B?

QwenLong-L1-32B is the first large-scale long-text reasoning model open-sourced by Alibaba Group's Qwen-Doc team. Based on progressive context expansion, course-guided reinforcement learning, and difficulty-aware retrospective sampling strategies, the model significantly improves reasoning capabilities in long-text scenarios. The model performs exceptionally well on multiple Document Question Answering (DocQA) benchmark tests, achieving an average accuracy of 70.7%, surpassing existing flagship models such as OpenAI-o3-mini and Qwen3-235B-A22B, and comparable to Claude-3.7-Sonnet-Thinking. QwenLong-L1-32B can handle complex multi-hop reasoning, logical reasoning, and mathematical reasoning problems, making it suitable for multiple fields such as law, finance, and scientific research, demonstrating powerful long-text processing and reasoning capabilities.

Main functions of QwenLong-L1-32B

• Long text reasoningIt can handle complex long text tasks, such as multi-hop reasoning, logical reasoning, and mathematical reasoning.
• Stable trainingBased on course-guided reinforcement learning and difficulty-aware retrospective sampling, the training process is kept stable.
• Mixed rewardsIt combines rule-based and model-based rewards to balance precision and recall.
• Wide applicabilityIt is suitable for a variety of practical applications, such as legal document analysis, financial report interpretation, and scientific research paper reading.
• High performanceIt outperforms existing flagship models such as OpenAI-o3-mini and Qwen3-235B-A22B in multiple long-text document question answering (DocQA) benchmark tests.

Technical Principles of QwenLong-L1-32B

• Progressive context expansionThe training process is divided into multiple stages, gradually increasing the context length to ensure the model can stably adapt to longer contexts at each stage. Sampling is performed based on the difficulty of the samples, prioritizing complex samples to encourage the model to explore more deeply.
• Hybrid reward mechanismThe accuracy of the model output is ensured by strictly matching the final answer and validating the format. A small language model is used as an evaluator to determine the semantic equivalence between the generated answer and the standard answer, thereby improving the model's recall rate.
• Reinforcement learning algorithmsThe strategy is optimized based on group relative advantage estimation, avoiding the use of an additional value network and reducing computational complexity. Combining a high shearing threshold, dynamic sampling strategy, label-by-label loss, and excessively long reward shaping ensures a more stable and efficient RL process.
• Pre-training and fine-tuningWe used pre-trained short text inference models (such as R1-Distill-Qwen-14B and R1-Distill-Qwen-32B) as the base model. Before RL training, we performed supervised fine-tuning based on high-quality labeled data to provide the model with a robust initial strategy.

QwenLong-L1-32B Project Address

• GitHub repository:https://github.com/Tongyi-Zhiwen/QwenLong-L1
• HuggingFace model library:https://huggingface.co/Tongyi-Zhiwen/QwenLong-L1-32B
• arXiv technical paper:https://arxiv.org/pdf/2505.17667

Application scenarios of QwenLong-L1-32B

• legal fieldIt analyzes legal documents, extracts key information, answers complex legal questions, and supports legal case analysis and judgment prediction.
• Financial sectorProcessing financial reports, conducting data analysis and forecasting, and supporting financial decision-making and risk management.
• scientific research fieldExtracting experimental results and conclusions from scientific research papers to assist in scientific research and academic writing.
• EducationIt assists in teaching, providing personalized learning content and answers, and supports online courses and intelligent tutoring.
• Intelligent Customer ServiceHandle complex user inquiries, provide accurate answers and suggestions, and support customer service in fields such as finance and technology.