DeepRAG - A novel search enhancement and generation framework jointly launched by the Chinese Academy of Sciences, the University of Science and Technology of China, and Tencent WeChat AI Department.

What is DeepRAG?

DeepRAG is a novel Retrieval Augmentation (RAG) framework jointly developed by the Chinese Academy of Sciences, the University of Science and Technology of China, and Tencent WeChat AI Department. It enhances the performance of large language models (LLMs) in reasoning and retrieval tasks. By modeling retrieval-enhanced reasoning as a Markov Decision Process (MDP), it introduces two key components: "retrieval narrative" and "atomic decision," achieving a structured and adaptive retrieval process. The core of DeepRAG lies in dynamic decision-making: at each step, the model decides whether to retrieve external knowledge or rely solely on its own parameter knowledge for reasoning based on the current state.

DeepRAG's main functions

• Dynamic retrieval decisionDeepRAG can dynamically determine at each step whether to retrieve external knowledge or rely solely on the model's own parameter knowledge for inference. This avoids unnecessary retrieval operations, improves retrieval efficiency, and reduces noise and inference latency.
• Structured search processBy introducing a "retrieval narrative," DeepRAG ensures that the retrieval process is structured and adaptive. The model generates new subqueries based on previously retrieved information, progressively advancing the reasoning process and better handling complex queries.
• Knowledge boundary calibrationDeepRAG uses a "chain of calibration" method to help models more accurately identify the boundaries of their own knowledge. This allows models to better determine when to retrieve external knowledge and rely on their own knowledge for reasoning, thus improving the accuracy and reliability of their answers.
• Improve search efficiency and answer accuracyExperiments show that DeepRAG significantly improves the accuracy of answers on multiple open-domain question-answering datasets (an average improvement of 21.99%), while reducing the number of searches and optimizing search efficiency.
• Generalization ability and robustnessDeepRAG demonstrates good generalization ability and robustness in time-sensitive and out-of-distribution question answering tasks, and can adapt to different question answering scenarios and data distributions.

DeepRAG's technical principles

• Markov Decision Process (MDP) ModelingDeepRAG models the retrieval augmented reasoning process as an MDP, representing a partial solution to the original problem. The state `st` includes the input problem `x` and the subqueries up to the present time and their intermediate answers. It includes two sub-decision steps:
  • Termination Decision: Decide whether to continue generating the next subquery or directly generate the final answer.
  • Atomic Decision: Decide whether to retrieve external knowledge or rely solely on the model's parameter knowledge.
• TransitionsUpdate the state based on the action. If a search is decided upon, the model will retrieve relevant documents and generate an intermediate answer; if no search is decided upon, an intermediate answer will be generated directly.
• RewardsThe evaluation state is based on the correctness of the answer and the retrieval cost. The reward function maximizes the correctness of the answer while minimizing the retrieval cost.
• Binary Tree SearchDeepRAG uses a binary tree search approach to construct an inference path for each subquery. For each subquery, the model generates two branches: one branch generates the answer directly based on parametric knowledge, and the other branch generates the answer by retrieving external knowledge. In this way, the model explores different retrieval strategies, generating a complete inference path from the initial question to the final answer. Binary tree search decomposes the question and thoroughly examines the impact of retrieval choices on the final answer.
• Imitation LearningThe goal of imitation learning is to enable the model to learn effective retrieval patterns. Priority queues are used to efficiently explore potential reasoning trajectories, prioritizing paths with lower retrieval costs. Synthetic data is searched using binary trees to extract the reasoning process leading to the correct final answer. The model is then fine-tuned using synthetic data to improve its ability to make terminating and atomic decisions, while also enhancing its query decomposition and generation of reliable intermediate answers.
• Chain of CalibrationThe calibration chain method further optimizes the model's perception of its own knowledge boundaries. It determines when retrieval is needed by synthesizing preference data. This data is generated based on the optimal path, indicating the preferred retrieval strategy for each subquery. Using this preference data, the model is fine-tuned, enhancing its atomic decision-making ability based on its internal knowledge boundaries. Through the calibration chain method, the model can more accurately identify when external knowledge needs to be retrieved and can rely on its own knowledge for reasoning.

DeepRAG's project address

• arXiv technical paper:https://arxiv.org/pdf/2502.01142

Application Scenarios of DeepRAG

• Open Domain QuestionsDeepRAG can handle complex multi-hop question-answering tasks by breaking down questions step by step and dynamically retrieving relevant information to generate accurate answers.
• Knowledge base Q&AIn knowledge base question answering, DeepRAG can combine external knowledge bases (such as Wikipedia) with the model's own knowledge to provide more accurate answers.
• Intelligent customer service and virtual assistantDeepRAG can be applied to intelligent customer service systems to provide more accurate and timely customer support through dynamic retrieval and reasoning.
• Education and learning supportIn the field of education, DeepRAG can help students and teachers acquire more accurate knowledge and information. It dynamically generates learning materials and exercises based on students' learning progress and needs.
• Medical and health consultationDeepRAG can be used for medical and health consultations, providing accurate health advice by retrieving the latest medical research and clinical guidelines.