MaskSearch - A retrieval enhancement pre-training framework launched by Alibaba Tongyi

What is MaskSearch?

MaskSearch is a novel general-purpose pre-training framework launched by Alibaba Tongyi Labs, enhancing the agent search capabilities of large language models (LLMs). Through the Retrieval Augmented Mask Prediction (RAMP) task, the model masks key information in the input text and uses an external knowledge base to call search tools to predict the masked segments. This covers key information such as named entities, dates, numbers, and ontology knowledge, increasing the task difficulty and prompting the model to process information more precisely. When generating supervised fine-tuning (SFT) data, a multi-agent system is employed, including roles such as planner, rewriter, and observer, collaboratively generating thought chain data to improve data quality. The training method combines SFT and reinforcement learning (RL), using the Dynamic Sampling Policy Optimization (DAPO) algorithm to construct a hybrid reward system. It employs a course-based learning approach, grading the sample difficulty according to the number of masks, allowing the model to learn progressively.

MaskSearch's main functions

• Improve question answering performanceSignificantly enhances the performance of LLM in open-domain multi-hop question answering scenarios, especially in downstream tasks both within and outside the domain, improving the model's ability to understand and answer complex questions.
• Adaptable to multiple tasksBy using RAMP tasks and thought chain data generated by multi-agent systems, the model can better adapt to various question-answering tasks and improve its performance in different scenarios.
• Compatible with multiple training methodsIt is compatible with both SFT and RL training methods, and can select the appropriate training strategy according to different task requirements.
• Dataset ExpansionImprove the training performance and scalability of models by building large-scale pre-trained datasets (such as 10 million samples).

MaskSearch's technical principles

• Retrieval Enhanced Mask Prediction (RAMP) taskThe core of MaskSearch is the RAMP task, inspired by the BERT masking mechanism. It allows the model to mask key information in the input text sequence and then actively leverages external knowledge bases and search tools to predict these masked text fragments. In addition to common named entities, dates, and numbers, it also considers ontology knowledge, specific terms, and numerical values, increasing the task difficulty and prompting the model to process information more finely during retrieval and inference.
• Multi-agent collaborative generation of thought chain (CoT) dataTo generate thought chain data for Supervised Fine-Tuning (SFT), MaskSearch employs a multi-agent system, including planners, rewriters, and observers, to collaboratively generate thought chains. Ultimately, an LLM (Limited Learning Model) is responsible for answer judgment, retaining only the thought chains of the correct answers. To rapidly expand the dataset and maintain high quality, a teacher model trained on existing data is used to directly generate inference trajectories. The teacher model is iterated progressively to gradually improve data quality.
• Reinforcement Learning (RL)In the reinforcement learning part, MaskSearch employs the Dynamic Sampling Policy Optimization (DAPO) algorithm to construct a hybrid reward system, including format rewards and answer rewards. Format rewards check whether the model output conforms to a specified format, while answer rewards evaluate the consistency between the generated answer and the standard answer. Finally, a model-based reward function is selected, using the Qwen2.5-72B-Instruct model as the criterion to score the consistency between the generated answer and the standard answer.
• Course LearningMaskSearch introduces a course learning strategy that classifies training samples into difficulty levels based on the number of masks, allowing the model to learn from easy to difficult and gradually improve its capabilities.

MaskSearch's project address

• Github repository:https://github.com/Alibaba-NLP/MaskSearch
• arXiv technical paper:https://arxiv.org/pdf/2505.20285

Application scenarios of MaskSearch

• Intelligent Customer ServiceIn the field of intelligent customer service, MaskSearch can help customer service systems understand user questions more accurately, quickly retrieve relevant answers and information, and improve customer service efficiency and user satisfaction.
• EducationIn the education field, MaskSearch can be used to build intelligent tutoring systems that help students better understand and solve complex academic problems. It can retrieve relevant knowledge points and answers based on students' questions, providing personalized learning support.
• Enterprise-level search systemEnterprise search systems need to handle large amounts of internal data and complex query requirements. MaskSearch can enhance the retrieval capabilities of enterprise search systems, more accurately understand user query intent, quickly retrieve relevant information from massive amounts of data, and improve enterprise decision-making efficiency.
• Debugging and Optimization of Machine Learning ModelsMaskSearch can also be used for debugging and optimizing machine learning models. For example, when debugging image classification models, MaskSearch can help users query image databases by mask attributes, identify spurious relevance learned by the model, and explore the differences between model salience and human attention.