AtomThink - A multimodal mathematical reasoning framework jointly developed by Huawei Noah's Ark and several universities.
AtomThink is a multimodal mathematical reasoning framework jointly developed by researchers from Sun Yat-sen University, Hong Kong University of Science and Technology, Shanghai Jiao Tong University, the University of Hong Kong, and Huawei Noah's Ark Lab. The framework is based on the concept of building long chains of thought (CoT)...
What is AtomThink?
AtomThink is a multimodal mathematical reasoning framework jointly developed by researchers from Sun Yat-sen University, Hong Kong University of Science and Technology, Shanghai Jiao Tong University, the University of Hong Kong, and Huawei Noah's Ark Lab. The framework guides large multimodal language models (MLLMs) to perform complex reasoning based on Constructing Long Chains of Thought (CoT), and includes an automatic CoT annotation engine, atomic step fine-tuning, and various search strategies. By improving the quality of atomic steps, AtomThink significantly enhances the reasoning ability of MLLMs in solving mathematical problems, providing a new direction for developing general-purpose slow-thinking models.
Main functions of AtomThink
- CoT annotation engineAutomatically generates high-quality Chain-of-Thought (CoT) annotations, addressing the issue of insufficient quality in visual mathematical data.
- Atomic step fine-tuning strategy: Jointly optimize the multimodal large language model (MLLM) and the policy reward model (PRM) to achieve stepwise reasoning.
- Search StrategyIt offers four different search strategies that can be used in conjunction with PRM to complete complex reasoning tasks.
- Dataset ConstructionWe propose AtomMATH, a large-scale multimodal dataset containing long CoTs, for training and evaluating models.
- Atomic ability assessmentDesign a result-supervised atomic capability evaluation method to evaluate the capability of MLLMs in generating each atomic step.
AtomThink's technical principles
- Slow Thinking FrameworkThe core idea of the AtomThink framework is "slow thinking," which guides MLLMs to perform complex inferences by gradually building long CoTs, without relying on fast and direct predictions.
- Dynamic prompting strategyBased on a dynamic prompting strategy, MLLMs are driven to iteratively construct state reasoning paths. Each path node represents a reasoning step, including the previous stage, the current state, and possible actions.
- Short CoT enhancementBased on LLMs, the existing short CoT annotation semantics are segmented into multiple discrete steps, focusing on solving individual atomic problems in the reasoning process.
- Multimodal datasetsMathematical data is sampled from multiple data sources, and multi-step inference paths are generated based on dynamic prompts and short CoT enhancements to construct the AtomMATH dataset.
- Atomic step fine-tuningFine-tuning on the AtomMATH dataset allows MLLM to learn inference patterns based on atomic steps.
AtomThink's project address
- GitHub repository:https://github.com/Quinn777/AtomThink
- arXiv technical paper:https://arxiv.org/pdf/2411.11930
Application scenarios of AtomThink
- Educational SupportAs an intelligent tutoring system, it provides students with step-by-step solutions and explanations for mathematical problems.
- Automated testing and evaluationAutomatically generate and grade math test questions in an online examination system.
- academic researchIt assists researchers in exploring and solving complex mathematical problems.
- Software developmentIt helps developers automatically generate and debug code related to mathematical calculations.
- Intelligent customer service and technical supportProvide support in customer service scenarios that require mathematical calculations or reasoning.