HuatuoGPT-o1 - A large-scale medical inference model open-sourced by the Chinese University of Hong Kong and the Shenzhen Big Data Research Institute.

What is HuatuoGPT-o1?

HuatuoGPT-o1, jointly developed by the Chinese University of Hong Kong, Shenzhen and the Shenzhen Big Data Research Institute, is a complex reasoning model for the medical field. It enhances performance in solving medical problems through sophisticated reasoning capabilities. The model employs a two-stage training method: first, a medical validator guides the search for the correct reasoning path to fine-tune the model; second, reinforcement learning based on validator feedback further enhances the model's complex reasoning ability. HuatuoGPT-o1 can generate long chains of thought processes, identify errors, and refine answers by trying different strategies. Experimental results show that the model outperforms general and medical-specific baseline models in multiple medical benchmarks and significantly benefits from complex reasoning and reinforcement learning.

Main functions of HuatuoGPT-o1

• Complex ReasoningThe HuatuoGPT-o1 can perform complex reasoning and solve complex problems in the medical field.
• Error identification and correctionThe model can identify errors in its answers and try different strategies to correct and optimize them.
• Long Chain ThinkingThe HuatuoGPT-o1 can generate long chains of thought (CoT) to demonstrate the reasoning process.
• Self-improvementBased on reinforcement learning (RL), the model can improve itself and further enhance its complex reasoning ability.

Technical Principles of HuatuoGPT-o1

• Two-stage training method:
  • Phase 1: Learning Complex Reasoning: Construct complex inference trajectories based on validator feedback (correct or incorrect) guided by policy search, and fine-tune the LLM.
  • Phase Two: Enhancing Complex Reasoning Through RLAfter acquiring complex reasoning skills in the first stage, the model is further optimized using validator-based sparse rewards.
• Verifiable medical issuesThe system constructs 40,000 verifiable medical questions, each with an objective and unique correct answer, to support the model in validating the correctness of the solution.
• Medical VerifierUse GPT-4o as a validator to check whether the model-generated answers (CoT and results) match the real answers, providing binary feedback.
• Reinforcement Learning (RL)The Proximal Policy Optimization (PPO) algorithm is used for RL training. Based on the reward provided by the validator, the model improves itself and optimizes complex inference paths.
• Chain Thinking (CoT)The CoT generated by the model consists of three parts: "internal thinking", "final conclusion" and "validation", which simulates the human problem-solving thinking process.

HuatuoGPT-o1 Project Address

• GitHub repository:https://github.com/FreedomIntelligence/HuatuoGPT-o1
• HuggingFace model library:https://huggingface.co/collections/FreedomIntelligence/huatuogpt-o1
• arXiv technical paper:https://arxiv.org/pdf/2412.18925

Application scenarios of HuatuoGPT-o1

• Medical diagnostic aidsIt assists doctors in diagnosing diseases by providing possible diagnostic suggestions based on the analysis of patients' symptoms, signs, and laboratory test results.
• Treatment plan developmentIt helps doctors develop personalized treatment plans, taking into account the patient's specific situation and the latest medical research.
• Medical education and trainingIn medical education, it serves as a teaching aid to help students understand complex medical concepts and reasoning processes.
• Medical researchIt supports medical researchers in making complex inferences during literature reviews and data analysis, accelerating the research process.
• Drug development consultingWe provide consultation on drug mechanisms of action, side effects, and clinical trial design during the drug development process.