OpenAI o3 - OpenAI's next-generation, most powerful inference model

What is OpenAI o3?

The OpenAI o3 model is the next-generation inference model following o1. It's OpenAI's first model to integrate image reasoning into its thought process, allowing it to directly consider images within the thought chain. Users can upload blurry, inverted, or low-quality images, and the model can still interpret them and perform operations such as cropping, enlarging, and rotating. o3 can intelligently select and combine various tools built into ChatGPT, such as web search, Python code execution, and file analysis, to solve complex problems. o3 has performed exceptionally well in programming, mathematics, and science, breaking numerous benchmark records. For example, in the Codeforces programming competition, o3 scored over 2700 points, ranking among the top 200 globally. OpenAI has conducted a comprehensive security assessment and training on o3, adding rejection prompts for biometric threats, malware generation, and jailbreak attacks.

Main functions of OpenAI o3

• Image Thinking:o3 is the first to directly integrate images into the thought process, enabling reasoning using pictures. Users can upload blurry, upside-down, or low-quality images, and the model can still interpret them and perform operations such as cropping, enlarging, and rotating.
• Multimodal processingo3 supports web search, file analysis, Python code execution, visual input deep reasoning, and image generation. This makes it excellent at handling multimodal tasks, enabling a more comprehensive understanding and problem-solving approach.
• Tool call:o3 can independently select and use tools to quickly generate detailed and thoughtful answers. For example, it can call a search engine to obtain information, use Python for data analysis, and even generate images.
• Efficient Reasoningo3 offers superior performance at the same latency and cost, making it suitable for complex queries. It sets new records in multiple benchmarks, including programming, mathematics, science, and visual perception.
• reinforcement learning:o3 enhances reasoning ability through large-scale reinforcement learning. This training method enables the model to think and solve problems more effectively when faced with complex issues.
• open source toolsOpenAI also launched Codex CLI, a lightweight, terminal-based programming agent. It supports multimodal input and native code execution, allowing users to program efficiently via the command line.
• Security:o3 has made significant improvements in security, reconstructed the security training dataset, and enhanced its defenses against biological threats, malware generation, and jailbreak attacks.
• Cost efficiencyThe cost of inputting o3 is $10 per million tokens, and the cost of outputting it is $40 per million tokens. Compared to its predecessor, o3 offers better performance at the same latency and cost.
• Excellent programming performanceIt achieved an ELO score of 2727 on the CodeForces programming competition platform, surpassing the level of top programmers, and supports code generation and execution for complex tasks, automatically optimizing code logic and improving development efficiency.
• Transparent reasoning pathIt provides a clear reasoning process, demonstrating the logical flow and intermediate conclusions at each step, thus enhancing the credibility and explainability of decisions.

The technical principles of OpenAI o3

• Hybrid reasoning frameworko3 employs a hybrid reasoning framework, combining neural symbolic learning with probabilistic logic. It integrates the powerful pattern recognition capabilities of neural networks with the logical rigor of symbolic reasoning, enabling the model to better handle structured data and knowledge. Probabilistic logic allows the model to make reasonable inferences when faced with ambiguous or incomplete information.
• Multi-way reasoning and search mechanismo3 implements multi-path inference, generating multiple possible paths during the inference process, evaluating and selecting these paths to find the optimal solution. Similar to Monte Carlo Tree Search (MCTS), it can effectively utilize computational resources and improve the model's performance in complex tasks.
• Problem decomposition and iterative optimizationo3 can break down complex problems into smaller, more manageable parts for step-by-step analysis and resolution. This "divide and conquer" strategy makes the model perform exceptionally well when handling multi-step reasoning tasks. o3 also possesses the ability to iteratively optimize, continuously improving and refining the answer through multiple rounds of reasoning.
• powerful contextual memoryThe o3 model features extended memory, allowing it to retain contextual information during extended interactions. When handling tasks with strong continuity, the model can better understand and respond to previous dialogues or data, providing users with more coherent and accurate answers.
• Optimize model architecture:o3 employs a more advanced multi-layer transformer architecture, which significantly enhances the model's ability to process long text sequences and captures complex semantic relationships and logical structures in the text.
• Strengthen learning and training:o3 extensively utilizes reinforcement learning during training, continuously optimizing the model's reasoning ability through interaction with the environment. This training method enables the model to think and solve problems more effectively when faced with complex issues.
• High-efficiency inference and key-value cachingo3 employs efficient key-value caching technology during the inference phase, which reduces redundant calculations and improves inference efficiency by storing and reusing intermediate results.
• Mind Chain Search and ExecutionThe o3 model overcomes the limitations of traditional large language models in handling new problems by generating and executing its own programs. The program itself (the thought chain) becomes a concrete manifestation of knowledge reorganization.
• Supervised fine-tuning (SFT) and reinforcement learning (RL)OpenAI uses supervised fine-tuning and human feedback reinforcement learning for safety training. This allows the model to learn ideal behavior from examples, and reinforcement learning trains the model to use its thought processes more effectively.
• Deliberative AlignmentThis is a novel security assessment method that directly teaches a new paradigm of model security specifications. By inferring the user's input intent, it significantly improves the model's ability to identify potentially insecure requests.
• Adaptive Thinking TimeThe o3 model supports three inference time modes: low, medium, and high. Users can flexibly adjust the model's think time according to the complexity of the task to achieve optimal performance.

OpenAI o3 benchmark

• existARC-AGIIn the test: o3 achieved [result] under the high reasoning ability setting.87.5%The score, even under low reasoning ability settings, is as high as 0.1.3 times.
• In programming competitionsCodeforcesIn the middle: O1's score is 1891, while O3 can reach [a higher score] under high reasoning settings.2727The scores for low-inference settings also exceeded 0.1.
• AIME 2024:In mathematical benchmark testsAIME 2024In the middle, the accuracy of o3 reached96.7%
• EpochAI Frontier Math:o3 is also touted as the industry's strongest mathematical benchmark, jointly developed by Terence Tao and over 60 other global mathematicians.EpochAI Frontier MathChina set a new record, with a score of 100.25.2None of the other models exceeded 2.0.

Test type	o3 performance	Human expert level	Remark
ARC-AGI Test	87.5%	85%	Even with low reasoning ability settings, the score is as high as O1.3 times
CodeForces Elo rating	2727	–	Surpassing 99.99% of human programmers, the O1 score is 1891.
AIME 2024 Mathematics Competition	96.7%	–	Almost perfect score
GPQA Diamond Test	87.7%	70%	Significantly exceeding the average level of human experts
EpochAI Frontier Math	25.2%	–	Other models did not exceed 2.0%.

Difference between OpenAI o3 and o1

• Performance improvementAccording to the SWE-bench Verified code generation evaluation benchmark, o3 achieved an accuracy score of 71.7, surpassing o1's 48.9 and o1 preview's 41.3. In the 2024 AIME Mathematics Competition problem test, o3 achieved an accuracy score of 96.7, surpassing o1's 83.3 and o1 preview's 56.7.
• costIn low computational mode, each ARC-AGI task costs $17-20, while in high computational mode, each task costs several thousand dollars.
• Safety and AlignmentOpenAI is using a new technology called "deliberative alignment" to make models like o3 conform to its safety principles. o3 is trained to "think" before reacting, reasoning about the task and planning ahead, executing a series of actions over a longer period of time to help find solutions.

OpenAI o3 project address

• Project official website:https://openai.com/index/introducing-o3-and-o4-mini/

How to use OpenAI o3

• For ordinary usersAccess via ChatGPTChatGPT Plus, Pro, and Team users will see o3, o4-mini, and o4-mini-high in the model selector, replacing the previous o1, o3-mini, and o3-mini-high. ChatGPT Enterprise and Edu users will gain access in one week. Free users can select "Think" to try o4-mini before submitting queries. Rate limits for all plans remain unchanged from the previous set of models.
• For developersUse via APIDevelopers can use o3 and o4-mini through the Chat Completions API and Responses API.

Application scenarios of OpenAI o3

• Mathematical Reasoning and EducationThe O3 model performs exceptionally well on complex mathematical problems and is well-suited for use in education, helping students solve mathematical problems and providing problem-solving strategies and methods.
• Programming and Software DevelopmentIt supports code generation and execution for complex tasks, automatically optimizes code logic, and improves development efficiency. o3 can act as a programming assistant, helping developers with code writing, debugging, and optimization.
• Scientific research and data analysiso3 is suitable for data analysis and problem modeling in scientific research, helping scientists deal with complex scientific problems.
• Multimodal problem solvingThe o3 model can handle mixed text and image inputs, providing strong support for multimodal reasoning scenarios, such as visual reasoning and cross-modal problem solving. o3 can be effective in scenarios that require combining visual and textual information, such as image recognition and description generation.
• Transparent reasoning patho3 provides a clear reasoning process, demonstrating the logical flow and intermediate conclusions at each step, enhancing the credibility and explainability of decisions. It is particularly useful in applications requiring explanation and verification of reasoning processes, such as legal analysis and financial risk assessment.
• High-efficiency multitaskingThe o3 supports long context input and can handle complex multi-step instructions, making it suitable for programming, scientific, and multimodal problem-solving scenarios.