Game-TARS - A general-purpose game AI agent launched by ByteDance

What is Game-TARS?

Game-TARS is a general-purpose game agent developed by ByteDance's Seed team. Based on a unified keyboard-mouse action space training, it can be pre-trained on a large scale in operating systems, web pages, and simulation environments. Leveraging over 500 billion labeled multimodal training data points, combined with sparse inference and decaying persistence loss, it significantly improves the agent's scalability and generalization. The core innovation of Game-TARS lies in enabling the agent to operate like a human using a keyboard and mouse, completing every action in the game by simulating human operation, achieving direct alignment with the physical interaction methods of human users. In tasks such as FPS, open-world, and web games, it outperforms GPT-5, Gemini-2.5-Pro, and Claude-4-Sonnet.

Main functions of Game-TARS

• Cross-platform game operationGame-TARS enables game operation on different platforms (such as PC, web, and emulation environments) through a unified keyboard-mouse action space, without the need to write specific scripts for each platform, achieving cross-platform automated testing and game interaction.
• Multimodal data pre-trainingBased on pre-training with over 500 billion labeled multimodal data, covering various data types such as game trajectories and graphical user interface interactions, the intelligent agent has strong generalization and adaptability, enabling it to handle various complex game tasks.
• Efficient Reasoning and Sparse ThinkingIt employs a sparse reasoning strategy, performing deep reasoning only at key decision points, and combines this with rejection fine-tuning techniques to optimize the reasoning process, improve reasoning efficiency and the operability of actions, enabling the agent to make more accurate decisions in complex environments.
• Combining long-term and short-term memoryThe system introduces a dual-layer memory mechanism: short-term memory stores the latest image information, while long-term memory retains refined sparse textual thoughts. This helps the agent maintain its memory of key information during long-term tasks, improving task completion rate and interactive intelligence.
• Zero-sample transfer capabilityAfter large-scale pre-training, Game-TARS can achieve zero-shot transfer in unseen 3D web games, quickly adapting to new environments and completing tasks without additional training, demonstrating strong generalization ability.
• Instruction compliance and action semantic understandingBy randomly replacing key bindings, the ability of the intelligent agent to follow instructions is enhanced, enabling it to understand the semantics of actions based on system prompts, accurately execute task instructions, and improve adaptability and operational accuracy in different game environments.

The technical principles of Game-TARS

• Unified Action SpaceEmploys native keyboard-mouse input actions aligned with human input methods, such as...mouseMove,mouseClickandkeyPressThis allows the action instruction set of an intelligent agent to be decoupled from a specific application or operating system, achieving cross-platform versatility.
• Multimodal data pre-trainingPre-training is performed on more than 500 billion labeled multimodal data, including various data types such as game trajectories, graphical user interface interactions, code generation, and scientific research tasks, to improve the generalization ability and adaptability of intelligent agents.
• Sparse reasoning strategiesBy using sparse-thinking techniques, deep reasoning is performed only at key decision points, combined with rejection fine-tuning, to optimize the reasoning process and improve reasoning efficiency and the operability of actions.
• Visual language model integrationIt integrates visual perception, strategic reasoning, action execution, and long-term memory into a single visual language model (VLM), eliminating the need to write specific code or rules for each game. The agent can learn operations and complete tasks autonomously.
• Continuous pre-training frameworkWe adopt a unified single-stage continuous pre-training method, which integrates all data sources to perform large-scale pre-training of the model. At the same time, we further enhance the agent's execution ability and interactive intelligence in specific tasks during the post-training stage.
• Long-term and short-term memory mechanismsThe system introduces a dual-layer memory mechanism: short-term memory stores the latest image information, while long-term memory retains refined sparse thought text, helping the agent maintain the memory of key information in long-term tasks, thereby improving task completion rate and interactive intelligence.

Game-TARS project address

• Project official websitehttps://seed-tars.com/game-tars/
• arXiv technical paper: https://arxiv.org/pdf/2510.23691

Application scenarios of Game-TARS

• Game Automated TestingGame-TARS can be used to automate testing of various games, including FPS (first-person shooter) games, open-world games, and web games, helping developers quickly identify problems in games and improve testing efficiency and quality.
• Cross-platform compatibility testingDue to its unified action space design, Game-TARS can conduct adaptation tests on different platforms (such as PC, mobile, and web), ensuring the game's compatibility and consistency across multiple platforms.
• Complex task executionGame-TARS can handle complex game tasks, such as building and exploring in Minecraft, and even achieve zero-sample transfer in unseen 3D web games, demonstrating strong generalization capabilities.
• Abnormal scenario handlingGame-TARS has an error recovery mechanism that can learn through self-supervised learning from failure cases, handle abnormal scenarios in the game, and improve the robustness of the agent.
• Game development assistanceIn addition to testing, Game-TARS can also assist game development, such as providing interactive prototype testing during the game design phase to help developers optimize game design and user experience.
• Research and EducationGame-TARS's open-source nature makes it a valuable resource for research and education. Researchers can use its framework for research in fields such as artificial intelligence and machine learning, while educational institutions can use it as a teaching case to cultivate students' practical skills.