AB
AB
AiBoss

All Categories

project

TuriX-CUA - An open-source AI desktop agent that automatically performs screen recognition and operations.

TuriX-CUA is an open-source AI agent based on Python that automates desktop operations using AI technology. TuriX-CUA "sees" the screen by taking screenshots, uses a multimodal large model to "think" about the next action, and completes user-instructed tasks...

What is TuriX-CUA?

TuriX-CUA is an open-source AI agent based on Python that automates desktop operations using AI technology. TuriX-CUA "sees" the screen by taking screenshots, uses a multimodal model to "think" about the next action, and completes user-specified tasks such as searching for information and processing documents. TuriX-CUA supports Windows and macOS systems, is compatible with various AI models, and can be integrated with other AI tools via the MCP protocol to automate complex tasks, providing users with an efficient and intelligent desktop experience.

Main functions of TuriX-CUA

  • Screen recognition and operationIt automates tasks by capturing screenshots, recognizing screen content, and controlling mouse clicks and keyboard input.
  • Multimodal task executionIt supports a variety of complex tasks, such as booking flights, searching for information, generating charts, and inserting documents.
  • Multi-model architectureBy employing a division of labor between Planner and Executor, the success rate and stability of task execution can be improved.
  • Cross-platform supportIt is compatible with Windows and macOS systems, catering to the operating system needs of different users.
  • MCP Protocol IntegrationIt supports collaboration with other AI tools such as Claude via the MCP protocol, expanding functionality and application scenarios.
  • Flexible model configurationUsers can replace the underlying AI model, such as OpenAI or locally deployed Qwen3-VL, to meet their individual needs.
  • Task planning and decompositionBy using the Planner, complex tasks can be broken down into multiple steps and executed one by one, thereby improving task completion.

How to use TuriX-CUA

  • Environmental preparationInstall Python and Conda, create the Python environment, and install the dependencies required for the project.
  • Model ConfigurationSet the interface information of the AI model in the configuration file and define the task description to be executed.
  • Permission settingsGrant the system permissions to take screenshots and control the mouse and keyboard, ensuring that the software can operate desktop elements normally.
  • Start AgentRun the project startup script to allow the AI to begin executing the configured tasks.
  • Configuration taskModify the task description according to actual needs to ensure that the task is clearly and unambiguously described, thereby improving the success rate of execution.

TuriX-CUA project address

  • GitHub repository: https://github.com/TurixAI/TuriX-CUA

Application scenarios of TuriX-CUA

  • Document processingAutomatically create, edit, and send documents (such as Word, Excel, and PowerPoint), generate reports based on user instructions, insert charts, and send them to colleagues.
  • Email ManagementAutomatically compose, send, and reply to emails, handling daily email tasks according to preset rules.
  • Data collectionAutomatically extracts data from web pages or applications, such as searching and recording product prices, stock information, etc.
  • Content search and browsingAutomatically search for content, browse web pages, and perform actions such as liking, commenting, or sharing in the browser.
  • Interface Testing: Automatically test the software's user interface to verify that the functions are working properly.
Back