MemGUI-Agent - A long-range mobile GUI intelligent agent jointly launched by Zhejiang University and Kuaishou.

What is MemGUI-Agent?

MemGUI-Agent is a long-range mobile GUI intelligent agent jointly developed by Zhejiang University and Kuaishou. It is designed for mobile automation tasks that span multiple apps, involve multiple steps, and have long chains. Traditional ReAct-style agents suffer from the dilution of key information in long-range tasks due to the linear expansion of historical records. MemGUI-Agent, through the ConAct protocol, allows the model to simultaneously decide how to operate the UI and how to organize memories at each step, achieving compact and structured context management.

Main functions of MemGUI-Agent

• Long-range task executionIt supports complex automated processes across apps and pages, and can handle an average of more than 30 consecutive operations.
• Active Context ManagementThrough the ConAct protocol, the Agent autonomously decides when to compress history, when to write UI facts, and how to describe the current step.
• Structured memory operationsIt supports memory operations such as memory_add, memory_update, and memory_delete, and persistently saves key UI information such as price, phone number, and specifications.
• End-to-end model trainingWe provide a LoRA fine-tuning model MemGUI-8B-SFT based on Qwen3-VL-8B, which can handle long-range tasks with 8B parameters.
• Full-chain open sourceThe code, MemGUI-3K dataset, training/evaluation pipeline, and model weights are all available.

Technical Principles of MemGUI-Agent

• ConAct (Context-as-Action) Core FrameworkMemGUI-Agent elevates context management from a traditional external rule module or post-processing summarizer to a first-class action on par with UI clicks, swipes, and inputs. In each step of inference, the model proactively decides what history to compress, what UI facts to remember, and how to describe the current step, making context updates an executable and trainable policy output.
• Three structured state fieldsThe agent splits the context into three complementary fields that work together.Folded Action History It is responsible for compressing completed subtask segments into reusable summaries to avoid linear expansion of history; Folded UI State is dedicated to persistently storing key facts across screens, ensuring that information can still be accurately retrieved dozens of steps later; Recent Step Record records the screen observation, action intent, executed action and result of the most recent step, providing reliable local raw materials for subsequent history folding and memory writing.
• Five-segment structured output and memory operationIn each step, the model outputs five structured paragraphs: thinking, folding, tool_call, ui_observation, and action_intent. Among them, tool_call can be a regular UI operation or an explicit memory operation such as memory_add, memory_update, and memory_delete.

Follow us on WeChat and reply with "open source",join inAI open source project discussion group

How to use MemGUI-Agent

• Cloning open source codeFrom GitHub repository github.com/kwai/MemGUI-Agent Pull the complete code and training/evaluation pipeline.
• Download datasetGet the MemGUI-3K dataset from HuggingFace.
• Loading model weightsDownload MemGUI-8B-SFT from HuggingFace.
• Configure Android environmentConnect the device or start the simulator to deploy the Agent execution environment.
• Enter task commandThe Agent describes long-term goals in natural language and then automatically plans and executes them.

MemGUI-Agent's core advantages

• Context costs are significantly reducedIn a long-range region of 150 steps, ConAct saves approximately 1500 input tokens on average compared to ReAct, suppressing linear growth.
• Accurate retention of key informationPersistently preserve the complete facts through UI State to avoid information dilution and illusions in long-running tasks.
• Small Model, Big CapabilitiesThe 8B model, after being trained with MemGUI-3K, outperforms most 32B and even 235B baselines on long-range benchmarks, enabling efficient deployment.
• Excellent out-generalizationMemGUI-8B-SFT achieved a 17.9% success rate on MobileWorld, an improvement of 8.5 percentage points over the baseline, demonstrating strong cross-scenario migration capabilities.
• End-to-end reproducibleThe entire process, from data construction and model training to evaluation protocols, is completely open source, lowering the research threshold.

MemGUI-Agent project address

• Project official websitehttps://memgui-agent.github.io/
• HuggingFace model libraryhttps://huggingface.co/collections/lgy0404/memgui-agent
• arXiv technical paper: https://arxiv.org/pdf/2606.19926

Comparison of MemGUI-Agent with similar products

Application scenarios of MemGUI-Agent

• Cross-App Information IntegrationAutomatically collects information from e-commerce, maps, and social apps and aggregates it into note-taking software.
• Long form fillingIt can automatically complete complex registration or reimbursement processes based on information from multiple sources (email, address book, calendar).
• Contact Management and UpdatesExtract new phone numbers/emails from social media posts, automatically update your contacts, and send text messages.
• Product price comparison and recordBrowse multiple e-commerce platforms, record prices and specifications, and generate comparison documents.
• Automated Testing and RPA: Perform long-chain UI regression testing or enterprise-level process automation on mobile devices.