MemHarness - an intelligent agent memory reconstruction framework launched by Shanghai AI Lab and others. - AiBoss

What is MemHarness?

MemHarness is an LLM agent memory reconstruction framework developed by the Shanghai AI Lab in collaboration with universities such as Zhejiang University, Fudan University, and Shanghai Jiao Tong University. Existing memory enhancement agents often directly inject retrieved historical experiences into the context, which can lead to negative transfer if the old experience does not match the current state. Inspired by the human memory reconstruction mechanism, MemHarness inserts an explicit critique and reconstruction step between retrieval and action. It retains, rewrites, or discards historical experiences based on the current context and is trained end-to-end using GRPO, requiring no additional manual annotation.

MemHarness's main functions

Memory retrievalThe Agent generates a query based on the current observations and retrieves top-k relevant historical experiences and their source states from the Milvus vector memory.
Criticism and ReconstructionThe unified strategy model compares the source state of memory with the current state, critiques its applicability, rewrites the experience into state-aligned guidance information, or discards it as inapplicable.
Action generationBased on the reconstructed guidance information or autonomous reasoning, generate executable environmental actions.
Experience writing and pruningAfter each round of interaction, the trajectory is summarized as experience and written into the memory bank. Semantic deduplication is performed, and low-value memories are pruned periodically according to the utility of the experience.
End-to-end training: By jointly optimizing retrieval, reconstruction, and action generation through GRPO, independent labeled data is not required in the reconstruction stage.

MemHarness's technical principles

Five-stage decision-making process:MemHarness breaks down memory-guided decision-making into five consecutive stages: environmental observation, experience retrieval, memory critique, contextual memory reconstruction, and action generation. It simulates the human cognitive process of retrieval, evaluation, and reconstruction, replacing the static paradigm of traditional agents that directly replay memories.
Contextual memory reconstruction mechanism:The strategy model concatenates the task description, current history, retrieved experience, and their source states into a reconstruction context. By comparing the historical source states with the current state, it identifies differences, retains transferable knowledge, rewrites mismatched content, or outputs... <EMPTY> The memory is rejected and the process reverts to autonomous reasoning.
Unified strategy model architecture: The three stages of query generation, memory critical reconstruction, and action generation share the same policy model parameters, eliminating the need to train a value network or supervise data separately for the reconstruction module, thus tightly coupling memory utilization and decision reasoning within the same model.
GRPO end-to-end training:Since the reconstruction guidance information lacks truth labels, MemHarness employs GRPO to optimize the entire retrieval-reconstruction-action chain end-to-end. The reward consists of sparse task results and format rewards, and trajectory-level credits are allocated to all tokens through group normalization advantages, while training thinking, reconstruction, and action generation capabilities.

How to use MemHarness

Clone the repository and create the environment:implement git clone https://github.com/KnowledgeXLab/MemHarness.git and create python==3.12 In the Conda environment, install vLLM, Flash Attention 2, and MemHarness in sequence.
Deploy embedded services:pass vllm serve BAAI/bge-m3 --port 8001 Start the BGE-M3 embedding model to provide vector encoding services for the Milvus memory.
Install target environmentInstall the ALFWorld or WebShop interactive environment according to the task requirements, and download the corresponding game files and pre-trained detectors.
Cold start SFT (optional):run scripts/build_*_coldstart_data.py Build cold start data and execute scripts/cold_start_sft.sh Align the model with the interactive format and the memory summary format.
GRPO end-to-end training:run run_scripts/train_alfworld.sh or run_scripts/train_webshop.shThe framework will automatically start the memory vector database, perform agent retrieval, experience write-back and pruning, and complete GRPO training.

MemHarness's core advantages

Reconstruction is better than playbackExplicitly insert critique and reconstruction steps to transform static memory fragments into context-sensitive state alignment guidelines, thus avoiding negative transfer.
Smaller models surpass larger modelsThe 7B parameter size outperforms the Gemini-2.5-Pro by 23.1% and 39.7% on ALFWorld and WebShop, respectively.
OOD is robustThe average success rate in out-of-distribution scenarios reached 85.9%, significantly higher than the 76.3% of original memory playback.
Implicit reasoning enhancementEven with memory disabled during testing, reconstructing the training objective still increased the success rate of the basic policy from 76.4% to 83.0%, fundamentally enhancing the agent's intrinsic reasoning ability.
No additional annotations requiredThe reconstruction capability emerges naturally through end-to-end training of GRPO, without relying on manually written reconstruction supervision data.

MemHarness's project address

GitHub repository：https://github.com/KnowledgeXLab/MemHarness
HuggingFace model library：https://huggingface.co/KnowledgeXLab/MemHarness
arXiv technical paper：https://arxiv.org/pdf/2607.28272

Comparison of MemHarness's similar products

Dimension	MemHarness	EvolveR
Memory Paradigm	Explicit memory bank + state condition reconstruction	Explicit memory bank + experiential evolution
Core Mechanism	Critical reconstruction after retrieval: retain/rewrite/discard	Direct injection after retrieval, relying on empirical evolution and iteration.
Training methods	GRPO end-to-end, no need to refactor annotations	RL training requires the gradual evolution and accumulation of memory experience.
OOD performance	85.9%（ALFWorld OOD）	OOD robustness not highlighted
Explainability	The refactoring process is inspectable and supports EMPTY rejection.	The memory injection process is relatively black box
Model size	7B surpasses Gemini-2.5-Pro	Typically, a larger scale or more training steps are required.

Application scenarios of MemHarness

Smart Home ManagementIn home environments such as ALFWorld, Agent can reconstruct past experiences of retrieving items and cleaning, adapting to different room layouts to complete complex household chores.
Independent online shoppingIn e-commerce platforms such as WebShop, the current product search and filtering strategy is reconstructed based on historical shopping experience to improve the success rate of goal-oriented shopping.
Intelligent customer service dialogueCustomer service agents reconstruct solutions after searching for similar historical work orders, avoiding the direct application of old responses that lead to irrelevant answers.
Code-assisted developmentThe programming agent refactors past bug fixing experience, adapts it to the current code context, and provides accurate fix suggestions rather than simply copying old solutions.
Scientific research experiment planningThe Experiment Agent reconstructs optimal configuration suggestions for the current experimental conditions based on historical experimental parameters and results, reducing trial and error costs.