HY-WU - Tencent's Hunyuan open-source functional neural memory framework

What is HY-WU?

HY-WU (Hunyuan Wuxiang) is a functional neural memory framework launched by Tencent Hunyuan, enabling AI to generate personalized parameters in real time during inference. The framework uses a Transformer parameter generator to synthesize personalized LoRA weights in real time based on the input during inference, dynamically injecting frozen schemas and achieving instance-level adaptation without fine-tuning. The framework has achieved leading levels among open-source models in over 60 tasks, including image editing, paving a new path for personalized and continuous learning in general AI.

Main functions of HY-WU

• Real-time parameter generationHY-WU can synthesize a set of exclusive LoRA adaptation parameters in real time based on the currently input image and editing instructions during the model inference process, without the need for pre-training or storing a large number of fixed weights.
• Instance-level personalizationThe system dynamically generates customized operator parameters for each specific sample or task, achieving true "personalized adaptation for each individual".
• Zero Forgotten AdaptationBy dynamically injecting the generated parameters into the frozen base model, HY-WU avoids direct modification of shared weights, thus completely solving the catastrophic forgetting problem of "learning new and forgetting old" caused by traditional fine-tuning.
• Functional decouplingWhen faced with conflicting task objectives, HY-WU eliminates the performance seesaw effect of traditional models by routing different requirements to different regions of the parameter update family, thus avoiding compromises that sacrifice one aspect for another.
• End-to-end trainingUnlike traditional hypernetworks that rely on a large number of pre-trained checkpoints, HY-WU uses an end-to-end training approach to directly optimize the parameter generator without the need to collect and store massive amounts of fine-tuned weights.
• Large-scale compatibilityThrough structured parameter segmentation and Factorized Self-Attention optimization, HY-WU can efficiently support pedestal models with 80B parameters, and the inference generation process can be completed in just a few seconds.

HY-WU Technical Principles

• Functional neuromemory paradigmHY-WU redefines the "memory interface" of model adaptation. Traditional methods view adaptation as finding a fixed point in a static parameter space, while HY-WU transforms it into learning a parameter generation function—that is, synthesizing a mapping relationship of specific operators in real time based on input conditions. Adopting the paradigm of "memory as neural network," the model can dynamically route within the weight space based on different instances, fundamentally avoiding repeated erasures and conflicts on shared parameters, and achieving conditional parameter updates using non-isolated operators.
• Hybrid Conditional CodingThe system first extracts mixed conditional features of the current input image and editing instructions through a multimodal encoder, fusing visual content and language instructions into a unified semantic representation. This conditional encoding captures the task intent of "what to do" and perceives the instance characteristics of "to whom to do it," providing fine-grained contextual basis for subsequent parameter generation and ensuring that the generated parameters can accurately match the unique needs of the current sample.
• Transformer parameter generatorHY-WU introduces a parameter generation network (8B parameters) based on the Transformer architecture. It takes mixed conditional features as input, models the complex interactions between conditions through a self-attention mechanism, and outputs structured LoRA weight parameters. To address the computational challenges of large-scale parameter generation, the team designed a Factorized Self-Attention mechanism, which significantly reduces time complexity by decomposing attention computation, making it feasible to generate parameters in real time on billions of primitive models (Rank-16, approximately 720 million parameters).
• Dynamic parameter injection and frozen model coordinationThe generated LoRA parameters are immediately injected into the linear modules of the frozen base model (80B parameters, 13B activations), forming temporary, instance-specific computational paths. The base model remains frozen to ensure the complete preservation of its original capabilities, while dynamically mounted personalized parameters are responsible for performing specific edit transformations. This "backbone + plug-in" architecture decouples versatility from adaptability, allowing the model to maintain robust basic capabilities while flexibly handling diverse downstream tasks.
• Structured parametric manifolds and semantic routingDuring training, HY-WU induces a parametric manifold with a geometric structure, where functionally similar editing operations (such as animal deformation and style transfer) automatically cluster in the weight space. The semantic consistency of the weight space indicates that functional memory successfully captures the transformation rules behind the task. When faced with conflicting objectives, the system can resolve interference by routing to different regions of the manifold, avoiding forced performance compromises at single parameter points and achieving true functional modularity and decoupling.

HY-WU project address

• Project official websitehttps://tencent-hy-wu.github.io/
• GitHub repositoryhttps://github.com/Tencent-Hunyuan/HY-WU
• HuggingFace model libraryhttps://huggingface.co/tencent/HY-WU
• Technical Papers: https://github.com/Tencent-Hunyuan/HY-WU/blob/main/assets/report.pdf

Application scenarios of HY-WU

• Image editingHY-WU can accurately understand complex instructions and achieve high-quality personalized generation while maintaining the character's identity, posture, and background, meeting the diverse visual content production needs such as social sharing and advertising creativity.
• Game character customizationIn the gaming and digital entertainment field, HY-WU enables high-fidelity facial identity transfer and cross-domain clothing integration, allowing players to transform real-life photos into game character images with one click, significantly enhancing the immersion of virtual role-playing and creative cosplay, as well as the efficiency of UGC content creation.
• e-commerce virtual try-onFor fashion retail scenarios, HY-WU can naturally and seamlessly transfer target clothing onto mannequins of different body types and postures, maintaining consistency in texture details and lighting, providing a low-cost and high-efficiency visualization solution for online product display, and accelerating the entire process from design to shelf.
• Film and television advertising productionIn the professional content production process, HY-WU supports rapid iteration of multiple versions of visual solutions, helping designers achieve high-quality texture synthesis and scene integration effects, significantly reducing the technical threshold of post-production and shortening the conversion cycle from concept to finished product.