ToonComposer - An AI animation production tool jointly launched by Tencent, CUHK, and Peking University

What is ToonComposer?

ToonComposer is a generative AI tool jointly developed by researchers from the Chinese University of Hong Kong, Tencent PCG ARC Lab, and Peking University. It can transform sketches into professional-grade animations in seconds. Based on generative post-keyframe technology, ToonComposer automates the in-between frame generation and coloring processes in traditional animation production. It requires only a sketch and a coloring reference frame to generate high-quality animated videos. The tool supports sparse sketch injection and region control, allowing artists precise control over animation effects, significantly reducing manual workload, improving creative efficiency, and revolutionizing animation production.

Main functions of ToonComposer

• Generative post-keyframeIt integrates the intermediate frame generation and coloring processes in animation production into an automated process, requiring only a sketch and a coloring reference frame to create a complete cartoon video, significantly reducing manual workload.
• Precise sketch keyframe controlArtists can guide animation at any point on the timeline using sparse keyframe sketches, maintaining high precision and quality when handling complex movements.
• Area controlUsers can selectively leave blank areas in the sketch and mark these areas with the brush tool. ToonComposer can intelligently fill in the blank areas based on the context or prompts, further reducing the workload for artists.

ToonComposer's technical principles

• Sparse sketch injection mechanismBased on positional encoding mapping, sketch frames are converted into model-compatible markers and injected into the latent space of the video generation model, ensuring that the sketches have temporal awareness during the generation process. Simultaneously, the positional-aware residual module allows users to dynamically adjust the control strength of the sketches during inference, enabling more flexible animation generation.
• Cartoon adaptation methodsBased on the Spatial Low-Rank Adapter (SLRA), this approach adjusts only the spatial behavior of the model without altering its temporal behavior, adapting modern video foundation models to the cartoon genre. It ensures that the generated animations visually conform to a cartoon style while maintaining smooth motion.
• Regional control mechanismDuring training, random masking is applied to sketch frames, and the model learns to generate reasonable content within these masked areas. Users can specify which areas the model should automatically generate content for during creation, eliminating the need to draw complete sketches, further reducing workload and increasing creative flexibility.

ToonComposer's project address

• Project official websitehttps://lg-li.github.io/project/tooncomposer/
• GitHub repositoryhttps://github.com/TencentARC/ToonComposer
• HuggingFace model libraryhttps://huggingface.co/TencentARC/ToonComposer
• arXiv technical paper: https://arxiv.org/pdf/2508.10881
• Experience the demo onlinehttps://huggingface.co/spaces/TencentARC/ToonComposer

Application scenarios of ToonComposer

• Animation ProductionIndependent animators can quickly generate animation prototypes, reducing tedious intermediate frame drawing and coloring, and focusing their energy on creativity and storytelling.
• Game developmentGame developers can quickly generate animation resources for their games, such as character movements and scene animations, reducing manual production costs and improving development efficiency.
• Education and TrainingIn animation education, it can be used as a teaching tool to help students quickly understand the animation production process and enhance their interest in learning.
• Advertising and MarketingAdvertising companies can quickly generate animated ads to meet different client needs, saving time and costs when rapid iteration and modification of animation content are required.
• Film and television productionIn the production of animated films and television series, it generates preliminary animation clips to help directors and production teams quickly evaluate creative ideas and scene effects, reducing pre-production costs.