FairyGen - An AI-powered animated story video generation framework, ensuring consistent style and coherent narrative.

What is FairyGen?

FairyGen, developed by universities in the Greater Bay Area, is an animation story video generation framework that supports generating animated story videos with coherent narratives and consistent styles, starting from a single hand-drawn character sketch. The framework utilizes a multimodal large language model (MLLM) for story planning, applies the character's visual style to the background using a style propagation adapter, reconstructs the character using a 3D agent to generate realistic motion sequences, and optimizes the coherence and naturalness of the video animation using a two-stage motion adapter. FairyGen excels in style consistency, narrative coherence, and motion quality, opening new avenues for personalized animation creation.

FairyGen's main functions

• Animated story video generationStarting from a single hand-drawn character sketch, generate animated story videos with a coherent narrative and consistent style.
• Maintaining style consistencyBased on the style propagation adapter, the visual style of the hand-drawn characters is applied to the background to ensure style consistency throughout the video.
• Complex motion generationReconstruct characters using 3D proxies to generate physically plausible motion sequences, supporting complex yet natural motion performance.
• Narrative coherence support: Use a multimodal large language model (MLLM) for story planning to generate structured storyboards and ensure the narrative coherence of the video.

FairyGen's technical principles

• Story Planning: Use a multimodal large language model (MLLM) to generate structured storyboards from a single hand-drawn character sketch, define a global narrative overview and detailed shot-level storyboards, and provide the foundation for the narrative framework of the video.
• Style disseminationBased on the style propagation adapter, the visual style of the hand-drawn character is learned and applied to the background to ensure the consistency of the background and the character in visual style, thereby enhancing the overall visual coherence of the video.
• 3D motion modelingThe character is reconstructed using a 3D Agent, and based on skeletal rigging and motion retargeting technology, a physically plausible motion sequence is generated to provide natural and smooth motion for the animation.
• Two-stage motion adapterBased on the image-to-video diffusion model, a two-stage training strategy is employed. The first stage learns the spatial features of the character to remove temporal bias; the second stage learns the motion dynamics based on a time-stepping strategy to ensure the continuity and naturalness of the motion.
• Video generation and optimizationBy combining the above components and using a finely tuned image-to-video diffusion model, diverse and coherent video scenes aligned with the storyboard are directly rendered, ultimately generating a complete animated story video.

FairyGen's project address

• Project official websitehttps://jayleejia.github.io/FairyGen/
• GitHub repositoryhttps://github.com/GVCLab/FairyGen
• arXiv technical paper: https://arxiv.org/pdf/2506.21272

Application scenarios of FairyGen

• EducationTeachers can use FairyGen to transform students' hand-drawn characters into animated stories, stimulating students' writing and creativity.
• Digital art creationArtists use FairyGen to quickly generate animated videos from hand-drawn sketches, efficiently realizing their creative ideas while saving time and costs.
• Mental health therapyTherapists use FairyGen to help patients transform hand-drawn characters into animated stories, promoting emotional expression and psychological recovery.
• Children's Creative DevelopmentParents and children can use FairyGen together to create animations from children's hand-drawn characters, fostering imagination and creativity and enhancing parent-child interaction.
• Advertising and MarketingAdvertisers use FairyGen to create personalized animated ads, attracting audiences and enhancing brand communication.