Animated Drawings - An AI-powered tool for converting hand-drawn artwork into animation, developed by Meta AI.

What are Animated Drawings?

Animated Drawings is an open-source tool from Meta AI that uses AI technology to automatically convert hand-drawn portraits into animations. Animated Drawings provides a complete set of tools and algorithms to detect and segment figures in drawings, generating animation effects. Users can flexibly control the animation's scenes, actions, and characters through configuration files. The tool includes an automatic annotation generation tool to help users quickly create the annotation files needed for the animation. Animated Drawings supports multiple output formats, such as MP4 video and transparent GIFs.

Main functions of Animated Drawings

• Automatically animated hand-drawn charactersIt automatically converts user-uploaded hand-drawn character illustrations into animations, supporting various actions and scenes.
• Flexible configuration filesIt controls the details of the animation based on configuration files (such as YAML format), including character movements, scene settings, etc.
• Automatic annotation generationAI models can be used to automatically detect figures in drawings and generate annotation files (such as joint positions and segmentation masks), simplifying the animation production process.
• Multi-format outputIt supports outputting in formats such as MP4 video and transparent GIF to meet different usage needs.
• Adding multiple characters and backgroundsIt supports adding multiple characters and background images to animations, enriching the animation content.
• Custom BVH file supportUsers can drive animations using their own BVH (bone animation) files, adapting to different skeleton movements.
• bug fixing toolsIt provides a user interface tool that allows users to manually correct erroneous annotations generated by the AI model, ensuring more accurate animation effects.

The technical principles of Animated Drawings

• Image detection and segmentationThis system uses computer vision technology to detect and segment hand-drawn portraits uploaded by users. Through a pre-trained object detection model, the system can identify the outlines of figures in the paintings and their positions within the image. A segmentation algorithm is then used to generate a segmentation mask, separating the figures from the background.
• Attitude estimationAfter detecting the character's outline, the system uses a pose estimation model (such as the deep learning-based OpenPose model) to detect key points, such as the joint positions of the head and limbs. The detection results of these key points are saved as annotation files, providing necessary data support for animation generation. With accurate joint position detection, the system can better understand the character's posture and structure, achieving more natural motion mapping.
• Animation generationTo apply motion data to hand-drawn figures, BVH (Bone Void) files are used as the motion source. These files contain skeletal data for the figure's movements. The system uses motion redirection technology to map the motion data from the BVH files to the joints of the drawn figure, enabling it to perform corresponding actions. The system employs an As-Rigid-As-Possible shape deformation algorithm to ensure that the figure's shape remains natural and consistent during motion execution.
• Configuration and renderingUsers can flexibly define specific animation parameters, including characters, actions, and scenes, through configuration files (such as YAML format). The configuration file provides detailed instructions for animation generation, enabling the system to generate animations according to the user's personalized needs. Finally, the system renders the animation as a video or GIF based on the configuration file, supporting various output options such as transparent backgrounds to meet the needs of different users.

Project address for Animated Drawings

• Project official websitehttps://sketch.metademolab.com/canvas
• GitHub repositoryhttps://github.com/facebookresearch/AnimatedDrawings

Application scenarios of Animated Drawings

• Creative and Arts EducationIt inspires the creativity of children and art lovers by turning paintings into animations, enhancing the fun and interactivity of art education.
• Content creation and entertainmentCreators can quickly produce hand-drawn animated short videos or films, share them on social media, attract viewers, and enhance the appeal of their content.
• Interactive experiences and exhibitionsIn museums, exhibitions, and offline events, audience drawings can be animated in real time to increase interactivity, fun, and engagement.
• Advertising and MarketingBrands create personalized hand-drawn animated ads to attract consumer attention and increase social media exposure.
• Game developmentQuickly generate character animation prototypes, develop interactive games, and allow players to control game character movements through drawing.