Code2Video - An AI-powered video generation framework developed by a national university

What is Code2Video?

Code2Video is an AI-powered educational video generation framework developed by the Show Lab team at the National University of Singapore. It automatically generates high-quality educational videos using Python code. Its core innovation lies in its three-agent collaborative model: Planner transforms knowledge points into structured storyboards, Coder converts these storyboards into executable Manifest code, and Critic optimizes the visual layout. Compared to traditional video generation tools, it outperforms traditional tools in knowledge transfer efficiency (40% improvement in MMMC benchmark tests) and image stability, making it particularly suitable for fields such as mathematics and programming that require precise visual expression.

Code2Video's main functions

• Code Center Generation ParadigmUsing Manim code as a unified medium, the temporal sequence and spatial layout of the video are implemented to ensure that the content is clear, coherent and reproducible.
• Modular three-agent cooperationThe planner (storyboard extension), encoder (debuggable code synthesis), and reviewer (layout optimization) work together to achieve structured video generation.
• Multi-dimensional evaluation systemThe quality of the generated content is comprehensively evaluated based on efficiency, aesthetics, and end-to-end knowledge transfer effectiveness (such as TeachQuiz and AES metrics).
• Flexible script generationSupports single-concept and batch video generation, with configurable parameters such as API selection, output directory, and parallel processing.
• Rich visual resource integrationCombine with APIs such as IconFinder/Icons8 to obtain icons and other materials, thereby improving the visual effect of videos.

The technical principles of Code2Video

• Code Center Generation ParadigmUsing Manim code as a unified medium, it controls the temporal sequence and spatial layout of the video by generating executable animation scripts (rather than directly manipulating pixels), ensuring that the content structure is clear and reproducible.
• Multi-agent collaborative frameworkThe three modules—planner, encoder, and reviewer—work collaboratively.
  • Planner: Parse the input content (such as text or concepts), generate a storyboard (storyboard), and clarify the logical structure and keyframe sequence of the video.
  • encoderBased on the storyboard, it calls LLM (such as GPT-4) to generate animation code that conforms to Manim syntax, supporting dynamic debugging and iterative optimization.
  • ReviewerPerform layout and visual consistency checks on the generated code, and correct errors (such as object position conflicts and unreasonable animation timing) through feedback loops.
• Manim engine renderingThe generated code is rendered into video using the Manim engine, leveraging its vector graphics processing capabilities to achieve high-precision mathematical formulas, dynamic charts, and smooth animation transitions. The rendering process includes frame segmentation, transformation, quantization, and entropy encoding, ultimately outputting in formats such as MP4.
• External resource integrationIt combines icon libraries (such as IconFinder) and multimedia APIs to automatically insert visual elements (such as icons and backgrounds), enhancing the richness and aesthetics of videos.
• Evaluation and optimization mechanismThe code generation quality is evaluated from multiple dimensions, including knowledge transfer accuracy, visual fluency, and user learning effectiveness, using metrics such as TeachQuiz and AES (Automatic Evaluation System), and the code generation strategy is iteratively optimized.

Code2Video's project address

• Project official websitehttps://showlab.github.io/Code2Video/
• Github repositoryhttps://github.com/showlab/Code2Video
• arXiv technical paper: https://arxiv.org/pdf/2510.01174

Application scenarios of Code2Video

• Educational video generationTeachers can quickly transform abstract knowledge (such as mathematical formulas and physical laws) into dynamic and visual teaching videos. Through animated demonstrations and step-by-step explanations, the efficiency of knowledge transfer is improved. It is suitable for K-12, higher education and vocational training.
• Scientific research and academic presentationResearchers can generate animations of technical principles, simulations of experimental processes, or videos demonstrating paper results to help intuitively showcase complex models (such as algorithm processes and scientific experiments), enhancing the clarity and appeal of academic communication.
• Corporate training and skills instructionUsed to create standardized operating instructions (such as equipment usage, software tutorials), safety demonstrations, or product training videos. Code control ensures the accuracy and consistency of the content, reducing training costs.
• Personalized learning content creationIt generates customized videos based on learners' needs (such as language learning and programming instruction) and combines interactive elements (such as embedded exercises and dynamic feedback) to adapt to different learning paces and levels.
• Popular science and knowledge disseminationMedia outlets or creators can quickly generate science animations (such as explanations of astronomical phenomena and reconstructions of historical events), transforming professional knowledge into easily understandable video content for the general public, thereby enhancing the fun and dissemination of science popularization works.