FlowDirector - A video editing framework jointly developed by Westlake University and Central South University

What is FlowDirector?

FlowDirector is a novel training-free video editing framework developed by the AGI Lab team at Westlake University in collaboration with Central South University. It is specifically designed for precise editing of video content based on natural language instructions. The framework directly models the editing process in the data space, using smooth transition paths driven by ordinary differential equations (ODEs) to avoid the temporal inconsistencies and structural distortions inherent in traditional latent space inverse mapping methods. FlowDirector introduces a Spatial Attention Flow Correction (SAFC) mechanism to precisely protect the spatiotemporal consistency of unedited regions and enhances semantic alignment capabilities based on a Differential Averaging Guided (DAG) strategy. The framework performs exceptionally well in multiple video editing benchmarks, significantly improving instruction compliance, temporal consistency, and background protection, providing a new solution for efficient and coherent video editing.

FlowDirector's main functions

• Precise semantic editing: Modify the semantics of video content based on natural language instructions, such as replacing "bear" with "dinosaur" in the video.
• Spatiotemporal consistency protectionDuring the editing process, maintain the temporal continuity and spatial structure of the video to avoid content misalignment or style inconsistency.
• Local editing and global protectionBased on spatial attention mechanism, only the target area is edited, preserving the original content and dynamics of the unedited area.
• High-efficiency training-free editingNo additional training is required; editing can be performed directly using a pre-trained text-to-video (T2V) model, reducing editing costs and improving efficiency.
• Supports multiple editing tasksIt supports handling a variety of complex video editing tasks, such as object replacement, texture conversion, local attribute modification, and object addition/deletion.

FlowDirector's technical principles

• Editing Flow GenerationBased on a pre-trained text-to-video (T2V) model, the velocity field difference between the source and target videos is calculated to generate an editing path directly from the source to the target video. A smooth transition path driven by ordinary differential equations (ODEs) avoids the structural distortion problem caused by latent space inverse mapping in traditional methods.
• Spatial Attentive Flow Correction (SAFC)An attention-guided masking mechanism is introduced. By extracting an attention map relevant to the editing task, a mask is generated to precisely control the editing area. During ODE-driven editing, the mask is applied to the velocity field to freeze non-target areas, ensuring that the area remains unchanged during editing.
• Differential Averaging Guidance (DAG)Inspired by Classifier-Free Guidance (CFG), this approach enhances semantic alignment by generating multiple candidate edit streams and calculating the difference signals between them. It adjusts the edit trajectory based on the difference signals, making the edit results closer to the target semantics while maintaining structural consistency.

FlowDirector project address

• Project official website:https://flowdirector-edit.github.io/
• GitHub repository:https://github.com/Westlake-AGI-Lab/FlowDirector
• arXiv technical paper:https://arxiv.org/pdf/2506.05046

Application scenarios of FlowDirector

• Video effects productionIt generates special effects quickly based on simple text commands, replacing objects in ordinary scenes with fantasy elements (such as replacing "car" with "dragon"), adding creativity to film and television works.
• Advertising video productionQuickly adjust video content based on advertising copy, replacing product appearance or background environment with elements that match the advertising theme to enhance the appeal and relevance of the advertisement.
• Animation ProductionQuickly modify characters or scenes in animated videos, such as changing a character's clothing from "red" to "blue" or changing a scene from "forest" to "city," thus accelerating the animation creation process.
• Personalized video editingUsers can quickly edit short videos according to their needs, replacing pets in the video with other animals, or adding interesting special effects to enhance the video's fun and appeal.
• Creative content generationCreators can quickly generate video content that fits a specific theme or style, transforming ordinary landscape videos into a "cyberpunk style" to meet the needs of social media users for creative content.