Alibaba Cloud WAN2.1 Local Deployment Tutorial - 8GB VRAM Can Be Generated into Video

Today I'm sharing some recipes suitable for local cooking. AI Video content.

To be honest, nowadays AI Video generation technology is already quite mature, but it has only one drawback: it's expensive. The fees are basically charged per second, and it's hard to guarantee a perfect shot in one take. My wallet really can't handle it.

Today's Alibaba Wan2.1 project on GitHub is quite ingenious; it can run locally with a minimum of 8GB of video memory, which I believe most computers can meet now.

Wan2.1 is from the Ali Tongyi team.open sourceA complete set of basic models for video generation.

Project address: https://github.com/Wan-Video/Wan2.1

Wan2.1 not only includes text-to-video generation, but also covers tasks such as image-to-video generation, frame interpolation at the beginning and end, video editing, reference subject generation, and text-to-image generation.

Although Wan2.2 has been released, Wan2.1 is more complete in terms of low VRAM gameplay, and is especially suitable for local experimentation and secondary development.

Many video generation projects only release a single model, such as those that only generate text-to-video or image-to-video. Wan2.1 is more like a toolbox of video models, with different models responsible for different tasks.

• Wensheng Video can start with Wan2.1-T2V-1.3B, which mainly features 480P and is suitable for local testing and devices with low video memory;
• For better image detail and motion performance, choose Wan2.1-T2V-14B, which supports 480P and 720P.
• The video is compatible with Wan2.1-I2V-14B, and the official website provides 480P and 720P versions respectively.
• To control the start and end of the video feed, you can use Wan2.1-FLF2V-14B-720P.
• Wan2.1-VACE-1.3B is geared towards video creation and editing, targeting 480P scenarios, and has a lower barrier to entry.
• Wan2.1-VACE-14B supports 480P and 720P, making it suitable for generating reference subjects, making partial modifications, changing video subjects, and more complex video editing.

The official documentation provides several integration methods, including Hugging Face, ModelScope, Diffusers, ComfyUI, and Gradio.

This time, I chose the Wan2.1-T2V-1.3B, which only requires 8GB of VRAM to run. On an RTX 4090, it generated approximately 5 seconds of 480p video, while the official reference time is about 4 minutes. The 1.3B can also attempt 720p, but the official documentation states that 720p is less stable than 480p.

User inputPrompt wordsNext, Wan2.1 uses the T5 encoder to convert the text into semantic vectors. The semantic vectors describe the content of the image, the main action, the scene, the camera movement, the style of the image, the lighting, and the color.

Each subsequent Transformer Block will inject textual semantics into the video generation process through Cross-Attention. Wan2.1 supports multilingual input, with the official documentation particularly emphasizing Chinese and English.Prompt wordsability.

Wan2.1 also supportsPrompt wordsExpand upon it. For example, you could write:

A cat is making a cake in the kitchen.

Expanding the model can add details such as the cat's appearance, kitchen environment, animation, camera distance, lighting, style, and sequence of actions.

But in actual writingPrompt wordshour,recommendThe structure is:

Subject + Action + Scene + Camera + Lighting + Style + Movement Rhythm

A girl in a yellow raincoat stands on a rainy street, slowly turning her head to look at the camera. Cars drive by in the distance, and neon lights are reflected in puddles. The camera slowly pans forward from a medium shot, with shallow depth of field, a realistic cinematic style, and a cool blue tone.

Wan2.1 uses Diffusion Transformer, and the generation process can be roughly understood as follows:

• Create a piece of random noise.
• The model is based onPrompt wordsDetermine what should appear in the noise.
• Noise reduction through multiple iterations.
• Gradually, the main body, background, and actions are formed.
• VAE decodes latent variables into video.

Before running Wan2.1, you need to prepare the corresponding Python environment. The official dependencies mainly include:

• PyTorch ≥ 2.4.0
• torchvision ≥ 0.19.0
• Diffusers ≥ 0.31.0
• Transformers ≥ 4.49.0
• Accelerate
• Flash Attention
• Gradio
• OpenCV
• NumPy 1.x

The basic installation method is as follows, taking Windows PowerShell as an example:

1 Set-Location "想安装 Wan 2.1的文件夹路径" 2 git clone https://github.com/Wan-Video/Wan2.1.git 3 Set-Location .\Wan2.1 4 5 python -m venv .venv 6 .\.venv\Scripts\python.exe -m pip install --upgrade pip 7 .\.venv\Scripts\python.exe -m pip install -r requirements.txt

The .venv file here is the Wan2.1 project's own Python virtual environment. It is recommended to use the Python file in .venv for subsequent dependency installations and script execution to avoid polluting the system's Python environment.

Then download the 1.3B model:

1 Set-Location "Wan 2.1的文件夹路径" 2 .\.venv\Scripts\python.exe -m pip install "huggingface_hub[cli]" 3 4 .\.venv\Scripts\huggingface-cli.exe download Wan-AI/Wan2.1-T2V-1.3B ` 5 --local-dir .\Wan2.1-T2V-1.3B

After installation, you can use it by entering the .venv virtual environment in the Wan2.1 project directory. In PowerShell, enter the following format:

Set-Location "Wan 2.1的文件夹路径" # 进入 Wan2.1 项目目录。后面的模型路径、输出路径、generate.py 都基于这个目录执行。 New-Item -ItemType Directory -Force .\outputs # 创建 outputs 输出文件夹。如果文件夹已经存在，-Force 会让 PowerShell 继续执行，不报错。 .\.venv\Scripts\python.exe generate.py ` # 使用 Wan2.1 项目里的 .venv 虚拟环境 Python，运行 generate.py 生成脚本。末尾的 ` 表示命令还没结束，下一行继续。 --task t2v-1.3B ` # 指定任务类型。t2v 表示文生视频，1.3B 表示使用 13 亿参数版本。 --size 832*480 ` # 设置视频分辨率。这里是宽 832、高 480，也就是 480P 横屏。 --frame_num 65 ` # 设置生成帧数。Wan2.1 通常按 16 FPS 导出，65 帧大约是 4 秒。 --ckpt_dir .\Wan2.1-T2V-1.3B ` # 指定模型权重目录，也就是下载好的 Wan2.1-T2V-1.3B 文件夹。 --offload_model True ` # 开启模型卸载，把暂时不用的模型部分放到 CPU 内存里，降低显存压力，但会变慢。 --t5_cpu ` # 让 T5 文本编码器在 CPU 上运行，省显存。T5 负责理解提示词。 --sample_steps 20 ` # 设置采样步数。步数越高通常画面更稳，但耗时更久。20 步比较适合质量和速度平衡。 --sample_shift 8 ` # 控制采样过程的时间分布。1.3B 常用 8 到 12，8 是比较稳的默认选择。 --sample_guide_scale 6 ` # 控制提示词引导强度。数值越高越听提示词，但太高可能让画面变硬或变形。1.3B 用 6 比较稳。 --save_file .\outputs\moonlit_train_window.mp4 ` # 指定输出视频文件名和保存位置。这里会保存到 outputs 文件夹里。 --prompt "提示词放这里" # 视频提示词。描述主体、场景、动作、镜头、光线、风格和运动节奏。这个也要这种格式

When we worked on our first case, we started with a lower frame rate to gradually test the limits of our machine.

Wan2.1's default export speed is usually 16 FPS, so: 65 frames ÷ 16 FPS ≈ 4 seconds.

Common references: 33 frames ≈ 2 seconds, 49 frames ≈ 3 seconds, 65 frames ≈ 4 seconds, 81 frames ≈ 5 seconds.

Case 1: 33 frames, 12 sampling steps

PowerShell input:

Set-Location "D:\360MoveData\Users\win\Desktop\Distillation\Wan2.1"

.\.venv\Scripts\python.exe generate.py `

–task t2v-1.3B `

–size 832*480 `

–frame_num 33 `

–ckpt_dir .\Wan2.1-T2V-1.3B `

–offload_model True `

–t5_cpu `

–sample_steps 12 `

–sample_shift 8 `

–sample_guide_scale 6 `

–save_file .\outputs\quick_test.mp4 `

–prompt "A small robot walking through a rainy neon street, cinematic lighting."

At frame 33, after 12 steps, the main subject is clearly visible, the robot's shape is basically stable, and the neon street and wetland reflections are visible. The problem is that the image is rather empty, the rain effect is not obvious, and the robot's movements are stiff.

Case 2: 33 frames, 20 sampling steps

PowerShell input:

.\.venv\Scripts\python.exe generate.py `

–task t2v-1.3B `

–size 832*480 `

–frame_num 33 `

–ckpt_dir .\Wan2.1-T2V-1.3B `

–offload_model True `

–t5_cpu `

–sample_steps 20 `

–sample_shift 8 `

–sample_guide_scale 6 `

–save_file .\outputs\robot_neon_street.mp4 `

–prompt "A small friendly robot walking alone through a rainy neon street at night, wet asphalt reflecting red and blue signs, soft steam rising from street vents, cinematic cyberpunk atmosphere, low-angle tracking shot, slow camera movement, detailed metal body, glowing eyes, realistic rain, volumetric lighting, shallow depth of field, high detail, smooth motion, 35mm film look"

The footage is significantly better than before. The red and blue neon lights, wet roads, and nighttime atmosphere are all more stable, the robot is centered, and the subject is easily recognizable. However, 2 seconds is too short, and the movement is insufficient; the robot appears to be standing on the street making slight movements, rather than "walking through."

Case 3: 49 frames, 20 sampling steps

PowerShell input:

Set-Location "D:\360MoveData\Users\win\Desktop\Distillation\Wan2.1"

.\.venv\Scripts\python.exe generate.py `

–task t2v-1.3B `

–size 832*480 `

–frame_num 49 `

–ckpt_dir .\Wan2.1-T2V-1.3B `

–offload_model True `

–t5_cpu `

–sample_steps 20 `

–sample_shift 8 `

–sample_guide_scale 6 `

–save_file .\outputs\robot_neon_street_v2.mp4 `

–prompt "A cinematic 3-second video of a small friendly robot walking alone through a rain-soaked neon street at night. The robot has a compact metal body, round glowing eyes, and subtle reflections on its wet surface. Red, blue, and purple neon signs reflect on the asphalt. Light rain falls through the frame, steam rises from street vents, and distant shop lights glow softly in the background. Low-angle tracking shot, slow forward camera movement, realistic rain, volumetric lighting, shallow depth of field, smooth walking motion, detailed cyberpunk street, moody atmosphere, 35mm film look, high detail."

The sense of space is much better in this version. The fog, ground reflections, and distant shop lights are all more complete, and the robot blends into the environment more naturally. The 3.06-second clip is smoother to move than the previous two, and the footage feels more like a complete shot.

Case 4: 65 frames, 20 sampling steps

PowerShell input:

Set-Location "D:\360MoveData\Users\win\Desktop\Distillation\Wan2.1"

.\.venv\Scripts\python.exe generate.py `

–task t2v-1.3B `

–size 832*480 `

–frame_num 65 `

–ckpt_dir .\Wan2.1-T2V-1.3B `

–offload_model True `

–t5_cpu `

–sample_steps 20 `

–sample_shift 8 `

–sample_guide_scale 6 `

–save_file .\outputs\moonlit_train_window.mp4 `

–prompt "A cinematic 4-second video seen from inside an old night train. A young traveler sits beside a rain-covered window, watching a quiet countryside pass by under moonlight. Reflections of warm carriage lamps shimmer on the glass. Outside the window, distant trees, small houses, and silver mist slide slowly across the frame. The camera begins with a close-up of raindrops on the window, then gently shifts focus to the traveler's calm face and the moving landscape beyond. Soft film grain, realistic rain, warm interior light, cool blue moonlight outside, slow emotional camera movement, shallow depth of field, natural motion, detailed train cabin, quiet poetic atmosphere, 35mm film look, high detail."

As the video ran longer, the narrative improved. The warm lighting inside the carriage, the cool blue light outside the window, the raindrops, and the traveler's profile were all well-done; the figures didn't appear distorted, and the overall effect was very stable. However, the fact that it took so long to run indicates that my machine's limits are pretty much reached.

AI Videos are no longer just a tool for trying out new things.

Goldman Sachs predicts that the creator economy could grow from approximately $250 billion to $480 billion by 2027. While demand for video content is rising, the biggest pressures for creators remain cost, trial-and-error timelines, and production speed.

Wan2.1 cannot replace a professional video team, nor can it guarantee that every video will be a perfect shot on the first try. However, Wan2.1-T2V-1.3B brings the process of creating video from a pay-per-second platform back to a workflow where local graphics cards can be used for repeated testing.Prompt wordsYou can modify it gradually, adjust the parameters yourself, and the cost of failure is much lower.

Grand View Research predicts that globally AI The video generation market is projected to grow from $788.5 million in 2025 to $3.4416 billion in 2033, representing a CAGR of approximately 20.3%.

Wan2.1 is suitable for placement at the front end of the actual production chain: first, generate sample images at low cost, verify lenses, and test.Prompt wordsBased on style and other factors, decide which segments are worth retouching, editing, frame interpolation, or outsourcing to more expensive commercial models.

Wan2.1 transforms the previously expensive, low-frequency, budget-intensive video trial-and-error process into a routine production workflow that can be participated in by ordinary computers.

Original link:Achieved 16,000 stars on GitHub; local video generation possible with only 8GB of VRAM.