Vidu S1 - A real-time interactive video platform developed by Shengshu Technology

What is Vidu S1?

Vidu S1, launched by Shengshu Technology, is a globally leading real-time interactive video foundation model, marking the transition of AI video from offline generation to the era of real-time two-way interaction. Based on an autoregressive diffusion architecture, it supports real-time video generation at 540P resolution and 25FPS (up to 42FPS), and can run on consumer-grade GPUs. Users only need to upload a single image to create a digital character with zero training, and can drive the character's facial expressions, lip movements, gestures, and full-body movements in real time through voice commands, achieving stable interaction for an unlimited duration. Vidu S1 has scene understanding capabilities, can perceive camera images and respond in real time, and is widely used in AI companionship, virtual idols, interactive live streaming, game NPCs, and other scenarios, redefining the evolutionary path of digital humans from static content assets to continuously online intelligent interactive portals.

Main functions of Vidu S1

• Real-time two-way video interactionIt supports real-time dialogue with AI characters, similar to video calls, allowing them to understand, generate, and output simultaneously. Users can interrupt and change commands at any time, and the model adjusts the screen content instantly.
• Voice-driven full-dimensional behaviorVoice not only drives lip movements, but also understands semantics, intentions and emotions, generating matching facial expressions, eye contact, gestures and complete body movements in real time.
• Unlimited duration real-time generationBased on an autoregressive diffusion (AR + Diffusion) architecture, it can continuously generate for hours, with character identity and actions remaining consistent, without drifting or breaking down.
• 540P High Definition Real-Time Picture QualitySupports 540P (960×540) resolution, 25 FPS real-time generation, up to 42 FPS, and can run smoothly on consumer-grade GPUs.
• Single-image zero-training character creationNo 3D modeling or specialized training is required. Simply upload an image (real person, anime, cute pet, etc.), and the model will automatically understand the character's appearance and style, instantly enabling interaction.
• Custom toneIt supports selecting system preset tones or recording your own voice to achieve a unified and personalized visual image and sound.
• Scene perception and understandingOnce the camera is turned on, the model can recognize environmental information such as the number of people and their movement status in the scene, and provide real-time feedback accordingly.
• Streaming Real-Time ResponseIt employs the TurboServe inference engine to achieve efficient streaming scheduling, ensuring a low-latency, highly stable, and continuous online interactive experience.

How to use Vidu S1

• Access Experience EntryVisit the Vidu AI official website https://www.vidu.cn/vidu-stream or download the "Vidu AI Pro" APP. You can also access the platform for development through the API.
• Create a new characterClick "Create Character", upload a first frame image (supports any image such as real people, anime, cute pets, game characters, etc.), and enter the character name and description.
• Configure toneChoose a system preset tone, or click "Record Sound" to record your own voice to achieve a personalized consistency between your image and voice. Click "Submit" when finished.
• Initiate a conversationSelect an existing or preset role, grant microphone and camera permissions, and click "Allow" to enter the real-time interactive interface.
• Real-time voice interactionVoice commands can be issued directly through a microphone, and the character will understand the meaning in real time and generate corresponding lip movements, facial expressions, gestures, and full-body motion feedback.
• Adjust instructions at any timeDuring the dialogue, the instructions can be changed at any time (such as "make a heart shape", "adjust glasses", "get angry", etc.), and the model will respond instantly and adjust the subsequent screen content.
• Enable camera to enhance interactionAfter the camera is turned on, the model can identify the number of people and their movement status in the scene, and provide richer real-time feedback by combining environmental perception.

Vidu S1 official website address

• Official website addresshttps://www.vidu.cn/vidu-stream
• API Platformhttps://platform.vidu.cn/live/landing
• Technical Report: https://jt-zhang.github.io/files/Vidu_S1.pdf

Vidu S1's core advantages

• Real-time two-way interactionIt upgrades from traditional offline production to real-time video call-level dialogue, allowing users to interrupt and modify commands at any time, with the model responding instantly.
• Voice-driven full-dimensional behaviorBreaking through the limitations of audio-driven lip-reading, voice directly controls facial expressions, eye movements, gestures, and full-body posture, achieving true "understandable and accurate movement".
• Stable generation for unlimited durationBased on an autoregressive diffusion architecture, the game can sustain interaction for hours, with character identity and actions remaining consistent without drifting or breaking down.
• High-quality, high-frame-rate real-time outputIt supports real-time generation of 540P + 25FPS, with a maximum of 42FPS, placing it among the industry leaders in similar real-time interactive solutions.
• It can run on consumer-grade hardwareLeveraging the collaborative optimization of TurboDiffusion and TurboServe, it enables smooth real-time interaction on consumer-grade GPUs with a low deployment threshold.
• Single-image zero-training character creationNo 3D modeling or specialized training is required; simply upload any image (real person, anime, cute pet, etc.) to create an interactive digital human in seconds.
• Long-term consistency of rolesIt maintains the consistency of character appearance, style and timbre during long-term real-time generation, solving the pain points of traditional video generation that are prone to corruption and distortion.
• Scene awareness capabilityIt supports camera input and can recognize environmental information such as the number of people and their actions in the scene, achieving physical environment perception that surpasses voice commands.
• Leading quantitative indicatorsIt outperforms mainstream solutions such as OmniAvatar and HeyGen in core audio-driven digital human evaluation metrics such as CSIM (0.9192) and Sync-D (7.8470).

Comparison of Vidu S1 with similar competing products

Application scenarios of Vidu S1

• AI-powered emotional companionshipIt transforms real-life photos, anime characters, or pet pictures into virtual companion characters that can engage in real-time conversations and provide emotional feedback, offering 24/7 online emotional interaction.
• AI Virtual Idols and Interactive Live StreamingVirtual anchors can respond to comments and tipping commands in real time, and adjust their performance actions and expressions according to the audience's voice, creating a truly interactive live streaming experience.
• Game NPCs and Role-PlayingGame characters no longer rely on preset scripts; they can understand player voice commands in real time and generate corresponding actions and dialogues, greatly enhancing immersion and freedom.
• Brand Digital Humans and Virtual Customer ServiceBusinesses can quickly transform their brand IP into real-time online digital employees for scenarios such as receiving inquiries, explaining products, and providing intelligent customer service, thereby reducing labor costs.
• Online Education and Smart TutoringHistorical figures and subject tutors can be "awakened" for real-time Q&A and interactive teaching; real-time dialogue companions can be created in language learning.
• XR / Metaverse ExperienceIt provides VR/AR devices with low-latency, high-frame-rate real-time digital human rendering capabilities, supporting real-time social interaction and virtual meetings in the metaverse.