Lyria 2 - DeepMind's AI music generation model

What is Lyria 2?

Lyria 2 is Google DeepMind's third-generation AI music generation model. As a core component of the Vertex AI platform, it boasts high-fidelity audio generation capabilities, outputting professional-grade stereo audio at 48kHz and 24-bit. It supports various music styles, including pop, rock, and classical, enabling cross-style fusion. Lyria 2 expands upon the Lyria RealTime model, supporting real-time creation and adjustment. Creators can trigger AI composition in real-time via MIDI controllers or microphones. New features include "Create," "Extend," and "Edit," used for generating music snippets from text, creating extensions based on existing audio, and adjusting audio mood and style, respectively.

Main features of Lyria 2

• High-fidelity audio generationLyria 2 can generate professional-grade 48kHz high-fidelity stereo audio, adaptable to a variety of music styles, including pop, classical, and electronic.
• Real-time creation and adjustmentIt expands the Lyria RealTime model, allowing users to create and adjust music in real time, instantly changing the style, rhythm, and emotion of the music, making it suitable for live performances and rapid prototyping.
• Text-driven creationThe newly added "Create" function allows users to generate music snippets using text descriptions or lyrics, greatly lowering the barrier to creation.
• Audio continuation generationThe "Extend" feature generates extended content based on existing audio clips, helping creators expand their musical works.
• Audio Editing and FusionThe "Edit" function allows users to adjust the mood and style of audio through preset options or text prompts, and can blend different music segments.
• Digital watermarking technologyAll music generated by Lyria 2 is embedded with SynthID watermark technology to ensure the recognizability of AI-generated content and prevent copyright disputes.

The technical principles of Lyria 2

• Multimodal input and parameterized controlLyria 2 supports multiple input methods, including text, MIDI, and audio clips. Users can control music generation through descriptions (such as "cheerful jazz piano music") or parameters (such as BPM and mode). The generation engine combines a diffusion model and a generative adversarial network (GAN) to achieve style decoupling and recombination.
• Real-time interaction and dynamic creationLyria's RealTime feature, through TensorRT acceleration and streaming inference, compresses generation latency to under 500ms, supporting live performances and real-time creation. Users can trigger AI-generated compositions in real time via MIDI controllers or microphones.
• Modular creation platformLyria 2 is integrated into Music AI Sandbox, providing a modular toolchain from melody generation to chord generation, supporting drag-and-drop combination of functional modules, and lowering the creative threshold.

Lyria 2 project address

• Project official website:deepmind.google/technologies/lyria/

Application scenarios of Lyria 2

• Professional music productionLyria 2 can generate professional-grade 48kHz high-fidelity stereo audio, supporting a variety of music styles, including pop, classical, and electronic, helping musicians quickly generate background music, sound effects, or complete soundtracks.
• InspirationUsers can generate musical fragments through text descriptions (such as "cheerful jazz piano music") or lyrics, breaking through creative bottlenecks.
• Film and television scoresLyria 2 can generate dynamic background music that matches the mood of the scene based on script keywords, and supports real-time adjustments, significantly improving creative efficiency.
• Advertising background musicBy quickly generating customized music using "brand keywords + emotion tags", advertising teams can iterate rapidly and reduce costs.
• AI Music CourseCollaborating with institutions such as Berklee College of Music and Stanford CCRMA to develop AI music courses and promote innovation in music education.