SongGeneration 2 - A music generation model jointly developed by Tencent and Tsinghua University

What is SongGeneration 2?

SongGeneration 2 is a 4B-parameter music generation model jointly open-sourced by Tencent and Tsinghua University. It employs a hybrid LLM-diffusion architecture and hierarchical representation design to achieve commercial-grade music generation quality. The model achieves a lyric accuracy of 8.55% PER, surpassing commercial models such as Suno v5. SongGeneration 2 supports multiple languages, including Chinese and English, and complex multi-track arrangements, allowing for precise style control through text descriptions and audio cues. The model can run locally on consumer-grade hardware with 22GB of VRAM and can generate a complete song within one minute.

Main functions of SongGeneration 2

• High-quality song generationA complete song with a length of 4 minutes and 30 seconds, beautiful melody, and rich arrangement was generated based on a hybrid LLM-diffusion architecture.
• Multilingual lyrics synthesisIt supports lyrics in multiple languages, including Chinese, English, Spanish, and Japanese, and achieves clear and accurate pronunciation with an ultra-low frequency error rate of 8.55%.
• Diverse generation modesIt can flexibly output complete songs, instrumental music, pure vocals, or separate vocal and accompaniment dual audio tracks.
• Precise style controlPrecise control over the generated style is achieved through text descriptions (gender, genre, mood, instrument) or 10-second audio prompts.
• Flexible deployment optionsSupports local operation with 22GB of video memory and fast online experience via HuggingFace Space, and provides a low video memory adaptation solution.

The technical principles of SongGeneration 2

• Hybrid LLM-Diffusion ArchitectureThe Language Model (LeLM) acts as the "composition brain," coordinating the overall musical structure and performance details, while the Diffusion Model, as the "high-fidelity renderer," synthesizes complex acoustic details under the guidance of the Language Model. The two work together to balance musicality and sound quality.
• Hierarchical representation modelingThe design employs parallel modeling, with mixed tokens capturing high-level melodic and structural semantics, and dual-track tokens modeling fine-grained acoustic variations in vocal and accompaniment tracks respectively.
• Automated Aesthetic EvaluationA fine-grained evaluation framework was constructed based on 11,717 expert-annotated samples to inject musicality prior knowledge into model training and to introduce a CFG strategy based on musicality labels in the inference stage.
• Three-stage progressive post-training: By narrowing the data distribution through SFT to build a high-quality foundation, large-scale offline DPO uses 200,000 pairs of strict positive and negative samples to eliminate lyric illusions, and semi-online DPO periodically updates the model to break through the upper limit of musicality.

SongGeneration 2 project address

• GitHub repositoryhttps://github.com/tencent-ailab/songgeneration
• HuggingFace model libraryhttps://huggingface.co/tencent/SongGeneration

Application scenarios of SongGeneration 2

• Music creation assistanceIndependent musicians can quickly generate demos to validate their inspiration, lowering the barrier to entry for arrangement and reducing production costs.
• Video music productionSelf-media creators can instantly generate customized background music based on the video style and emotional needs.
• Game audio developmentGame developers can generate suitable theme music and sound effects in batches for different scenes, characters and storylines.
• Advertising and Marketing ContentBrands can generate custom jingles based on campaign tone, enabling rapid iteration of audio assets.
• Educational and entertainment applicationsThe music teaching platform offers stylized accompaniment generation, while karaoke and social applications support personalized song creation for users.