Step-Audio-EditX - Step-Star's open-source audio editing platform

What is Step-Audio-EditX?

Step-Audio-EditX is the world's first LLM-level audio editing model, open-sourced by StepStar. It features fine-grained iterative control across three axes: emotion, speaking style, and paralinguistics. It allows for arbitrary increases or decreases in the intensity of emotions such as anger, happiness, and sadness, and can layer styles like coquettishness, whispering, and elderly speech multiple times. It can also insert 10 types of natural paralinguistic tokens, such as breathing, laughter, and sighs, much like subtitles. It includes zero-sample TTS, cloning voices without requiring the target speaker's voice, and instantly switching dialects by adding tags like "[Sichuanese]" or "[Cantonese]" before the text. The model is entirely based on SFT+PPO performed on large-interval synthetic data, achieving attribute decoupling and iterative control without the need for additional encoders or adapters during the post-training phase.

Main functions of Step-Audio-EditX

• Emotional EditorDozens of tags, such as anger, happiness, sadness, excitement, fear, surprise, and disgust, can be used to iterate and increase or decrease their intensity.
• Style EditorIt offers more than ten speaking styles, including coquettish, whispering, elderly, childlike, serious, generous, and exaggerated, and supports layering and fine-tuning.
• Sub-language insertionPrecisely insert 10 types of natural tokens, including breathing, laughter, sighing, surprise (oh/ah), confirmation (en), dissatisfaction (hnn), question (ei), and um (uhm).
• Zero-sample TTSIt can clone voice without requiring the target person's voice, and directly switch dialects by adding tags such as "[Sichuan dialect]" or "[Cantonese]" before the text.
• Iterative controlThe same voice can be edited repeatedly, attributes are decoupled to avoid crosstalk, and the effect is enhanced step by step.
• Open source and lightweightOffers an 8-bit quantized version, runs on a single card with 8 GB of video memory, and achieves the best sound quality with 4×A800/H800; includes inference/training code, Grado Demo, and HF Space.

The technical principles of Step-Audio-EditX

• Dual-codebook audio word segmentationThe parallel processing of 16.7 Hz/1024 "language codebooks" and 25 Hz/4096 "semantic codebooks" in a 2:3 staggered slice converts any speech into discrete tokens while preserving emotional and prosodic information, providing "speech vocabulary" for subsequent LLM direct operations.
• 3B Audio LLMThe 3B model is pre-trained with text and then warmed up. The text token and the dual-codebook audio token are concatenated into the input according to the chat format, and only the audio token is output. The training data is text:audio = 1:1, making full use of the existing text LLM ecosystem for post-training.
• Large-interval synthetic data drivenWithout introducing additional encoders or adapters, SFT+PPO is performed using only paired data with "same text, different attributes (sentiment/style/sub-language)"; the large gap forces the model to learn attribute decoupling, enabling iterative intensity increase/decrease and multi-attribute superposition.
• Stream matching + BigVGANv2 decodingThe dual-codebook token output by the audio LLM is used to generate a Mel spectrum by the DiT-stream matching module, and then the waveform is restored by the BigVGANv2 vocoder; 200,000 hours of high-quality training ensures pronunciation accuracy and timbre similarity.
• Unified frameworkThe same "word segmentation → LLM → decoding" pipeline simultaneously supports zero-sample TTS, emotion/style/paralanguage editing, speech rate adjustment and noise reduction, without the need for task-specific modules, significantly reducing system complexity and inference costs.

Step-Audio-EditX project address

• Project official websitehttps://stepaudiollm.github.io/step-audio-editx/
• Github repositoryhttps://github.com/stepfun-ai/Step-Audio-EditX
• HuggingFace model libraryhttps://huggingface.co/stepfun-ai/Step-Audio-EditX
• arXiv technical paper: https://arxiv.org/pdf/2511.03601

Application scenarios of Step-Audio-EditX

• Audio content upgradeAudiobooks, podcasts, and news readings can be overlaid with emotions or styles such as "happy/sad/whispering" with one click, quickly generating multiple versions of audio without re-recording, enhancing the immersive experience.
• Video and advertising voiceoverShort videos, animations, and commercials can clone character voices with zero samples, and then iterate to add styles such as "coquettish, exaggerated, and serious" to achieve low-cost, multi-character, and multi-emotion automatic dubbing.
• Games/Virtual IdolsFor NPCs, virtual anchors, and VTubers, the voice is cloned using a reference voice line, and then laughter, breathing, sighs, and other paralinguistic sounds are inserted in real time to create more vivid and sustainable interactive character voices.
• Intelligent customer service and voice assistantThe customer service robot can improve the user experience by editing existing TTS (Text-to-Speech) messages to transform bland responses into warm and reassuring ones; it also supports dialect tags to meet localized service needs.
• Education/Language LearningOnline courses and language apps can generate age-appropriate pronunciations using a "senior/child/whisper" style, or instantly switch from standard Mandarin to Cantonese or Sichuan dialect to help students follow along and imitate, reducing recording costs for teachers.
• Meeting minutes and accessibilityFor meeting recordings containing noise or excessively long pauses, first perform "noise reduction + mute trimming" editing, then increase the speaking speed or add emotion as needed to generate clear and easy-to-read meeting minutes audio.