FS-DFM - A diffusion language model developed by Apple in collaboration with The Ohio State University

What is FS-DFM?

FS-DFM (Few-Step Discrete Flow-Matching) is a diffusion language model developed by Apple in collaboration with Ohio State University for the rapid generation of long texts. The model is trained by using the number of sampling steps as an explicit parameter, enabling it to generate high-quality text in fewer steps. FS-DFM combines reliable update rules with strong teacher guidance to ensure accurate probability updates without overfitting. In language modeling benchmarks, FS-DFM achieves the perplexity level of a 1024-step discrete flow baseline with only 8 sampling steps, while increasing sampling speed by 128 times, significantly improving efficiency and throughput.

Main functions of FS-DFM

• High-efficiency samplingAchieving the effect of the traditional 1024-step diffusion model with only 8 sampling steps, increasing speed by 128 times.
• Long text generationIt is suitable for generating long texts and solves the efficiency bottleneck of traditional autoregressive models in generating long sequences.
• Stability and controllabilityBy optimizing sampling rules and providing teacher guidance, we ensure that the generation process is stable and easy to control.

Technical principles of FS-DFM

• Based on Discrete Flow Matching (DFM) FrameworkThe technical basis of FS-DFM is Discrete Flow-Matching (DFM), which generates text by learning a probabilistic path from the noise distribution to the target distribution. DFM uses the properties of continuous-time Markov chains (CTMC) to provide a method for parallel text generation, which can significantly improve generation efficiency compared to traditional autoregressive models.
• Explicit sampling stepsFS-DFM uses the number of sampling steps as an explicit parameter, training the model to maintain consistent generation performance across different step budgets. This means the model achieves comparable quality to traditional methods (e.g., 1024 steps) with fewer steps (e.g., 8 steps), significantly reducing the computational resources and time required for generation.
• Reliable update rulesTo ensure the stability and accuracy of few-step generation, FS-DFM introduces a reliable update rule. This rule controls the direction and magnitude of probability updates, avoiding overshooting and making the generation process more stable.
• Cumulative ScalarFS-DFM introduces the concept of a cumulative scalar, providing the correct probability flow for each finite step by adjusting the rate of the integrator scheduler over the time interval. This enables the model to achieve efficient updates in a few generation steps, maintaining sufficient update momentum in early steps and preventing the generation process from stalling.

FS-DFM project address

• arXiv technical paper: https://arxiv.org/pdf/2509.20624

Application scenarios of FS-DFM

• Content creationIt can quickly generate high-quality long texts for use in creating articles, stories, news reports, etc., helping content creators improve efficiency.
• Intelligent Customer ServiceThe intelligent customer service system can quickly generate detailed responses, improving response speed and user experience.
• Language translationIt efficiently generates long texts, which can be used to translate long documents, improving translation efficiency and quality.
• Creative WritingIt provides inspiration for writers and creative professionals, quickly generating story outlines, scripts, or poems to stimulate creative ideas.
• EducationGenerate educational materials, such as course outlines, teaching cases, or exercises, to assist teachers in preparing teaching content.