DSpark is an open-source speculative decoding acceleration framework jointly developed by DeepSeek and Peking University.

What is DSpark?

DSpark is an open-source speculative decoding acceleration framework developed by DeepSeek in collaboration with Peking University. It specifically addresses the pain point of slow, incremental autoregressive generation for large models. Employing a semi-autoregressive generation architecture, it models token dependencies using lightweight Markov heads, balancing the speed and coherence of parallel drafts. It also introduces confidence-based scheduling verification, dynamically allocating verification resources based on system load. The framework has been deployed in the DeepSeek-V4-Flash/Pro production environment, achieving a 57%–85% improvement in generation speed per user and a maximum throughput improvement of 400%. The project is open-source under the MIT license, compatible with mainstream models such as Qwen and Gemma, and provides a practical engineering solution for efficient inference of large models.

DSpark's main functions

• Semi-autoregressive draft generationIt retains the high-speed characteristics of the parallel draft model, while adding a lightweight Markov head (or RNN head) to model the dependency of adjacent tokens, which alleviates the suffix decay problem of traditional parallel schemes and makes the candidate sequence more coherent.
• Confidence score predictionOutput a confidence score for each candidate token and estimate the probability that the token will be accepted by the target model in real time, providing a basis for subsequent scheduling decisions.
• Hardware-aware prefix schedulingBased on the system's concurrent load, candidate confidence level, and engine throughput curve, the token length that should be verified for each request is dynamically determined. More verifications are performed when the system is idle, and low-confidence requests are streamlined when the system is busy.
• Production-grade inference accelerationIt has been integrated into the DeepSeek-V4-Flash/Pro online service, achieving a 57%–85% improvement in single-user generation speed and a maximum 400% improvement in aggregate throughput under real high-concurrency traffic.
• Multi-model compatibility supportIn addition to DeepSeek's self-developed models, it is compatible with mainstream open-source large models such as Qwen and Gemma.
• Full-stack open sourceThe complete code, papers, training scripts, and model checkpoints are open-sourced under the MIT license, lowering the barrier to entry for developers.

How to use DSpark

• Cloning projectClone the DeepSpec open-source repository and configure the runtime environment, installing the relevant dependencies.
• Download ModelDownload the target model (such as DeepSeek-V4, Qwen3, or Gemma4) and the corresponding DSpark draft model checkpoints.
• Select dependent modules as needed.Load the semi-autoregressive draft model, and select either the Markov head or the RNN head as the sequential dependency module as needed.
• Scheduling verificationEnable confidence-based scheduling verification and configure a hardware-aware prefix scheduler to adapt to the current concurrent load and throughput curves of the GPU cluster.
• Integrated EngineIntegrate DSpark into existing inference engines (such as vLLM or self-developed service frameworks) to replace traditional MTP-1 or standard autoregressive decoding processes.
• Initiate a request:By initiating a request via API or command line, the system automatically executes an accelerated inference process: "draft generation → confidence assessment → dynamic verification → result return".

DSpark's official website address

• GitHub addresshttps://github.com/deepseek-ai/DeepSpec
• Hugging Facehttps://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-DSpark
• Technical Papers: https://github.com/deepseek-ai/DeepSpec/blob/main/DSpark_paper.pdf

DSpark's core advantages

• Semi-autoregressive architecture, balancing speed and consistencyIt retains the high throughput advantage of parallel draft models, while effectively alleviating the suffix decay problem of traditional parallel schemes by modeling the dependencies between tokens through a lightweight Markov head (or RNN head), resulting in more coherent drafts and higher acceptance rates.
• Confidence-based dynamic scheduling enables smarter resource utilization.The system introduces confidence score prediction and hardware-aware prefix scheduler to dynamically adjust the verification length based on system concurrent load, candidate survival probability and engine throughput curve. More verifications are performed when the system is idle and low-confidence requests are streamlined when the system is busy, thus avoiding wasting batch capacity.
• Significant improvement in production-grade performanceIt has been deployed on the DeepSeek-V4-Flash/Pro online service, achieving a speed improvement in single-user generation under real high-concurrency traffic. 57%–85%Maximum increase in aggregate throughput 400%.
• Broad model compatibilityIt not only supports DeepSeek's self-developed models, but is also compatible with mainstream open-source large models such as Qwen and Gemma, making it suitable for a variety of scenarios.
• Full-stack open source, low barrier to entryThe complete code, paper, training scripts, and model checkpoints are open-sourced under the MIT license, allowing developers to quickly integrate them into vLLM or their own inference engines.
• Zero mass loss accelerationBased on the speculative decoding mechanism, the output distribution of the target model remains unchanged, achieving a significant speedup without sacrificing the accuracy and quality of the generated content.

Comparison of DSpark with similar competing products

Application scenarios of DSpark

• Real-time chat and dialogue systemFor low-latency requirements in highly interactive scenarios, DSpark can significantly improve single-user generation speed, enhance dialogue fluency, and improve user experience.
• Code assistants and programming toolsThe candidate token acceptance rate for code generation tasks is high (average accepted length of 5.12), and DSpark can accelerate code completion, automatic error correction, and multi-file generation.
• Multi-round Agent WorkflowIn complex tasks involving multiple rounds of calls and interconnected toolchains, reduce response latency in each round and prevent latency from accumulating and amplifying with each round.
• Mathematical Reasoning and Online EducationMathematical tasks (GSM8K, MATH, AIME, etc.) have the highest candidate acceptance rate (average 5.57), making them suitable for problem-solving scenarios with long reasoning steps and strong structure.
• High-concurrency cloud API serviceBy dynamically adapting GPU load through a hardware-aware prefix scheduler, throughput can be increased by up to 400% under real high-concurrency traffic, reducing the cost per request.
• Local deployment of open source modelsIt is compatible with mainstream models such as Qwen and Gemma, and is open source under the MIT license, making it easy for small and medium-sized enterprises and developers to integrate efficient inference capabilities into their self-developed frameworks or engines such as vLLM.