What is a Chain-of-Draft (CoD)? - AI Encyclopedia
Chain-of-Draft (CoD) is a new AI reasoning paradigm proposed by Zoom's research team. It improves reasoning efficiency and saves costs by mimicking the concise thought process of humans. This approach is inspired by the human reliance on drafts or...
Chain-of-Draft (CoD) is a new type of...AIDraft Chain improves reasoning efficiency through concise intermediate reasoning steps. It mimics the concise thinking of humans when solving problems, limiting the key information output at each step to no more than five words. Compared to traditional CoT (Cooperation of the Thinker), Draft Chain significantly reduces token usage, substantially lowering reasoning costs and latency while maintaining high accuracy. Draft Chain performs exceptionally well in various reasoning tasks, such as arithmetic, common sense, and symbolic reasoning, making it suitable for real-time processing.AIApplications, resource-constrained environments, and cost-sensitive scenarios.
What is a draft chain?
Chain-of-Draft (CoD) is a new technology proposed by Zoom's research team.AIThis reasoning paradigm improves reasoning efficiency and saves costs by mimicking the concise human thought process. It is inspired by how humans rely on drafts or shorthand to capture key insights when solving problems. Compared to traditional chain-of-thought (CoT), draft chains encourage models to generate extremely simple intermediate reasoning steps, capturing only the crucial information needed to solve the problem.
How draft chains work
Chain-of-Draft (CoD) works by mimicking the concise thinking process of humans when solving problems, allowing large language models (LLMs) Generate concise yet information-rich intermediate steps during the inference process. Specifically, the CoD strategy requires the model to limit the number of words used in each inference step, typically no more than five words. This approach is not a mandatory restriction but a guiding suggestion that promotes concise inference steps.
CoD encourages models to generate minimal and information-rich intermediate results at each step of reasoning, focusing on key computational or transformation steps. By reducing verbose outputs, CoD significantly reduces token usage, decreasing the overall output length and latency. Despite reduced token usage, CoD maintains comparable accuracy to traditional Chain of Thought (CoT) in various reasoning tasks, such as arithmetic, commonsense, and symbolic reasoning.
Main applications of draft chains
- Real-time customer supportIn the realm of real-time customer support, response speed is crucial for user experience. DraftChain significantly reduces latency by decreasing the number of tokens involved in the inference process, enabling...AIMorefastProvide solutions locally.
- Education and learning supportIn the field of education, draft chains can be used to provide...fastFor example, in mathematics education, CoD (Coding in Disputes) can quickly demonstrate the key steps to solve a problem, rather than a detailed reasoning process, helping students understand the core of the problem more quickly.
- conversationalAIsystemDialogueAISystems, such as chatbots, needfastAnd it accurately understands user intent and responds accordingly. The draft chain generates concise intermediate reasoning steps, enabling...AIThe system can process user input and generate responses much faster.
- large-scaleAIdeploy: When large-scale deployment is requiredAIIn scenarios where models are used, such as cloud computing services, cost control is a crucial consideration. DraftChain significantly reduces the cost of inference tasks by minimizing the amount of tokens used.
- Resource-constrained environmentIn resource-constrained environments, such as mobile devices or edge computing devices, computing power and storage space can be very limited. Draft chains enable...AIThe model can run on these devices with lower resource consumption.
- Summarize and extract key informationWhen needing to extract from a large amount of textfastIn scenarios where key information is extracted, such as news summaries or research reports...fastReading and drafting chains effectively extract key points. By limiting the number of words in each reasoning step, CoD helps users...fastGrasp the main points of the text; there's no need to delve into every detail.
- High-frequency trading in financeIn the financial sector, especially in high-frequency trading, decision speed is crucial for success. DraftChain reduces inference latency, enabling...AIThe system can analyze market data and make trading decisions more quickly.
- automaticDriving Decisions:automaticDriving systems need to make complex decisions in a very short time. Draft chains provide...fastFurthermore, accurate reasoning ability helpsautomaticThe driving system reacts quickly when faced with complex traffic situations.
Challenges facing draft chains
- Performance degradation under zero-sample settingThe draft chain exhibits a significant performance degradation in zero-shot settings where few examples are not provided. This is likely because the training data for large language models lacks CoD-style inference patterns, making it difficult to generate concise and insightful "drafts" without sample guidance.
- Performance gap on small modelsWhen testing CoD on small language models with fewer than 3B parameters, although CoD can reduce the number of tokens required per response and improve accuracy, the performance gap is more significant compared to CoT.
- Limitations in complex tasksDraft chains may not be suitable for complex tasks that require extensive reflection, self-correction, or external knowledge retrieval. In these cases, the simplicity of CoD may limit the inference depth and accuracy of the model.
- Balancing Reasoning Depth and ConcisenessWhile CoD reduces latency and computational costs by minimizing redundancy and focusing on key insights, this simplicity can come at the expense of inference depth. In some cases, detailed intermediate steps are crucial for understanding and validating the inference process. CoD's minimalist approach can make the inference process less transparent, making it difficult to trace and understand the model's thought process.
- Cost-performance trade-offWhile CoD excels at cost reduction, its cost advantage may come at the expense of accuracy in certain high-performance applications. For example, in fields requiring extremely high accuracy, such as financial analysis or medical diagnostics, CoD's performance may be insufficient.
- Model adaptabilityThe CoD (Coding on Demand) requires models to limit the number of words used in each inference step, and this limitation may not be suitable for all types of models. Different models may require different strategies to adapt to the CoD requirement, which may increase the complexity of model training and tuning.
- Challenges of real-time applicationsWhile CoD significantly reduces latency by decreasing token usage, in real-time applications, such as...automaticIn driving or high-frequency trading, even minor delays can affect the quality of decision-making. Therefore, CoD needs to ensure the accuracy and reliability of inference while maintaining low latency.
The Development Prospects of Draft Chain
Chain-of-Draft (CoD) is an emerging technology.AIThe reasoning paradigm has a very broad development prospect. CoD, by mimicking the concise human thought process, breaks down complex reasoning tasks into simple and information-intensive intermediate steps, significantly reducing token usage and reasoning latency. CoD's...High efficiencyCost-effectiveness and efficiency have significant application potential in multiple fields. For enterprises processing 1 million inference queries per month, using CoD can reduce costs from $3,800 to $760. This cost advantage is particularly prominent in latency-sensitive scenarios such as real-time customer support, education, and financial services. CoD Implementation MethodsSimpleSimply adjust the hints from the existing model.SimpleThe switch can be easily made with modifications, making it highly feasible for large-scale deployments. Overall, CoD is...AIReasoning provides a moreHigh efficiencyMore economical solutions are expected in the future.AIIt will be more widely promoted and applied in applications. With continuous technological advancements and optimizations, CoD is expected to overcome current limitations and further improve its performance and applicability.