Less is more: How ‘chain of draft’ could cut AI costs by 90% while improving performance

A team of researchers to Zoom has developed a revolutionary technique that could considerably reduce the cost and the calculation resources necessary for AI systems to solve complex reasoning problems, potentially transform the way companies deploy large scale AI.

The method, called draft chain (COD), allows major language models (LLM) to solve problems with a minimum of words – using as little as 7.6% of the text required by current methods while maintaining or even improving precision. The results were published in an article last week on the Arxiv research frame of reference.

“By reducing verbity and focusing on critical information, the COD corresponds or exceeds the COT (chain of thoughts) in precision while using as little as 7.6% of the tokens, considerably reducing the cost and the latency through various reasoning tasks”, write the authors, led by Silei Xu, a zoom researcher.

How “less is more” transforms the reasoning of the AI ​​without sacrificing precision

Cod is inspired by the way humans solve complex problems. Rather than articulating every detail when working on a mathematical problem or a logical puzzle, people generally note only essential information in abbreviated form.

“When resolving complex tasks – whether mathematical problems, writing tests or coding – we often only notify the critical information that helps us to progress,” explain the researchers. “By emulating this behavior, LLM can focus on progression to solutions without the general costs of verbose reasoning.”

The team has tested its approach on many benchmarks, including arithmetic reasoning (GSM8K), common sense reasoning (understanding of dates and understanding of sports) and symbolic reasoning (tasks of reversing parts).

In a striking example in which Claude 3.5 SONNET Transformed questions related to sport, the COD approach reduced the average production of 189.4 tokens to only 14.3 tokens – a reduction of 92.4% – while improving the accuracy of 93.2% to 97.3%.

IA costs of the reduced company: the profitability analysis for the concise reasoning of the machine

“For business treatment 1 million requests for mensually, the DCO could reduce costs by $ 3,800 (COT) to $ 760, which saves more than $ 3,000 per month”, researcher IA Ajith Vallath Prabhakar written in an analysis of the document.

Research comes at a critical time for the deployment of corporate AI. While companies are increasingly incorporating sophisticated AI systems into their operations, calculation costs and response times have become significant obstacles to general adoption.

Current cutting -edge reasoning techniques as (Bed bed)), which was introduced in 2022, considerably improved the capacity of the AI ​​to solve complex problems by breaking them into reasoning step by step. But this approach generates long explanations that consume substantial calculation resources and increase the response latency.

“The verbal nature of the COT causes substantial calculation fees, increased latency and higher operational expenses,” writes Prabhakar.

What makes The cod is particularly remarkable For companies is its simplicity of implementation. Unlike many IA advances which require recycling of expensive models or architectural modifications, the DCO can be deployed immediately with existing models thanks to a simple modification of the prompt.

“Organizations that already use COT can move to cod with a simple change in the prompt,” explains Prabhakar.

The technique could prove to be particularly precious for applications sensitive to latency such as real -time customer support, mobile AI, educational tools and financial services, where even small delays can have a significant impact on the user experience.

Industry experts suggest, however, that the implications extend beyond cost savings. By making an advanced and affordable advanced AI reasoning, the DCO could democratize access to sophisticated AI capabilities for small organizations and resources related to resources.

While AI systems continue to evolve, techniques like the DCO highlight an increasing accent on efficiency alongside raw capabilities. For companies navigating in the landscape of rapidly evolving AI, such optimizations could prove as precious as improvements in the underlying models themselves.

“While the models of AI continue to evolve, the optimization of the effectiveness of reasoning will be as critical as to improve their raw capacities,” concluded Prabhakar.

The research code and the data were created accessible to the public On Github, allowing organizations to implement and test the approach with their own AI systems.