LLM-optimized code: faster, but more energy-efficient than humans in 42 of 44 configurations

A study measures the energy of 11 open models on EffiBench's 1,000 problems: the "optimized" code consumes up to 886% more, and optimization itself is almost never profitable.

Does having your code optimized by an LLM mean saving energy? A team of researchers (Sadra Emami, Benjamin Weigell, Mohammad Reza Mousavi and their colleagues) tested this preconceived idea by measuring the consumption of 11 open source models, under four decoding configurations, out of the 1,000 problems in the EffiBench performance benchmark.

An unfavourable balance sheet

The inference required for optimization consumes between 0.79 and 8.68 megajoules depending on the model, the equivalent of 250 to 2,800 GPT-4.1 requests. The resulting code spends more energy per problem than the reference human solution in 42 out of 44 configurations, with an additional cost of 4% to 886%. The two exceptions gain only marginally and would only amortize the cost of optimization after one to ten million executions.

Two levers stand out: capping the number of output tokens reduces consumption by up to 69%, and architectures with a mix of experts spend significantly less than dense models of equivalent size, without losing accuracy. For the authors, speed and energy sobriety do not necessarily go hand in hand, and the choice of architecture matters more than the adjustment of the decoding.