Extraordinary article on the energy usage of generative AI from BloombergNEF founder Michael Liebreich - absolutely worth spending some time with this: about.bnef.com/blog/liebreich-…
I wrote up some of my own notes on the article here: simonwillison.net/2025/Jan/12/…
Liebreich: Generative AI – The Power and the Glory | BloombergNEF
This year will go down in history as the year the energy sector woke up to AI. This is also the year AI woke up to energy. Is the data center power frenzy just the latest of a long line of energy sector bubbles, or is it the dawning of a new normal?BloombergNEF
Matt Campbell
in reply to Simon Willison • • •> In 2007 the EPA predicted data center energy usage would double: it didn't, thanks to efficiency gains from better servers and the shift from in-house to cloud hosting.
That latter shift can also be seen as a shift to greater centralization, which isn't necessarily a good thing. I hope centralization, a concentration of hosting onto big servers at a few providers, isn't necessary to keep energy consumption at a reasonable level.
Simon Willison
in reply to Matt Campbell • • •Eleanor Saitta
in reply to Matt Campbell • • •@matt
It's a few things. Large cloud providers have done a lot of work to reduce inefficiencies in their ecosystems, including moving to large-scale liquid cooling on some cases, whole rack-aisle airflow calculations, custom chassis designed for power efficiency, etc., meaning that in many cases, less than ten percent of the total DC power is going to things other than compute, storage, and networking. They also have enough load to balance that they can keep most of their machines running at right around the 80% load mark, because systems are more efficient when they're running a heavy workload.
Very few companies with small on-prem DC footprints do anything like this much work, and the systems that are sold to them are generic, because they need to work anywhere, so that kind of extreme optimization just isn't available to them. Loads for individual companies are also often bursty, given peaks in customer demand, monthly batch jobs, etc., and the rest of the time that spare capacity sits idle. Cold-booting hardware carries some risk of systems not coming back online, so servers sit mostly idle, still consuming most of their peak power.
So basically yes, on-prem is hard to make as efficient as cloud systems, for most realistic companies. However, one of the things you can do is get rid of useless compute. Taking the time to work with profiling tools and taking some care at the business level in how and when jobs are run can have a large impact on performance. Getting rid of needlessly complex systems also helps — for instance, the entire targeted advertising ecosystem, if turned off and all related code removed from systems, would likely cover efficiency losses from returning to on-prem systems many times over.
@simon
Matt Campbell
in reply to Eleanor Saitta • • •Sensitive content
Matt Campbell
in reply to Matt Campbell • • •Eleanor Saitta
in reply to Matt Campbell • • •Yes, basically, especially with full lifecycle analysis. We can federalize services politically and take advantage of industrial efficiencies of scale without trying to be digital homesteaders, and most of our options for efficiency come from doing less dumb useless stuff, like data collection, behavioral analysis, personalized advertising, and the entire suite of useless LLM tools.
@simon
Eleanor Saitta
in reply to Matt Campbell • • •So, a) if we're talking about large models, they cannot be trained in that context and many of them won't even be able to do inference there. b) For a phone, the embodied carbon (ignoring all other resource impact) of manufacturing is larger than its lifetime energy carbon footprint. c) A machine like that serving a single user is almost always going to be idle, and even if it has significant idle power reduction, it's still going to do much lower useful work per watt than a more computationally capable per watt large system that's running at a steady 75-95% load.
@simon