Where is all the power going to come from for AI/ML

[Arthur Hanson]

AI/ML consume power like it's an unlimited resource and it isn't. I feel a combination of power saving technologies on the semiconductor end and small modular nuclear power plants like we have in submarines and aircraft carriers will be the best answer to this current and coming challenge. Also, will the cost of power be the next driver of increasing cost for using AI/ML. Any thoughts or comments appreciated. With TSM going full blast on production, this is a critical question for the future of the semi industry as a whole.

 

[AAPL]

Last Monday, Bloomberg's Odd Lots podcast was about this question: (How Electric Utilities Will Handle Booming AI Datacenter Demand). Worth a listen if you're interested in this issue.

 

[Barnsley]

AI/ML consume power like it's an unlimited resource and it isn't. I feel a combination of power saving technologies on the semiconductor end and small modular nuclear power plants like we have in submarines and aircraft carriers will be the best answer to this current and coming challenge. Also, will the cost of power be the next driver of increasing cost for using AI/ML. Any thoughts or comments appreciated. With TSM going full blast on production, this is a critical question for the future of the semi industry as a whole.

In the grand scheme it doesnt appear to have been thought out.
Reality is whatever concerns there are will be ignored in the name of "progress"

 

[milesgehm]

small modular nuclear power plants like we have in submarines and aircraft carriers

Why have these never been deployed commercially? I assume they aren’t economical

 

[milesgehm]

https://www.datacenterdynamics.com/...enter-focused-thermal-energy-company-exowatt/

Seems like hype to me

 

[blueone]

Why have these never been deployed commercially? I assume they aren’t economical

Though SMRs have the backing of people like Bill Gates, as far as I can tell every progressive environmental group is dead-set against nuclear energy of any kind. And BS arguments made against SMRs get me shaking my head every time I read one. For example, this one I've chosen randomly:

Why Small Modular Nuclear Reactors Won’t Help Counter the Climate Crisis

Small modular nuclear reactors, or SMRs, are designed to generate less than 300 megawatts of electricity – several times less than typical reactors, which have a range of 1,000 to 1,600 MW. While the individual standardized modules would be small, plans typically call for several modules to be...

www.ewg.org

The authors foolishly use Boeing's manufacturing quality problems with the 737 and 787 aircraft to argue that common design manufacturing in a factory won't improve field reliability and may propagate design flaws. Don't these esteemed authors know that the 737 and 787 didn't suffer from design problems, they are manufacturing problems? (The 737-MAX MCAS issue was a software issue built to purposeful design specs to avoid pilot retraining from the previous generation 737NG planes, not a hardware design problem.) And they go on to discuss problems that might be encountered in large-scale manufacturing, though neither of the authors have any manufacturing experience, or commercial nuclear experience at all. And their reference list is completely self-referencing? Very unimpressive. If any of the semiconductor experts here saw similar nonsense in chip fabrication articles there would be long threads about it.

Over the past several months I'm becoming convinced that the environmental lobbies will try to block any nuclear development in favor of wind and solar, two technologies which have very significant environmental problems themselves, and require equal amounts of back-up power for when the sun isn't shining and the wind isn't blowing. And they can't be distributed due to geographic and meteorological considerations like nuclear plants can, so you need ultrahigh voltage transmission lines for power distribution to population centers, and there are endless challenges in the US and other western countries to building unsightly and land-hungry transmission lines too.

I think it's going to take an Elon Musk-style do-whatever-it-takes character to push a company to defy the political odds to get a commercial SMR in production. I don't follow the nuclear industry very closely, but I haven't noticed anyone like that appear yet.

 

[Arthur Hanson]

Though SMRs have the backing of people like Bill Gates, as far as I can tell every progressive environmental group is dead-set against nuclear energy of any kind. And BS arguments made against SMRs get me shaking my head every time I read one. For example, this one I've chosen randomly:

Why Small Modular Nuclear Reactors Won’t Help Counter the Climate Crisis

Small modular nuclear reactors, or SMRs, are designed to generate less than 300 megawatts of electricity – several times less than typical reactors, which have a range of 1,000 to 1,600 MW. While the individual standardized modules would be small, plans typically call for several modules to be...

www.ewg.org

The authors foolishly use Boeing's manufacturing quality problems with the 737 and 787 aircraft to argue that common design manufacturing in a factory won't improve field reliability and may propagate design flaws. Don't these esteemed authors know that the 737 and 787 didn't suffer from design problems, they are manufacturing problems? (The 737-MAX MCAS issue was a software issue built to purposeful design specs to avoid pilot retraining from the previous generation 737NG planes, not a hardware design problem.) And they go on to discuss problems that might be encountered in large-scale manufacturing, though neither of the authors have any manufacturing experience, or commercial nuclear experience at all. And their reference list is completely self-referencing? Very unimpressive. If any of the semiconductor experts here saw similar nonsense in chip fabrication articles there would be long threads about it.

Over the past several months I'm becoming convinced that the environmental lobbies will try to block any nuclear development in favor of wind and solar, two technologies which have very significant environmental problems themselves, and require equal amounts of back-up power for when the sun isn't shining and the wind isn't blowing. And they can't be distributed due to geographic and meteorological considerations like nuclear plants can, so you need ultrahigh voltage transmission lines for power distribution to population centers, and there are endless challenges in the US and other western countries to building unsightly and land-hungry transmission lines too.

I think it's going to take an Elon Musk-style do-whatever-it-takes character to push a company to defy the political odds to get a commercial SMR in production. I don't follow the nuclear industry very closely, but I haven't noticed anyone like that appear yet.

More like when they can't use their appliances, heater, air conditioning and most of all their television or charge their cell phone.

 

[Xebec]

AI/ML consume power like it's an unlimited resource and it isn't. I feel a combination of power saving technologies on the semiconductor end and small modular nuclear power plants like we have in submarines and aircraft carriers will be the best answer to this current and coming challenge. Also, will the cost of power be the next driver of increasing cost for using AI/ML. Any thoughts or comments appreciated. With TSM going full blast on production, this is a critical question for the future of the semi industry as a whole.

The gradual winding down of coin mining?

 

[MKWVentures]

What percentage of the worlds energy is used by AI datacenters?
Is overall electricity usage going up a lot in the last 10 years?
What % capacity are large power plants running at today? I believe there is 20% more usage in July and August since most electricity use is heating and cooling
What is the impact of growth or EV cars and Electric appliances and Electric heat (heat pumps) over the next 20 years

I would be interested in what the data says. @Arthur Hanson

 

[blueone]

What percentage of the worlds energy is used by AI datacenters?

This data is for the US only. The rest of the world is someone else's problem.

https://www.datacenterdynamics.com/en/news/us-data-center-power-consumption/#:~:text=Data%20centers%20switch%20on%20to%20AI&text=While%20the%20hyperscalers%20typically%20need,up%20from%2017GW%20in%202022.

Is overall electricity usage going up a lot in the last 10 years?

Use of electricity - U.S. Energy Information Administration (EIA)

How and how much electricity is used by major types of end uses in homes, commercial buildings, and manufacturing facilities.

www.eia.gov

What % capacity are large power plants running at today? I believe there is 20% more usage in July and August since most electricity use is heating and cooling

Electric Power Monthly - U.S. Energy Information Administration (EIA)

www.eia.gov

What is the impact of growth or EV cars and Electric appliances and Electric heat (heat pumps) over the next 20 years

The EV Transition Explained: Can the Grid Cope?

There have been vigorous debates pro and con in the US and elsewhere over whether the electric grids can support EVs at scale. The answer is a nuanced “perhaps.” It depends on several factors, including the speed of grid component modernization, the volume of EV sales, where they occur and when...

spectrum.ieee.org

I would be interested in what the data says. @Arthur Hanson

The data seems to say US grid armageddon seems to be coming in about 2030 or so, so we better get off our butts and get real about what our priorities should be.

 

[blueone]

It just occurred to me to mention that EV long-haul trucking makes the personal EV problem look tractable. And even if the grid could handle it, the extra weight of batteries reduces the amount of freight trucks could haul and stay under the federal weight limit, so we'll need a lot more trucks. I think the EV long haul trucking problems make AI datacenters look easy.

 

[milesgehm]

It just occurred to me to mention that EV long-haul trucking makes the personal EV problem look tractable. And even if the grid could handle it, the extra weight of batteries reduces the amount of freight trucks could haul and stay under the federal weight limit, so we'll need a lot more trucks. I think the EV long haul trucking problems make AI datacenters look easy.

EV semi weight limit is 2000 lbs higher, and the motors/drive train are lighter. I think it works out to needing maybe 20% more trucks. Not such a big deal.

 

[blueone]

EV semi weight limit is 2000 lbs higher, and the motors/drive train are lighter. I think it works out to needing maybe 20% more trucks. Not such a big deal.

A Tesla Semi weighs about 27,000lbs. A typical diesel tractor weighs about 19,000lbs with fuel. If you consider recharging time versus refueling time, and the lower payload capacity we will need a lot more than 20% more trucks. For example, Tesla was/is estimating 30 minutes of recharging on dedicated 1000 volt chargers every 500 miles. Most diesel trucks will go over 1000 miles (many will go 2000 miles) on a fuel fill. I remember reading a logistics article after the Tesla Semi was released that speculated that without a breakthrough in battery technology the only viable solution to the range issue will be to have tractor-switching depots on long-distance routes with a fleet of charged EV tractors. Even then freight costs will rise significantly due to the need for more drivers and labor, and of course a much larger fleet of more expensive tractors. This looks like an under-estimated mess to me.

 

[Xebec]

Keep in mind not all trucking is long haul (more than 300 miles). The Tesla semis are extremely well suited for this use case. Also the owners can only drive for so many miles before they are required to take breaks so it's not like these are operating 24 hours continuously either.

I don't have any math to add at the moment without research but I think it's only 10-20% of trucking that is a hard sell for electrification right now.

Of course in the US we fail at efficient transportation in general. Water and rail are a lot more efficient than anything with tires but cheap gasoline made us take the lazier, poorer long term option.

 

[blueone]

Keep in mind not all trucking is long haul (more than 300 miles).

Yeah, but current fleets use common tractors. What you're suggesting is a more tiered tractor fleet.

The Tesla semis are extremely well suited for this use case. Also the owners can only drive for so many miles before they are required to take breaks so it's not like these are operating 24 hours continuously either.

I don't have any math to add at the moment without research but I think it's only 10-20% of trucking that is a hard sell for electrification right now.

I think you're way off. I'll ask someone I know in the logistics industry for another view.

Of course in the US we fail at efficient transportation in general. Water and rail are a lot more efficient than anything with tires but cheap gasoline made us take the lazier, poorer long term option.

You can't use water to go from Alabama to Arizona, unless you use the Panama Canal. ;-)

Railroads already account for about 40% of long distance domestic freight.

Freight Rail Data Center - AAR

Find traffic data, new Policy & Economics releases, fact sheets, infographics, charts, reports and more about the North American freight rail industry.

www.aar.org

Actually, US railroads are world leaders, if the industry players are to be believed.

U.S. Freight Rail: The Envy of the World

When it comes to shipping by rail, America reigns supreme

www.up.com

 

[tooLongInEDA]

Though SMRs have the backing of people like Bill Gates, as far as I can tell every progressive environmental group is dead-set against nuclear energy of any kind. And BS arguments made against SMRs get me shaking my head every time I read one. For example, this one I've chosen randomly:

Why Small Modular Nuclear Reactors Won’t Help Counter the Climate Crisis

Small modular nuclear reactors, or SMRs, are designed to generate less than 300 megawatts of electricity – several times less than typical reactors, which have a range of 1,000 to 1,600 MW. While the individual standardized modules would be small, plans typically call for several modules to be...

www.ewg.org

The authors foolishly use Boeing's manufacturing quality problems with the 737 and 787 aircraft to argue that common design manufacturing in a factory won't improve field reliability and may propagate design flaws. Don't these esteemed authors know that the 737 and 787 didn't suffer from design problems, they are manufacturing problems? (The 737-MAX MCAS issue was a software issue built to purposeful design specs to avoid pilot retraining from the previous generation 737NG planes, not a hardware design problem.) And they go on to discuss problems that might be encountered in large-scale manufacturing, though neither of the authors have any manufacturing experience, or commercial nuclear experience at all. And their reference list is completely self-referencing? Very unimpressive. If any of the semiconductor experts here saw similar nonsense in chip fabrication articles there would be long threads about it.

Over the past several months I'm becoming convinced that the environmental lobbies will try to block any nuclear development in favor of wind and solar, two technologies which have very significant environmental problems themselves, and require equal amounts of back-up power for when the sun isn't shining and the wind isn't blowing. And they can't be distributed due to geographic and meteorological considerations like nuclear plants can, so you need ultrahigh voltage transmission lines for power distribution to population centers, and there are endless challenges in the US and other western countries to building unsightly and land-hungry transmission lines too.

I think it's going to take an Elon Musk-style do-whatever-it-takes character to push a company to defy the political odds to get a commercial SMR in production. I don't follow the nuclear industry very closely, but I haven't noticed anyone like that appear yet.

It is astonishing just how many people are employed in countries like the US and UK simply to *prevent things being done*. Not simply to check stuff/tick boxes and "ensure safety", but all too frequently with the explicit aim of actually preventing things being done - even when they are improvements (or perhaps, especially because they are improvements or challenge the perceived wisdom).

We are constantly told here in the UK that we have a "productivity crisis". Look no further, I say. Too many people in actively value destroying jobs.

 

[Xebec]

Yeah, but current fleets use common tractors. What you're suggesting is a more tiered tractor fleet.

I think you're way off. I'll ask someone I know in the logistics industry for another view.

You can't use water to go from Alabama to Arizona, unless you use the Panama Canal. ;-)

Railroads already account for about 40% of long distance domestic freight.

Freight Rail Data Center - AAR

Find traffic data, new Policy & Economics releases, fact sheets, infographics, charts, reports and more about the North American freight rail industry.

www.aar.org

Actually, US railroads are world leaders, if the industry players are to be believed.

U.S. Freight Rail: The Envy of the World

When it comes to shipping by rail, America reigns supreme

www.up.com

Hmm I’ll have to re-read up on rail, but I think the US has steadily decreased it’s railway miles over the years, so many “rails to trails” projects in the Northeast..

From a trucking company site:
Department of Transportation workers are likely to question any mileage that is greater than 500 miles per day, as it is difficult to get these miles in while staying in compliance with the time requirements. The four rules that will help truck drivers determine how they can lawfully conduct business in more than 97% of the cases include:

1. 11-hour rule. Simply stated, a driver must take at least a 10-hour sleeper break or off-duty rest period after driving for 11 consecutive hours.

…

This implies if truckers are doing over 500 miles a day, it’s not much more. Forums seem to indicate 650 as a practical upper limit. If you start your day with a fully charged Tesla Semi, you only need 20-30 minutes of charging to add enough miles to hit that 650. It’s not ideal, but the running costs of the truck are a LOT lower, and if you read anecdotes, truckers certainly love the newfound torque. (Also there’s less downtime with an electric vehicle than a gas or diesel one due to less maintenance).

Related - the current Tesla Semi uses about 1.7 kWh per mile. Musk/Tesla have stated they have an easy path to 1.6, and they think they can get that to 1.5 kWh per mile. That implies another 6-12% range FWIW.

I do agree a transition is going to require a ‘tiered fleet’ approach on the semi side. I think that’s true though with any major transition. Horse and Buggy to Gas cars, Rail to modern trucks, etc.. We’re tiered in the aspect that we have multiple methods of transport (and packaging) already.

LOL - agreed water can’t solve everything.. but we are under utilizing water for transport in the US, thanks to the Jones Act.. There are a lot of former canals that are just no longer used too..

 

[blueone]

It is astonishing just how many people are employed in countries like the US and UK simply to *prevent things being done*. Not simply to check stuff/tick boxes and "ensure safety", but all too frequently with the explicit aim of actually preventing things being done - even when they are improvements (or perhaps, especially because they are improvements or challenge the perceived wisdom).

We are constantly told here in the UK that we have a "productivity crisis". Look no further, I say. Too many people in actively value destroying jobs.

Well stated.

 

[blueone]

Hmm I’ll have to re-read up on rail, but I think the US has steadily decreased it’s railway miles over the years, so many “rails to trails” projects in the Northeast..

Not consequentially, especially on main lines. US rail traffic looks +/- a few percent for the past ten years based on carloads:

Freight Rail Traffic Data - AAR

Rail traffic is a useful gauge of broader economic activity, both for specific industries and for the economy as a whole.

www.aar.org

Department of Transportation workers are likely to question any mileage that is greater than 500 miles per day, as it is difficult to get these miles in while staying in compliance with the time requirements. The four rules that will help truck drivers determine how they can lawfully conduct business in more than 97% of the cases include:

1. 11-hour rule. Simply stated, a driver must take at least a 10-hour sleeper break or off-duty rest period after driving for 11 consecutive hours.

…

This implies if truckers are doing over 500 miles a day, it’s not much more. Forums seem to indicate 650 as a practical upper limit. If you start your day with a fully charged Tesla Semi, you only need 20-30 minutes of charging to add enough miles to hit that 650. It’s not ideal, but the running costs of the truck are a LOT lower, and if you read anecdotes, truckers certainly love the newfound torque. (Also there’s less downtime with an electric vehicle than a gas or diesel one due to less maintenance).

The miles per day limit is not primary in the problems with electric trucking, though cold weather range isn't discussed much. Charging time just increases costs. Building the charging infrastructure is the biggest problem, although tractor supply is too. Charging stations outside of major metropolitan areas will take a lot of investment and time.

Megawatt truck charging arrives - albeit slowly - FreightWaves

Megawatt charging is too much for pretty much every electric truck save for the purpose-built Tesla Semi. But that is slowly changing.

www.freightwaves.com

Related - the current Tesla Semi uses about 1.7 kWh per mile. Musk/Tesla have stated they have an easy path to 1.6, and they think they can get that to 1.5 kWh per mile. That implies another 6-12% range FWIW.

I do agree a transition is going to require a ‘tiered fleet’ approach on the semi side. I think that’s true though with any major transition. Horse and Buggy to Gas cars, Rail to modern trucks, etc.. We’re tiered in the aspect that we have multiple methods of transport (and packaging) already.

LOL - agreed water can’t solve everything.. but we are under utilizing water for transport in the US, thanks to the Jones Act.. There are a lot of former canals that are just no longer used too..

That's quite an agenda you're packing based on emotion, not factual analysis. Personally, I think the data shows long distance electric trucking is a complete waste of money for a decade or more, probably a lot more. In metro areas, the biggest argument for electric trucks is pollution reduction, which is very significant and easily quantifiable, and metro trucking is a far easier problem to solve than the entire freight ecosystem in countries as large as the continental US. In rural areas the benefits are negative.

The impact on climate change is minimal too. In 2021, the entire transportation sector in the US was 29% of US GHG emissions, and medium & heavy duty trucks were 23% of the 29%, so 6.7% of US GHG emissions for the entire trucking sector, which is a lot bigger than just long distance trucking. The US is responsible for about 11% of global GHG emissions, so 6.7% of 11% = 0.737% of global GHG emissions. So the entire US long distance trucking electrification problem looks to me like a huge waste of money for no substantial benefit.

Fast Facts on Transportation Greenhouse Gas Emissions | US EPA

Basic facts and data on greenhouse gas emissions from transportation.

www.epa.gov

 

[count]

For electric trucks, I think eventually you'll have greater movement towards them as batteries improve and prices continue to decline. It's not quite Moore's law but the cost/performance of batters has been steadily improving year after year, I think at around 7% a year, with some ups and downs based on the fluctuations in metals pricing.

Back to the topic of the OP, I don't see a huge issue. Big industrial power consumers sign power purchase agreements with nearby generators, and in many cases the power purchase agreements lead to significant amounts of new generation being built next to the load. In many cases, the really big power consumers, like large factories or mines, will generate their own power on site.