Curated by THEOUTPOST
On Fri, 8 Nov, 8:03 AM UTC
3 Sources
[1]
Up to 30% of the power used to train AI is wasted: Here's how to fix it
A less wasteful way to train large language models, such as the GPT series, finishes in the same amount of time for up to 30% less energy, according to a new study from the University of Michigan. The approach could save enough energy to power 1.1 million U.S. homes in 2026, based on Wells Fargo's projections of AI power demand. It could also take a bite out of the International Monetary Fund's prediction that data centers could account for 1.2% of the world's carbon emissions by 2027 -- and the water demands that come with that energy use. Some experts say that these costs could be outweighed by environmental benefits. They argue that AI could be a "game changer" for fighting climate change by identifying ways to optimize supply chains and the grid, manage our energy needs, and improve research on climate change. Still, that doesn't excuse squandering energy, and some of the power used to train AI has zero impact on training time and model accuracy. "Why spend something when there's no point?" said Mosharaf Chowdhury, U-M associate professor of computer science and engineering and the corresponding author of the study presented at the 30th Symposium on Operating Systems Principles. "We can't keep building bigger and bigger data centers because we won't have the power to run them. If we can reduce the energy consumed by AI, we can reduce AI's carbon footprint and cooling requirements and allow for more computation to fit within our current energy constraints." The energy waste is created when AI training is unequally divided between GPUs, which are computer processors specialized for large data and graphics applications. Although it opens the door for waste, splitting the work is necessary for processing huge datasets. "AI models today are so large, they cannot fit inside a single computer processor," said Jae-Won Chung, U-M doctoral student in computer science and engineering and the first author of the study. "They need to be divided into tens of thousands of processors to be trained, but dividing the models in perfectly equal sizes across all processors is practically impossible." The training jobs are so difficult to evenly split up because some tasks need to be grouped together on the same processor -- like how each installment of a book series will be grouped together in an organized shelf. Depending on how the tasks are grouped, some processors might get stuck with the AI-training equivalent of the Encyclopedia Britannica while others get assigned a fantasy trilogy. Because current training methods run each processor at top speed, processors with a lighter load will finish their calculations before other processors. This doesn't speed up training, which isn't complete until every processor finishes its job -- but it is wasteful because faster calculations require more energy. In addition, problems such as faulty hardware or network delays create energy waste by slowing down a single processor's computing speed. To save energy, the researchers developed a software tool, called Perseus, that identifies a critical path, or a series of subtasks that will take the longest time to complete. Then, Perseus slows down processors that aren't on the critical path so that they all finish their jobs around the same time -- eliminating unnecessary power use. "Reducing the power cost of AI can have important implications for equitable AI access," Chowdhury said. "If a country doesn't have enough power to run a big model, they might need to use services from far away, or be stuck running smaller, less accurate models. This gap could further perpetuate disparity between different communities." The team tested Perseus by training GPT-3, three other large language models and one computer vision model. Perseus is an open-sourced tool available as part of Zeus, a tool for measuring and optimizing AI energy consumption. The research was funded by the National Science Foundation, Dutch Research Council (NWO) Talent Programme, VMware, Mozilla Foundation, Salesforce and Kwanjeong Educational Foundation. Chameleon Cloud and CloudLab supported the research by providing computational resources.
[2]
Up to 30% of the power used to train AI is wasted: | Newswise
Smarter use of processor speeds saves energy without compromising training speed and performance A less wasteful way to train large language models, such as the GPT series, finishes in the same amount of time for up to 30% less energy, according to a new study from the University of Michigan. The approach could save enough energy to power 1.1 million U.S. homes in 2026, based on Wells Fargo's projections of AI power demand. It could also take a bite out of the International Monetary Fund's prediction that data centers could account for 1.2% of the world's carbon emissions by 2027 -- and the water demands that come with that energy use. Some experts say that these costs could be outweighed by environmental benefits. They argue that AI could be a "game changer" for fighting climate change by identifying ways to optimize supply chains and the grid, manage our energy needs, and improve research on climate change. Still, that doesn't excuse squandering energy, and some of the power used to train AI has zero impact on training time and model accuracy. "Why spend something when there's no point?" said Mosharaf Chowdhury, U-M associate professor of computer science and engineering and the corresponding author of the study presented at the 30th Symposium on Operating Systems Principles. "We can't keep building bigger and bigger data centers because we won't have the power to run them. If we can reduce the energy consumed by AI, we can reduce AI's carbon footprint and cooling requirements and allow for more computation to fit within our current energy constraints." The energy waste is created when AI training is unequally divided between GPUs, which are computer processors specialized for large data and graphics applications. Although it opens the door for waste, splitting the work is necessary for processing huge datasets. "AI models today are so large, they cannot fit inside a single computer processor," said Jae-Won Chung, U-M doctoral student in computer science and engineering and the first author of the study. "They need to be divided into tens of thousands of processors to be trained, but dividing the models in perfectly equal sizes across all processors is practically impossible." The training jobs are so difficult to evenly split up because some tasks need to be grouped together on the same processor -- like how each installment of a book series will be grouped together in an organized shelf. Depending on how the tasks are grouped, some processors might get stuck with the AI-training equivalent of the Encyclopedia Britannica while others get assigned a fantasy trilogy. Because current training methods run each processor at top speed, processors with a lighter load will finish their calculations before other processors. This doesn't speed up training, which isn't complete until every processor finishes its job -- but it is wasteful because faster calculations require more energy. In addition, problems such as faulty hardware or network delays create energy waste by slowing down a single processor's computing speed. To save energy, the researchers developed a software tool, called Perseus, that identifies a critical path, or a series of subtasks that will take the longest time to complete. Then, Perseus slows down processors that aren't on the critical path so that they all finish their jobs around the same time -- eliminating unnecessary power use. "Reducing the power cost of AI can have important implications for equitable AI access," Chowdhury said. "If a country doesn't have enough power to run a big model, they might need to use services from far away, or be stuck running smaller, less accurate models. This gap could further perpetuate disparity between different communities." The team tested Perseus by training GPT-3, three other large language models and one computer vision model. Perseus is an open-sourced tool available as part of Zeus, a tool for measuring and optimizing AI energy consumption. The research was funded by the National Science Foundation, Dutch Research Council (NWO) Talent Programme, VMware, Mozilla Foundation, Salesforce and Kwanjeong Educational Foundation. Chameleon Cloud and CloudLab supported the research by providing computational resources.
[3]
Up to 30% of the power used to train AI is wasted: A software tool could help fix that
A less wasteful way to train large language models, such as the GPT series, finishes in the same amount of time for up to 30% less energy, according to a new study from the University of Michigan. The approach could save enough energy to power 1.1 million U.S. homes in 2026, based on Wells Fargo's projections of AI power demand. It could also take a bite out of the International Monetary Fund's prediction that data centers could account for 1.2% of the world's carbon emissions by 2027 -- and the water demands that come with that energy use. Some experts say that these costs could be outweighed by environmental benefits. They argue that AI could be a "game changer" for fighting climate change by identifying ways to optimize supply chains and the grid, manage our energy needs, and improve research on climate change. Still, that doesn't excuse squandering energy, and some of the power used to train AI has zero impact on training time and model accuracy. "Why spend something when there's no point?" said Mosharaf Chowdhury, an associate professor of computer science and engineering and the corresponding author of the study presented at the 30th Symposium on Operating Systems Principles last Monday. "We can't keep building bigger and bigger data centers because we won't have the power to run them," said Chowdhury. "If we can reduce the energy consumed by AI, we can reduce AI's carbon footprint and cooling requirements and allow for more computation to fit within our current energy constraints." The energy waste is created when AI training is unequally divided between GPUs, which are computer processors specialized for large data and graphics applications. Although it opens the door for waste, splitting the work is necessary for processing huge datasets. "AI models today are so large, they cannot fit inside a single computer processor," said Jae-Won Chung, a doctoral student in computer science and engineering and the first author of the study. "They need to be divided into tens of thousands of processors to be trained, but dividing the models into perfectly equal sizes across all processors is practically impossible." The training jobs are so difficult to evenly split up because some tasks need to be grouped together on the same processor -- like how each installment of a book series will be grouped together in an organized shelf. Depending on how the tasks are grouped, some processors might get stuck with the AI-training equivalent of the Encyclopedia Britannica while others get assigned a fantasy trilogy. Because current training methods run each processor at top speed, processors with a lighter load will finish their calculations before other processors. This doesn't speed up training, which isn't complete until every processor finishes its job -- but it is wasteful because faster calculations require more energy. In addition, problems such as faulty hardware or network delays create energy waste by slowing down a single processor's computing speed. To save energy, the researchers developed a software tool called Perseus that identifies a critical path, or a series of subtasks that will take the longest time to complete. Then, Perseus slows down processors that aren't on the critical path so that they all finish their jobs around the same time -- eliminating unnecessary power use. "Reducing the power cost of AI can have important implications for equitable AI access," said Chowdhury. "If a country doesn't have enough power to run a big model, they might need to use services from far away, or be stuck running smaller, less accurate models. This gap could further perpetuate disparity between different communities." The team tested Perseus by training GPT-3, three other large language models and one computer vision model. Perseus is an open-sourced tool available as part of Zeus, a tool for measuring and optimizing AI energy consumption.
Share
Share
Copy Link
Researchers at the University of Michigan have developed Perseus, a software tool that can reduce energy consumption in AI training by up to 30% without compromising speed or performance, potentially saving enough energy to power 1.1 million U.S. homes by 2026.
A new study from the University of Michigan has revealed that up to 30% of the power used to train large AI models, such as GPT-3, is wasted. This inefficiency stems from the unequal distribution of workload across multiple GPUs (Graphics Processing Units) during the training process 1.
To address this issue, researchers have developed a software tool called Perseus. This innovative solution identifies the critical path in AI training tasks and adjusts processor speeds accordingly, ensuring all processors finish their jobs simultaneously. By doing so, Perseus can reduce energy consumption by up to 30% without compromising training speed or model accuracy 2.
The energy savings achieved by Perseus could be substantial. Based on Wells Fargo's projections of AI power demand, the approach could save enough energy to power 1.1 million U.S. homes in 2026. This reduction in energy consumption could also help mitigate the environmental impact of data centers, which the International Monetary Fund predicts could account for 1.2% of global carbon emissions by 2027 3.
Mosharaf Chowdhury, associate professor of computer science and engineering at the University of Michigan, emphasizes the importance of this development: "We can't keep building bigger and bigger data centers because we won't have the power to run them. If we can reduce the energy consumed by AI, we can reduce AI's carbon footprint and cooling requirements and allow for more computation to fit within our current energy constraints" 1.
Perseus tackles the inefficiency created when AI training tasks are unevenly distributed across multiple processors. Current methods run all processors at top speed, resulting in some finishing their calculations before others. Perseus identifies the longest series of subtasks (the critical path) and slows down processors not on this path, ensuring all processors complete their work simultaneously and eliminating unnecessary power use 2.
The researchers argue that reducing AI power costs could have significant implications for equitable AI access. Chowdhury notes, "If a country doesn't have enough power to run a big model, they might need to use services from far away, or be stuck running smaller, less accurate models. This gap could further perpetuate disparity between different communities" 3.
The team has tested Perseus by training GPT-3, three other large language models, and one computer vision model. Perseus is now available as an open-source tool, part of Zeus, which measures and optimizes AI energy consumption 1.
Chinese startup DeepSeek claims to have created an AI model that matches the performance of established rivals at a fraction of the cost and carbon footprint. However, experts warn that increased efficiency might lead to higher overall energy consumption due to the Jevons paradox.
5 Sources
5 Sources
Researchers develop innovative methods to significantly reduce AI's energy consumption, potentially revolutionizing the industry's environmental impact and operational costs.
2 Sources
2 Sources
As AI technology advances, concerns grow over its environmental impact. ChatGPT and other AI models are consuming enormous amounts of energy and water, raising questions about sustainability and resource management in the tech industry.
3 Sources
3 Sources
As artificial intelligence continues to advance, concerns grow about its energy consumption and environmental impact. This story explores the challenges and potential solutions in managing AI's carbon footprint.
5 Sources
5 Sources
The rapid growth of AI technology has raised concerns about its environmental sustainability. This story explores the energy consumption of AI models, their carbon footprint, and potential solutions for a greener AI industry.
2 Sources
2 Sources
The Outpost is a comprehensive collection of curated artificial intelligence software tools that cater to the needs of small business owners, bloggers, artists, musicians, entrepreneurs, marketers, writers, and researchers.
© 2025 TheOutpost.AI All rights reserved