Alibaba Group Holding (NYSE: BABA) has introduced a new computing pooling system called Aegaeon, which dramatically reduces the reliance on Nvidia (NASDAQ: NVDA) GPUs by 82% for AI models.

**Three Months In Testing**

This innovation was tested in Alibaba Cloud’s model marketplace for over three months, according to a research paper presented this week at the 31st Symposium on Operating Systems Principles (SOSP) in Seoul, South Korea. The Aegaeon system has successfully decreased the number of Nvidia H20 GPUs required from 1,192 to just 213 for serving models with up to 72 billion parameters.

“Aegaeon is the first work to reveal the excessive costs associated with serving concurrent LLM workloads on the market,” the researchers stated in the paper. Researchers from Peking University and Alibaba Cloud emphasized the high costs involved in serving concurrent large language model workloads.

**One GPU For Multiple Models**

Alibaba Cloud, the AI and cloud services division of the Hangzhou-based Alibaba, aims to boost efficiency by pooling GPU resources, allowing a single GPU to support multiple models. This system addresses resource inefficiency, as previously, 17.7% of GPUs were allocated to serve only 1.35% of requests in Alibaba Cloud’s marketplace.

Cloud service providers like Alibaba Cloud and ByteDance’s Volcano Engine manage thousands of AI models simultaneously, often leading to inefficiencies. The Aegaeon system seeks to optimize this process by significantly reducing the number of GPUs needed.

**Growing Concerns Over Nvidia’s China Presence**

This development comes amid growing concerns over Nvidia’s presence in China. Recently, China raised security concerns about Nvidia’s H20 chips, particularly regarding potential backdoor risks.

As part of its deal with Nvidia, the Trump administration struck an agreement for a 15% revenue share from the company’s chip sales to China. Nvidia CEO Jensen Huang revealed that Nvidia’s market share in China has dropped from 95% to zero. He expressed concerns about the impact of U.S. policies on Nvidia’s market presence in China.

Despite these challenges, Nvidia has financially insulated itself from potential escalations. Huang stated that the company’s guidance assumes zero revenue from China.

—

*Disclaimer: This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors.*

*Photo courtesy: Shutterstock.*
https://www.benzinga.com/markets/tech/25/10/48293177/alibaba-clouds-new-system-cuts-nvidia-gpu-usage-by-82-amid-trumps-flip-flop-on-ai-chip-ban-on-china