China makes AI breakthrough, reportedly trains generative AI model across multiple data centers and GPU architectures

[Beijingwalker]

China makes AI breakthrough, reportedly trains generative AI model across multiple data centers and GPU architectures​

News
By Jowi Morales
September 29 2024

(Image credit: Shutterstock)

An industry analyst recently revealed that China has developed a single generative AI (GAI) model across multiple data centers — a massive feat considering the complexity of using different GPUs in a single data center, let alone using servers in multiple geographic locations. Patrick Moorhead, Chief Analyst at Moor Insights & Strategy, said on X (formerly Twitter) that China was the first country to manage this achievement and that he discovered it during a conversation about a presumably unrelated NDA meeting.

This technique of training GAIs across different locations/architectures is essential for China to keep its AI dreams moving forward, especially as American sanctions have stopped it from acquiring the latest, most powerful chips to drive its research and development. Since Nvidia does not want to lose the Chinese market, it created the less powerful H20 AI chips that fall within Washington’s restrictive performance parameters. However, there are rumors that even these down-tuned chips might be banned soon, highlighting the uncertainty Chinese tech companies face in the current political climate.

Because of this uncertainty, Chinese researchers have been working on melding GPUs from different brands into one training cluster. By doing so, the institutions could combine their limited stocks of sanctioned high-end, high-performance chips, like the Nvidia A100, with less powerful but readily available GPUs, like Huawei’s Ascend 910B or the aforementioned Nvidia H20. This technique could help them combat the high-end GPU shortage within China, although it has historically come with large drops in efficiency.

However, it seems that China has found ways to solve this issue, especially with the news of the single GAI development across multiple data centers. Although we don’t have any information on this GAI yet, it shows the lengths that Chinese researchers will go to, to ensure that they can continue driving China’s AI ambitions forward. As Huawei said, China would find ways to continue moving its AI development despite American sanctions. After all, necessity is the mother of invention.