The market for artificial intelligence (AI) hardware is expected to grow rapidly in the coming years, as more applications and industries adopt AI solutions for various tasks. According to a report by McKinsey, AI could create up to $4.4 trillion of value for the global economy by 20301. However, the AI hardware market is also highly competitive, with Nvidia being the clear leader in the field of graphics processing units (GPUs), which are widely used for AI training and inference.
Nvidia’s GPUs are popular among cloud service providers, data centers, and researchers, who use them to run complex machine learning (ML) algorithms and deep neural networks (DNNs). Nvidia’s CUDA programming model and software ecosystem also provide a rich set of tools and libraries for developers to optimize their AI workloads. Nvidia’s revenue from its data center segment, which includes its AI products, reached $10 billion in Q2 FY20242, more than double that of Intel’s data center revenue of $4 billion.
However, Nvidia’s dominance in the AI hardware market does not deter Chinese GPU companies from pursuing their own opportunities in the field. According to a report by DigiTimes, Chinese GPU developers see a golden opportunity to fill the void in their domestic market, especially after the U.S. restrictions on high-end AI and high-performance computing (HPC) exports to China2. These restrictions have limited the availability of Nvidia’s latest generation of Instinct GPUs, the MI200 series, which are based on the Hopper architecture and offer significant performance improvements over the previous generation.
Chinese firms such as ILuvatar CoreX, Moore Threads, and BirenTech are actively collaborating with local cloud computing providers like Bidu to run their large language models (LLMs) services2. LLMs are advanced AI models that can generate natural language texts and understand human queries. Other companies like DenglinAI, Vast AI Tech, and MetaX have developed products that run LLMs from the start. For instance, MetaX’s M6 chip is designed specifically for LLMs and claims to offer 10 times higher performance per watt than Nvidia’s A100 GPU3.
Some Chinese enterprises are also exploring alternative routes to compete in the AI-accelerated computing sector. For instance, Alibaba subsidiary T-Head and Enflame are investigating non-GPU solutions as a way to gain a foothold in the market2. T-Head has developed its own RISC-V-based CPU core called Xuantie, which can be customized for different AI scenarios. Enflame has developed its own DNN processor called CloudBlazer, which can handle both training and inference workloads.
The Chinese GPU companies are optimistic about their prospects in the AI hardware market, as they believe Nvidia’s success is essentially laying the groundwork for them as the market shifts from CPUs to compute GPUs2. They also hope to leverage their local advantages, such as lower costs, faster time-to-market, and better understanding of customer needs. However, they also face many challenges, such as competing with other CPU, GPU, and accelerator vendors in the global arena; developing their own software ecosystems and standards; and ensuring the quality and reliability of their products.
The GPU for AI market is expected to witness significant growth and innovation in the near future, as more applications and industries adopt AI solutions for various tasks. The market will also be influenced by the trends and roadmaps of GPU technology across different segments, such as cloud inference, edge inference, and training. According to a report by Grand View Research, the global GPU market size is expected to reach $172.08 billion by 2032, registering a CAGR of 32.70% from 2023 to 20324.
Cloud inference refers to the process of running trained AI models on cloud servers to provide answers or actions based on user queries or data inputs. Cloud inference is widely used for applications such as natural language processing (NLP), computer vision (CV), speech recognition, and recommendation systems. Cloud inference requires high-performance GPUs that can handle large amounts of data and complex computations in parallel. Nvidia’s GPUs are currently the dominant choice for cloud inference, but they face competition from other vendors such as AMD, Intel, Google (with its Tensor Processing Units or TPUs), Amazon (with its Inferentia chips), and Huawei (with its Ascend chips).
Edge inference refers to the process of running trained AI models on edge devices, such as smartphones, tablets, laptops, cameras, robots, cars, drones, etc., to provide answers or actions based on user queries or data inputs. Edge inference is used for applications that require low latency, high privacy, or limited connectivity. Edge inference requires low-power GPUs that can balance performance and efficiency while operating within thermal and battery constraints. Nvidia’s GPUs are also widely used for edge inference, especially for applications that require high graphics and computing capabilities, such as gaming, AR/VR, and autonomous driving. However, Nvidia also faces competition from other vendors such as Qualcomm, Apple, Samsung, MediaTek, and Huawei, who integrate GPUs into their SoCs for mobile and embedded devices.
Training refers to the process of creating and optimizing AI models using large amounts of data and ML algorithms. Training requires high-performance GPUs that can handle massive amounts of data and complex computations in parallel. Training is usually done on cloud servers or data centers, but it can also be done on edge devices for some applications. Nvidia’s GPUs are the leading choice for training, as they offer high performance, scalability, and compatibility with popular ML frameworks and libraries. However, Nvidia also faces competition from other vendors such as AMD, Intel, Google (with its TPUs), Graphcore (with its Intelligence Processing Units or IPUs), and Cerebras (with its Wafer Scale Engine or WSE).
The GPU for AI market is expected to witness significant growth and innovation in the near future, as more applications and industries adopt AI solutions for various tasks. The market will also be influenced by the trends and roadmaps of GPU technology across different segments, such as cloud inference, edge inference, and training. The Chinese GPU companies see a golden opportunity to fill the void in their domestic market, especially after the U.S. restrictions on high-end AI and HPC exports to China. However, they also face many challenges, such as competing with other CPU, GPU, and accelerator vendors in the global arena; developing their own software ecosystems and standards; and ensuring the quality and reliability of their products.
