Nvidia has turned the AI boom into an infrastructure story. In its latest quarter, it reported $81.6 billion in revenue, with $75.2 billion coming from data centers. CEO Jensen Huang described the buildout of AI factories as “the largest infrastructure expansion in human history.”
Most AI infrastructure coverage still treats compute as something that lives almost exclusively inside hyperscale data centers. Yet a large share of the hardware base that could support AI’s broader deployment already exists. Roughly 3 billion PC-based GPUs are already sitting inside consumer and prosumer machines in bedrooms, gaming rigs, creator workstations and home offices around the world, and only a small share of that installed capacity is being used for AI work.
The next phase of AI infrastructure may depend on turning existing compute into something usable.
What Is the Next Phase of AI Infrastructure?
The next phase of AI infrastructure must focus on distributed compute and orchestration — activating the world’s estimated 3 billion existing PC GPUs. As data center inference costs soar, shifting smaller workloads (like distilled models and local copilots) to underused consumer gaming rigs and workstations can drastically lower delivery costs and reduce latency by bringing compute closer to the user.
We Want to Know.
Inference Makes AI Expensive
Training still dominates public attention because it is tied to frontier models and massive capital expenditure. The next cost pressure is forming around inference.
McKinsey expects inference to surpass training by 2030 and represent more than half of all AI compute in data centers. Each prompt, recommendation, generated image and agent action adds another compute charge to the cost of delivery.
OpenAI shows how difficult it is to deliver AI models at acceptable latency and sustainable cost. Reuters has reported that the company generated $13 billion in revenue in 2025 while holding expenses to about $8 billion. It has also reported that OpenAI projects roughly $50 billion in computing-power spending in 2026 alone. HSBC estimates suggest the company could still lose about $14 billion that year and need to raise at least $207 billion by 2030 to keep absorbing losses driven by soaring compute costs.
Recurring inference demand should change what “infrastructure” means. Some workloads require the largest clusters available. Many others can run on cheaper, more distributed compute closer to the user.
The Installed Base Everyone Ignores
Today’s inference market is still built around heavy capital expenditure. Providers buy GPUs, build or lease data center capacity, and keep upgrading hardware to stay competitive. In practice, that has concentrated AI serving around enterprise-grade chips such as Nvidia’s A100, H100 and H200.
Frontier systems and the largest workloads still require enterprise-grade chips. Assistants, copilots, media tools and lightweight agents often have different requirements. Many useful AI tasks need available compute, acceptable latency and a cost structure that lets the product be used often.
Jon Peddie Research has put the PC GPU installed base in the multibillion-unit range and has projected continued growth toward 2026. PC GPUs form an uneven pool. Integrated graphics, older chips and high-end discrete GPUs have different limits. Even with those distinctions, the installed base changes the denominator.
Consumer and prosumer machines, especially gaming PCs and creator workstations, represent a large, underused compute layer that has been purchased, is powered and sits idle for much of the day. Smaller tasks, including distilled models, local copilots, media tools and software assistants, require far less than the hardware used for frontier model serving.
Activating even part of that installed base would move AI infrastructure beyond a pure buildout story and into deployment and orchestration.
What Distributed Compute Changes
Cost is the most immediate pressure point. Centralized GPU clusters are designed for high-memory, high-performance workloads, including models that can reach hundreds of gigabytes in size. The largest models require those systems, along with expensive infrastructure and constant hardware upgrades.
Distributed compute expands the range of devices that can participate in AI serving. Consumer-grade hardware has rarely been treated as infrastructure in its own right, even though capable machines already sit in homes, studios and small offices. High-end gaming PCs and creator workstations can support smaller inference jobs outside the data center.
Latency creates another opening for distributed compute. On-device or nearby inference can reduce the delay that comes from routing every request back to a distant cluster. For many applications, better infrastructure means compute that sits closer to the user.
Cheaper inference widens the market AI can actually serve. AI adoption remains narrower than the industry narrative often suggests, and cost is one reason why. An analysis highlighted by AIQuinta puts the gap in stark terms, suggesting that roughly 84 percent of the world has never meaningfully used AI. At the same time, only about 15 to 25 million people pay for premium AI tools. Lower delivery costs make more use cases economically viable, especially outside the small group of users already paying for premium AI products.
The Hard Part Is Coordination
Gaming PCs vary too widely to function as reliable AI nodes by default. Security, orchestration, hardware heterogeneity, reliability and bandwidth remain real constraints. Distributed compute is harder to coordinate than centralized infrastructure.
A workable model would start with a small client on each participating device. The client would identify the GPU, check available memory, test performance and report when the machine is free to run inference jobs. The network would then know which devices are useful for which tasks.
The next step is matching the work to the machine. A gaming PC should not be treated like a data center GPU. Smaller jobs, such as distilled models, local copilots, media tools and software assistants, are better candidates for distributed compute. Larger or more sensitive workloads can stay in centralized clusters.
Security and reliability would decide whether this can work at scale. Inference jobs would need to run in isolated environments. Data would need to be encrypted. Device owners would need limits on what their machines can access or expose. Since consumer devices can go offline at any time, the network would also need backup nodes, availability scores and fallback paths to cloud infrastructure.
Giant data centers, sovereign compute plans and premium Nvidia chips will remain central to the AI race. A second layer is starting to matter as well, built from compute that is already online, already paid for, and already much closer to users than any hyperscale campus can be.
The next AI infrastructure market will reward companies that can both build massive AI factories while coordinating the machines people already own.
