There’s an AI arms race taking place, with companies investing billions of dollars into developing and adopting AI technology. While plenty of ink has been spilled explaining how intellectual property issues, potential regulatory frameworks and the shortage of GPUs could slow the development of AI applications, no one seems to be acknowledging the much bigger bottleneck: the shortage of data center power and space required to train and run AI applications.
4 Data Center Storage Challenges That Impact the Future of AI
- Cost to build new data centers.
- Time it takes to build new data centers.
- A lack of space in existing data centers.
- Power constraints.
Training artificial intelligence models generates an enormous resource drain, requiring mind-boggling amounts of capital, time, space and power to deploy. In the end, AI advancement may come down to solving this data center shortage.
Why Data Centers Are Important to AI
Data centers play a critical role in our everyday lives, yet they’re often taken for granted or overlooked entirely. Whenever you scroll through social media, send a Slack message, order a vacuum on Amazon or send money through Zelle, you’re relying on a data center to process each one of those digital actions. They’re the cornerstone of our IT infrastructure.
But with the explosion in demand for AI, data center capacity is dwindling. A 2022 Market Vacancy report from Data Center Hawk revealed a vacancy rate of just 4.4 percent, which was the lowest ever— until now. The latest report from earlier this year revealed a vacancy rate of just 2.88 percent in the top 10 North American markets.
Demand is significantly outpacing supply. While conversations about AI focus on chips and ChatGPT and how AI products will add billions in revenue, what we should focus on instead is answering the question: “What happens when there is more demand for AI than what our current infrastructure can support?”
The answer seems simple: overcome a lack of capacity by building more capacity. In practice, however, it becomes much more difficult.
4 Reasons for the Data Center Shortage
The demand for computing power has exploded with AI, which requires a dense, complex computational ability to train.
The data center industry has already been pressured to meet the demand of existing enterprise digital transformation and public cloud usage. Because of the high costs and time to deploy new builds, data centers are typically built with just-in-time delivery based on forecasted demand. On top of an already strained supply, we now have this unprecedented demand being driven by the exponential adoption of AI.
Generative AI demands as much as five times the power of traditional workloads. ChatGPT-4, which is said to have markedly better intelligence, requires even more power to train. The AI development race comes down to the four key supply constraints: capital, time, space and power.
1. Capital
Data centers are expensive to build. A single medium-sized data center building of 250,000 square feet, can cost close to half a billion dollars to fully construct.
With projected demand for AI in the hundreds of megawatts to gigawatts, the full price tag will be tens of billions of dollars. That’s a lot of capital. When interest rates are high and capital is harder to raise, the price tag of data centers increases, making it more challenging to raise the necessary funds and keep pace with the demand. It also takes longer to raise the funds needed, which extends the construction time and slows everything down even further.
As of late, a number of legacy data center providers have been distracted or stalled by restructuring or bankruptcies, further challenging the timely delivery of capacity. The capital is still available for data center operators to fund new capacity, it’s just not going to come as easily as in the days of zero interest rates. The winners will be those who have a track record of deploying capital efficiently and operating effectively.
2. Time
One data center takes anywhere from 24 to 36 months to build. Sure, providers can have multiple projects in progress at once, but the majority are still months to years away from completion. And many were built based on far more modest demand projections, before the current demand skyrocketed.
On top of that, the industry is still struggling with supply chain challenges to supply the required mechanical, electrical and cooling systems that power and cool modern data centers. Those timelines have gone from 12-to-24 weeks of lead time two years ago to 52-to-80 weeks of lead time.
It will take time for new capacity to come online, and it may not be enough even when it does. However, operators who own the real estate beneath their data centers, and pursue multi-facility campus strategies, will be best positioned to scale up quickly by controlling their land use and reducing time to market.
3. Space
There is not a lot of available space left across the industry, especially after customers who have right of first refusal (ROFR) on any additional capacity are calling in those options now as they see the demand rising faster than supply.
High-performance computing (HPC) clusters powered by GPUs from companies like Nvidia to train AI applications are the current rage, but they’re not the only driver of demand for data center space. The natural growth of hyperscale public cloud and traditional enterprise technology deployments are still going strong, too, which will put even more pressure on supply in the next 12 to 18 months. And that’s if we continue on this path as expected without any surprise technology innovations.
The solution to this challenge will be for data center providers to design their new facilities to accommodate workloads ranging from traditional raised-floor, air-cooled enterprise applications to slab-floor and water-cooled hyperscale cloud or HPC workloads. This ensures that whatever capacity is brought online can be quickly adapted to the market’s most critical chokepoints.
4. Power
Everything ties back to power. The GPUs used in these HPC clusters for AI use as much as five times the power of traditional workloads.
Power was already in short supply, as we have seen major data center markets announce constraints on delivering power that was already promised for expansions. This trend is expected to continue with increased demand putting a bigger strain on power grids along with the transition to renewable energy, which takes time to adjust and perfect, will create more bottlenecks.
Smart data center providers are getting ahead of this by both increasing efficiency and siting their multi-facility campuses near power-generation sources or even deploying their own sub-stations.
What Happens If the Data Center Shortage Persists?
After examining these four areas, one question still remains: “What happens if we’re unable to overcome these constraints?” The implications for this are as profound as they are overlooked.
First, it could hamper the speed at which AI application developers can deploy the GPU-powered HPC clusters required to train underlying large language models (LLMs), and/or the eventual inference phases, where we see unprecedented demand for access to the applications built upon these LLMs.
It could also inhibit the ability of data center capacity to meet the demand of cloud providers and enterprises who are growing their existing workloads. These are the ones everyone already relies on today to power SaaS applications, social media, video streaming, gaming, e-commerce, the conveniences of our modern interconnected world, let alone what AI-powered apps the future holds.
Finally, it could have an adverse effect on the world’s newest trillion-dollar company. Nvidia reached its new valuation on the premise of unyielding demand for its GPUs in the months and years ahead. What happens if, after all of this, there is nowhere to put them?
