OpenAI is exploring whether custom AI chips could help it manage processor shortages and the high cost of running generative AI services. The company has not made a final decision, and any move into chipmaking would require substantial investment and time.
Why OpenAI is looking at hardware
OpenAI’s services depend on specialized processors that are in short supply. The company has been assessing ways to address that constraint, including designing its own chips, acquiring a chipmaking company, or working more closely with manufacturers such as Nvidia.
The issue is also financial. Bernstein analyst Stacy Rasgon estimates that each ChatGPT query costs approximately 4 cents to run. That cost makes the computing infrastructure behind a widely used AI service a significant operating concern.
The scale of possible demand could make the hardware bill much larger. If ChatGPT queries reached even a tenth of the scale of Google search, the initial GPU investment could be around $48.1 billion, with annual maintenance costs of about $16 billion, according to the source report.
A major reliance on Nvidia processors
OpenAI currently relies on a massive supercomputer built by Microsoft, one of its largest backers. Reuters reported that the system uses 10,000 Nvidia graphics processing units, or GPUs.
Nvidia holds more than 80 percent of the global market for processors best suited to AI applications. OpenAI CEO Sam Altman has publicly raised concerns about the scarcity and cost of these chips, making access to computing capacity a strategic issue for the company.
Building a custom chip capability could give OpenAI another way to plan for its hardware needs. But exploring that path does not immediately solve current shortages: the company would still need processors to serve its products while any new design is developed.
Custom chips would take time
Developing a chip is a long and difficult undertaking. Reuters reported that creating a custom processor, even after acquiring an existing chip company, would likely take several years. OpenAI would therefore remain dependent on commercial providers such as Nvidia and AMD in the interim.
An acquisition could potentially accelerate the effort by bringing an existing chip business into the company. The source points to Amazon’s acquisition of Annapurna Labs in 2015 as an example of a purchase that helped advance custom chip projects.
There are no guarantees that internal development will proceed smoothly. Meta’s attempts to create its own AI chips have encountered problems, according to Reuters. OpenAI would face the same broad challenge of turning a strategic ambition into hardware that can be built and used at scale.
Several paths remain open
The company’s discussions have been ongoing since at least last year, but the options remain under consideration. OpenAI could pursue an acquisition, deepen partnerships with chip manufacturers, or eventually invest in designing its own processors.
Microsoft is also reportedly working on a custom AI chip that OpenAI is testing. The Information reported that Microsoft planned to reveal the chip next month. That effort could be relevant to OpenAI’s hardware needs, although the source does not say whether it would replace or supplement the company’s existing reliance on Nvidia GPUs.
For now, OpenAI faces a choice between continuing to depend on outside suppliers and committing significant resources to a longer-term hardware strategy. Custom chips could offer a way to address supply and cost pressures, but the company has not announced a final course of action.