Google is reportedly in discussions with Marvell Technology to develop new AI chips, including a memory-focused chip designed to support existing tensor processing units, and a TPU tailored specifically for the demands of AI inference.
Google has long worked with Broadcom on chip design, but rising demand and cost pressures are prompting diversification. Partnering with Marvell could provide additional capacity and design flexibility at a time when AI infrastructure is under strain across the industry.
The potential partnership coincides with Google’s upcoming Cloud Next conference, where the company is expected to outline the next generation of its AI infrastructure. While details are limited, executives have indicated that improving the speed and efficiency of AI deployments will be a key focus. Any announcement will likely build on the company’s existing TPU roadmap while introducing more customized chip hardware.
Shift Toward Inference
Beyond the immediate news, the Google-Marvell news is part of a major shift in the AI sector. The focus for the past several years has been on training increasingly large models, which demand massive compute resources. Now, the growing priority is AI inference, the process of delivering answers, which is all-important for user experience.
Google’s tensor processing units have gained a high profile in this shift. Developed in-house over more than a decade, TPUs have become foundational to its cloud offering, and attracted customers like Meta and Anthropic, both of which are exploring alternatives to GPU-based infrastructure.
NVIDIA, of course, remains the top player in the GPU sector, with its pricey chips widely used for training large AI models. But inference, which calls for efficiency and speed over raw computer power, gives Google’s specialized chips an opening to gain market share.
Furthermore, Google designs both AI models and the hardware that runs them, which gives it a competitive advantage. Engineers can design across the full stack, refining chips based on the behavior of real-world applications. This feedback loop has supported successive TPU generations and is likely to influence any new designs emerging from the Marvell collaboration.
Need for Flexibility
While Google’s profile in the chip sector is increasing, it still faces challenges. Chip development cycles can take years, while AI models evolve on a much shorter timeline. Predicting the requirements of future systems is an uncertain venture, particularly as new AI use cases emerge. Splitting development across multiple partners may help mitigate that risk, allowing Google to explore parallel design paths.
Also, demand for AI hardware exceeds supply, even among the biggest tech companies. As Google expands its customer base, it needs to balance its in-house needs with customer demand, a juggling act for which there is no clear guide.
Still, Google’s focus on custom chips designed for the ever-changing needs of inference should be a major plus for the company. The potential Google-Marvell collaboration supports this trend, highlighting the need for flexibility to keep pace with the rapidly evolving AI sector.
