PyTorch is at the heart of AI development, and increasingly the software layer through which hardware vendors compete for AI workloads.
SAN JOSE, Calif. — At the PyTorch Conference, the Big AI movers and shakers discussed the current state of PyTorch. And, well, they should since, as Amazon pointed out, “85% of AI researchers and developers write their code in PyTorch.”
The New PyTorch Chips
At this week’s conference, the biggest news was that Google’s new TorchTPU backend and Amazon’s TorchNeuron both now support PyTorch to port existing PyTorch workloads to their accelerators. TorchTPU brings native PyTorch execution to Google’s Tensor Processing Units (TPUs), while TorchNeuron does essentially the same thing with Amazon’s AWS Trainium chips.
This is more than just supporting new chips with a familiar application programming interface (API). It also demands compilers, kernels, and distributed-training tools that can withstand expensive, long-running workloads.
As Chris Lattner, Qualcomm’s Executive VP of AI, said in his keynote, “We all know that PyTorch has done an amazing job at unifying the modeling layer, making it easy to build and train models for a wide variety of different applications, but we still have the bottom part of the stack that the hardware vendors have to make for themselves. This includes the kernels, the compilers, and low-level system software.”
That gap between a common modeling interface and fragmented execution infrastructure is becoming a central battleground in AI. Lattner added that creating a common framework is Qualcomm’s goal with the Modular software stack. Lattner described it as “something that’s like LLVM, [a collection of modular and reusable compiler and toolchain technologies for Linux] but for AI chips, and that’s what we’ve been building at Modular.”
He added Modular’s stack supports hardware from NVIDIA, AMD, Apple, Qualcomm, Amazon and Google. Making PyTorch as reusable as the framework above it on all chips is still a work in progress.
Success Is Pressuring PyTorch’s Maintainers
PyTorch co-maintainer Alban Desmaison told attendees that more than 600,000 GitHub projects build on PyTorch and that the project receives roughly 150 pull requests daily. As a result, he said growing usage is straining testing, infrastructure, and release operations.
That’s a lot by any reckoning. PyTorch is growing to meet the demand. Mark Collier, the PyTorch Foundation executive director, reported that active contributors had increased 60% year over year, participating organizations 30% and pull requests 40%.
Desmaison also described shared hardware abstractions, backend integration and validation as essential framework responsibilities.
PyTorch in Practice
PyTorch co-maintainer Edward Yang said closer work with large language model (LLM) training teams had reshaped his view of framework priorities. “At the end of the day, what you care about is how long it takes for you to get the final checkpoint that you’re actually going to serve.”
Peak utilization matters, but so do crashes, numerical errors, and code that engineers can understand and repair. Yang added users want codebases that let them resolve problems themselves when something goes wrong on the critical path.
His presentation emphasized the need to test large distributed workloads with fewer resources, improve profiling, and develop smaller, composable building blocks. We shouldn’t be, he continued, imposing an all-encompassing training system. Yang explained, “It’s impossible for us to anticipate any given problem any given user might have, so we want to build building blocks that you can easily adapt to your circumstances.”
Yang also previewed a lower-overhead profiler intended to remain enabled throughout training. He explicitly said it wasn’t publicly available yet.
Shopify Shows What PyTorch Can Do in Business.
Shopify offered a concrete example of PyTorch’s role beyond frontier-model development: repeatedly improving a model for a narrowly defined business task.
Cody Mazza-Anthony, a Shopify Machine Learning (ML) engineer, described how the company provides its all-in-one cloud e-commerce platform used PyTorch to create a continual-learning workflow. It begins with a trusted ground-truth dataset and a model-based evaluator calibrated against human judgments. Once a system reaches production, Shopify identifies failures, repairs examples, adds successful corrections to its training data, and trains another model checkpoint.
In its buyer-profile application, the system converts shopping activity into structured representations of customers’ tastes and interests. Mazza-Anthony said Shopify began with approximately 29,000 examples and improved the specialized model through that production-feedback loop.
“It’s important to keep in mind that human agreement is the ceiling,” he said. “We’re not looking for perfect here. We’re looking for as good as human.”
Shopify’s presentation also described compressing system prompts into learned representations to reduce inference overhead. Together, the techniques show how businesses can use an open software stack to optimize task quality, latency and cost rather than continually substitute a larger general-purpose model.
An Open Framework Is Not Open Source.
Meta’s presentation reinforced the view that PyTorch is becoming a collection of interoperable components spanning training, inference and optimization. Jana van Greunen, Meta’s Director of PyTorch Engineering, called Meta’s recently released Muse Spark Open models “open source models.” It’s not.
Its Muse Model License imposes usage restrictions and requires a separate agreement for qualifying model-as-a-service businesses above a $20-million revenue threshold. Those terms conflict with the Open Source Initiative’s (OSI) requirement that an open-source AI system be usable for any purpose without seeking permission.
That said, Muse Spark Open is an open-weight model.
Openness Is AI’s Future.
Earlier, Collier had argued that open is the right way forward for both AI and PyTorch. He observed a DeepSeek model reportedly matched capability at 60% lower cost after 37 days; At the same time, NVIDIA work increased LLM inference performance by 68% on the same hardware in 33 days. This was all, he claimed, because they’d embraced the open approach.
He added, “company after company and individuals from those companies that are really committed to working in the open, and that starts with open weights.” You can see the proof yourself. “PyTorch trains most of the models that we use every day,” and 70% of the new models use open weights. Other studies have shown that in just a few months, 56% of models processing AI Gateway tokens are open-weight models.
The moral of the story is that the future of AI is open, and PyTorch, itself an open-source language, is leading the way.
