The era of AI PCs has begun, and a new arms race has shaped out around it.
Back in May, Microsoft debuted its first crop of AI PCs – Copilot+. Touted “the fastest, most intelligent Windows PCs ever built,” Copilot+ PCs have stirred up the competition, renewing hopes for the revival of the PC industry.
Copilot+ PCs represent a complete AI PC profile. According to Microsoft, Copilot+ systems can handle 40+ trillion operations per second (TOPS). No traditional computer holds a candle to that.
“It can do a lot of things in order to make AI happen on the PC,” said Stephen Foskett, president of Tech Field Day, a unit of The Futurum Group, while giving a talk on AI PCs at an Ignite Talk at the AppDev Field Day event in California. “It has full integration of datasets, AI-powered assistance and more.”
The thin, lightweight Copilot+ devices from Microsoft Surface demonstrate incredible pace and performance.
“Microsoft is clearly putting in a ton of money to support and drive the Copilot+ PC directive, and the first systems we have tested using Qualcomm and AMD silicon offered not just new AI capabilities, but a complete overhaul of performance and battery-life expectancy,” says Ryan Shrout, president and general manager of Signal65, a division of Futurum, that tests data center, enterprise and client computing technologies.
Shrout’s works – A Modern AI PC Design and The Samsung Galaxy Book4 Edge PC powered by Snapdragon X Elite – provide deep analyses of the performance and energy efficiency of Windows and Copilot+ devices.
Original equipment manufacturers (OEMs) like Dell, Samsung Electronics, HPE, Lenovo, Acer and ASUSTeK Computer have unveiled their own brands of Copilot+ computers bringing Microsoft shoulder to shoulder with its biggest contender, Apple.
Apple already has a lead in the race with 60% of the market share under its command, an advantage it has gained by starting early. With the Mac portfolio, Apple has been shipping AI-enabled PCs since 2020.
“Every Apple M series machine has had matrix multiplication acceleration, neural processing units, and GPUs on the PC side,” Foskett said.
Intel too has been adding Advanced Vector Extensions (AVX) to its x86 instruction set architecture for years, in preparation for the AI wave.
At Dell Technologies World, Dell projected that AI PCs would fuel a “super cycle” of buying in the business sector where sales have been on a steady decline since the pandemic, delivering the industry out of the slump.
Canalys, a research firm, predicts that in 2024, AI PCs will make up 18% of the total PC shipments worldwide, a number that will rise to 40% in 2025. According to their forecast, the AI PC market will see an estimated CAGR of 44% between 2024 and 2028.
On-Device AI
An AI PC is not just a buzzy new AI-plastered technology. It is a slam-dunk for a proliferating body of compute-intensive AI tasks that are taking over enterprises.
AI PCs pack incredible performance that allow huge volumes of AI tasks – typically done on servers – to be computed directly on the device. As businesses look to supplement bigger AI models hosted in the cloud with private, fine-tuned models, AI PCs will allow them to bypass cloud-based AI applications and perform AI tasks locally on the device.
“The idea that these machines would be able to run AI-accelerated tasks locally as part of our regular everyday use of them, has the potential to be a revolutionary thing,” Foskett said. “As people are starting to use various AI tools – and not just having a chatbot to interact with – if all of those things can be done more locally, then that could spark a new wave of customer demand for PCs.”
AI PCs integrate a distinguished hardware makeup that gives them the horsepower to handle high volumes of intense computation comparable to that of a server. Key among them is a specialized component called NPU.
“Neural Processing Unit is sort of an accelerator that usually sits on the same die as the CPU, and sometimes part of the same chip, but it operates independently,” Foskett told.
A NPU or neural processing unit, in theory, can stimulate the neural network seen in a human brain. Inside the AI PC architecture, the NPU behaves like a specialized coprocessor working together with the CPU to execute AI and ML algorithms and perform complex mathematical computations.
“You don’t necessarily need a discrete NPU in order to do AI processing, but most of the chips that are going to be used for these things are going to have a discrete NPU.”
Storage, memory and connectivity too play pivotal roles. “For example, if you query and it needs to hit a database on the Internet or look something up in the calendar or in a business application, it’s going to need that always-on connectivity, and this is going to be one of the hallmarks of AI PCs,” he explained.
NPU often get pitched as the new holy grail of the AI PC, but the component has existed since before the AI wave. Apple has been using them in Macs for years.
Besides Apple and Microsoft, other industry players too have joined the fray. Intel’s announcement of Core Ultra processors, aka, Lunar Lake, and AMD’s Ryzen AI 300 series, nicknamed Strix Point, have made significant noise further strengthening the narrative of on-device AI.
“Lunar Lake and Strix Point are going to deliver AI PC functionality especially in use cases that don’t demand the battery life and lower power consumption of Snapdragon X,” said Foskett.
An Area of Concern
Amid the hype, the flagship feature “Recall” in Copilot+ has caught some bad press. Recall is a capability embedded in Copilot+ PCs that allows the embedded AI assistant to search and find information based on past activities.
This requires Recall to track the history of voice chats and web browsing activities on the device, and build a detailed record. This has raised significant privacy concerns among users.
Adding to the fear is security concerns around Recall’s on-device storing of the historical data.
“It’s just a demonstration of the capability of the AI PC,” argues Foskett, “What a lot of people missed is that it’s an optional feature, maybe a tone deaf one, but its Microsoft’s way of saying that these AI features will be integrated all the way throughout Windows giving the PCs the next-level capability that they never had before.”
Regardless, the total addressable market of AI PCs is growing at a rapid pace.
“This is going to be a banner year for PC OEMs because products like Windows Copilot+ PCs are so good that businesses will buy them hand over fist. The Qualcomm Snapdragon X laptops are so compelling in terms of performance, battery-life and formfactor that even if people don’t use the AI features, they’re still going to buy those machines because they’re that much better than anything that business users have been able to buy besides Mac.”
The effect on the consumer market however is unknown, and discussions are speculative.
“A clear majority of the people do not use PCs on a daily basis. This will blunt the impact in the consumer space, because consumers are increasingly going to abandon PCs in favor of mobile devices because they already have the AI technology they need.”
