Arm today added a processor to its portfolio that is optimized for running inference engines that will drive deployments of artificial intelligence (AI) applications at the network edge.
The Arm Cortex-M52 is designed to enable inference engines to run without requiring any additional processors that would increase the overall footprint of the platform. The capability is enabled using an extension to the Cortex family of processors dubbed Helium to accelerate the performance of machine learning algorithms.
In effect, Arm is streamlining a software development process that would otherwise require separate compilers and debugging tools to build a platform, says Paul Williamson, senior vice president and general manager for the IoT line of business at Arm. “Developer enablement is really critical,” he adds.
As part of that effort, Arm will make the Cortex-M52 available on Arm Virtual Hardware, a cloud-based service that provides developers with access to next-generation Arm processors.
There is now a race on to optimize the performance of inference engines at the network edge as more AI models are being deployed as close as possible to the point where data is being created and consumed. Most inference engines today run on x86 processors, but Arm sees an opportunity at the network edge for a class of processors that while being both smaller and more energy efficient are also faster.
At this juncture it’s already apparent that inference engines running AI models will soon be pervasively deployed at the network edge. The challenge is that use cases involving, for example, vehicles require low-energy processors capable of processing massive amounts of data in real time. Most AI applications running at the network edge because of latency issues are not going to be able to transfer massive amounts of data over the air to a cloud service to process data. Instead, AI applications that are compute intensive will need to access local resources that are as powerful as any server found in a data center.
It’s not clear how much it will cost to build and deploy these platforms, but Arm is betting its existing partnerships with foundries around the world will drive the cost of processors optimized of AI workloads down faster than rivals can match.
In the meantime, organizations of all sizes should, at the very least, be experimenting with AI applications for the network edge. It will undoubtedly be more challenging to deploy AI models that need to be regularly updated at the network edge, but as IT teams start to define best practices for integrating machine learning operations (MLOps) with DevOps, more of these tasks will become increasingly automated.
The challenge, of course, is many edge computing platforms today are managed by operational technology (OT) rather than IT teams. The level of exposure OT professionals have to AI is even more limited than traditional IT teams, so it may be a while before AI models are pervasively deployed at the network edge. However, the foundation for building AI applications is already starting to fall into place.
