AI Leadership Insights: Computer Vision at the Edge

Mike Vizard: Hello, and welcome to the latest edition of the Techstrong.ai video series. I’m your host Mike Vizard. Today we’re with Fabrizio Del Maffeo, who’s CEO for Axelera AI, and they just raised $50 million to drive computer vision out to the edge. And we’re going to talk about what is the state of computer vision at the edge these days. Fabrizio, welcome to the show.

Fabrizio Del Maffeo: Thank you so much for having me here today, Mike. Well, you know, computer vision and artificial intelligence are already all around us. If you think about it, your car is a computer on wheels with a battery. If you think about the medical devices, if you think about retail surveillance, you have computer vision everywhere. But algorithms are becoming bigger and bigger, and the technology that is today available at the edge is actually designed for cloud computing with no constraints of power, and no constraints of cost. And then it’s clear that if you want to deploy at the edge, these new technologies, these new algorithms – we need AI native edge technologies. And this is what Axelera AI is doing is building up any AI hardware and software platform, which can deliver high performance and usability at a fraction of the cost of today’s solution available.

Mike Vizard: Is that for running, say, some sort of inference engine at the edge, or are you also training the AI model in real time to learn what’s happening around it?

Fabrizio Del Maffeo: Thanks for asking. It is for inference, because you train – all the models are trained in the cloud, as you know. The latest models need thousands to be trained. But the beauty of the technology is that once the model is trained, you can then convert it and make it lighter and deploy it at the edge if you know how to do it. And here Axelera comes to play this game – we are here to enable companies to train models in the cloud, run them in the cloud, but also deploy them in the edge. And our focus is now computer vision. But we can already see that in a couple of years, these large language models, these generic models will come to the edge, and we want to play a role in it.

Mike Vizard: It seems that at the edge, there’s always cost sensitivity. So is part of the equation going to be that we’re going to have these models, and they’re going to have a lot of data – they’re going to be learning something or updated in real time – and we just don’t have the right set of processes in place to enable that?  Because an x86 wasn’t designed for that.

Fabrizio Del Maffeo: It’s correct. It’s completely correct. Because if you look inside any model, actually any family of neural networks, you see that 70 to 90% of operations are vector matrix multiplications. Essentially, you’re multiplying numbers. And the rest are more sophisticated operations that you can run on the CPU. Then, you need a new kind of technology, which can optimize the vector matrix multiplications. And it’s what we are doing in Axelera. We are designing a custom technology for which they care about these operations, and it’s called digitally memory computing, and then all the other operations can be run using a RISC-V architecture or X axis architecture.

Mike Vizard: What is the challenge then, in making computer vision pervasive? Is it just cost or are there other factors that go into that? It would seem to me that I have to be a pretty darn good developer/data scientist to kind of build these things successfully. So what do I need to succeed as an organization?

Fabrizio Del Maffeo: It’s true, it’s correct. Cost is an important factor. But it’s not the only one because when you get to the edge, you have thousands or hundreds of thousands of customers which are building up devices. I mean, and if you look inside these companies, you may find one or two machine learning engineers but you cannot find thousands, like in Google and Facebook, for support. Therefore, you need to have a technology which is usable – a technology with plug and play technology, which allows customer to drag and drop and build up applications in a very simple way. A no-code or low-code way. And since day one, since we built up our platform, we thought about it. We talked about that out there, there are 23- 24 million software developers, but how many machine learnings engineers are there? How many deep learning engineers are there? There is definitely just a small portion. And if a customer, if a company, wants to deploy technology – wants to develop a platform like ours – it’s for sure going to consider this; the fact that you have to provide the tools that are simple to be used and allows tools that can allow customers to deploy easily this model, and use them easily in a simple way.

Mike Vizard: There’s a lot of concern about privacy regulations, and all those things, and I don’t think a machine can un-see something once they’ve seen it. So how do we kind of protect that? Or how do I limit what the machine might see or not see?

Fabrizio Del Maffeo: I think that edge is going to play an important role on this, because the beauty to run AI at the edge is exactly this – is that you don’t have to share your data with a cloud, actually. Thinking about a camera, if I have to recognize you in your house, and the algorithm is running in your house, you feel more safe compared to today, when you know that the algorithm is running in the cloud and the image of you is sent to the cloud to be recognized, right? Then I think edge can help in each sector – can be medical imaging, can be security, can help to preserve the the security and the privacy, clearly, of the data. And in the long run, in the long term, I can think that also the training part, partially the training part, will run along the edge – not totally – but that fine tuning, which is required to recognize you, for example, can can be run at the edge and can be done at the edge without sharing your data with the cloud.

Mike Vizard: Do you think this adds a different dimension to computing, because we’re talking about vision and location and combining that in a way that lets me throw in some analytics and data science? So, when do we start to discover patterns and processes that we were just unaware of?

Fabrizio Del Maffeo: Yeah, it’s a good question. Definitely we are going to discover patterns for which we are not aware. And we have to also give a sense to this. And that’s why we still need human interaction. And this, I think we have plenty of data. We can extract the information from that data. But the quality of data is impacting the information we’re extracting. And in general, we have to understand whether that information is correct or not. And it’s not so simple.

Mike Vizard: So what’s your best advice to folks? What do you see companies who were implementing computer vision today doing successfully that you wish other people would copy?

Fabrizio Del Maffeo: I think that if you look at the computer vision – it depends on which market segment – but in general, I think that if you want to run computer vision at the end, you have to get expertise on board and really quantize networks in specific market domains. For example, pandas network, which allow you to take any network, make it smaller, and run it in almost any architecture. Clearly, if you have a specific AI native architecture, you can extract way more from the data; from the algorithms. But in general, if you can get expertise to quantize networks and to compress networks, then you can unleash thousands of new scenarios and create the real value at the edge. And today, I don’t see this expertise. And we are here to help, but to spread it around and make aware to the customer that it’s easier and easier to deploy AI at the edge to take advantage of computer vision to serve a myriad of problems out there.

Mike Vizard: Alright, folks, there’s an old saying that says, “Hey, if you can’t see it, you can’t manage it.” So, I think we need a little more computer vision here and there. Fabrizio, thanks for being on the show.

Fabrizio Del Maffeo: Thank you so much, Mike for inviting me.

Mike Vizard: And thank you all for watching the latest episode of Techstrong.ai. You can find this and other episodes on our website. We invite you to check them all out. And once again, thanks for spending some time with us.