AI Leadership Insights: How to Operationalize AI

Mike Vizard: Hello and welcome to the latest edition of the Techstrong AI series. I’m your host, Mike Vizard. Today we’re with Amir Husain, who is CEO for a company called SparkCognition, and he is also author of a book called The Sentient Machine. We’re going to be talking about how to operationalize all this AI stuff that’s out there because while it’s awesome, it’s hard to wrap your arms around it. Amir, welcome to show.

Amir Husain: Thank you very much, Michael. It’s a pleasure to be with you.

Mike Vizard: What are you seeing when you talk to other organizations out there about the challenges that they’re running into? I think there’s a huge amount of enthusiasm, but nobody seems to understand exactly where do I put together the LLMs? How do I organize the various services? Do I need a vector database? What do I do with it? Where do I get the data? It seems like this is going to be a longer journey than anybody anticipated, but what’s your take?

Amir Husain: Yeah, well, I think many of the problems are common to just simple artificial intelligence deployments, even prior to generative AI being a mainstream phenomenon that a lot of companies are trying to get behind. Now, the issue with AI was do I have the data? Is the data clean? Can I have access to the data? There’s a ton of data governance challenges around that. And what’s even, I guess one step even behind that is people don’t really have a good sense of what they can do with artificial intelligence techniques if they have a lot of great data and they’ve got that data infrastructure. And if they don’t, there are still things you can do with AI in your company. So for example, at SparkCognition, we have a product that’s focused on visual AI. That’s a product that comes with a pre-trained model, and you can deploy that in your environment and you can optimize your factory flow, your warehouse and so on and so forth without needing any data at your end.
It is artificial intelligence, it is adding value, it is making your organization more optimal. So one, I think it’s education. It’s like trying to figure out, okay, I’ve got to drive some efficiencies. What matters? What types of efficiencies do I really need to drive in order to move the top line, bottom line? And then for that, do I need data? Because a lot of conversations I have in the enterprise stop with what come back in a year when I have the data. Another aspect I think which is challenging and problematic is human flows versus artificial intelligence flows. At the end of the day, if with generative AI you’re going to solve a problem that’s going to be meaningful, what that means is that it either reduces the cost of a workflow of a transaction that runs through your company or it increases the velocity of that transaction, or it increases the quality or the accuracy of that transaction.
And generally, that end-to-end flow, in order to do that properly, you’ve got to go across silos, you’ve got to go across departments, and right now that’s a human problem, not so much a technology problem. So those to me are some of the challenges. I think one is around education, being ready to deploy AI, even when you think internally you don’t have the data, there’s still a lot to do. When you do have some data, you should definitely get in and figure out what part of the key workflows overlap with areas where you have data, start to put that infrastructure in, and think about the deployment as an end-to-end cross-silo, cross-department workflow that ultimately does one of three things, increases velocity, increases quality, and reduces cost.

Mike Vizard: When you think about this, is this a variation of the old, do we build it, buy it or rent it kind of philosophy? Is that what we need to work through first?

Amir Husain: I agree with you completely. There are lots of industrial companies, dozens and dozens of large industrial companies that I’ve come across, each of which are very large, they have tens of thousands, sometimes a hundred thousand plus employees working for them. And then the general sense is, oh, where are we going to have company, we can do this. There’s also some protectionism going on generally inside these companies because after you acquire a certain level of scale, this is normal corporate behavior. You are worried about your budgets and you want to control how much you have and whether you get the opportunity to show that you got something done. Those things become important. And sometimes those goals are organizational goals, but not company-wide goals. They might be department goals, but they’re not at the top level driving the benefit for the company.
And that’s where I think leaders need to be very careful, because the internal team is nine times out of 10 going to tell you, we can just do it all. We can do it all. And the reality is that if you’re an industrial company, if you’re a utility company, can you really retain an interest and bring on the world’s top AI guys? Can you really bring on people that are very super rare and generally want to work in an environment which is a pure software environment? Because that’s what’s driving their exposure to multifaceted problems. It’s the software culture that they’re familiar with, that they’re comfortable with. So I think yes, it’s build or buy. There’s a pressure internally in these large companies, which is, again, I’m the data analytics team at this big co. I can do it all. That’s seldom true.
And then secondly, a self-assessment, which is that I run a utility. Do I really want to be a software company? Do I really want to build the software infrastructure that’s really leverageable product style that software companies are in the sole business of doing? So yes, I definitely believe that build versus buy is a very important thing to think about. There’s a lot of challenges right now. This issue hasn’t broadly been resolved. We are not at the point where there are brands that people could point to and say what IBM was to the mainframe era. There isn’t an equivalent company now that you can point to and you say, “If I buy X, Y, Z AI, then I’ll never get fired.” We’re not quite there yet. So it has to be a thoughtful analysis of what you as a company bring to the table, what capacities you have, and where it’s better to partner.

Mike Vizard: Are we having an irrational exuberance moment where it seems like everybody I know wants to be working on an AI project regardless of whether or not it makes sense for the business yet, and we’re going to have to have some sort of rational moment where we start picking the projects that are going to win?

Amir Husain: Look, overall, if you ask me, there’s nobody more exuberant and optimistic about AI than me perhaps. I do think it’s a fundamental change. It’s going to make a huge difference, and I’ve not been just saying this since the advent of the GPTs and the ChatGPT, et cetera, which have made it mainstream in consumer, but I’ve been saying it for years and years and I’ve written about it over many years. So I think this is finally the coming of the age of many types of AI algorithms and technologies which can truly drive results. So overall optimism, absolutely. Now, that doesn’t mean that if there’s merit in an idea, but you pursue it in silly ways, that the idea will provide payback. So there’s an execution element to it and then there’s an overall quality to the idea.
The quality of this idea is very high. The potential of this idea is very high, but the execution mechanisms that you exercise internally at a company, first of all figuring out what should you be solving? You can’t solve everything. So what should you really be solving? And the way that I generally advise boards and CEOs is to say, “How would you describe three workflows that happen in your company that contribute the bulk of the revenue?” So if you’re an insurance company, what principally contributes your revenue? What are the three flows? Now let’s break these out and figure out how we could do these with AI and drive one of those three benefits. Now, when you put it in those terms, the first thing that happens is you win the interest of the top level people at the firm, the C-level, the board level, in order to seriously think about this and drive that change organizationally, because like I said, people are siloed. The data is cross-department. You really need some silo breakers, generally the board and the C-level management, to come in and do that.
If you do it that way, then I think you are beginning to execute in the proper fashion. Then the second thing is, okay, now the company’s bought in. Now, who does it? If you can do certain things internally, certainly, but most companies that are not pure software AI companies cannot execute the entire plethora of activities that are necessary in order to get to a win. So that’s where partnerships become very important and creating a partnership culture within the company. You’re not just partnering in a way that you’re allowing somebody to come in, but you’re really setting them up for not success. You’re setting them up for failure because you want to make the point to your management that, look, I told you I could have done that. Instead, it’s a partnership culture where there’s enough on both sides that can be shared in the case of a win to where people are incentivized internally to look for partnerships and then they’re not threatened by partnerships.

Mike Vizard: Is it your sense that the business leaders understand what’s at stake here and what the issues are? Because I talk to some of them and they’re just thinking that all this stuff is magically going to work tomorrow and they’re already making business plans for cost of labor and revenue outlooks, but it doesn’t seem that they’ve actually taken a hard look at understanding there’s a long way from here to there.

Amir Husain: Yeah, there was a recent survey going back to August where generally 30% of the leaders that were interviewed said that they felt that generative AI was ready for a deployment. So I think you still have a minority of executives thinking that it’s ready for deployment, but again, that’s not granular enough for me to really make much of that comment because, again, generative AI is not a thing. Generative AI is a core technology, a big chunk of artificial intelligence that can drive thousands and thousands of use cases. I would much rather think at a more practical and useful level, which is for your business, Ms. insurance company CEO, or for your business, defense production big prime company CEO, here are the top use cases. And if they tie in with the three, four workflows that you really have to rinse and repeat in order to make your business far more profitable, differentiated, developed modes, all of those things, then it makes sense for you now.
So the question is not is generative AI ready for me? The question is are use cases that I have a fairly high confidence on that generative AI has been shown to drive now, are those use cases part of my most important flows? And if that’s the case, you should go. There’s no excuse. You should go. Because in your area, somebody else gets to that first, I think they develop a pretty substantial edge. This is one of those capabilities where there’s a leapfrog moment. You see what’s happening in military affairs now with autonomous technology, AI-driven technology, that small 10, $20,000 systems are taking on three, five, 10, $20 million systems and doing so in an advantaged way. So it’s a completely different paradigm and I think there are civilian parallels to that. You’ve got to get there first.

Mike Vizard: Is that part of the motivation? It seems like everybody is worried about being left behind even though they’re not quite sure what it is that they’re going to be behind of.

Amir Husain: Well, in anyone’s business, you can certainly describe what it means to be left behind. So as an example, take insurance. There’s fundamental metrics in the insurance business. Take consumer banking, lending. There’s fundamental metrics in those businesses, and so if you can use artificial intelligence to improve those metrics, you have left your competition behind because that entire industry is based on a cost structure. Take engineering. Essentially, large aviation companies are doing three things. They’re doing CAD to design the thing. They’re doing code to control the thing. They’re doing CAM to manufacture the thing. Code, CAD, CAM. I talk about this often in my conversations with aviation boards and companies and leadership.
Now, that is really the bulk of what an aviation company does, and aviation companies, the large ones, employ hundreds of thousands of people. So now if you think about that and you say, “How can I apply generative AI to CAD? How can I apply generative AI to the kind of code I need?” There are efforts today to do that. So what would it mean to be left behind, going back to your question? In this area, what it would mean is that aviation company A gets to a 100 the cost design process, a 100 the cost software development process, than a competitor. At that point, you have been left behind and in many cases it’ll be hard to catch up

Mike Vizard: As you think it through some more, how sustainable are your so-called competitive advantages, even if you go out and do something? It used to be you might be able to do something and maintain an edge for a year, two, before somebody else would clone it, but I feel like the way tech is going at the moment that I might only have two or three months before somebody else is able to do something very similar.

Amir Husain: You’re absolutely right, Michael. If you really look at LLMs as the sole manifestation of the technology, then you’re absolutely right. We saw this. Less than a year ago, ChatGPT came out, or about a year ago it came out, and since then there’s been this parade of open source models that anybody can use and embed and so on and so forth. So there’s also that famous Google paper which was leaked. I wonder if you read the email that was leaked, which said, “We have no [inaudible 00:15:16].” And so the idea is of course, if you just focus on LLMs as the thing, you’re right. You’re right. But they’re not the thing. And that’s what, in my business and in all the customers that I talk to, I say, “Guys, look.” So we built a foundation model for seismic. Rather than taking data that’s all over the internet, we chose to take a dataset that’s highly proprietary, that’s not over the internet, so people can’t build another model based on that dataset, but even if they had access to that data, that’s not the only thing.
The thing is that seismic surveys aren’t as voluminous, aren’t as common as, let’s say, text on the internet or pictures on the internet. There are not enough seismic surveys to go and scrape off of even proprietary databases to take the kind of approach that an LLM takes. So what you have to do is you have to develop smarter AI, you have to develop artificial intelligence systems and models that are infused with data that learn from data, but are also aware of the physics of the thing. They’re actually aware of what they’re doing. That gets into algorithmic innovation again. And my view is that that kind of stuff is not easy to do, which is why what you see in the open source community are replicants of a ChatGPT-like text LLM. Now, one could argue that it’s not the same level of quality, et cetera, that’s fine, but in nature it’s essentially the same thing.
But what I want to spend my time building are things that are super hard to build and they’re not easy to replicate. So, like with all technology areas, eventually certain things get commoditized. You’ve been writing for 20 plus years, you’ve seen so many things change in technology, and so at one time the hottest thing was who can build the best OS? Steve Jobs was out building next step and Jean-Louis Gassée was out building BOS, and Apple had an internal project that they were funding and OS2 and who would win that? Xenex and Unix, and all that’s gone. It’s all now essentially commoditized. I think Windows now is number four on the list of things that makes money for Microsoft and it’s a fraction of Azure as an example. Linux is free, so ultimately the core LLM becomes free. What you do on top of it that’s different and unique and what it can drive is where the differentiation is.

Mike Vizard: I think there’s also going to be a lot of these LLMs because the first wave was very general purpose in nature and it’s hard to keep collecting all that data to update those LLMs, but I’ll have smaller, more domain-specific LLMs that are trained on a narrow set of data, and I might get better results from a smaller LLM than from a bigger LLM for a specific process. Is that a fair assessment?

Amir Husain: I think that’s a fair assessment, but I’ll add two things just for clarity. One is that it’s not just because the LLM was trained on a smaller amount of data and therefore the intuition is that the LLM isn’t confounded by things that it has nothing to do with, that have nothing to do with that domain. It’s also the fact that you can then enrich the LLM by additional models. You can give the LLM tools of the domain. For example, you can tell the LLM that here is a book of all the formulas in physics, and if you find data that does not fit with the list of formulas in this book of physics, there’s probably something wrong. If you hallucinate and ever project an outcome that doesn’t fit with this book of physics, it is probably wrong. Now, you might still want to say, depending on the application, but provide a reference to the calculated form of that outcome as well so that the user can look at it and compare and actually drive some value.
So I’m giving you a very simplistic example obviously, but my idea is to say the book of physics and my example is the model. It’s the model of the thing, and it’s technology that is not an LLM. It’s a tool available to the LLM. So when you build these smaller LLMs, it’s very hard to build tools for everything in the world. It’s very hard to build tools for ChatGPT that just cover everything in the world. But in one area, one domain, you can absolutely reduce hallucinations down to a point where they’re a non-issue. You can definitely add very focused tools that improve the outcome, the speed, the veracity of an LLM. And yes, those are differentiated models because they’re not just in an LLM. There are a lot of things all put together to solve a problem.

Mike Vizard: We see a lot of people are working with vector databases to extend an LLM. The LLM as I understand it is trained up until a certain date and you’re basically trying to extend it by showing it additional data that’s housed in a vector database that they can understand. Is this a short-term solution or is this going to be something where you really need to operationalize for the long term?

Amir Husain: So, let’s think about what that is. What that is that you have a limited context window on LLMs. So when you’re having a conversation with an LLM, it can only remember so much. Now, there are a couple of approaches. One, people are trying to increase that context window to let’s say a novel length, but even then a novel is still limited. It’s not everything. And so when you put it as if it’s a novel, then the mind says, oh, well, what’s going to be longer than a novel? But I’ll tell you what’s going to be longer than a novel. Just logs off a server, logs off of a physical machine that might be gigabytes. And if I want an LLM to become an analyst for machine logs, I can’t dump all the machine logs. They’re far longer than a novel. There’s an alternate technique that’s being developed, which is called streaming LLMs, where the idea is that you focus in on the data keeps coming in and streams, and you focus in on what you think is important, but it’s compression.
It’s not all the data that’s going into the LLM in that case also. So there are some techniques being developed to figure out how you can get more relevant information inside an LLM. Another way to do that is what you described, which is that you augment the response of a network with an in-time query going back to a database which happens to use vectors to match the query with the content in the database. Rather than doing a relational query, you essentially convert the question to an embedding and you match that with the embeddings of all of the records in that database, typically a text database, and you pull that context and you give that into the LLM as context for the query. These, to me, are very workable things in certain applications, but one could see that they essentially will merge into one block over time. This idea of memory will probably be taken care of with a construct, because embeddings, the networks already understand. The networks implement compression. In this case, we don’t want compression of the raw content.
So maybe this all merges and it becomes one thing. And so from a developer standpoint, the block is going to become larger, but I no longer have to think of them as two different things. That’s one potential path that things could go down. It’ll be interesting to see how it happens, but I don’t think five years down the road we’ll be worrying about manually setting these things up.

Mike Vizard: All right, folks. Well, you heard it here. Things are moving fast, but one of the first things I ever learned in this business is never mistake a clear view for a short distance. Hey, Amir, thanks for being on the show.

Amir Husain: Good to see you, Michael. Take care.

Mike Vizard: All right. Thank you for all watching the latest episode of the Techstrong AI video series. You can find this episode and others on our website. Please check them all out. And until then, we’ll see you next time.