By now, we’ve all seen the findings from the recent MIT report: 95 percent of AI pilots are failing to generate real value. This may come as a surprise to some, given the hype surrounding AI and the amount of talent, time, and money companies are pouring into the technology.
But personally? This stat didn’t shock me in the slightest. The reality is, when AI doesn’t have context around the information it’s processing, it tends to generate a lot of garbage.
I experienced this recently when I used a large language model (LLM) to conduct deep research into how to improve the product at our startup. The answers it provided read with conviction, but the hallucinations were so extreme that acting on the output could have been detrimental.
The LLM confidently described fantastical product features that didn’t exist and imagined capabilities that were illogical and unnecessary. And other AI tools weren’t any better; they generally cap out at around 50 sources and present incomplete information.
Despite AI’s propensity to hallucinate, and that striking stat, this technology is indeed the future. You’re already using it, your teams are already using it, and its transformative potential (when leveraged correctly) cannot be ignored.
However, technology leaders need to take a different approach to realize its full value. It’s time to start applying engineering principles to AI, just like one would with any other piece of software infrastructure.
Here’s how to “think like an engineer” when implementing AI for better outcomes:
Prioritize Repeatability, Testability, and Trust
When engineers build systems, they test those systems and then run them in repeatable, predictable ways. As technology leaders, we need to make sure that any implementation of AI follows those same principles.
This is especially critical since AI is now more accessible than ever. In theory, someone could spin up a chatbot, plug in an API or two, and have a working AI prototype within a matter of hours. But once that prototype reaches production, they’ll experience a slew of issues (unpredictable costs, a lack of observability, brittleness, hallucinations, etc.) if they didn’t take the time to apply sound engineering principles from the beginning.
At our own startup, the hardest part of building our AI assistant wasn’t connecting to a model, it was building the “scaffolding” to support it. The real work was in writing the model into our tech stack, creating usable APIs (more on this shortly), and managing the constant iteration and debugging that subsequently took place.
Applying engineering principles from the start was integral to our success, and it’s the key to other companies finally realizing the value of their AI investments.
Get Your APIs AI-ready
APIs serve as the building blocks for every successful AI application: if your AI tools aren’t connected to the right services and data sources, they can’t produce useful outputs. And even more alarming: if your APIs are poorly structured or documented, the AI tool isn’t going to pause—it’s going to fail.
These issues at scale put the entire organization at risk. Despite this fact, companies are still designing APIs for use by human developers, when they should actually be tailoring them to collaborate with humans, agents, and models.
Get your APIs AI-ready by moving away from fragmented documentation and ad hoc processes. Instead, opt for structured, standardized systems that both humans and machines can easily understand.
The first step is to conduct an audit of your current API landscape to uncover any gaps that will impede AI integration. Then, you should move all collaboration to shared workspaces, and enforce consistent, machine-readable documentation so AI systems can reliably discover, understand, and consume your APIs.
With these foundations in place, your APIs will be primed and ready for use by AI agents so they can drive innovation across the business.
Go Back to Engineering Basics
Before embarking on AI projects, ask yourself: How would an engineer approach this?
They’d set goals, plan sprints, report results along the way, and pivot as necessary. Applying engineering basics to AI is the best way to ensure the ROI of these initiatives and build trust in the products themselves.
It’s crucial to zero in on areas that engineers hyperfocus on, such as cost, latency, error rates, and user outcomes, in order to win the confidence of investors and customers alike. These aren’t new challenges, but the explosion of AI has underscored their importance.
As technology leaders, we must lead the charge by encouraging teams to perfect the basics instead of getting caught up in the latest new models. This means focusing on building clear interfaces and modular architectures, conducting rigorous testing, and creating tight feedback loops.
AI will undoubtedly power the most significant technological breakthroughs we see in the coming years. But just as software demanded the right approach to achieve breakthroughs, so too will AI. And it all starts with thinking like an engineer. Here’s to ushering in a new era where humans, AI agents, and APIs come together to drive innovation like never before.
