There is no question that generative AI and deep learning are the defining technological changes of the 2020s. As we look at the trajectory this technology took over the past several years, and identify upcoming trends for the next few years, what do we learn? How can we maximize the positive impact on the world from these technologies?
I would argue that AI is making an accelerated transition from being a “white coat in research lab” pursuit to a ubiquitous underpinning for activities across all walks of life. Whereas researchers in academia have been debating AI since the 1950s and the technology has had an increasing level of use in actual products ever since, the level of maturity (and the capabilities of today’s technology, let alone those of tomorrow) are such that everyone wants to be in on the game. How will this transition work? Relatedly, a lot of investment has gone into developing the core technologies enabling this revolution. How are we going to add value widely to the world to recoup those investments?
Infrastructure for Experts vs. Infrastructure for All
First, let’s examine the compute infrastructure that has developed in the ML/AI field and how it can better support this transition from the lab to the world. It is interesting to start at this level because the development to date has been very focused on the compute requirements of deep learning. This is for a good reason. Advances in accelerated large-scale computing are what enabled much of the progress in the core technologies that have driven the AI trend, and much of the value created has been in this space (look no further than NVIDIA, though there are other winners).
Yesterday’s AI compute infrastructure was optimized for exploration. When you explore a technology, you focus on expanding the envelope of what’s possible. You want to be able to tweak all aspects of the technology, and even small improvements in certain areas can be of value, as they advance the state of the art. As a result, the way ML/AI has been operating to date has been what I would characterize as “the HPC approach”: Tightly coupled, specifically architected compute infrastructure that is often dedicated (and built to suit) for a task (like pre-training a foundational model) and where the software used is largely built in-house. This is similar to how supercomputers in places like national labs have been built and used in the past.
The challenge with this type of infrastructure is that it is both difficult to use and expensive to build. It is expensive because of the lack of economies of scale, and because the hardware is built to live on the bleeding edge of technology, where there’s a huge premium to pay for the “latest and greatest”. It is difficult to use because it exposes a lot of low-level tuning knobs, and is often immature and therefore brittle to maintain. To be widely available and useful, AI has to make the transition to a type of infrastructure that is more typical of what we see in classic cloud compute today. Such infrastructure emphasizes economies of scale, maintainability and commoditization/standardization over pushing the boundaries in all dimensions all at once. What would that look like for deep learning/GenAI?
Commoditization and Abstraction
To make a technology accessible to the masses, a level of independence needs to be created between the supplier of the technology and the consumer. One of my favorite examples of this in my mind is the standardization of the electrical grid. In the dawn of the electrical revolution, different power companies had different grid structures, voltage levels and even DC vs. AC grids. Today, we know that if the plug fits the socket, our appliance will work. While compute infrastructure is somewhat more complex, the concepts are the same: The basic infrastructure needs to be abstracted and commoditized. The main reason is that this allows the proliferation of the basic technology, which will spur innovation in its use, separately from innovation in the infrastructure itself (which can and will, continue). The electrical grid analogy is that once a grid was standardized, innovation could happen in the appliance space, and in quick succession, we got radios, refrigerators, etc. All those could not have been successful without a commoditized and abstracted infrastructure.
If we go back to AI and its proliferation, the question to ask is what can we do, as infrastructure providers, to hasten this process? Who are the innovators? What are they looking for?
One of the reasons for the success we’re seeing with LLMs is that they are very versatile: A single model can be used for summarization, translation, reasoning, and many other tasks, in many contexts. At the same time, as we have agreed, LLMs are difficult and expensive to build, and the expertise required to train and build them is in far shorter supply than the expertise of building software products. We are starting to see innovation happening in how and where Generative AI is used rather than in the base LLM. These innovations still require large amounts of compute, large amounts of data, etc., but the innovators are no longer experts at compute or large-scale data. The first round of commoditization and abstraction that we see in this space is things like the proliferation of the OpenAI inference API as a de facto standard for using LLMs. This is definitely a step in the right direction, and one that allows innovation to happen below that surface (such as by chip startups) independently of innovation above it, but it covers a tiny fraction of the abstraction that we need for a complete end-to-end ML ecosystem.
The New AI Innovation Ecosystem
It is clear that to create true value and innovation in ML/AI, we need to establish a new ecosystem. One in which innovations can happen in parallel in different layers of the stack, enabled by clear, stable interfaces between those layers. The other component of the ecosystem, beyond abstractions and interfaces, is a place where the ecosystem can thrive and grow – a place where innovators in different areas and levels of the stack can come together with the consumer of these innovations to bring real value to society.
