Generative AI is changing faster than perhaps any other digital technology in history. Over the past year, it’s dazzled the world with support for all sorts of groundbreaking business use cases. Looking ahead to the next few years, LLMs will continue to mature and move beyond web and traditional enterprise data. The next frontier for GenAI is at the edge.
In these environments, generative AI will learn from sensor data just as it currently does from text data, giving rise to exciting new applications well beyond traditional AI/ML at the edge. It will also allow organizations to ask questions and interact with sensor data in ways that haven’t been possible.
Gen AI at the Edge: Evolving Faster Than Nature
The leading LLMs are not just getting smarter, they’re also getting smaller, enabling them to run in edge environments. At the same time, improvements in hardware have lowered the price and energy consumption of edge computing while also speeding it up. Putting all of these developments together, it means that soon generative AI at the edge will not just be technically possible, but also economically viable.
So how will these models progress to support edge deployments? It’s helpful to look at biology. In nature, streams of data are processed in layers. Humans perceive the world with sight, sound, smell and touch. For example, the human eye detects edges (a pattern in space) and movement (a pattern in time) before the “signal” ever reaches the “thinking” part of the brain. Large language models do the same, with the GPT families notably starting around 100 layers.
Text models like GPT-4 (before they could analyze multi-modalities like video or sound) receive highly processed signals at their lowest level. A single word of text is already highly processed – it’s a concept representing an object that may be made of other objects. As these models learn to process streaming sensor data, these capabilities will push AI closer and closer to reality itself – to lower-level sensors.
The patterns appearing on these sensors – whatever their complexity – will serve to train new lower layers. This is the power of “auto encoding” which is at the heart of self-teaching neural networks that learn from experience, without the need for direction.
Lower layers are more tightly coupled to the physical world. This allows efficient, fast and accurate capture of reality. The lowest layers of an LLM similarly need to be more aware of the real world (i.e. movement, position), while higher layers make “sense” of it (i.e. danger, opportunity). In fact, Anthropic recently pointed out that their Sonnet model does exactly that, opening a gateway to the elusive “interpretability” of LLMs that scientists have been seeking.
Biology took millions of years to evolve these capabilities, but generative AI will reach them in just a few years.
Edge LLMs in Action
GenAI will enable a range of powerful use cases at the edge. Consider the following example in manufacturing. A factory manager is using LLMs to analyze production data. They’ve connected sensors to their machinery, gauging signals like temperature, vibration and operational status. In addition, they’ve set up cameras to monitor production lines and detect defects. They do this for 1,000 machines and ask a model to learn.
The model will learn factors that influence production efficiency, their combinations, the impact of operational timing and more. If the factory manager then gives the model control over machine operations, maintenance schedules, quality control processes, etc., they can ask it to optimize for production output, minimize downtime or find a balance that maximizes both efficiency and product quality.
Here are some other examples of the insights LLMs at the edge will deliver:
- Retail: Should we adjust our product placement in stores? The LLM will answer based on data from in-store sensors tracking customer movements, dwell times and product interactions.
- Supply chain: Are there bottlenecks in our warehouse operations? The LLM will answer based on data from sensors monitoring the movement of goods, equipment usage and worker activities within the warehouse.
- Logistics: Which delivery trucks need servicing? The LLM will answer using data from sensors monitoring vehicle health, such as engine performance, tire pressure, brake wear and oil levels.
Gen AI vs. Traditional Machine Learning at the Edge
Of course, AI and machine learning have existed at the edge for a decade, supporting things like self-driving cars and smart surveillance. At their core, LLMs fundamentally are traditional machine learning.
However, LLMs are a very specific kind of machine learning known as Transformers. These are different from traditional machine learning algorithms like Support Vector Machines or Decision Trees in a few ways. But above all, Transformers are auto-encoding, meaning they distill their own learning from whatever data is provided, while other approaches generally require curated, “labeled” data.
This is a bit like saying that Transformers can learn from experience, while other machine learning models only learn from school.
For sensor applications, the auto-encoding nature of LLMs is important because we don’t yet have good training sets for sensor data. We have mountains of data that we traditionally wire into a screen for a human to look at. Unless some human has recorded thoughts that go with the data, we don’t really have a way of training a model that can’t teach itself.
A Transformation Still in the Making
Generative AI is advancing rapidly, but it will be a few years until LLMs can effectively interact with sensor data. However, years of machine learning development – long before the birth of ChatGPT – have provided a foundation that will allow us to bring large language models to the edge and deploy them in full production this decade.
And that will be utterly transformational. LLMs at the edge unlock use cases in almost every vertical – from manufacturing to retail to supply chain management – that traditional machine learning at the edge can’t support.
