It was a beautiful week in Boca Raton, as experts from all over the world arrived for Operationalizing AI, where they put their heads together to come up with new ideas relating to AI, with a focus on DevOps. As I listened to the different speakers, I learned that one important aspect to a large language model is a topic known as, “in-context learning.”
What is In-Context Learning?
At a most fundamental level, in-context learning simply means that a generative AI, after it’s been initially trained, will continue to learn. Indeed, even humans do this; even after we learn something, if we allow ourselves to, we continue learning more about a topic.
An AI tool such as ChatGPT uses an architecture called “generative pre-trained transformer,” or GPT for short. An important aspect of this is pre-training, where the system is trained with massive amounts of information before it can be used. Once the system is up and running, as users continue to interact with it, it continues to learn new information via the context of the question and the answers that take place with the users.
This is a radical departure from such traditional means of “looking up data” such as Google, where the “conversation” consists of one search and then one response (a list of sites). ChatGPT, however, is capable of continuing a conversation. Savvy users can interact with a GPT by providing additional data that gets added into the GPT’s overall knowledge base. This happens through multiple Q&A sessions.
A ChatGPT Example of In-Context Learning
For example, a user might ask ChatGPT what its favorite movie is. ChatGPT, of course, isn’t a human, and doesn’t have a favorite movie. So instead, ChatGPT would likely state that it doesn’t have a favorite movie. But it might prod the user by asking a question of its own: “What is your favorite movie?” The user might provide an answer in the input box, after which ChatGPT might provide additional information about that movie. But it might follow that with a question, such as, “What is your favorite part of the movie?” And the conversation would continue as ChatGPT continues learning.
How Reliable is In-Context Learning?
Because some of what ChatGPT knows comes from the users, this means we can’t always rely on tools like ChatGPT to always give us correct results. Interestingly, ChatGPT allows for corrections. You can ask it pretty much anything; but if you find that it’s wrong, you can tell it that it’s wrong and why. It will then attempt to correct itself.
But of course, ChatGPT doesn’t particularly know if the correction you’re giving it is truly correct. Remember, a GPT is based strictly on language models and textual predictions. It doesn’t technically “know” things. And as users provide it more data that gets absorbed into its massive model, that data may or may not be right.
This is why we can’t yet rely on tools like ChatGPT to always be correct.
Consider, for example, what ChatGPT knows about prime numbers. When asked to list prime numbers, does ChatGPT actually calculate them? Not at all. Everything it knows is stored as word embeddings, including anything it might know about math. ChatGPT is not capable of testing out whether a number is a prime. It doesn’t have built into its code an algorithm for determining whether a number is prime.
To answer a question on whether a number is prime, ChatGPT uses language processing, looking for associations and drawing on its body of word embeddings to locate the answer from what it has learned. Indeed, I asked ChatGPT whether a seven-digit number is prime, and it gave me the wrong answer. It said the number is not prime; I replied that the number is, in fact, prime. ChatGPT’s response was worded as an apology, and it said that it was wrong and that I was correct – the number is indeed prime. It likely added that to its knowledge.
Except I typed the number wrong. And ChatGPT was in fact correct to begin with; the number I gave it was not prime, yet I may have convinced ChatGPT the number is prime. Can ChatGPT actually verify if my assertion was correct? No. It does not possess code to determine if a number is prime. Everything it knows is about word associations.
You Need Governance
Hence, while in-context learning is an important part of an LLM AI… proceed with caution. Clearly ChatGPT has “learned” about prime numbers, but that information may or may not be correct.
What this means is you probably won’t want to rely on a public-facing GPT for processing your information; instead, your business or organization would likely want to leverage an internal GPT where only specific, well-vetted information is fed into it. From there, the database will grow and be able to make predictions based only on the data it was given. Those predictions, then, must be tested for accuracy and integrity, with strict security policies – in other words, strong data governance.
