Thore Graepel, the former Google DeepMind research lead who helped create the AlphaGo AI program, wants to apply that technology’s reasoning powers to compete against the Large Language Models (LLMs) dominating today’s AI services.
Graepel has helped launch a London-based 12-person startup called Metis Reasoning, along with British biotech entrepreneur Piers Ingram. Metis’s team includes experts with roots in DeepMind, DARPA robotics and Cambridge Consultants.
The company’s goals are ambitious: to introduce, in its words, a “neuro-symbolic architecture” that teases apart world knowledge from the reasoning process needed to interact with the world. It’s a radically different approach from LLMs, which Graepel has argued are not capable of trustworthy machine reasoning.
Metis has developed a prototype that it claims is already tackling PhD-level scientific problems. It plans to refine the technology so that it will be ready for external testing within six months and will be able to target difficult problems in drug discovery, engineering and legal analysis.
Bloomberg has reported that the company is seeking tens of millions of dollars in additional investment, though the company has already secured sufficient funding to cover initial operations.
How AlphaGo Is Different from Today’s LLMs
The DeepMind team made AI history in 2016 when its AI system beat champion Lee Sedol in a much-publicized match of Go. Go is a complex strategy game played on a 19-by-19 grid of intersections from which the player must carve out their own territory.
In their post-game analysis, the researchers focused on one game move that the computer made – later called “Move 37” – that a human player would not typically think of taking. In fact, researchers estimated that there was only a 1 in 10,000 chance that an expert human Go player would make such a move. It would prove to be a pivotal move that helped the machine win the match.
The researchers concluded that AlphaGo made the move not because it would be one that a human might make (as an LLM’s next-token prediction process might suggest), but rather because it evaluated the consequences of possible actions and mapped out the most promising ones. In other words, AlphaGo won through analytic reasoning.
The Limits of LLMs
This is the approach that Metis plans to take with its own technology. The company’s still somewhat bare website includes a foundational paper, entitled “The Age of Reasoning,” which goes into great detail about why this sort of reasoning can be superior to LLMs.
Perhaps the biggest weakness of LLMs is that they don’t maintain the “state” of the task they are trying to complete. Instead, this is the work of the harness or the agent.
“Next-token inference does not by itself provide a persistent state of knowledge,” the researchers write.
LLMs are trained to predict the next token rather than independently verify the truth of their outputs, the researchers argue. They contend that this approach contributes to hallucinations.
Worse yet, LLMs are black boxes. The answers they arrive at are inscrutable. The user doesn’t have any information about how the LLM arrived at its conclusion. If you ask an LLM how it arrived at its answer, it will construct a series of post-hoc rationalizations after the fact, the Metis researchers argue.
This flaw is particularly fatal for industries that require a high level of transparency and accountability. LLMs work well for coding, they go on to explain, because the output is easily tested. But other endeavors can suffer more from this black box mentality.
Metis’s Approach
Metis’s technology aims to maintain an independent record of what has been established, what remains uncertain and what evidence supports its conclusions. Such a record could guide progress and keep track of which portions of the problem remain to be completed.
LLMs “are particularly good at problems that can be expressed and solved through natural language, especially when they can exploit patterns learned from the very large datasets that are used for training. Their abilities improve predictably with scale,” the researchers write. “There exists, however, a significant set of problems that no amount of scaling can solve.”
We’ll see if Metis can jump to the next level of the AI game board and deliver on its promise of more reliable machine reasoning.
