The number of AI models is rapidly accelerating, and they are becoming more open and more expensive to run, and are being embraced by industry more than by government or academic institutions.
They’re also racing toward or doing better than humans in a range of benchmarks and while private investment in AI is slowing a bit, it’s exploding when it comes to generative AI. In addition, the technology is causing people to worry about its long-term impact on society and fueling concerns among governments around everything from privacy and misinformation to employment and intellectual property rights, which are leading to more regulations.
Those are some of the top findings by researchers at Stanford University in its massive annual AI Index, a 300-plus page report aimed at giving AI players, policymakers, business leaders and the general public rolling snapshots of what’s happening in the emerging field in areas like AI capabilities, investments and deployment and development.
In introducing the report, the researchers looked at the accelerated pace of innovation in AI since the introduction of generative AI to the public in late 2022, noting that 10 years ago, even the top AI systems couldn’t classify objects in images at a human level, struggled with comprehending language and couldn’t solve math problems. These days, AI systems routinely surpass human capabilities in such areas as image classification, basic-level reading comprehension and understanding the English language.
Improving Performance
AI also does better at visual reading and multitask language understanding, and is closing in on human performance in competition-level math.
“As AI has improved, it has increasingly forced its way into our lives,” AI Index co-directors Ray Perrault and Jack Clark wrote. “Companies are racing to build AI-based products, and AI is increasingly being used by the general public. But current AI technology still has significant problems. It cannot reliably deal with facts, perform complex reasoning, or explain its conclusions.”
AI – particularly generative AI – is in its early phases, with Perrault and Clark writing that it “faces two interrelated futures. First, technology continues to improve and is increasingly used, having major consequences for productivity and employment. It can be put to both good and bad uses. In the second future, the adoption of AI is constrained by the limitations of the technology.”
Either way, the development will continue, as will adoption, investments and government involvement, from encouraging university R&D and private investments to addressing security, privacy and other concerns.
Open Models Come with Performance Hit
The notable findings in Stanford’s AI Index include the turn toward open source AI models, which will make it easier for organizations to adopt the technology, even if such models don’t always perform at the same level as closed models.
In 2023, organizations released 149 foundation models – more than twice the number the year before – and 65.7% of them were open, able to be freely used and modified by anyone. In 2022, 44.4% of such models were open; in 2021, it was 33.3%.
That said, closed models still performed better. Looking at 10 benchmarks, closed models did 24.2% better with a range of differences with individual benchmarks, from 4% better on mathematical jobs like GSM8K to 317.7% better on agentic tasks like AgentBench.
Google outpaced its rivals in releasing foundation models last year, with 18 in the fold – including Gemini and RT-2 – compared with 11 for Meta, nine for Microsoft and seven for OpenAI. The biggest beneficiaries were corporations, with the industry space accounting for 72% of new foundation models with 108. Academia had 28.
Costs Keep Going Up
Companies are implementing AI in various parts of their businesses, including contact centers, personalized content and customer acquisition. About 55% said in surveys last year said they were using AI, up from 50% in 2022 and 20% in 2017.
The dominant use by industry in part can be attributed to rapidly rising costs, according to the researchers.
“One of the reasons academia and government have been edged out of the AI race: The exponential increase in cost of training these giant models,” they wrote in an accompanying blog post. “Google’s Gemini Ultra cost an estimated $191 million worth of compute to train, while OpenAI’s GPT-4 cost an estimated $78 million. In comparison, in 2017, the original Transformer model, which introduced the architecture that underpins virtually every modern LLM, cost around $900.”
While governments may continue to encourage private investment in AI, the total amount invested has dropped since 2021 – when it hit $132.36 billion – to $95.99 billion last year. However, that’s not the whole story. Perrault and Clark wrote that “investment in generative AI skyrocketed. More Fortune 500 earning calls mentioned AI than ever before, and new studies show that AI tangibly boosts worker productivity.”
Private investment in generative AI 2022 – OpenAI kicked off the craze when it released ChatGPT in November of that year – reached $2.85 billion. Last year, private investors put $25.23 billion into the emerging technology.
U.S. is Investing the Most in AI
The United States was by far the largest region for funding AI, with $67.22 billion of private investments. China, the U.S.’s chief economic rival, was a distant second, with $7.76 billion. It looks the same when viewing through a 10-year lens, with private AI investments in the United States reaching $335.2 billion, to China’s $103.7 billion.
The AI Index also tracked individual and governments concerns about AI, finding that 66% of Gen Z ages – those born between the late 1990s and early 2010s – believe AI will significantly affect their current jobs, while only 46% of Baby Boomers (born between 1946 and 1964) feel the same. And the nervousness about AI products and services is spread around the globe, with more than 60% of people in such countries as Australia, Great Britain, Canada and the United States concerned.
In countries like India, France, Spain and Belgium, the rates are more than 50%.
The Stanford researchers also saw a growing trend of government regulation in the United States, pointing to copyright guidance issued by both the Copyright Office and Library of Congress and the Securities and Exchange Commission creating a cybersecurity risk management strategy and incident disclosure plan.
