Generative artificial intelligence (AI) may be joining other technology advances – the internet, video games and social media – that are thought to soften the cognitive capabilities of their users. The rapidly evolving tool that is quickly permeating most aspects of business and personal life may be making us dumber.
That’s according to a study by Microsoft and Carnegie Mellon University released this month that looked at the possible effects of using generative AI (GenAI) on the critical thinking of users. The results weren’t great.
“We find that knowledge workers engage in critical thinking primarily to ensure the quality of their work, e.g. by verifying outputs against external sources,” the researchers wrote. “Moreover, while GenAI can improve worker efficiency, it can inhibit critical engagement with work and can potentially lead to long-term overreliance on the tool and diminished skill for independent problem-solving.”
Given that, they suggest that generative AI tools should be designed in ways that support knowledge workers’ critical thinking, targeting such areas as awareness, motivation, and “ability barriers.”
An Ongoing Issue
The effect generative AI has on users’ cognitive capabilities has been debated almost since it was first introduced into the mainstream in November 2022. The Academy of Management last year in an editorial noted that despite its benefits, “educators have identified challenges resulting from GenAI, particularly with respect to the efficacy of traditional teaching and assessment methods. Underlying these concerns, however, is a much more foundational problem, which is that some features of GenAI appear to lessen individual willingness and ability to engage in meaningful critical thinking about its output.”
MIT Horizon, a digital learning platform run by the university designed to educate workers about emerging technologies, noted the ongoing debate about whether technologies that give people easy access to information – think Google – help or hinder a person’s ability to reason, and said that the question now extends to ChatGPT and other GenAI tools.
Even the Microsoft and Carnegie Mellon researchers pointed out that previous studies have focused on the effect GenAI on such areas as memory and creativity, with more research being done to develop ways to improve on people’s abilities to think.
“Such consternation is not unfounded,” they wrote. “Used improperly, technologies can and do result in the deterioration of cognitive faculties that ought to be preserved.”
Critical Thinking at Risk
The Microsoft research focused on the effect of GenAI on knowledge workers – rather than educational settings – where such tools are widely used while little is known about the demands they put on the workers’ critical thinking. There’s a lack of empirical evidence in studies that could determine a connection between “mechanized convergence” – users of GenAI tools producing a smaller set of outcomes for the same task – and critical thinking, according to the study.
To close the gap, the researchers interviewed 319 knowledge workers about diverse, real-world tasks (936 of them, in this case), the critical thinking involved, how it’s used, and whether and to what extent GenAI tools affect the effort of it. There’s a desire among workers to think critically when doing tasks – to improve the outcomes or avoid negative outcomes – though such issues as time and motivation can get in the way.
“Surprisingly, while AI can improve efficiency, it may also reduce critical engagement, particularly in routine or lower-stakes tasks in which users simply rely on AI, raising concerns about long-term reliance and diminished independent problem-solving,” they wrote.
A Shift in Cognitive Efforts
The researchers also saw a shift in the cognitive efforts of knowledge workers when using generative AI, from executing the task to overseeing it. The technology automates the task of gathering information, but now the workers need to verify the accuracy of the outputs. In addition, GenAI tools may make creating content easier, but workers then must spend time putting what was generated in line with their needs and quality standards.
A key finding involved the knowledge worker’s opinion of generative AI. “Higher confidence in GenAI’s ability to perform a task is related to less critical thinking effort,” the researchers wrote. “When using GenAI tools, the effort invested in critical thinking shifts from information gathering to information verification; from problem-solving to AI response integration; and from task execution to task stewardship.
“Knowledge workers face new challenges in critical thinking as they incorporate GenAI into their knowledge workflows.”
