ZAP // Dall-E-2
Leading Artificial Intelligence models demonstrate cognitive impairments similar to early symptoms of dementia when assessed with the MoCA test.
A new study has revealed that almost all major Artificial Intelligence (AI) models, or “chatbots,” show signs of mild cognitive impairment when tested with assessments used to detect early dementia.
According to , scientists evaluated the cognitive abilities of the main large language models (LLMs) publicly available – ChatGPT versions 4 and 4o (developed by OpenAI), Claude 3.5 “Sonnet” (developed by Anthropic) and Gemini versions 1 and 1.5 (developed by Alphabet) – using the test Montreal Cognitive Assessment (MoCA).
This test is widely used to detect cognitive impairment and early signs of dementia, usually in older adults. Through a series of tasks and short questions, it assesses individuals’ abilities, namely attention, memory, language, visuospatial abilities and executive functions.
The maximum score is 30 points, with a score of 26 or more being generally considered normal.
In the assessment, the ChatGPT 4th scored highest in MoCA test (26 out of 30), followed by ChatGPT 4 and Claude (25 out of 30). Gemini 1.0 had the lowest score (16 out of 30).
All chatbots performed poorly in terms of visuospatial abilities and executive tasks, how to connect circled numbers and letters in ascending order, and the clock drawing test. The Gemini models failed the delayed recall task (remembering a sequence of five words).
In the visuospatial tests that followed, chatbots failed to demonstrate empathy or accurately interpret complex visual scenes.
In addition to these findings being just observational, it is necessary to take into account that there are numerous differences between the human brain and the great language models.
However, the team points out that the uniform failure of all major language models in tasks that require visual abstraction and executive function he can prevent its use in clinical settings.
The scientific article with the results was published in The BMJ.
