AI is misleading humans: Study reveals unexpected risk

With the development of modern technologies, AI systems are becoming an integral part of our daily lives. However, a new study conducted by scientists from three prestigious universities in France and Italy has presented unexpectedly alarming conclusions. It turns out that relying on AI advice significantly impairs a person's ability to objectively assess their own knowledge. This is reported by Ixbt.com reports. states.

During the experiment, the correlation between participants' confidence in their knowledge and their actual results was studied. The results showed that individuals using AI assistance struggled to admit they were wrong and tended to accept incorrect answers as truth. Most interestingly, while their confidence in their own answers increased sharply, their accuracy decreased.

Confidence is rising, while accuracy is falling

Researchers asked participants complex questions and divided them into two groups. Members of the group working without AI assistance openly admitted they did not know the answer in 44 percent of cases. However, in the group using system prompts, this figure dropped to just 3 percent. In other words, instead of admitting they did not know, people preferred to blindly trust the incorrect information provided by AI.

The numbers are even more surprising: the accuracy of those using AI dropped from 27 percent to 9 percent, but their confidence in being correct rose from an average of 30 percent to 76 percent. According to ixbt.com, the experiment deliberately used questions that modern AI models might get wrong. For example, visual details such as the color of the sports uniforms of the characters in the movie "Bend It Like Beckham" were requested.

Financial incentives did not help either

Scientists used the Gemini 1.5 Flash model in the experiment because it is known that this model often makes mistakes in these types of tasks. The authors of the study even tried to incentivize participants with a cash reward for correct answers. Even with a financial interest, the situation did not change significantly: the share of those who admitted they did not know rose from 3 percent to 8 percent, and accuracy rose to only 16 percent. These figures remained significantly lower than the results of the group without AI.

According to scientists, a person's ability to say "I don't know" is very important, as it means understanding the limits of one's own knowledge. The constant presence of AI and its immediate answers to any question dampen a person's desire to acknowledge their own doubts. This leads to a decline in critical thinking skills.

This issue can be particularly dangerous for children and young people whose critical thinking has not yet fully developed. It is warned that excessive reliance on AI systems could completely eliminate people's ability to verify information and recognize their own mistakes in the future. The authors of the study urge not to lose vigilance when using technology.