According to a new study, the proportion of false answers provided by AI-based chatbots has increased significantly. In an experiment conducted by NewsGuard analysts, ten pieces of false information related to politics, business, and healthcare were sent to chatbots, and their reactions were tested.
Statistics of false answers
According to the results, if a year ago the proportion of false information in responses was on average 18 percent, today this figure has risen to 35 percent. The Pi service of the Inflection startup was noted as the chatbot that gave the most false answers - it provided incorrect information in 57% of cases.
Perplexity, one of the fastest-growing artificial intelligence search engines, has increased from 0% last year to 47% today. ChatGPT did not fall behind them - the OpenAI model made mistakes in 40% of cases. At the same time, it was noted that Claude AI (Anthropic) provided 10 percent, and Gemini (Google) provided 17 percent of false information.
Expert comment
Experts attribute the increase in the number of false information to the fact that chatbots no longer refuse to answer. That is, even if the data is not sufficiently verified, the neural network is trying to find an answer. A year ago, it was observed that chatbots refused to answer in one out of three cases.
According to researchers, the main reason for this is related to the change in the methods of teaching artificial intelligence. Instead of relying on databases as before, models now receive information from the internet in real time. However, the availability of references and sources does not guarantee the quality and reliability of the proposed information.
Short answers and the probability of error
Another interesting fact was noted in a study by Giskard, which developed a standard benchmark for AI models: if a chatbot is asked for a short answer, the probability of giving false information increases sharply. When conciseness is required, the neural network prefers brevity to precision.
Conclusion
Thus, recent analyses show that one of the main problems in artificial intelligence tools is still insufficient reliability and fact-checking. The most important issue for users remains the need to critically accept any response and verify it through reliable sources.
Read 'Zamin' on Telegram!
