ChatGPT has surpassed humans at writing short stories

A new scientific study led by researchers at Villanova University found that short stories created by artificial intelligence were rated higher by readers than texts written by humans. During the experiment, ChatGPT's texts scored particularly well for their appeal and ability to engage readers. This once again confirms how rapidly the text-generation capabilities of modern generative artificial intelligence technologies are advancing. This was reported by Ixbt.com.

According to ixbt.com, the first experiment involved 1,682 participants aged 18 to 81. They were shown three human-written stories taken from well-known literary magazines and collections, along with three corresponding texts written with the help of artificial intelligence. The readers were told who had written each text, although this information was deliberately presented incorrectly in some cases. On average, the stories generated by the neural network received higher scores than the human-written ones for quality and engagement.

The Author Deception Effect and Psychological Factors

The study showed that assumptions about who had written a text played a decisive role in its evaluation. Participants gave higher ratings to texts they believed had been written by humans, regardless of their actual origin. As a result, the highest scores went to ChatGPT stories that participants mistakenly believed were written by humans. This indicates that human psychology contains a certain stereotype regarding products created by artificial intelligence.

In the next stage, the researchers tested whether readers could identify the origin of a text on their own without any clues about its author. One additional experiment involved 424 participants, whose accuracy in identifying the origin was just 39.93 percent. This result was even lower than the level expected from random guessing.

AI-Generated Texts Are Becoming Harder to Distinguish

In a second similar experiment, 481 people were tested, and the accuracy rate was 51.97 percent. Experts noted that this figure was also statistically indistinguishable from random guessing. This means that, for the average reader, distinguishing between a text written by a neural network and one written by a human is an almost impossible task.

The study's authors specifically noted that the experiment was not designed to compare literary skill, originality, or the ability to write large-scale works. It examined only two aspects: readers' assessment of text quality and how difficult it was to determine the text's origin. The results indicate that, without specialized detection methods, people are unable to reliably distinguish texts written by generative artificial intelligence.