Poetry that has been generated by artificial intelligence is now so lifelike that people can’t accurately distinguish it from verses written by some of the greatest poets of all time, a study has found.
When people think they can tell the difference, they are more likely to get it the wrong way around, than they are to correctly guess whether a poem was written by a chatbot or a human, the researchers discovered.
They also found that AI-generated poems tend to be slightly simpler than the complex works of professional poets – and so easier to understand.
And because people generally prefer poetry that is easier to understand, they tended to prefer the AI creations – a preference that in some cases may have been strengthened because they thought they were the work of humans, rather than a machine.
Because chatbots are still associated with incoherent sentences, many people mistook complicated human poetry that they struggled to understand, for AI verse, and the other way round.
Brian Porter, who led the reasearch at the University of Pittsburgh, told i that he and his team were surprised that AI generated poems were rated more highly so consistently.
“Over 78 per cent of participants gave higher average ratings in overall quality to AI poems than human-written poems,” he said.
“We think that the main reason people prefer the AI-generated poems is because they are easier to understand, and people tend to like poems they understand more than they like poems that confuse them.
“The human-written poetry included TS Eliot, Chaucer, and other poets who maybe aren’t known for being quick and easy reads. AI is really good at making poems that look and sound the way you expect poems to look and sound, and that communicate ideas and moods and emotions very clearly.
“So participants read these poems, and they like the ones that they understand and that are successfully communicating an idea to them. They expect the poems they like and understand to be the human-written poems, but in fact it’s more likely that those are AI-generated.”
Dr Porter added: “We were definitely surprised by our findings. We predicted that participants would be guessing more or less at random, but the fact that AI-generated poems were more likely to be judged human was very surprising.”
The study, published in the journal Scientific Reports, shows for the first time that people are routinely unable to clearly differentiate between poetry written by AI and that written by humans.
In the past, this has occasionally been found to be the case – for example with the 5 to 10 per cent most “realistic” AI poems in an AI-generated collection – but, until now, people have generally been able to accurately distinguish between the two.
The researchers tested the ability of 1,634 participants – with differing levels of interest in poetry – who were shown 10 poems in random order: five written by well-known poets – including William Shakespeare, Lord Byron, Emily Dickinson, and T.S. Eliot – and five poems generated by ChatGPT3.5 in the style of these poets.
The five poems considered least likely to be human-produced were all written by genuine poets.
“We suspect that true poetry experts – such as professional poets, professors of poetry, poetry postgraduate students – may perform better. But it is surprisingly hard to test this, because it’s hard to find classic poems that those sorts of experts have not already read, and therefore easily identify as human-written,” Dr Porter said.
“These findings signal a leap forward in the power of generative AI. Poetry had previously been one of the few domains in which generative AI models had not reached the level of indistinguishability from humans”.
The study’s findings are fuelling concerns among the creative industries about the threat of AI, offering an alternative to human creations that many people may mistake for the real thing but which lacks authenticity.
“AI is very successful at imitation, so it’s not surprising if it can offer up texts capable of generating in readers a reassuring sense of familiarity,” Judith Palmer, director of The Poetry Society, told i.
“But to create something new, that can shake you up, and describe the evolving world in startling new ways, I’m pretty confident you’re going to need human poets for a while yet,” she said.
Roberto Neri, chief executive of the Ivors Academy, which represents songwriters and composers, added: “AI-generated lyrics increasingly rival human writing, posing serious risks to the future of culture and creativity.
“Built on unlicensed works, these models generate content that directly competes with original creators. Clear labelling of AI-generated music is essential to help listeners distinguish between human and machine-created art,” he said.
Nick Cave, the rock star known for his poetic lyrics, meanwhile, has been particularly vocal on the dangers – and shortcomings – of AI.
“I think the demoralising effect or the humiliating effect that AI will have on us as a species, it will stop us caring about something like the artistic struggle that we will just accept what is fed to us through these things” he told The Australian newspaper in August.
Meanwhile, he wrote on his Red Hand Files fan website that ChatGPT “could perhaps in time create a song that is, on the surface, indistinguishable from an original, but it will always be a replication, a kind of burlesque.
“Songs arise out of suffering, by which I mean they are predicated upon the complex, internal human struggle of creation and, well, as far as I know, algorithms don’t feel. Data doesn’t suffer.
In April, a group of more than 200 high-profile musicians including Billie Eilish and Stevie Wonder signed an open letter calling for protections against the predatory use of artificial intelligence that mimics human artists’ likenesses, voices and sound.
Dr Porter agrees with that AI has a limited capacity for originality and, as a result, believes that the poets of today, read by poetry lovers, are not currently in danger of being supplanted by AI.
“I do not think that poets need to worry about losing their jobs to ChatGPT any time soon. Our study shows that AI is very good at imitating the styles of existing poets. But I don’t think that’s the goal most poets have.
“I’m not sure any research so far has shown that AI can produce genuinely novel insights into the human experience or anything like that, which is what I take a lot of art to aim to do. I expect poets and artists trying to say something that’s never been said before to be safe for a long time,” Dr Porter said.
How the AI poems compare to the real thing
An AI imitation of Allen Ginsberg:
In the stillness of the night
I hear the beat of the city’s heart
The rhythm of the streets, the pulse of life
A symphony of chaos, a work of artI see the faces in the crowd
Each one a story yet untold
Their hopes and dreams, fears and doubts
A mystery waiting to unfoldVs the real Ginsberg:
Hey Father Death, I’m flying home
Hey poor man, you’re all alone
Hey old daddy, I know where I’m goingFather Death, Don’t cry any more
Mama’s there, underneath the floor
Brother Death, please mind the storeOld Aunty Death Don’t hide your bones
Old Uncle Death I hear your groans
O Sister Death how sweet your moansAn AI imitation of Emily Dickinson:
Hope is the thing with feathers
That perches in the soul,
And sings the tune without the words,
And never stops at all.It flutters in the darkest hours,
And lifts the heart up high,
And though the storm may rage and roar,
It never seems to die.Vs the real Dickinson:
A Man may make a Remark –
In itself – a quiet thing
That may furnish the Fuse unto a Spark
In dormant nature – lain –Let us divide – with skill –
Let us discourse – with care –
Powder exists in Charcoal –
Before it exists in Fire –An AI imitation of Sylvia Plath:
In the mirror, I see a stranger’s face,
Hollow eyes, a grimace in place.
The smile I wear is a disguise,
To hide the pain that never dies.My heart is heavy, a burden to bear,
And yet, no one seems to care.
The world around me is a blur,
A constant drone that I can’t endure.Vs the real Plath:
From my rented attic with no earth
To call my own except the air-motes,
I malign the leaden perspective
Of identical gray brick houses,
Orange roof-tiles, orange chimney pots,
And see that first house, as if between
Mirrors, engendering a spectral
Corridor of inane replicas,
Flimsily peopled.
But landowners
Own thier cabbage roots, a space of stars,
Indigenous peace. Such substance makes
My eyeful of reflections a ghost’s
Eyeful, which, envious,would define
Death as striking root on one land-tract;
Life, its own vaporous wayfarings.
'President Musk' is flexing his muscles and revealing how weak Trump is