Chatbot ‘Eugene Goostman’ passes Turing Test, Warwick claims

June 9, 2014

(Credit: Vladimir Veselov and Eugene Demchenko)

The Turing Test was passed for the first time by a chatbot called “Eugene Goostman” on Saturday by convincing 33% of the human judges that it was human, according to Professor Kevin Warwick, a Visiting Professor at the University of Reading and Deputy Vice-Chancellor for Research at Coventry University, in a statement.

The Turing Test, proposed by mathematician and code-breaker Alan Turing in a 1950 paper, investigates whether people can detect if they are talking to machines — in this case, a chatbot simulating a 13-year-old boy — or humans. If a computer is mistaken for a human more than 30% of the time during a series of five minute keyboard conversations it passes the test.

Beyond that, the test design and protocols are not precisely defined, including the knowledge domain of the bot (a 13-year-old would not have to show as much knowledge as an adult, Eugene’s creator Vladimir Veselov points out), the competence of judges, and other factors. The “Turing Test” words have been applied to similar competitions around the world, Warwick notes.

“However, this event involved the most simultaneous comparison tests than ever before [sic], was independently verified and, crucially, the conversations were unrestricted,” said Warwick. “A true Turing Test does not set the questions or topics prior to the conversations. We are therefore proud to declare that Alan Turing’s Test was passed for the first time on Saturday.”

The test was staged during Turing Test 2014 held at the Royal Society in London on the 60th anniversary of Turing’s death, and produced by the University of Reading  in partnership with RoboLaw, an EU-funded organization examining the regulation of emerging robotic technologies. The Eugene software was developed in Saint Petersburg, Russia by Veselov, Eugene Demchenko, and Sergey Ulasen.

Among the judges tasked with separating the human and computer participants were the actor Robert Llewellyn, who played robot Kryten in the sci-fi comedy TV series Red Dwarf, and Lord Sharkey, who led the successful campaign for Alan Turing’s posthumous pardon last year.

“Of course the Test has implications for society today,” said Warwick. “Having a computer that can trick a human into thinking that someone, or even something, is a person we trust is a wake-up call to cybercrime. The Turing Test is a vital tool for combating that threat. It is important to understand more fully how online, real-time communication of this type can influence an individual human in such a way that they are fooled into believing something is true…when in fact it is not.”

You can test Eugene Goostman yourself here (if you can get through the congestion).

A related Turing Test event, the 2014 Loebner Prize Competition, will be held Nov. 14, 2014 in Bletchley Park, UK, the former central site of the United Kingdom’s Government Code and Cypher School (GC&CS).

Trailer to University of Reading Turing test event, The Royal Society, 7 June 2014. Animated short created by Harjit Mehroke.