In 2010, Google launched a universal translation tool called Google Translate on its Android and Chrome platforms. IOS versions soon followed and before long it was most readily available translation tool in the world. Input any text into the search bar and the software would instantly detect both the user’s language and the language of the written text, providing a full translation in seconds. It seemed revolutionary. Every smart phone user in the world would have a access to 133 languages at the touch of button. For professional translators, it looked like the writing was on the wall. In Roboto font.
They needn’t have worried. Whilst mostly accurate, Google Translate soon began to acquire a reputation for mistakes. And not just small mistakes. As powerful as the programme was, it struggled with the idiosyncrasies of human language. It couldn’t grasp context. It was baffled by double meanings. Whilst it could be relied upon to be accurate mostof the time, when it made mistakes it had the potential to make absolute howlers, many of which are simply too rude to reproduce here.
Since last November everyone has been talking about an even more powerful online tool; ChatGPT. Unlike Google Translate, this software promises a dazzling array of functionality across a range of disciplines. It can write code, draft contracts, compose poems or give you directions to a good fish restaurant in Copenhagen. It has been heralded an epoch defining piece of software, an AI so powerful that it will fundamentally change the way we work and, in the case of education, the way we learn.
So last week I thought I would put it to the test. Not a particularly rigorous or scientific test I grant you, but I simply wanted to see if it would work as a revision aid for the History A Level syllabus I have been teaching. I asked it: “ChatGPT, what was Richard III’s foreign policy towards France?” Without delay, it produced the following answer:
On first reading, this looks like an articulate, well-structured response. It gives specifics; names, dates, facts and figures seem to substantiate its analysis. Job done! Rather like Google Translate.
If it feels too good to be true, that’s because it is. Like one of Google Translate’s howlers, the software has managed to mangle the whole thing. In the opening sentence it refers to the “ongoing conflict” of the Hundred Years War, a conflict that had ended 30 years before Richard III became King. In the second paragraph, it refers to “the Lancastrians who sought to regain power”, when Richard’s greatest dynastic threat came from Henry Tudor (unmentioned). But perhaps best of all, it refers to “Richard’s…ally, Richard Neville, Earl of Warwick, who sought to install Henry VI back on the English throne”. At least it acknowledges that “these efforts were ultimately unsuccessful”, which is unsurprising given that the Earl of Warwick was killed at the Battle of Barnet in 1471 and Henry VI in the Tower of London shortly thereafter. In other words, it’s the historical equivalent of Google Translate confusing the English for “duck” with the Portuguese for “traffic warden”.
It highlights that what is most compelling about ChatGPT is not that it has all the answers, but that it feels like it has all the answers. It seems to offer just what we want, when we want it and in just the right language to convince, but without any means of verifying whether what it is saying is actually true. Ask it to provide references or footnotes for its responses and it calmly explains that it isn’t designed to do so. Other than doing the legwork to cross reference every claim, therefore, there is simply no way of knowing how many howlers each particular answer is going to contain.
ChatGPT’s genius is in using language for language’s sake. As Sam Altman, CEO of Open AI, the company behind ChatGPT, said in an interview recently: “what this system is, is a system that takes in some text, does some complicated statistics on it and puts out some more text”. Its command of style and tone is remarkable and as a drafting tool, with the right inputs, it can be incredibly powerful (although students beware, you can ask ChatGPT whether it has written something; it remembers!)
Beyond this, however, don’t be beguiled. Enjoy asking it to write the shipping forecast in the style of TS Eliot because the linguistic pyrotechnics are quite something. But for things that require real analysis and where accuracy is key, such as preparing for your GCSEs or A Levels, we would be fools to trust AI just yet.