Oh expensive gruntbuggly…
There, in the wood, among trees,
He sees shadows on the way, hears voices,
hears the wind and the rustling of leaves;
Through an open glade
He sees a shape and the shape hears:
It waits as he waits,
Waits as the shadows wait,
As the voices wait;
Shadows on the way, voices in the wind.
It’s not going to fool anyone into thinking a new book of Shakespearean sonnets has been discovered, but OpenAI’s GPT-3 text generator AI is capable of creating better poetry than the teenage drivel I handed in for GCSE coursework. The example above comes from a new research paper about the natural language processing capabilities of the model and how much better it is at words than its predecessor, GPT-2. The paper digs in to what GPT-3 can teach us about racial and gender bias in our languages, but it’s especially good, apparently, at teaching itself to correct grammar. Such pedantry comes at a cost, however:around $12m worth of compute time, apparently.