Here are two poems by Wallace Stephens, except one is written by AI. Guess which one was written by Stephens?

2

I must have shadows on the way

If I am to walk I must have

Each step taken slowly but alone

To have it ready made

And I must think in lines of grey

To have dim thoughts to be my guide

Must look on blue and green

And never let my eye forget

That color is my friend

And purple Must surround me too

The yellow of the sun is no more

Intrusive than the bluish snow

That falls on all of us. I must know

Grey thoughts and blue thoughts walk with me

If I am to go away at all.

AI continues to do things we never thought possible. GPT-3 does what your brain is doing now or more accurately what your brain is doing when you write. It is producing the next word in the sentence. Except it’s not. It is using brute force and a huge amount of data from the internet, to process what your brain does with relative ease. Nevertheless what GPT-3 does is astounding.

GPT-2 was impressive, a Natural Language Processing model that took a massive amount of text from the internet, at least the quality text, and it performed well, really well. So well, they didn’t release the software, for fear of its power. Open AI kept on going and created GTP-3, which is 117 times bigger that GTP-2. Bigger appears to be better.

Benchmarking GTP-3, on writing articles, humans can distinguish ‘human from AI’ created content accurately 52% of the time. In other words only as good as guessing, in other words, indistinguishable. Whether it’s a poem by Wallace Stephens or an essay, this is a stunning, if not frightening, achievement. Basically, AI is producing texts that are indistinguishable from human texts. Let that sink in, as the consequences of that competence are massive.

The researchers claim that GTP-3 has learnt how to learn and that that the bigger the model, he better its adaptive abilities. This hints at the possibility that we are moving slowly towards AGI (Artificial General Intelligence) here. The rick is to focus on the few shot problem. In standard machine learning, you give it loads of data. However, for many real-world problems, the data is small, only a few examples.  An example of a few shot learning problem, its digital ID. You give it only a few photos or image of your face, and it recognises your face to give you access to your phone. The larger models, like GPT-3, make better use of fewer examples in the data. What’s more, when it gets things wrong, it tends to get them wrong in plausible ways. Now that is interesting. We may not only get right answers but common misconceptions, which are useful in teaching and learning. 

It is easy to anthropomorphise GTP-3 but the deeper you delve and extrapolate, the more scary it becomes. It feels like a child, a smart, fast learning, precocious child, feeling its way forward in the world. Such power, creeping towards AGI, from just predicting the next word (or token) in a sentence. As the efficacy of the model is still trending upwards, it may still have a long way to go, a very long way. It would seem that ‘competence without comprehension’ is with us in a big way. We could even just stumble upon AGI if the internal ‘learning how to learn’ feature, that seems to be emerging, gets exponential. What a time to be living.