I have for some time been complaining that progress in many fields peaked around 1972 or so, that many important fields have gone backwards. Last man on the moon 1972, cars and clothes washing machines have been getting crappier. The skyline on big western cities is starting to look less and less like the future, as is the interior of your neighbourhood shopping mall. The highest flying and fastest flying warplane retired in the early eighties.
But there has been a major breakthrough in AI. The methods proposed in “attention is all you need” have been applied to a variety of problems, and are yielding interesting, important, and impressive results.
The breakthrough is that generative techniques can generate endless instances as instantiations of a word, or set of words, and can also recognise a particular instance as an instantiation of a word or words. In other words, handles words as reference to concepts.
This has been the show stopper problem in philosophy, ai, and the philosophy of ai for a long time. That GPT works as well as it does tells us something important about meaning, thought, and words. What it is telling us is not clear, but whatever it is telling us, it is a reply to an issue first raised by Aristotle.
GPT type models can generate an unlimited number of instances corresponding to a concept or set of concepts, and can recognize the goodness of match of a particular instance to concept or set of concepts. Or at least is acting like it can in some important cases, quite a lot of important cases.
What we could do with this tool is take an enormous pile of conversations, and for each entity in the conversation, predict his response to any previous comment.
The question then is, would a generated conversation indicate a sentient response to novel prompts?
One of the things gpt can do is represent a very large body of knowledge, by predicting the response to a query about it from existing similar, but far from identical, queries.
But because it does not understand the information it is representing, the responses suffer from “hallucination” reflecting the fact that its model of the knowledge is not the knowledge, but a model of words about the knowledge, words about words. Sometimes, they superficially sounded very like a correct answer but were utter nonsense.
ChatGPT makes errors because its universe consists of words referring to words. Its errors do not necessarily reveal a lack of consciousness, but rather reveal it does not understand the words refer to real physical things.
When it makes a completely stupid error, and gives a meaningless nonsense response, it sounds very like a sensible and correct response, and you have to think about it a bit before you realise it is utter nonsense and meaningless gibberish.
ChatGPT is very very good at writing code. Not so good at knowing what code to write.
Suppose it had been trained on words referring to words, and on words referring to diagrams, and on diagrams and words referring to twodee and threedee images, and on words, diagrams, two dee and three dee images referring to full motion videos.
From the quality of the performance on words about words, and words about artistic images, one might plausibly hope for true perception. What we now have is quite clearly not conscious. But it has taken an impressively large step in the direction of consciousness. We have an algorithm that successfully handles the long standing central big hard problem in philosophy, AI, and the philosophy of AI, at least in a whole lot of useful, important, and interesting cases.
Quite likely we will find it only handles a subset of interesting cases. That is what happened with every previous AI breakthrough. After a while, people banged into the hard limits that revealed no one at home, that consciousness was being emulated, but not present. People anthropomorphise their pets, because their pet really is conscious. They do not anthropomorphise their Teslas, because the Tesla really is not, and endlessly polishing up the Tesla’s driving algorithms and giving the more computing power and data is not getting them any closer.
But we are not running into hard limits of GPT yet.
Perception is starting to look soluble. Not solved, but definitely looking like a solution may well be merely a matter of polishing up what we now have.
Will, intent, purpose, and desire still conspicuously missing. But they are problems very similar to perception, hard in the same way and for the same reasons perception is hard.
We do not yet have a robot that can take a beer out of a fridge, pop open the can and pour me a drink, or can fold a shirt in a reasonable time. And the way the wind blows, we are likely to get an AI that knows all the knowledge in the world, and can provide meaningful and useful answers about it before we can get a robot that can make me a ham sandwich. But it now starting to look a whole lot more likely that we can get an AI that knows all the knowledge in the world and can provide meaningful and useful answers.
