ChatGPT is More Like Google Maps Than a Chimpanzee
A few weeks ago, the headline of an essay in the New York Times caught my eye: “I Finally Figured Out Who ChatGPT Reminds Me Of.” The writer, Elizabeth Spiers, observes that talking to generative AI models is somewhat akin to giving instructions to her eight-year old son.
A.I. is in the phase when kids live like tiny energetic monsters, before they’ve learned to be thoughtful about the world and responsible for others. That’s why I’ve come to feel that A.I. needs to be socialized the way young children are — trained not to be a jerk, to adhere to ethical standards, to recognize and excise racial and gender biases. It needs, in short, to be parented.
In my day job, I am responsible for “parenting” a large language model like ChatGPT. I work on evaluating the model’s responses in various situations, and trying to steer it away from spewing obscenities, making insensitive or inappropriate remarks, and suggesting or providing guidance on dangerous or illegal conduct. At work, we have invoked the parent-child analogy many times. I once expressed to a colleague that, despite my team’s efforts, I wasn’t overly confident about the behavior of a particular model that we were hoping to launch into production. I imagine it was something like the equivalent of taking an unruly child over to a friend’s house for dinner. We went through training, carefully selecting good responses, and encouraging the model to emulate those, but at the end of the day we don’t, can’t know exactly what it will do — we just hope it doesn’t embarrass us.
In some ways AI models are much smarter than even adult humans, but in important ways they are still far inferior to human children. Children are much better at generalizing, for example; they require many fewer examples to understand what trucks are and what dogs are. But what is perhaps a bit misleading about the analogy of AI as children (I’d imagine, as a non-parent) is that typically in other humans, even if we are surprised by their choices or actions, we can recognize their motivations and thought processes. Individual variations from to nature and nurture mean that each brain is unique, but they fundamentally all work the same way. We don’t fully understand how the brain works, but we have some level of insight into how others are thinking, because we are capable of putting ourselves in their shoes.
AI models do not work the same way as human brains. It is another analogy, one that has yielded extremely productive results, since neural networks were inspired by the brain. But neural networks, which make up today’s advanced AI models, are only abstractions of how neurons work; the internal representations of knowledge that these deep neural networks build are not interpretable to us. Lots of people are researching, for example, techniques to examine what words or concepts models like ChatGPT are paying attention to when they respond to a question, but it’s difficult to assess why they respond the way they do at a higher level, and hard to explain why certain techniques like Chain-of-Thought affect responses so significantly.
I heard an alternative analogy offered on a podcast today, that training an AI is not as much like raising a child as it is like trying to raise a lion cub, or a chimpanzee, or some kind of wild animal. Granted, both of the discussants were Effective Altruists and much more concerned than I am with advanced AI risks, and the likelihood that an AI system causes human extinction. But Ajeya Cotra, who introduced the scenario, described a case in which a primatologist had adopted a baby chimp, separated from its family and too young to survive on its own. The primatologist fed, cared for, and played with the chimp, which became effectively a member of the family and known in the community. But even after years of friendly interactions with humans, something spooked the chimp, maybe unlocked some deep-seated fear, anxiety, or instint, and it killed its adoptive parent.
I have no idea whether this story is apocryphal, but it seems plausible enough, and gets the point across — a wild animal can’t be tamed, and if you think you’ve done so, you don’t understand its nature and you’re setting yourself up for a surprise at best, a horrific fate at worst. The implication is, unless or until we understand AI models more completely, they are no better than chimpanzees we’ve adopted and expected to live in harmony with people (or in the case of AI, put in charge of key commercial functions). Cotra argued that as a community, we should slow down, and possibly not produce any models larger or more powerful than GPT-4 until we better comprehend whether these models are aligned, develop their own goals, deceive humans, etc.
Interestingly, Cotra and the host, Rob Wiblin, seemed to agree that the primary reason others may not agree with their views is having a different mental model of what advanced AI is like. I think that may well be true — although I found the wild animal variation illustrative, I have problems with that analogy as well. A lion cub will, from birth, have leonine instincts; An untrained AI is a random guesser. Like a child or a chimpanzee, a cub can make inferences in a way that is unavailable to the large language models of today. And of course, these are differences in cognitive performance alone; the cub is also an agent in the world, interacting with its environment, listening and learning. It doesn’t call itself a lion, but it doesn’t need to. It recognizes its own species, predators, and prey; it plans its attacks, anticipates the movements of antelope and zebras, and recognizes the results of its own actions. Cotra speaks of “situational awareness” as a property of models, but none of today’s models have any meaningful degree of true situational awareness. Yes, ChatGPT will gladly tell users that it is a language model, but this is only because it has been trained repeatedly to produce this explanation in contexts where people ask it to do things that it can’t. My intuition would be that some type of agentic behavior is required to develop situational awareness.
According to Wiblin, those who brush aside the existential threats of AI do so because they understand these models to be more like “a can opener,” or Google Maps. Of course, an LLM is hard to compare to a can opener, because the latter tool is completely defined: there is no randomness involved. It is a mechanical function. There might be variations in that function (the teeth might fail to pierce a particularly thick lid, the cog might get stuck…) that mean the can doesn’t get opened, but a can opener never exhibits unexpected behavior.
On the other hand, sufficiently advanced machine learning systems, including supervised systems, including Google Maps, are often not that easy to interpret. Like ChatGPT, Google Maps sees hundreds of thousands of examples of the task it is trained to do, only its objective is to produce routes between two points instead of to generate freeform text responses. There are differences, but they are in large part differences of scale: large language models like GPT-4 are much, much bigger, meaning that they can learn more complex relationships from their training data. They also require much more training data, since language is less constrained than routes on a map — there are only so many ways to go from A to B, but hundreds of thousands of words, and infinitely many ways to answer an open-ended query. But differences of scale and training objective alone do not mean that these two systems are fundamentally different. ChatGPT is still much more like Google Maps than a chimpanzee.
The primary reason that people disagree with that statement is because of emergence, a term that refers to capabilities of large language models that are not present in smaller models. The idea is that, for tasks a small model is trained to do, we can both measure that model’s performance on the task and predict how much better we could do with a larger model, a concept referred to as scaling laws. But for some tasks the smaller model is not trained to do, a larger model trained the same way can do them. The ability “emerges,” despite no particular reason to expect that. An example of an emergent behavior is language models solving math or reasoning problems, considering that they are trained only to produce text. Some people — including a team at Microsoft testing GPT-4 — believe that these represent the first, baby steps towards artificial general intelligence, or human-level intelligence across all cognitive tasks that humans can do. Others — I would include myself in this latter category — are not convinced that large language models as a class could produce AGI at all. Researchers at Stanford have tested many of the claims of emergence and discovered that they could be explained by a combination of training data (e.g., including lots of math data in training helps make language models more likely to produce the correct responses when asked) and consistency in evaluations across models. These abilities are still impressive, of course, but perhaps not as surprising or alien as originally believed.
The reason why I think the analogy that we use for AI is important is because it does affect public perception and reactions to these systems. Humans use analogies for sense-making, and it is part of what makes us learn so quickly. It is this very notion that inspires at least one of the alternative approaches to artificial intelligence than the enormous and data-intensive deep neural networks that have yielded so much success in recent years. Melanie Mitchell of the Santa Fe Institute writes of the problem of “the barrier of meaning,” or LLMs that produce extremely convincing answers, without necessarily an internal conception of meaning. Such models emulate understanding, but often in a quite superficial way, as anyone who has asked a lot of follow-up questions to ChatGPT can attest. Mitchell believes that meaning holds the key to intelligence, and analogies hold the key to meaning:
It’s a fundamental mechanism of thought that will help AI get to where we want it to be. Some people say that being able to predict the future is what’s key for AI, or being able to have common sense, or the ability to retrieve memories that are useful in a current situation. But in each of these things, analogy is very central.
For example, we want self-driving cars, but one of the problems is that if they face some situation that’s just slightly distant from what they’ve been trained on they don’t know what to do. How do we humans know what to do in situations we haven’t encountered before? Well, we use analogies to previous experience. And that’s something that we’re going to need these AI systems in the real world to be able to do, too.
The way that large language models solve the “some situation that’s just slightly distant from what they’ve been trained on” problem is by training on most every word ever written on the internet, and it’s hard to argue with the great success of throwing data and GPUs at a problem. But these models aren’t like a human, or an animal. They’re like models. Unfortunately, this means that I can’t scold or cajole the model that I work on into compliance; we define policies and collect data for instructions that would be common sense for a child. While I don’t mean to downplay the impact that large-scale AI could have, I don’t expect them to “think” like us anytime soon.