In my last piece, I laid out an argument from What Computers Still Can’t Do by Hubert Dreyfus. To quickly sum up, nature seems to be able to obey mathematical laws without having to expend computational power to do so. For instance, planets orbit stars without computing a differential equation, and soap bubbles take on a minimum surface area shape without minimizing an integral. The question, then, is what if the brain can likewise produce intelligent behavior without having to compute it? If it can, then the existence of the brain tells us nothing about how hard intelligence might be to compute, and artificial general intelligence (AGI) via digital computers could be any level of difficulty.
As interesting as I found this argument, I don’t think it holds in the end. It makes us scream the question, what is computation anyway? There is no single agreed-upon definition. Wikipedia says computation is “calculation that follows a well-defined model,” while calculation is a “deliberate mathematical process that transforms one or more inputs into one or more outputs.”
So, by this definition, is the brain a computer or not? It’s not very clear. And a mere definition can’t tell us how reality is, anyway. In this piece, I’ll develop my own notion of computation that I believe is relevant to the problem at hand. We’ll revisit this question later, so please hold onto it for now.
Before I go further, let’s get clear on exactly what Dreyfus thought.
What Is Computation?
Computation is a physical process by which one system comes to have mutual information with another system of interest.
What Did Hubert Dreyfus Actually Say?
Did Dreyfus himself explicitly claim that the brain doesn’t do computation? I looked back at his text as I wrote my last piece, and it’s a bit murky:
“[T]o say that the brain is necessarily going through a series of operations when it takes the texture gradient is as absurd as to say that the planets are necessarily solving differential equations when they stay in their orbits around the sun, or that a slide rule … goes through the same steps when computing a square root as does a digital computer when using the binary system to compute the same number.”
In regards to planets, it certainly sounds like he’s saying they don’t compute differential equations. By analogy, we are led to suppose that the brain doesn’t compute either. But on slide rules, it sounds more like he’s saying they simply compute in a different way. So, which did he believe in regards to the brain?
I managed to find him elsewhere explicitly saying that the mind is not a computer, and that computers are incapable of thought. This argument technically leaves open the possibility that the brain is a computer, but the mind is not. If the brain produces the mind, however, then computers could still produce minds, even if we for some reason did not wish to think of that mind itself as computing. This construction would make Dreyfus’s whole book fall flat, so I don’t think that’s what he’s trying to say.
What if the mind is not a product of the brain? I would bet that Dreyfus was open to this possibility, but he doesn’t lean on it all in his book, so I think his arguments are meant to show the possibility that the brain is indeed not a computer. In any case, a great number of people on Twitter agreed with the argument I laid out, so I don’t think I’m attacking a strawman now in arguing against it.
Do Planets and Soap Bubbles Compute?
The most interesting counterargument I received was that planets and soap bubbles do compute, as exemplified by this Saturday Morning Breakfast Cereal comic.
At first, I thought this was a silly, semantic argument. I started to feel differently when people pointed out all the ways in which these systems don’t behave “for free.” Before, I argued that these systems operate automatically and effortlessly, but this doesn’t really seem to be true. They require energy and they use up resources. For instance, bubbles have to perform thermodynamic work to take their minimum surface area form, which requires energy and increases entropy. Planetary orbits don’t require work to run, but they do require work to set up in the first place. None of this happens for free.
In my view, this is the crux of the whole argument. Setting up and running these systems increases entropy, and in a maximum entropy universe, I don’t think any interesting physics or computation could take place. So, the only reason we are able to do any computation in our universe is because it began in a low-entropy state. That is, the universe started off fairly orderly, and we use up this order in the process of doing computation. Eventually, we may run out, at which point no further computation will be possible. So, the question “Where does the universe get its computational power?” reduces to “Why did the universe start in a low-entropy state?” This remains an open question in physics.
I’m still a bit confused about how a maximum entropy universe can “afford” to run. There are still particles bopping around, even if they aren’t doing anything very interesting. How can that happen? How can a single particle, not doing any work, “afford” to exist? This feels much closer to the question of why anything exists at all, which I’m not going to be able to answer. But we’ve come a long way from just being confused by bubbles and orbits, so this seems like progress.
I also came up with a related limiting argument. As an example, consider the Antikythera mechanism, an analog computer from Ancient Greece used to predict the movements of celestial bodies. It does this by simulating these bodies through the movement of certain gears. The relative positions of these gears are analogous to the relative positions of the bodies in question. Clearly, this mechanism does computation.
Now, suppose we create a series of such computers, gradually becoming more and more analogous to the system under consideration. For example, we might start to use masses to represent the bodies, and allow the force of gravity to carry the mechanism forward, rather than a hand crank. Each step along the way, we’re doing computation.
Eventually, the “perfect” analog computer is just the system itself. Shouldn’t it also be doing computation? If not, at what point does our “computer” become too analogous to actually be doing computation? This just seems silly.
The Extended Church-Turing Thesis
Many people sent me work from Scott Aaronson, an expert in computational complexity and quantum computing. In this video, he discusses a lot of the foundational issues in computation. He touches on our soap bubble example, and explains that bubbles don’t always take the minimum surface area form, specifically when doing so would be too computationally expensive. Instead, the shape settles on some local minimum.
You can set up a problem in which finding the minimum surface area is equivalent to solving a certain NP-complete problem, and lo and behold, the bubble is unable to find the corresponding shape, bursting the whimsical idea that nature is solving all these problems effortlessly.
But my eureka moment actually came from reading the following question on his slides, which simply introduced the topic: “Can soap bubbles solve optimization problems faster than our fastest computers?” I suddenly realized, with some embarrassment, that I’d forgotten the first computational resource you ever learn about: time. We can compute orbits faster than just waiting and watching the planet. That’s why computers are useful. From that point of view, the real orbit isn’t so cheap after all.
The Extended Church-Turing Thesis
The standard (physical) Church-Turing thesis claims that any physical system is simulable by a digital computer. The extended version says this can be done efficiently.
The ultimate question is, when taking into account all the relevant resources, can nature outcompete our computers? That is, using similar amounts of time and energy, is there anything nature can do that digital computers can't? The Extended Church-Turing thesis addresses exactly this. The standard (physical) Church-Turing thesis claims that any physical system is simulable by a digital computer. The extended version says this can be done efficiently.
The standard thesis holds up well, but the extended version actually seems to be false. The counterexample is quantum computers. Quantum computers can, for example, factor numbers in polynomial time, which classical computers seemingly can’t (although this is unproven). According to Aaronson, the only known cases of a physical system being exponentially harder to simulate on a digital computer are certain quantum algorithms.
This point is important since a weaker form of Dreyfus’s argument would say that, while the brain may be a computer, it’s an analog computer, not a digital one. And that analog computer might be exponentially harder to simulate digitally. We have tried simulating analog systems, however, and we have not found such an exponential slowdown except for in the case of certain quantum computations.
This seems to indicate that the only plausible way the brain (and intelligence itself) would be infeasible on a digital computer would be if it relied on special quantum algorithms. No convincing evidence exists for this claim as of yet, and most scientists greatly doubt it.
The physicist Max Tegmark has a simple argument for its implausibility. Quantum computers require intense shielding to prevent interference from the environment, called decoherence. The brain has no such shielding, and the decoherence rates seem to be far faster than the time scale at which any useful computation could take place.
As such, despite much looking, we have found no reason to believe that there should be an exponential slowdown in simulating the brain. There are estimates of the compute power of the brain based on simulating its analog computation, and they are not infeasibly large. At this point, I see no real reason to doubt them.
What Is Computation?
On a more fundamental note, plenty of people asked what I mean by computation in the first place, to which I had no good answer. After a lot of thought, I settled on a working definition: Computation is a physical process by which one system comes to have mutual information with another system of interest.
Two systems have mutual information if learning the state of one tells you something about the other. For example, I can arrange a certain computation, and the final result on my computer corresponds to the position of a planet, or the shape of a bubble, or the order of a list, and so on. What I do on my computer corresponds to the system I’m curious about or answers some question I’m trying to ask.
Now, every physical system has perfect mutual information with itself, and it’s in this sense that everything is a perfect analog computer of itself. Orbits and bubbles, however, are very limited. A planet can only give you information about one orbit, and if you want it to tell you something about a different orbit, you must do work to change its path. Bubbles are similar; if you alter its boundary, it will compute a different shape, but it must perform work to do so.
This is a general fact, illustrated in decision theorist Eliezer Yudkowsky’s essay “The Second Law of Thermodynamics, and Engines of Cognition.” To change the state of a system so that it has mutual information with some other system always requires thermodynamic work and increases entropy. The Second Law of Thermodynamics requires it. The crux, then, is that the brain does this all the time.
Just looking around the room, your visual cortex is constantly updating its state to represent what you’re looking at in order to have mutual information with it. Further, the brain is a universal computer, capable of representing anything, while these other systems are not. That is why they can run comparatively effortlessly. You don’t have to expend any energy to just constantly represent one thing; being yourself costs nothing! But to constantly change what you represent, as the brain does, certainly requires computation.
AI as Alchemy
So, as I said, I don’t think Dreyfus’s argument holds in the end. But that doesn’t mean he had nothing important to say. His criticisms of the old AI paradigms were borne out, and he challenged AI to do better. On a personal note, Dreyfus forced me to grapple with computation much more deeply and did, in fact, alter my views on AI. Previously, I had little interest in what DeepMind calls neuroscience-inspired artificial intelligence, taking the human brain as inspiration for AI. Humans are riddled with flaws. Why would we want to model AI on ourselves, when we have much better models for ideal intelligence?
But we still have little idea how to build machines that robustly produce generally intelligent behavior. Even if we don’t want to copy the brain exactly, it still serves as our one and only example of a physical system that produces intelligent behavior. It surely behooves us to understand this system as thoroughly as possible if we intend to make a new or better one.
Dreyfus famously compared AI to alchemy, focusing on surface-level features at the expense of the underlying phenomena. He urged AI to grapple with the fundamentals:
“If the alchemist had stopped poring over his retorts and pentagrams and had spent his time looking for the deeper structure of the problem, as primitive man took his eyes off the moon, came out of the trees, and discovered fire and the wheel, things would have been set moving in a more promising direction. After all, three hundred years after the alchemists we did get gold from lead (and we have landed on the moon), but only after we abandoned work on the alchemic level, and worked to understand the chemical level and the even deeper nuclear level instead.”
Taking Drefyus’s advice, getting to grips with the deeper structure of the problem, with the physics, computation, and neuroscience of intelligence, now seems central to my growing understanding of AI.
