Episode 143: Is AGI Possible? (State of the Debate)

• Listen to this episode: Apple Podcasts

Unofficial, reviewed transcript. It may still contain mistakes; check the podcast when wording matters.

Speakers are labeled by first name where identified. Unidentified-speaker labels may refer to different people. Bracketed question marks indicate uncertain wording.

Transcript

[00:00:00]  Unidentified speaker: Hello out there.

[00:00:01]  Unidentified speaker: This week on The Theory of Anything Podcast, Bruce returns to AGI through the lens of a critical rationalist debate on whether or not AGI is theoretically possible.

[00:00:13]  Unidentified speaker: Can human intelligence be replicated?

[00:00:16]  Peter: Is this a physics or biology question? Is our universe and our minds like a compact disc or a vinyl record?

[00:00:26]  Unidentified speaker: I hope someone out there enjoys listening to Bruce on this topic as much as I did.

[00:00:38]  Bruce: Welcome to The Theory of Anything podcast.

[00:00:40]  Unidentified speaker: Hey, Peter.

[00:00:41]  Unidentified speaker: Hey, Bruce, how's it going today?

[00:00:44]  Unidentified speaker: Good.

[00:00:45]  Bruce: I'm excited for today's episode. This is, you know, we have kind of the episodes where I'm like muddling through ideas, and then we have episodes where I finally got something and it's come together for me, and I can do an episode where I explain what I now think I understand.

[00:01:01]  Unidentified speaker: This is one of those episodes.

[00:01:03]  Bruce: So where it pulls a lot of ideas together, and I think in some interesting ways. We're going to talk today about Ray Scott Percival versus David Deutsch on the question of is AGI possible on a computer, or is it possible at all, maybe even. So one of the most interesting debates amongst quote unquote high-ranking critical rationalists is between Ray Scott Percival, who we've done a bunch of episodes on him, his books, I love Ray, and David Deutsch who this show was largely inspired by on whether AGI can be built. Now Ray draws on Karl Popper, who argued against the possibility of artificial general intelligence, at least on a computer. So quote from Popper, I do not believe that computers will ever invent important new problems or new values. That's from All of Life is Problem Solving. End quote, I predict that we shall not be able to build electronic computers with conscious subjective experience. That's from The Self and Its Brain, both direct quotes from Popper. Ray therefore represents the classic Popperian view against AGI. Now Deutsch famously by contrast, argues via the Church-Turing, now Church-Turing-Deutsch thesis that quote, the very laws of physics imply that artificial intelligence must be possible. That's from his famous article, Creative Blocks from Aeon. And if you want our take on that, see episode 58 for our discussion on the article a decade later, and what I thought it got right and what I thought it got wrong, now that I had actually gone to school on the subject and studied it more intensely and more closely.

[00:02:55]  Peter: Well, if I can interject something before you get going here, I just want to say this is something I'm extremely interested in. I have this idea that AGI is definitely possible, and I would probably defend that more on the level of physics than biology, I guess. I think we both had the experience of talking to a neuroscientist, and they start talking about the different parts of the brain, and how it's nothing like a computer.

[00:03:39]  Unidentified speaker: That's exactly what the one I talked to said.

[00:03:41]  Unidentified speaker: Yeah, yeah.

[00:03:42]  Unidentified speaker: And I mean, it's hard to argue with someone who knows a lot more about the brain than at least I do.

[00:03:49]  Peter: But at the same time, like, I have just absorbed the perspective of so many physicists who see this as more of a foundational kind of physics issue, more like what Deutsch might argue with the Turing-Deutsch thesis, or at least something that's at the intersection of physics and machine learning. And, you know, on one hand, I find their perspective so convincing. Yet on the other hand, I think I might be accepting some arguments from authority there. Like, you know, John, I keep thinking about John Wheeler's thing of It from Bit. Reality is fundamentally information based.

[00:04:43]  Unidentified speaker: But, you know, there's another I think that's probably true.

[00:04:47]  Peter: Although I don't really have a way myself to get my mind around that, that feels completely coherent. I mean, the ones and zeros on a computer, I mean, they can simulate this, this analog, supposedly analog, reality, which actually probably indicates more that we might be living on something more like a compact disc than a vinyl record. But it's hard for me to really believe that, or really, I believe that, but it's hard for me to really take that, explain that to other people in a way that completely makes sense to them, and because it doesn't completely make sense to me, I guess. So maybe after this podcast, you'll clear it up for me.

[00:05:49]  Bruce: So actually, one of the things we're going to discover today is that, A, David Deutsch does not believe that we get it from bit.

[00:05:58]  Unidentified speaker: Oh, he doesn't.

[00:05:59]  Bruce: And B, he thinks the world's more like a vinyl record.

[00:06:02]  Unidentified speaker: Is that right?

[00:06:03]  Unidentified speaker: Okay.

[00:06:04]  Unidentified speaker: Yes.

[00:06:04]  Unidentified speaker: Okay.

[00:06:05]  Unidentified speaker: All right.

[00:06:05]  Unidentified speaker: All right.

[00:06:06]  Unidentified speaker: Okay.

[00:06:06]  Unidentified speaker: So I'm just like, I'm just confused.

[00:06:10]  Peter: I've just got confusion on top of confusion.

[00:06:12]  Unidentified speaker: To be honest-

[00:06:12]  Unidentified speaker: That's what you're here for, Bruce.

[00:06:14]  Bruce: To be honest, I'm not sure I fully understand David Deutsch's view. I feel like I understand the paper we're going to talk about pretty well, well enough that I can explain it.

[00:06:23]  Unidentified speaker: Okay.

[00:06:24]  Bruce: But like Mark Biros, who he came on here and he did the episode on disagreements with Deutsch because he has a lot of disagreements with David Deutsch. One of the things that he told me, not in that episode, but just elsewhere, is he said, Bruce, David Deutsch claims he doesn't agree with it from bit, but go read his paper, It from Qubit. He says, and then try to explain to me, Bruce, what the difference is. I'm like, well, I know about all sorts of differences between It from bit and It from Qubit. So I started to explain, he goes like, no, no, no, no, no, no. Don't give me your understanding, go read the actual paper. And I did and I couldn't tell. It was very, very confusing.

[00:07:04]  Unidentified speaker: So, um, yeah, I mean, it's hard to see in an intuitive way, at least for me, the connection between transist—transistors indicating, you know, ones and zeros, and this reality that we're experiencing all the—all the time, internally or externally.

[00:07:29]  Peter: But maybe I just don't understand enough about physics or machine learning.

[00:07:35]  Unidentified speaker: So, here I am.

[00:07:37]  Bruce: Let's admit—let's admit that this is a somewhat difficult topic. And that's literally why I went back to school, so I could upgrade my mathematical ability. And I still feel like I suck at this stuff. Getting a master's degree only barely helped. But one of the things that I do find frustrating, that I have expressed in this podcast with the CritRat community in general, is that I'm definitely left often with the impression that most of them are just accepting David Deutsch on authority on questions like this. Like they don't really, they have not earned the knowledge themselves. They don't understand it. They don't understand it well enough to even use it or work out implications with it or anything like that. I've had numerous, numerous conversations with CritRats that clearly do not understand David Deutsch's positions. And in fact, are at odds with it, but they don't realize they are because they've misunderstood it and they're just accepting what they think he said. This podcast, we've gone over a whole bunch of examples of that. So that shouldn't be surprising. But then you've got someone like Sam Kuypers, like who we just had on the podcast. He clearly understands it. And understands it better than I'm ever going to, because he's specifically studying physics and he's working with David Deutsch directly. So let's just admit that a lot of these topics are difficult. And while it's a mistake to believe in authority, in some ways you default to it because you just don't know how not to.

[00:09:15]  Bruce: And so like when I was talking to a neuroscientist, I mentioned to her that the brain was a computer. And she goes, no, it's not. It's nothing like a computer. So I said to her, the correct answer. No, I don't mean the brain is like a digital computer.

[00:09:33]  Unidentified speaker: It's not.

[00:09:34]  Bruce: I mean, the laws of physics are computational, and the brain is a physical object that follows the laws of physics. Blank stare. And it completely could not make sense of what I just said. Even though it's a really good answer. If she took the time to understand what I just said, she would very quickly realize why her position is wrong.

[00:09:56]  Unidentified speaker: No way was she ever going to do that.

[00:09:59]  Bruce: So in her mind, she just repeated, no, no, it's not a digital computer.

[00:10:05]  Unidentified speaker: I'm like, yeah, I know.

[00:10:05]  Bruce: I just said that. There was no way through. We just did an episode on dogmatism. Human beings are kind of inherently dogmatic.

[00:10:14]  Unidentified speaker: We have an idea in our head.

[00:10:16]  Bruce: We don't give it up easy.

[00:10:17]  Unidentified speaker: Our ideas are sticky.

[00:10:19]  Bruce: And even being given the right answer is not going to change your mind. Partially because it's hard to process Ray's point of view.

[00:10:27]  Unidentified speaker: He's right about that.

[00:10:28]  Bruce: Partially because you're probably not going to process it because you're dogmatic. My point of view, which I don't see as at odds with Ray's point of view at all.

[00:10:36]  Unidentified speaker: Maybe Ray would disagree.

[00:10:39]  Bruce: One thing just for convenience, I'm going to refer to Ray as Ray because I know him personally and consider him a friend. And I'm going to refer to Deutsch as Deutsch instead of David because I don't really know him personally. And so I've always called him Deutsch traditionally. I actually think most people call him David. But like, so that's a little unfair. I should probably say Percival and Deutsch. But like, I hope Ray will forgive me on this that I'm just going to call him Ray because that's what I'm used to calling him. And it was just so hard to keep trying to switch back and forth between what I was saying in my mind and trying to make it official. So I finally just gave up and just said Ray and Deutsch throughout the presentation.

[00:11:24]  Unidentified speaker: Just keep that in mind.

[00:11:25]  Bruce: I'm not trying to in any way downplay Ray just because I'm on a first name basis with him and I'm not with Deutsch.

[00:11:34]  Unidentified speaker: So here's the thing.

[00:11:35]  Bruce: What's interesting is, is that Ray, unlike Popper, is aware of Deutsch's AGI arguments that are grounded in critical rationalism, at least aware they exist. I don't know how well he understands them or not, but like he's aware that they exist. He knows that David Deutsch has an argument here, that it purports to be grounded in critical rationalism. Popper wouldn't have had that, wouldn't have been able to know any of that, right? Because he died before Deutsch produced a lot of these arguments. So he wouldn't have had any sort of awareness of the critical rationalist based counter arguments to his view. Whereas Ray is at least aware that they exist, okay? So despite years of discussion, Ray has never shifted from the old school Popperian view. Him and Deutsch know each other, they're aware of each other's arguments. Ray has never changed his mind.

[00:12:32]  Unidentified speaker: So-

[00:12:32]  Peter: And are these based on something he's written or is this, are these more private or public discussions with-

[00:12:38]  Unidentified speaker: Oh, yeah, no, no.

[00:12:39]  Bruce: This is all going to be taken straight from his Twitter account. So I'm going to quote what he said on Twitter and his arguments. And then he refers to a book which I then reviewed. So most of it will end up coming from the book that he pointed to as he thought this was a groundbreaking argument against David Deutsch's view.

[00:13:00]  Unidentified speaker: Okay.

[00:13:00]  Bruce: So now regular listeners know my position on this.

[00:13:04]  Unidentified speaker: Okay.

[00:13:04]  Bruce: So I'm going to say it up front. Bias is on the table. Ray is absolutely wrong.

[00:13:09]  Unidentified speaker: David Deutsch is absolutely right.

[00:13:11]  Unidentified speaker: Period.

[00:13:12]  Unidentified speaker: End of story.

[00:13:13]  Unidentified speaker: Okay.

[00:13:14]  Bruce: From a critical rationalist standpoint, our best theory is that general intelligence is possible on a computer. Popper was simply mistaken on this point. And throughout this episode, I will incidentally explain why that is the case. Here's the thing though. I know everyone's going to concentrate on, oh, this is Bruce's episode about why AGI is possible. That's not really what I'm arguing. The real aim is to explore Popperian style critical discussions. It's the epistemology that I find interesting. I'll make a bold claim in this episode that properly understood Popperian style critical discussions, or the scientific method, if you prefer, has an objective character. It focuses on what we intersubjectively agree upon, not on subjective feelings that we disagree upon. Or at least, that is the goal we aim towards as best we can. I'm sure in real life it doesn't work out that way, even with scientists. Is it possible that this disagreement between David and Ray is just a matter of opinion? Okay, so for example, might it be that given one's own knowledge state, that it is reasonable, assuming that both gentlemen have the same knowledge state, that it is reasonable to come up with either David Deutsch's position or Ray Percival's position? In other words, does the state of the critical discussion allow you to draw either conclusion? Okay, let's say David and Ray have different knowledge states. And under Ray's knowledge state, his conclusion is the correct one. And under David's knowledge state, his conclusion is the correct one. It's possible, okay?

[00:15:05]  Bruce: But if that's true, shouldn't one of them change their mind if they say, share their knowledge with each other so that they have the same knowledge state? Okay, why doesn't that happen? If they know each other, they can sit down, they can discuss it. One of them should adjust their knowledge state and they should then come to an agreement on this. So especially given Ray's view that we all have a robust reasoning module and that we aren't able to control our beliefs, if those are both true, why can't Ray and David just sit down, talk this out, and whoever is wrong changes their mind? Why doesn't that happen? Perhaps it is possible that the state of the critical discussion is such that maybe this is one of those up-for-grabs and neither side is really better than the other. Okay, so let's say they sit down, they discuss it out. Now they don't have separate knowledge states. They have the same knowledge state of what the state of the critical discussion is. And it turns out that one of them isn't really better than the other. It's just sort of up-for-grabs, maybe this one's better, maybe that one's better. If that were true, then that would make sense why, even though they have the same knowledge, that David and Ray have different opinions on the subject.

[00:16:18]  Unidentified speaker: Okay, but here's the thing.

[00:16:21]  Bruce: If that's the case, aren't really both of them wrong? It means, what I mean is, shouldn't they both concede, if they have the same knowledge [state/data?], and that's the case is that it's not clear, which is the better theory, shouldn't they both concede that their side isn't the obvious winner? Wouldn't that be the correct conclusion from the knowledge state? And wouldn't that be the correct critical rationalist view? Oh, I have an opinion, I think it's possible to do AGI, says David Deutsch. But I admit that given what we know, it's not obvious. And Ray goes, I have an opinion, my opinion is that it's not possible to do AGI, but I have to admit, given the state of the critical discussion, I don't think it's obvious. That would then be the correct answer, which would mean both of them are wrong. Okay, because David's not claiming that it's up for grabs. He's claiming, no, one one's a clear cut winner, it's AGI—it is possible on a computer, given our current understanding of these theories. And Ray's saying the exact opposite.

[00:17:22]  Unidentified speaker: Okay.

[00:17:23]  Bruce: So it seems to me that it must be the case, must be the case that at least one of the two men is mistaken, or possibly both. There's no way they can both be right. It's a logical impossibility.

[00:17:39]  Unidentified speaker: Okay.

[00:17:40]  Bruce: Now you might say, no crap, Sherlock. Obviously one of them is wrong and one of them is right. Oh, you're misunderstanding what I'm saying. That's true too, but that is obvious. I mean, it must be the case. So let's make a distinction here. A distinction between who is right about AGI, if it's possible on the computer or not, is a different question than the question of, given the state of the critical discussion, one of them has understood the state correctly and one of them has misunderstood the state. Okay, those are different questions. I'm talking about the latter.

[00:18:17]  Unidentified speaker: Okay.

[00:18:18]  Bruce: I mean, it must be the case that one of the two men is mistaken about, at least one of the two men, is mistaken about the objective state of the critical discussion. Okay, that's what I'm saying. So, I mean no disrespect to Popper and Ray in saying that they are mistaken on this point, though they are mistaken on this point. But it is not a matter of subjective opinion. Okay, under critical rationalism, I am claiming, there is a clear-cut answer to what our best current theories say. And those theories say human-like intelligence is possible on a computer, period. Now, if Ray or Popper claim otherwise, they are mistaken about what our best theories say. That does not mean that Ray and Popper—Ray or Popper—Ray and Popper are ultimately wrong. We could have a state of this critical discussion where our current best theories say AGI is possible on a computer, and it could still not be possible to have AGI on a computer, because there's some sort of flaw in our current theories that we don't yet understand. Okay, so I'm not make passing judgment on Ray and Popper in terms of if they're right or wrong about AGI. I am passing judgment on whether they have understood the state of the critical discussion, and that's all I'm passing judgment on. Until AGI is built, the question of whether it's possible or not remains open forever until it's built, okay? When we say, quote, our best theories say it is possible to build an AGI on a computer, that is a factually true claim about the theories, not strictly speaking about if AGI can be built.

[00:20:00]  Bruce: It does not assert that statement, the statement, our best theories say it is possible to build an AGI on a computer. Nothing in that statement asserts that AGI will be built on a computer, nor that it is more likely that it will be built, at least not in any simple probabilistic sense. It is a purely factual claim about the theories themselves. Because our best theories, in this case, quantum mechanics and Church-Turing-Deutsch thesis in particular, do imply that AGI is possible on a computer. That is why it is a factually true statement when one says, our best theories imply AGI is possible.

[00:20:40]  Unidentified speaker: That's not an opinion statement.

[00:20:42]  Bruce: It's not a claim that it is actually possible to build AGI on a computer. It is a claim about the theories and it is a fact. Note that there is no counter equivalent claim. The statement, according to our best theories, AGI is not possible on a computer, is a factually false statement. This distinction is the key to understanding the difference between a Popperian style critical discussion, or in other words, the scientific method, versus the average folk epistemology based critical discussion. Now, along the way, I want to introduce or reintroduce an idea that I think is important that I call intellectual lift. Let's first try to summarize Deutsch's argument as taken from his famous paper. This paper is Quantum Theory, The Church-Turing Principle, and The Universal Quantum Computer. This is the paper that made Deutsch famous. It created the entire field of quantum computing by defining a theory of quantum computation. The first part of Deutsch's argument is worth mentioning, though it isn't central to our main discussion. Deutsch's key creative argument is to redefine the Church-Turing Principle as a physical, not merely mathematical claim. So the standard formulation, mathematical formulation of Church-Turing says something like this. Every function which would naturally be regarded as computable can be computed by the universal Turing machine. Now, Deutsch reformulates this physically. He changes it to, quote, every finitely realizable physical system can be perfectly simulated by a universal model computing machine operating by finite means.

[00:22:27]  Bruce: Once framed this way, Deutsch refutes the principle by counter-example. Now, this is the thing that a lot of people are going to be surprised about. He shows that reality is continuous. It's more like a vinyl record. While digital computers are discrete, more like a CD. Therefore, classical systems cannot be perfectly simulated by a standard Turing machine. They can only be approximated, not perfectly reproduced. Now, classical physics involves uncountably many states, which a discrete machine, a computer, cannot fully represent. This argument may...

[00:23:04]  Unidentified speaker: go ahead.

[00:23:06]  Unidentified speaker: But isn't it true.

[00:23:12]  Peter: That digitally you can represent an analog signal at any desired degree of accuracy? I mean, doesn't that kind of play into it somehow?

[00:23:28]  Unidentified speaker: It does.

[00:23:31]  Bruce: So, you're actually asking a very good question. Let me actually work it into my explanation here, because what you're saying is actually true.

[00:23:40]  Unidentified speaker: Okay.

[00:23:41]  Bruce: So we're making a distinction here between, can you exactly represent an analog state? No, you can't. And can you finitely represent it discretely to any desired level of accuracy? Yes, you can.

[00:23:57]  Unidentified speaker: Okay.

[00:23:59]  Bruce: Deutsch is intentionally formulating.

[00:24:02]  Unidentified speaker: So let me put it this way.

[00:24:03]  Bruce: The argument that Deutsch just made, that I just repeated, may come as a surprise coming from Deutsch. And you might object, does exact simulation even matter? Which is what Peter just objected. I'm reading from my script and Peter actually always makes comments exactly at the right point.

[00:24:20]  Unidentified speaker: It's really kind of funny.

[00:24:21]  Bruce: Why isn't arbitrary precision enough? After all, Deutsch himself argues that the continuous versus digital debate is a red herring. So what's going on here? Why is Deutsch making a big deal about this? When really a digital representation can be finitely as exact as we need it to be, whatever that is. So what's going on here? Deutsch is making the precise argument he needs for his current point. His goal is to show that perfect physical simulation requires a quantum Turing machine, not a regular Turing machine, classical Turing machine. Therefore, the quantum computer, not the classical Turing machine, is the true universal computer. This is the argument he is at the moment making in the paper.

[00:25:08]  Unidentified speaker: Okay.

[00:25:09]  Bruce: Other objections may be valid. The one you just made may be valid, but they're not relevant to the specific question and context for the argument he's currently trying to make. So, Deutsch next defines a universal quantum computer. Now, I've tended to call this the quantum Turing machine, and I actually prefer that term. I think it is more evocative, so I tend to use it. So, Deutsch then proves that the universal quantum computer can perfectly simulate any real finite dissipative physical system. In essence, what Deutsch does is he maps every possible physical state, quantum states in this case, back to the quantum, to quantum computation. The argument relies on establishing a faithful description of a finitely realizable physical system within a finite dimensional state space.

[00:26:04]  Unidentified speaker: I know, that's a big mouthful.

[00:26:06]  Bruce: So, here's the key idea. All possible physical states, quantum physical states, can be mapped to quantum computations. So, quantum systems live in a finite dimensional state space suitable for computation. Even systems with energy loss or noise can be represented. As a result, quantum computers, not classical ones, satisfy the physical Church-Turing principle, whereas the classical Turing machine doesn't technically satisfy it, it only approximates it. So in lay terms, quantum systems are equivalent to quantum computations. So a quantum computer can simulate any quantum physical system. So, Deutsch disproves the classical Church-Turing thesis by counter-example. That is, he shows quantum systems can do things classical Turing machines can't. So, think here of Shor's algorithm as a concrete example, i.e. that the algorithm can break encryption, RSA encryption, that on a classic computer it is intractable, but on a quantum computer it isn't. But, Deutsch shows that a quantum Turing machine establishes a new principle, the Church-Turing-Deutsch thesis. So, unlike Turing's original thesis, which was essentially a conjecture that hadn't up to that point been falsified, this new thesis is proven to be true, provided quantum physics holds. So, it's a much stronger claim. The original Turing claim was something like this. We are guessing the Turing machine is universal, try to prove us wrong by counter example, which I want to point out is a perfectly valid scientific theory and a falsifiable one.

[00:27:48]  Bruce: And the fact that prior to Deutsch, no one had been able to disprove it was exactly why we embraced it in tentatively as true and what made it such a good theory. Now, Deutsch's revised claim goes something like this by comparison. If quantum theory is correct, the quantum Turing machine is universal. But go ahead and try to prove us wrong by counter example. So let's take a moment to talk about the strength of a theory or an argument. Now, a common modern crit-rat claim is that theories can't get stronger, that they're conjectural, and that you either prove them false or you don't. Deutsch's example of the Church-Turing-Deutsch thesis shows this view to be false. The original Church-Turing thesis was falsifiable, but for Deutsch, had never been shown to be false. To falsify it, you would either need a physically realizable algorithm that the Turing machine can't compute, or one that it computes much less efficiently, like exponentially slower or something like that. You could in theory come up with counter examples to it though. Turing's thesis was conjectural but falsifiable, yet was never falsified. Classic Popperian falsification. Deutsch's theory is similar. It still has that character. It could be falsified by a counter example, just exactly like Turing's thesis could be. But Deutsch goes further. He proves that producing a counter example would require new physics. This is why Deutsch's argument is stronger than Turing's.

[00:29:24]  Bruce: This shows that theories can be stronger or weaker, contrary to the modern crit-rat claim that there is no such thing as a stronger or weaker theory. Here is the key question. How did Deutsch make his theory stronger than Turing's? He increased the class of potential falsifiers. By tying his theory to quantum physics, you can now falsify this theory either by finding a counter example to the Church-Turing-Deutsch thesis or by refuting quantum physics itself. So the potential falsifiers just grew hugely larger. At first glance, if you're used to folk epistemology, this might seem to make Deutsch's theory weaker because it is riskier and potentially easier to falsify than Turing's. But here is Popper's whole point. Adding potential falsifiers actually strengthens the theory. It does not weaken it. Consider that to disprove the Church-Turing-Deutsch thesis, you now need to show quantum physics is false or potentially you could find a flaw in Deutsch's mapping, I suppose, but currently none is known to exist. In short, under Popperian epistemology and counterintuitively, a theory is stronger when it is bolder, riskier, and has more potential falsifiers. Why does having more potential falsifiers make a theory stronger and harder to overturn? This is actually very simple. There is now more to explain. Of course, I'm tacitly here assuming that we all accept Popper's no ad hoc rule, otherwise explaining, quote unquote, explaining things becomes trivially easy if I'm allowed to use ad hoc explanations. This is why the no ad hoc rule is central to making sense of Popper.

[00:31:14]  Bruce: With the no ad hoc rule assumed, and Popper is always assuming it, as a theory's content increases, competing theories must now explain more to stay competitive. That doesn't weaken your theory, it strengthens it. Let me introduce an idea that I call intellectual lift. And sometimes I've called this the burden of explanation. As a theory increases its class of potential falsifiers and continues to be corroborated, not falsified, it creates more problems that competing theories must also now explain. The more corroborated content, the heavier the intellectual lift, making it harder to even conceive alternative theories due to the increasing burden of explanation that they must now address. Deutsch's Church-Turing-Deutsch thesis created a huge, huge intellectual lift. To show Deutsch is wrong that AGI is possible on a computer, your intellectual lift now includes producing a theory of quantum gravity to replace quantum mechanics.

[00:32:17]  Unidentified speaker: Whoa!

[00:32:19]  Bruce: That's exactly what you get as you start to increase the class of falsifiers like this through your explanation, through your non-ad hoc explanation. This puts Deutsch's critics in an unenviable position. They now need to convert, convert to his view, become physicists and start working on quantum gravity, or give up on Popperian epistemology altogether, aka the scientific method, and turn to mysticism. There's really no other logical possibility open to them. Okay, so now, Ray is very much aware of Deutsch's argument. I don't know how deeply he understands it. I've never really talked with him about it.

[00:32:56]  Unidentified speaker: I had a chance to query him on it or anything like that.

[00:32:59]  Bruce: But he is at least high level aware that the paper exists and that Deutsch makes these claims. Ray is friends with David Deutsch, and both are hardcore critical rationalists. So in theory, they shared the same epistemology. Yet Deutsch, Deutsch's seemingly strong critical rationalist based argument does not faze Ray at all. So what does Ray argue instead? So let's take a look at his actual arguments. Now that we've finished looking at Deutsch's. Ray says in a tweet that he has, quote, crippling arguments against AGI for the studious acolytes of the brilliant David Deutsch and other keen thinkers. Then he offers a video of him reading a poem that he wrote that was based on the work of of [Doppst?] and Landgrebe. Landgrebe and their new book, Why Machines Will Never Rule the World, Artificial Intelligence Without Fear. And he says in his tweet, quote, a landmark book, he claims. Here's the actual poem, which I think is actually pretty telling.

[00:34:06]  Unidentified speaker: Okay.

[00:34:07]  Bruce: Turing can't map the Turing mind. The mind's a river, not to be mapped. No math can catch its eddies and flows. Through imagined space and times untapped, where logic lives, but passion knows. Conversation skips along its road, a dance unscripted, yet ruled by rules. It's led by aims we can't encode. It's binary, in binary, even those are fools. In Leibniz's lab, the stutter machine, it reckoned in steps, it reckoned all day. When asked what it might mean, it clicked and clacked and spat out a bean. For Landgrebe and Smith, AI's still a dream. The human mind won't fit in silicon. We can't yet model it nor build it alone. To some, it's just nonsense, a con dressed in chrome. You're not the map nor the map of the map.

[00:34:57]  Unidentified speaker: You're the unique bit that won't fit, just won't fit.

[00:35:00]  Bruce: Just accept it, that's being alive. Your mind doesn't live at E-5.

[00:35:08]  Unidentified speaker: By the way, I got one of the names wrong.

[00:35:10]  Bruce: It's Jobst Landgrebe and Barry Smith, who were the authors of the book. Here is the most relevant line to Ray's argument. It's, the mind's a river not to be mapped. No math can catch its eddies and flows. It's led by aims we can't encode. Ray's argument is that the mind can't be mapped or modeled by math or encoded. This is in fact a fair summary of Landgrebe's and Smith's book and their argument contained within. This poem was meant to capture their argument, and he's doing a pretty good job of capturing exactly what it is that they argue in the book. Ray makes a different critical rationalist based argument against AGI on a computer. In a separate tweet, I will read this in full. So he did a separate tweet. This one was against Elon Musk, and who also believes that you can do AGI on a computer, similar to David Deutsch. And this argument has some real obvious critical rationalist vibes to it. So I thought it was worth reading also. Ray's tweet said, so Ray put a video up, and the video is Duh Boy talking. So the tweet itself says, Duh Boy refutes Elon Musk and Geoffrey Hinton. Then there is an AI video, created video with Duh Boy. Duh Boy is his name. Duh Boy, get it?

[00:36:32]  Unidentified speaker: Duh.

[00:36:32]  Unidentified speaker: D-U-H.

[00:36:33]  Unidentified speaker: Duh.

[00:36:36]  Unidentified speaker: Duh Boy.

[00:36:37]  Unidentified speaker: Okay.

[00:36:37]  Unidentified speaker: Saying the following.

[00:36:39]  Bruce: No rule can dictate the wisdom or judgment required for its own application. When faced with a real-world problem, selecting the relevant rule and knowing how, when, and to what extent to apply it is always an act of conjectural judgment. This judgment draws on a tacit, context-sensitive grasp of the situation, an understanding that cannot be codified or fully articulated. It includes awareness of the rule's tolerances, the practical know-how needed to make it work, and the broader aims of the person or project at hand. Think of the skill of a plumber. No manual or algorithm, however detailed, can replace the lived experience and intuitive responsiveness that allows a plumber to improvise effectively when the textbook solutions fall short. This is why recent remarks from Elon Musk and Geoffrey Hinton are telling. Musk admitted, quote, AI is smart.

[00:37:36]  Unidentified speaker: It just doesn't have common sense.

[00:37:38]  Bruce: Hinton, when asked what young people should train for in the age of AGI, replied, quote, plumbing. Both recognize the limits of AI in a real world context. Outside narrow-defined probabilistically stable domains, AI is hopelessly clumsy. That last statement you're going to see comes very straight from the book that he cited previously. It is always human beings with their aims, imaginative leaps, situational awareness, and embodied know-how who provide machines with their tasks, formulate their rules, and decide when and how those rules are to be applied. And even then, such rules are viable only within tightly constrained domains. From a Popperian perspective, the application of a rule is a problem-solving activity, open to error, subject to revision, and informed by bold conjectures tested in practice. For Michael Polanyi [at angle?], much of what guides that application is tacit, not stated, indeed, unspecifiable, yet crucial. The skillful act is never fully captured by the rule.

[00:38:47]  Unidentified speaker: It transcends it.

[00:38:49]  Bruce: In conclusion, duh. Note that this tries to be an argument from critical rationalism, a claim that critical rationalism rules out any possibility of AGI. Also note that Ray is, at least jokingly, claiming that this is so obvious that it is a duh. Before diving into Landgrebe's and Smith's arguments, which Ray cited and is basing his arguments on, let's assess Ray's arguments so far in terms of intellectual lift. I'm going to claim that the intellectual lift between Ray and Deutsch is not comparable. Deutsch's argument exposes itself to falsification. It is testable and objective. Ray's claims, so far at least, are intuitive or subjective. You either resonate with, quote, the skillful act transcends the rule, or you don't. There's no clear way to test it independently. In other words, it is, in Popper's terms, ad hoc. Also, we do already capture many skillful acts via machine learning today. So clearly Ray doesn't mean no skillful act at all can be turned into algorithms. If he had claimed that, it would have been a universal claim and thus a falsifiable theory. And it would therefore be easy to show it to be false. But it isn't clear how to respond to Ray's actual argument since it makes no testable or falsifiable claims. How would one falsify, quote, it is impossible to capture every skillful act by an algorithm? Isn't that exactly equivalent to saying, there exists a skillful act that can't be turned into an algorithm, which is clearly now a purely existential claim, and thus can't be falsified?

[00:40:33]  Bruce: Wouldn't one have to prove that every single skillful act can be turned into an algorithm, to quote, refute Ray's position? Clearly an impossibility. That is what makes Ray's theory a purely metaphysical theory in Popperian terms. Okay, now maybe we could falsify it, though. Maybe we could falsify Ray's theory, something like this. What if we, I don't know, this is a stretch, bear with me. I don't know, what if we could prove that all of physics maps to computation? If we could prove that, that would falsify Ray's theory, right? Then we'd know for sure that Ray's theory must be wrong, and it would be falsified. But wait, that is Deutsch's argument, isn't it? But Ray doesn't accept it as a falsification of his theory. So what would Ray accept as a falsification? Note that only Ray can answer this question. I can't answer it for him. If I did, I'd consider his theory already falsified. He has to choose to formulate his theories in a falsifiable and thus testable way so that we can test it. By contrast, Deutsch provides a bridge. Even without intuition about quantum physics, you can learn in theory, you can learn the math and check the argument for yourself. In other words, Deutsch relies on an objective argument, whereas Ray is relying on a subjective one. Popper's epistemology can be summarized, in my view, as follows. Ignore subjective arguments and focus on what everyone can critically assess. This is the intersubjective nature of science in a nutshell.

[00:42:08]  Bruce: Now, so far, Deutsch's argument aligns with proper critical rationalist scientific methodology, while Ray's has a far more intuitive character.

[00:42:17]  Unidentified speaker: Consider this.

[00:42:18]  Bruce: How does Ray address the intellectual lift that Deutsch's argument demands? Does he, for example, point to a flaw in Deutsch's mapping between quantum physics and quantum computation? If so, where does he do that? Does he propose alternative physics? If so, where does he do that? Does he challenge the epistemology or scientific method underlying the argument? If so, where? In short, it is not clear how Ray engages with the intellectual lift Deutsch's argument requires. Now, of course, we're talking about two tweets. Hardly the depth of Ray's argument. And to be fair, Ray does point to a whole book on the subject by Landgrebe and Smith. By the way, Langerba is how I've chosen to pronounce it. I think it can also be pronounced Langer-B. When I looked it up online, it said both were proper pronunciations.

[00:43:08]  Unidentified speaker: So I'm going to go with Langerba.

[00:43:10]  Bruce: So it makes sense to next assess their arguments and see if they address any of these problems. So Jobst Landgrebe and Barry Smith argue that building an artificial general intelligence, an AGI, which would be an AI capable of equaling or exceeding human intellect, they claim it is barred by fundamental mathematical limitations. They claim this is not just an engineering challenge or a limitation of current computational theory. They claim that it is mathematically impossible to create an artificial mind. Their conclusion is AGI is impossible, meaning they state no future advances in technology or computing engineering can overcome this. They claim it is analogous to trying to build, this is exactly what they say, to build a perpetual motion machine. Their supposed conclusion rests on the alleged incompatibility between the complexity of human intelligence and the rigid nature of current mathematical models, which they claim form the foundation of computational AI. Now, let's give credit to Landgrebe and Smith here. They are attempting a claim—to make a claim with intellectual lift comparable to Deutsch's. Their claim implies that it should be mathematically provable that computers cannot instantiate minds. And this is what they are going to strive to prove. If they're correct, it should be possible to walk through the math and verify it personally, because that's what we can do with Deutsch's argument, correct? So this should also be possible to do. Do the arguments of Landgrebe and Smith interact with...

[00:44:58]  Bruce: How do the arguments of Landgrebe and Smith interact with Deutsch's arguments? Do they interact at all? No, not directly by name. They don't mention David Deutsch, they don't mention the Turing-Deutsch thesis, etc. But they do indirectly engage it, engage with his argument, by their analysis of the Church-Turing thesis. They don't seem to be aware of the Church-Turing-Deutsch thesis, but they're very aware of the Church-Turing thesis. So their core argument, as they see it, they see the Church-Turing thesis as in their favor. They see it not as a physical principle, as Deutsch formulates it, but as a limit on math itself that physics isn't bound by. One might rightly say that their whole argument is this. The Church-Turing thesis is a limit on math that nature does not have. Thus, nature can create minds, and we, using our math and computers, can't. They argue this is because all computational AI relies on Turing-compatible mathematical models, which, by the way, is true, which are limited to simple predictable systems and cannot capture complex dynamic systems like the human brain. Since all AI systems, regardless of future advances, they're arguing, are bound by these Turing-compatible constraints, they don't see how it will ever be possible to build AGI on a computer. Thus, they see the Church-Turing thesis as proving AGI is impossible on a computer, whether then, as Deutsch argues, it proves it is possible on a computer.

[00:46:34]  Peter: It seems like more people, why aren't more people aware of the Church-Turing-Deutsch thesis? Like, Nick Bostrom's simulation hypothesis, I hope this doesn't get us around a tangent. But I don't think he even mentions the Church-Turing-Deutsch thesis. That seemed like it would support his whole argument quite well.

[00:46:58]  Unidentified speaker: I wanted that, too.

[00:46:59]  Bruce: Like, how could you write an entire book on this subject and not even be aware of the Church-Turing-Deutsch thesis and what Deutsch's argument is and directly tackle it? Like, that blows my mind that Landgrebe and Smith did not even mention David Deutsch.

[00:47:13]  Unidentified speaker: Is it just not well known?

[00:47:16]  Bruce: So it clearly isn't super well known by comparison to the Church-Turing thesis.

[00:47:21]  Peter: But Deutsch invented quantum computers, I mean, at least theoretically.

[00:47:28]  Bruce: And this is an incredibly famous paper that we're reading from, right? Like it is the paper that led to the whole quantum computer drive that we're all going down. I just recently went to a CIO's lunch or something where they were talking about how quantum computers are going to change everything and because of quantum encryption is going to be broken. This is something that's become part of the mainstream thought of people just that are CIOs doing business, right? So it's a completely fair question.

[00:48:00]  Unidentified speaker: Like I don't, it does show a lapse in my opinion, that Landgrebe and Smith don't even mention the competing theory that they actually have to deal with, right?

[00:48:13]  Bruce: Like it shows a certain ignorance of the critical discussion. Because they do mention Church-Turing thesis and because that's similar enough to Church-Turing-Deutsch-Thesis, I think we can still actually work out what their arguments would be and address them, which is what I'm going to do.

[00:48:33]  Unidentified speaker: Okay.

[00:48:34]  Bruce: So with this in mind of everything I just said, let's review Landgrebe and Smith's three main arguments, at least what I consider the three main arguments. So I've reordered their arguments slightly to make the flow clearer, cleaner for this discussion. I'll present each argument in a steel man form. I'll try to make it sound as good as I can and try to make it sound the way they would say it. Then I'm going to explain what is problematic with each one. Finally, I will show why their interpretation of the Church-Turing thesis is deeply problematic compared to Deutsch's. So let's start with argument one. Landgrebe and Smith, they say that complex systems like the human mind do not produce stable reliable data needed to train AI. Modern AI, for example, deep neural networks, and that does seem to be what they have in mind, relies on patterns from large representative datasets. Complex systems violate the mathematical conditions needed for such data. So there is no reliable data for training. Complex systems do not generate data from a stable multivariate distribution. So truly representative samples cannot be drawn and then trained on. Context is always changing. Human systems are context dependent, lacking fixed boundaries. AI models assume context-free relations, which fails here if you're dealing with a complex system like the human brain. A concrete example they use is financial markets. Modeling the New York Stock Exchange price movements is impo... This is their example, by the way, is impossible because it's a hybrid complex system with unpredictable human behavior.

[00:50:22]  Bruce: Even accurate short-term models fail long-term due to chaotic, unpredictable dynamics that prevent stable statistical distributions for training. And because of model-induced escape, success of a model changes the system as participants react to the predictions. This is actually something that is known to be true, by the way, that if you try to predict... People obviously do write machine learning models to predict the stock market, and people do make money off of doing that. But it's hard to tell to what degree they just got lucky, and to what degree they actually captured something true about the stock market. And the moment if they did capture something true, reflexivity kicks in.

[00:51:04]  Unidentified speaker: That's what I was about to say.

[00:51:05]  Bruce: Yeah, so everybody immediately goes, oh, I can do that too, and then that causes the stock market to stop having that stability.

[00:51:12]  Unidentified speaker: Okay.

[00:51:12]  Peter: Yeah, it's almost like it would have to model its own behavior because it's affecting things.

[00:51:19]  Unidentified speaker: That's right.

[00:51:19]  Peter: It just kind of becomes a contradiction, I guess.

[00:51:22]  Unidentified speaker: That's how I understand it.

[00:51:23]  Unidentified speaker: Okay.

[00:51:24]  Bruce: That is the argument that Landgrebe and Smith are making.

[00:51:28]  Unidentified speaker: Okay.

[00:51:28]  Bruce: So summary, the mechanisms of human intelligence, they are chaotic, evolving, context-dependent, fundamentally defy fixed-rule data-averaging models, making their claiming AGI impossible. So let's talk through a rebuttal now. First of all, let me just point out, I really did try to put that as steel manned as I could in giving concrete examples of their point, agreeing with them where I could.

[00:51:56]  Unidentified speaker: Okay.

[00:51:57]  Bruce: I say this because I'm about to show you that pretty much this is a really stupid argument.

[00:52:01]  Unidentified speaker: Okay.

[00:52:02]  Unidentified speaker: So let's take the claim.

[00:52:04]  Bruce: Complex systems like the human mind do not produce stable, reliable data for training AI, making AGI impossible.

[00:52:11]  Unidentified speaker: Here's the counterpoint.

[00:52:13]  Bruce: We need to separate a narrow, specific view of AGI, of AI I mean, current deep learning approaches from the general argument that AGI is physically computable, which is Deutsch's actual view. So yes, you could find AI researchers that sincerely believe AGI is just a matter of a bigger neural network and more training data. Okay, you could find people who believe that. Jan LeCun, at least in the past, has made arguments to that effect. And as of today, he has suggested putting a deep learning neural network into a robot and training it on all sensors rather than using a large language model. And he thinks this might be the step we need to get to AGI. He's actually started a billion dollar company around that whole idea. But deep learning is really just one specific machine learning technique, not the only possible technique. AGI will likely require entirely new techniques. In fact, that is specifically what Deutsch argues in his AEON article. So if Deutsch is correct, the technique that we need for AGI may be very different from today's methods. So here's the key point. The fact that current deep learning has limitations that can make it so it cannot achieve AGI, true statement, says nothing whatsoever about whether AGI is physically computable. It only tells us that the current techniques are insufficient. A point Deutsch wholly agrees with them on. They are mistakenly assuming something like this. Deep learning can't get to AGI, as if that's the only possible way that you could do AI.

[00:54:01]  Bruce: Therefore, Landgrebe and Smith's argument really has no teeth at all against Deutsch's general thesis. So Landgrebe and Smith used the stock market as a concrete example of an unpredictable symptom. So observation here, humans can't consistently predict the stock market either, any more than a machine learning algorithm can. So there is no need for an AGI to predict the stock market, to be equivalent to a human, because humans can't predict it either. So this unpredictability argument has no teeth either. This is their real mistake. They assume AGI is about prediction when it isn't. It will actually be about explaining things. Quote, the core argument is that engineering an AI emulation of a complex system requires a mathematical model that enables prediction of the system's behaviors. However, the authors contend that it is impossible to build such models for complex systems for fundamental mathematical reasons. This impossibility is primarily understood as a mathematical impossibility, meaning a solution cannot be found due to an a priori reasons of mathematics, not just technical or physical limitations. But in reality, you don't need an AGI to predict the stock market. An AGI only needs to be one more person using the stock market. This misunderstanding around a model needs, the models need to predict something specific, is going to prove key to understanding what Landgrebe and Smith are getting wrong in their other arguments. So I want to really call attention to it just to make it clear. They are confused on a very important point.

[00:55:48]  Bruce: They think the fact that you can't predict the stock market proves AGI is not possible when a real general intelligence can't predict the stock market either, because prediction has got nothing to do with being a general intelligence. I want to make a point here about the large degree of actual factual agreement. This is something that I occasionally bring up, but I only want to kind of drive it home. It is not uncommon for the two sides of a critical discussion to factually agree on everything. So I see no disagreements over facts here at all so far. Landgrebe and Smith are stating that current AI techniques can't get you to AGI, and Deutsch and I are agreeing with them. I'm stating that there are other techniques and more will eventually be invented, so we can't draw any conclusions from that first statement.

[00:56:44]  Unidentified speaker: That's a stated fact.

[00:56:46]  Bruce: I can't say if they would agree with me factually on that or not, but if they don't agree with me, they would be factually incorrect and easy to disprove. Of course, we're going to discover new machine learning techniques in the future that don't exist today. For them to make a claim, no, there will be no more future discoveries in machine learning would be immediately falsifiable and would be immediately falsified the moment a new technique was discovered. They are pointing out that you can't model the stock market in the sense of predicting it. And I'm agreeing with them. I'm pointing out that humans can't model the stock market either in the sense of predicting it, either. And they are not disagreeing. So there's no actual factual disagreement in this critical discussion. So what is it we're disagreeing over? What we're disagreeing over is merely if one can infer from, quote, this is not a true quote, but an imagined quote, deep learning can't predict the stock market to, quote, there does not exist an AGI program. I'm going to say that again because I feel like this is important. Can you infer from the statement, deep learning can't predict the stock market to, there does not exist an AGI program? That is what we're disagreeing over. They think that's a valid inference. I don't think it is. The necessary bridge from the premises to the conclusions in their inference is entirely missing here. You either share the subjective intuition that this is a fair inference or you don't, and I don't happen to.

[00:58:22]  Bruce: This isn't like with Deutsch's argument, where I can go check for myself if I disagree with it because the entire argument is objective and not merely a subjective intuition. You have to basically just get the intuition that Landgrebe and Smith desire you to get. If you don't, there is nothing they can do to help you over that broken bridge because there is no logical inference here. It's just them intuitively saying, I think this link makes sense. This is a point I really want to drive home. It is actually quite rare for there to be at all that much that we are disagreeing over, other than the epistemology itself. For example, if you buy the Popperian idea that we constrain critical discussions to not include subjective uncheckable statements and only utilize the facts that we agree upon, or at least give those more weight, this seems like a pretty open and shut case. Their argument is invalid as a basis for the inferences that they're drawing from it. They would not be able to disagree with that statement if they're accepting Popper's epistemology because it is a mere factual statement due to the caveat of accepting Popper's epistemology. For example, it makes no claim that if we do not constrain ourselves to Popper's epistemology, that we couldn't draw a different conclusion. In fact, if we want to say go with mysticism, of course we could draw a different conclusion. And I don't see how they could factually argue with me that if we did constrain ourselves to only testable theories and factual statements, that their inference is anything but a leap of logic.

[01:00:02]  Bruce: Specifically, what I have in mind here is that if you don't feel you need to constrain yourself to testable theories and factual statements, surely they can easily here insert a subjective argument something like this. I personally feel that the failure of deep learning implies a future failure of all future AI techniques, including ones not yet discovered. Now, this is almost certainly what their real argument here is. One of a personal subjective feeling that I can't really meaningfully try to challenge. Can you see the fundamental difference in character between this argument and Deutsch's? The difference in character is what Popperian epistemology actually is. Okay, let's go to their second argument. Unlike simple systems, machines or planetary orbits, complex systems are dynamic and evolutionary. They have a creative character, producing new elements and patterns continuously. Mathematical models require static structures with fixed conditions for reliable predictions. Human systems, however, are constantly evolving, gaining new experiences, skills and intentions, making any model quickly obsolete. So a concrete example that they use, human language. Language is always changing. New words, phrases, styles and registers emerge continuously. This evolution is non-repetitive and erratic, making the stable structure needed, making—lacking the stable structure needed for modeling. AI cannot, they claim, obtain adequate training samples, as human language does not follow a stable statistical distribution.

[01:01:38]  Bruce: Human conversation depends on intention, content, context, and continuous change, making AI emulation fundamentally limited. As with argument one, Landgrebe and Smith seem to misunderstand the goal. They treat AGI as needing to predict complex systems when it only needs to participate in complex systems as a human would do. This problem is especially clear in their language argument that I just went over. Ironically, the rise of large language models, which were already undercut their argument. Even current deep learning systems emulate human language with remarkable proficiency. Now, their book appeared early in the LLM era. So they existed, but we weren't too far into the era yet. It was back when it was easy to point to embarrassing failures and say, see, language is a complex system, so machines can't model it. Consider this quote from their book, quote, trying to emulate human language with an AI is futile. Machines trained on past language patterns are fundamentally unable to cope with the variants and new structures that continuously emerge in human dialogue. But here's the catch, those failures can be fixed through fine tuning. And over time, LLMs have improved dramatically by tuning it. Not towards AGI, but enough to refute the claim that language is beyond computational modeling. One might here object, but LLMs still make mistakes today. But so do humans. They might then argue, but humans can learn. But so can LLMs via fine tuning, which is the machine analog to learning over time.

[01:03:22]  Bruce: The comparison Landgrebe and Smith want only works if you unfairly freeze the AI while allowing the humans to continue to learn. The key point is this, ordinary deep learning can already model a complex evolving system, like human language, at a level of proficiency that many humans, most humans envy. If that doesn't expose a flaw in their argument, then their arguments risk being unfalsifiable.

[01:03:51]  Unidentified speaker: Okay, argument three.

[01:03:52]  Bruce: The human brain and central nervous system are treated as a complex dynamic system. While obeying the laws of physics, they admit that much, they, the brain and the nervous system, possess properties that make them resistant to predictive mathematical modeling, again predictive, needed for AI emulation. So the key issue then is that the astronomical number of variables, trillions of molecular components across billions of neurons. Understanding how brain events relate to mental experiences, they claim requires navigating a system with greater than 10 to the 15th bodies. Based on that, they then give a concrete example, the n-body problem in classical physics. Calculating the motion of multiple celestial bodies becomes unsolvable for n greater than 2. This is the famous three-body problem, made famous by a series of books and an excellent Netflix adaption. Part of the plot of that story, meaning the three-body problem, was that it is impossible to use a computer to model even a three-body problem. So the Trisolarans couldn't figure out what their seasons would be, because no computer could model it unlike ours, where the seasons were predictable, thus leading to a chaotic life. So Landgrebe and Smith argue that modeling the entire system underlying human consciousness, which they claim is n greater than 10 to the 15th, is completely impossible because n equal 3 can't be modeled, much less n greater than 10 to the 15th. This is their strongest argument by far, precisely because it doesn't rest on subjective intuitions.

[01:05:38]  Bruce: Instead, it makes a clear falsifiable claim, one that as we will see, is in fact false and easy to falsify. One caution first, they have no principled way of knowing how many variables a human like mind requires. Appealing to numbers which they got from the number of atoms strikes me as question begging. Still, since their argument is meant to apply even to something as simple as a three-body problem, let's engage it on that concrete example, using their own framing of a three-body problem. Can you model an n-body problem on a computer? That's the question that we're now actually asking.

[01:06:16]  Unidentified speaker: Let's get more specific.

[01:06:17]  Bruce: Can you model a three-body problem on a computer?

[01:06:23]  Unidentified speaker: That's the question we need to ask.

[01:06:26]  Bruce: Turns out that the answer is not a strict yes or no, the way I just worded it. You would need additional context to be able to answer the question. If by model, you mean model a specific three-body problem, then I would agree that it is impossible to model any one specific three-body problem. So again, consider the three-body problem from the Netflix series or the books. The idea here is that trying to model three suns and a planet can't be done because you can't measure the initial positions, initial conditions, with enough precision to model them perfectly and they immediately start to diverge from the computation. In this sense, and only in this sense, it is, quote, impossible to model a three-body problem.

[01:07:15]  Unidentified speaker: And everybody knows this.

[01:07:16]  Bruce: Everybody knows, quote, it's impossible to model a three-body problem. But they actually mean it only in this sense that I've just clarified.

[01:07:23]  Unidentified speaker: A specific three-body problem.

[01:07:25]  Bruce: By the way, Deutsch admits that this is true in Fabric of Reality. So there's no actual factual disagreement here. This isn't some surprise reveal that Deutsch hasn't considered. Here's the actual quote from Fabric of Reality. I've already said that a perfectly accurate rendering of a roulette wheel need not, indeed should not, give the same sequence of numbers as the real one. Similarly, we cannot prepare in advance a virtual reality rendering of tomorrow's weather, but we can or shall one day be able to make a rendering of weather, which, though not the same as the real weather conditions prevailing on a historical day, is nevertheless so realistic in its behavior that no user, however expert, will be able to distinguish it from genuine weather. Okay, that was Deutsch's actual claim. Notice that it is not at odds with their claim about the three-body problem. So Landgrebe and Smith have again misunderstood something rather fundamental. They are wrongly trying to equate modeling a specific three-body problem with modeling any three-body problem.

[01:08:32]  Unidentified speaker: Let's ask the right question.

[01:08:34]  Bruce: Is it possible to model any three-body problem? Does the math stop us from doing this? No, not at all. This is trivially easy. In the books in the Netflix series, they actually do model the three-body problem in a VR simulation as part of the story. It just isn't the same one as the Trisolarans because that's what's impossible. In fact, it's so easy to model a three-body problem that a 10-year-old can do it in an afternoon on the Unity game engine. AGI is the same way. It would not be possible short of, let's say, an Omega point computer, to model a specific human mind that would, in fact, maybe be a 10 to the 15th body problem. But you could model a new AGI mind of your own making that isn't anyone specific and is its own person. This is a much, much easier thing to do because you aren't required to capture every single 10 to the 15th power of variables as Landgrebe and Smith believes you must. Instead, all you have to do is understand how the overall system works, and then let it develop its own 10 to the 15th variables over time.

[01:09:49]  Unidentified speaker: So that's, yeah, that's pretty much what the ChatGPT says here, is that it's about the rounding errors.

[01:09:57]  Unidentified speaker: That's right.

[01:09:58]  Unidentified speaker: They essentially accumulate over time.

[01:10:00]  Unidentified speaker: They have completely misunderstood this. They have completely misunderstood this.

[01:10:02]  Unidentified speaker: So if you're having to round anything, then the errors essentially magnify, I guess, whereas in reality...

[01:10:12]  Unidentified speaker: That's exactly what...

[01:10:13]  Bruce: That's why you can't model a specific three-body problem, but you can model a three-body problem.

[01:10:20]  Unidentified speaker: Wait, wait, say that one more time?

[01:10:22]  Unidentified speaker: You can't—you cannot model a specific three-body problem.

[01:10:25]  Bruce: You can't replicate a three-body problem that exists out in nature.

[01:10:28]  Unidentified speaker: Oh, okay.

[01:10:29]  Bruce: But you can make your own virtual three-body problem and it will work exactly like a three-body problem.

[01:10:35]  Unidentified speaker: Okay.

[01:10:36]  Unidentified speaker: And it will be indistinguishable from the one in nature.

[01:10:38]  Unidentified speaker: I see.

[01:10:39]  Unidentified speaker: It will be indistinguishable from one you found in nature.

[01:10:41]  Bruce: It would not be the same as any one in nature.

[01:10:45]  Peter: Because there's just too many multivariate conditions in real life.

[01:10:50]  Bruce: You would have to measure it to a degree of...

[01:10:53]  Peter: And you could just never theoretically do it, I suppose. But in practicality, you could never measure all of the positions of all the things that influence a solar system or a galaxy.

[01:11:11]  Unidentified speaker: You can't even just measure...

[01:11:13]  Bruce: Even if you just took the three suns, let's say.

[01:11:16]  Unidentified speaker: Yeah, yeah.

[01:11:17]  Bruce: And in fact, even if you assumed the three suns weren't putting off energy and they were just balls, you still would not be able to measure their position in space so accurately that you could replicate that specific three-body problem.

[01:11:30]  Unidentified speaker: I see.

[01:11:30]  Peter: But if you were to simulate your own solar system...

[01:11:36]  Unidentified speaker: It would be just fine...

[01:11:37]  Peter: .with presumably just as much complexity.

[01:11:43]  Bruce: Just as much complexity.

[01:11:44]  Unidentified speaker: That's correct.

[01:11:44]  Unidentified speaker: You could solve the n-body problem or whatever.

[01:11:50]  Unidentified speaker: You can.

[01:11:51]  Bruce: That's exactly right.

[01:11:52]  Unidentified speaker: That's a really interesting point, actually.

[01:11:54]  Unidentified speaker: Yes.

[01:11:55]  Unidentified speaker: Okay.

[01:11:56]  Bruce: So let me put this clearly and a bit bluntly. Longreba and Smith, I think I've pronounced Longreba every single possible way throughout this episode.

[01:12:05]  Unidentified speaker: So apologies to him.

[01:12:08]  Bruce: Landgrebe and Smith have fundamentally misunderstood the problem that they are trying to solve. Fundamentally misunderstood it. They see the three-body problem as, quote, unmodelable when really a 10-year-old can model it all the time. They are confused between the difference between modeling a specific three-body problem and modeling any three-body problem. So their argument doesn't merely have no teeth.

[01:12:32]  Unidentified speaker: It is self-undermining.

[01:12:34]  Bruce: There is no reason at all to believe we can't create an AGI, even if modeling 10 to the 15 complex variables is impossible mathematically.

[01:12:42]  Unidentified speaker: Why?

[01:12:42]  Bruce: Because you don't have to model anything to be an AGI. You have to merely be an AGI. To put this in the same terms, you don't have to model another three-body problem. You simply have to be a three-body problem. The exact same thing applies to AGI. Now, I hope I've convinced you that Landgrebe and Smith have failed in their task. They have not only not shown that AGI can't be programmed on a computer, they have instead demonstrated that they severely misunderstand what that even means. Put another way, none of their arguments carry any intellectual weight or intellectual lift at all. There is no actual problem here that needs to be solved. It was all just pseudo problems based on misunderstandings of the problem in question. However, I have a bigger concern with their argument that goes beyond even what seems like a serious misunderstanding of the problem to be solved. There is an outright incoherence to their arguments that I want to draw out now. To see why, consider this question. How do Landgrebe and Smith explain that nature regularly does create human minds in a completely replicable way?

[01:13:54]  Unidentified speaker: By giving birth to children.

[01:13:56]  Bruce: I have searched long and hard for a straightforward answer to this question in their book. And honestly, all I can find is a lot of slinging of philosophical jargon. Let me give you the actual quote where they asked, where they answered this question.

[01:14:10]  Unidentified speaker: Okay.

[01:14:11]  Bruce: Our own position can be called a no-layers approach, in that it embraces a materialistic monism, monist view according to which mental processes are physical processes. There are no layers in the human mind-body continuum, thus no underlying neurophysiological substrate, but rather a continuum of physical processes, which we can view at different levels of resolution. The mental processes are emanations of complex physical processes, down to the four basic interactions of matter, at which is the fundamental granular level of microparticles. Mental processes are themselves identical with the processes which conform to the laws of physics, but we have no way of describing, explaining, or predicting mental processes in terms of physical laws. I confess, I don't know what any of that means. And I also confess, I don't, I don't really believe they know what any of it means either. I don't even think that is a meaningful set of sentences. This reads a lot like a religious creed to me, complete with pedantic words and overall incoherence. Calling it a word salad would be doing word salads, be doing a disservice to the words word salad.

[01:15:26]  Unidentified speaker: Here is the line that...

[01:15:28]  Unidentified speaker: Shots fired, Bruce.

[01:15:29]  Unidentified speaker: Wow.

[01:15:31]  Bruce: Here is the line that really stands out to me though. We embrace a materialistic, [monist/modest?] view according to which mental processes are physical processes, but, quote, we have no way of describing, explaining or predicting mental processes in terms of physical laws. So they are insisting that they are materialists, but ones that believe it is impossible to have physical laws that can even just explain mental processes. To be clear, they are literally claiming it is impossible to explain mental processes in terms of physical laws. That is their stance. I am really, really unclear how this is anything but supernaturalism, with a faux-materialist label slapped on it. But given their view, they apparently feel no need to offer an explanation of how nature pulls this off, given that their theory is that nature can do it, but no physical laws can describe it or explain it. Their theory is, yes, nature can do it, but there are no physical laws that could ever explain it. We're not going to offer you an explanation of how this is possible because there is no explanation of how this is possible. This is their actual stance. To really drive this point home, how truly silly it is, how truly, stupidly, foolishly silly it is. Recall that they claimed creating an AGI is like trying to create a perpetual motion machine. Now suppose nature regularly created perpetual motion machines, even though our physics showed this to be impossible.

[01:17:07]  Bruce: Would it make sense to say, oh, well, nature can do things physics can't because nature is complex and physics is modeled by using math that is Turing compatible. And so it can only, it is only possible for us to build perpetual motion machines. Nature has no such bar. Would you consider this a valid explanation?

[01:17:27]  Unidentified speaker: I would not.

[01:17:30]  Bruce: Now let's look at why, from a critical rationalist standpoint, this isn't a good explanation. From a critical rationalist standpoint, the core problem is that their claim is their claim that mental processes have no possible explanation, and therefore they owe us none. I think this always comes down to epistemology. To make this clear, consider an analogy with jurisprudence, which I think is a very interesting analogy, trying to show us the similarities between jurisprudence and epistemology. In jurisprudence, the job isn't just interpreting the existing laws, which are often vague. It's often about the rules you are making or remaking. Judges can't simply say, I rule this way, and give no justification. They also know that whatever justification they give becomes a precedent. So we judge jurisprudence by whether its explanations can be applied to future cases without breaking the law as a coherent set of explanations. I'm arguing that epistemology works the same way. Landgrebe and Smith's explanation has to be evaluated by whether we can live with the precedent it sets. If they're allowed to say, we're materialists, but mental processes can't be explained by the laws of physics, so we owe no explanation of how nature builds minds, then that becomes the new precedent, rationally speaking. And once that precedent exists, we have to accept it as a valid future move in future arguments, even ones that we strongly disagree with. So for example, creationists now get to invoke that precedent against us. Can you live with that?

[01:19:15]  Bruce: If not, then you also have to accept that this is a bad explanation. This is the correct way of understanding epistemology and the correct way to do critical rationalism. You need to think of it not in terms of which theory you are declaring best, but in terms of the rules you had to invoke to get there, and whether those rules can themselves be explained, articulated, and then frankly handed over to those you disagree with, and let them invoke them too, and you're prepared to live with them. All of this was really a way for me to drive home why the concept of what I'm calling intellectual lift really matters in epistemology. Intellectual lift is not only rooted in the idea that we owe explanations, but also that these explanations are not ad hoc. That means that they are independently testable, and that the whole point of the exercise is to increase the whole point of the exercise, is to increase the explanatory power of our explanations. Intellectual lift is really the problems you leave for competing theories that your theory can explain and that any future theory must also now explain too. The goal of critical rationalism is not falsification of competing theories per se, but to demonstrate that your theory has more still unfalsified content than the competitors. So, what this means is that the burden of explanation is not something a critical rationalist wants to give away, like a burden of proof, but instead wants to take upon themselves.

[01:20:51]  Bruce: You win the critical rationalist battle not by showing your theory isn't refuted, that's just regular folk epistemology, and even creationists play that game. But by having the theory with the largest burden of explanation, that is the game you're trying to win.

[01:21:06]  Unidentified speaker: That's epistemology.

[01:21:10]  Bruce: Why pick the Deutsch version of CTD, Church-Turing-Deutsch thesis, over the Landgrebe and Smith version? Again, this is easily resolved via critical rationalism. Deutsch's version constrains more. You have more potential falsifiers. Just show the existence of a physical process that doesn't fit quantum computation. The Landgrebe version un-constrains the explanation. It merely says, there exists a physical process that doesn't fit quantum computation, but doesn't bother to tell you what that process is. Or worse, it claims that there is no possible explanation, so they aren't going to even bother to try to explain it. This is an existential theory, so it can't be falsified.

[01:21:51]  Unidentified speaker: It tries to make the burden of explanation the problem of the other theory, rather than show that their theory can handle that burden.

[01:21:58]  Unidentified speaker: It says, na-na-na-na-na-na, you can't prove I'm wrong, rather than, as Deutsch says, my theory says it's impossible.

[01:22:05]  Bruce: Now go try to prove me wrong by finding a counter example. I'm laying out for you exactly what you need to do, because this is exactly what a counter example would look like.

[01:22:14]  Unidentified speaker: Now, Ray himself said this well in his excellent book, The Myth of the Closed Mind.

[01:22:18]  Unidentified speaker: Quote, if someone rejects a theory's status as law, then it is incumbent upon them to supply alternatives that have at least the same truth content.

[01:22:27]  Unidentified speaker: It's from page 114.

[01:22:29]  Bruce: This is the correct understanding of the burden of explanation in a nutshell. He said this in referring to economic laws, by the way, but I think it can be easily applied to any theory. Put straightforwardly, if Landgrebe and Smith reject Deutsch's proof that quantum mechanics implies AGI is possible, it is incumbent upon them to supply an alternative theory that has at least the same truth content, which in this case means offering an alternative to quantum mechanics.

[01:22:58]  Unidentified speaker: They need to be physicists.

[01:22:59]  Bruce: They need to actually show the physics that explain this. Instead, they've supplied what is called an intuition pump. They've offered an intuitive argument as to why they find the idea of AGI intuitively wrong based on a series of vague analogies on the difficulties of modeling complex systems, which unfortunately is an argument only someone that subjectively shares the intuition will find to be a good argument, making it a purely subjective argument without any truth content that we can objectively check. We need a name for this rational fallacy. I propose we call this rational fallacy armchair philosophizing, meaning offering a vague philosophical alternative explanation, trying to displace a more solid, non-ad hoc, independently testable explanation. The idea is that when confronted with a strong counter-theory with high content, you don't address the theory directly because you can't, but instead you offer some sort of intuition pump with no independently testable content at all as an alternative theory. It is my understanding that this is precisely what Popper was by convention suggesting we disallow as a move. Because if we allow it, then every theory can be saved this way. Intuition pumps like this are a dime a dozen. Theories with actual testable content are not. They are hard to come by and rare. So in summary, the Percival-Deutsch debate over whether AGI can be built or not is not a matter of opinion. Under critical rationalism, our best theories give a clear verdict. So at least one side is mistaken about the objective state of the discussion.

[01:24:44]  Bruce: Deutsch's Church-Turing-Deutsch thesis re-casts Church-Turing as physics and proves that if quantum physics holds, AGI is possible on a computer. So the saying, our best theories imply AGI is possible, is a factual claim that no one, not even Ray Percival, is denying. This debate illustrates the idea of intellectual lift. A bolder theory with more potential falsifiers is stronger, not weaker, because rivals must now explain more, including, in this case, replacing quantum mechanics to overturn the Church-Turing-Deutsch thesis. Ray's case via Landgrebe and Smith is subjective and unfalsifiable. It's an intuition pump that the skillful act transcends the rule, carrying little intellectual lift compared to Deutsch's objective testable argument. All three Landgrebe and Smith arguments fail. They conflate AGI with prediction and confuse modeling a specific system with modeling any such system. AGI need not predict markets or brains. It need only participate as humans do. It need only be a brain. This position is also incoherent. Their position is also incoherent. Claiming materialism while denying any physical explanation of mind is supernaturalism in disguise and the precedent it sets would equally license creationism or really anything. So the bottom line, a critical rationalist debate is won by taking on the burden of explanation. You take on the burden of explanation. That's how you win a critical rationalist debate, by increasing the testable content of one's theories.

[01:26:20]  Bruce: Reject a high content theory like Church-Turing-Deutsch thesis, and you owe an alternative of equal truth content, not armchair philosophizing about why you subjectively feel the theory must be false. And that's the end of my summary.

[01:26:36]  Peter: Well, this was a fiery episode and I liked it, and I agree that it's above average, and thank you, Bruce.

[01:26:48]  Unidentified speaker: All right, you're welcome.

[01:26:57]  Unidentified speaker: Hello again.

[01:26:58]  Peter: If you've made it this far, please consider giving us a nice rating on whatever platform you use, or even making a financial contribution through the link provided in the show notes. As you probably know, we are a podcast loosely tied together by the Popper Deutsch Theory of Knowledge. We believe David Deutsch's Four Strands tie everything together. So we discuss science, knowledge, computation, politics, art, and especially the search for artificial general intelligence. Also, please consider connecting with Bruce on X at bnielson01. Also, please consider joining the Facebook group, The Many Worlds of David Deutsch, where Bruce and I first started connecting.

[01:27:41]  Unidentified speaker: Thank you.