Episode 58: Deutsch’s “Creative Blocks”: A Decade Later

• Links to this episode: Spotify / Apple Podcasts

Unofficial, reviewed transcript. It may still contain mistakes; check the podcast when wording matters.

Speakers are labeled by first name where identified. Unidentified-speaker labels may refer to different people. Bracketed question marks indicate uncertain wording.

Transcript

[00:00:11]  Bruce: Welcome to The Theory of Anything podcast. Hey, guys.

[00:00:14]  Unidentified speaker: Hi, Peter.

[00:00:15]  Unidentified speaker: Hi, Bruce.

[00:00:16]  Bruce: A number of years ago, David Deutsch wrote an article called Creative Blocks, or kind of the title in the URL is How Close Are We to Creating Artificial Intelligence? And he's written a number of articles like this. There's another really good one that was inside of, what was the name of that book you sent me, Peter?

[00:00:38]  Peter: The book I sent you.

[00:00:40]  Bruce: Yeah, you sent the link.

[00:00:42]  Peter: So we can cut this out.

[00:00:46]  Bruce: Possible Minds, yes. He also wrote an article in Possible Minds. We'll be covering that one today. But we wanted to kind of revisit that article. Now, I have to say those two articles are really important to me. They were a lot of the turning point in me wanting to study artificial general intelligence. I actually went back to school to study artificial intelligence, which may seem like a weird choice if you read this article, since he kind of points out that AGI and AI aren't the same thing. But it led to a number of changes to my career path, much as I could at this point in my life. And so these are kind of important articles to me because I found them quite inspiring. Having now studied this subject for a number of years, I actually have some criticisms of the articles also that I feel like are things where maybe certain parts of the article, he's not quite right on a few things.

[00:01:56]  Bruce: So I actually think it'd be a really interesting idea, and this was Peter's idea, to go back and to revisit these articles and to kind of talk about them, talk about why it inspired me to go back to school to study artificial intelligence, especially since it kind of attacked artificial intelligence. And how do I see it now based on having studied this more deeply? So I felt like that was an interesting subject idea. So that's what we're going to do today. So this article, Creative Blocks, David Deutsch in his books, he kind of developed a number of theories that are interconnected based on what we call the Four Strands. So the Four Strands would be many worlds, quantum physics, computational theory, Darwin's theory of evolution, really neo-Darwinian theory of evolution, and modern synthesis, obviously, not necessarily the original version of Darwin, which is long since we've advanced beyond.

[00:03:00]  Bruce: And then Karl Popper's Theory of Knowledge or Philosophy of Science, which is creative—sorry, critical rationalism. So he developed a number of interesting ideas that kind of came out of those. On this podcast that inspired a lot of this podcast, tons of subjects come from me researching various things that Deutsch said. And a lot of cases, particularly on the Four Strands, I completely agree with him. I was surprised, for instance, when I spent a number of years trying to study quantum physics and came to the realization, oh, he's actually right that we only have one explanation of quantum physics today and it's many worlds. Now, does that mean many worlds is true? That's not the way critical rationalism works, right? It just means that that's the only explanation we've been able to think up so far. So right now it's currently our best explanation.

[00:03:52]  Bruce: But I was surprised at that because it's well known they're supposed to be these other competing interpretations of quantum physics. So when I actually dug into them and found out that they did not pass even the most basic criteria for being considered good explanations under critical rationalism, I was like shocked. And the only one that actually passed was many worlds quantum physics. And I was really surprised when that happened. And it really actually bothered me because I don't particularly like many worlds quantum physics. But the deeper I got into it, the more I could see that the four strands, those four theories really are some really of our most powerful theories, and that when they're combined together, they become even more powerful explanations. So I ended up being completely in Deutsch's camp for the four strands. Now, if you've been listening to the podcast, I've criticized a whole bunch of ideas that have come from David Deutsch.

[00:04:51]  Bruce: So trying to figure out what are the implications of the four strands is not the easiest thing in the world. And just because you accept all four of the strands as being basic and our most important scientific and philosophical theories, doesn't mean that we're all going to agree on what those implications are. And that's where a lot of the room for criticism can come in, where we need to... we always have errors and mistakes in our theories. And we have, even when we accept them, we have errors in our understandings of them, errors in how we apply them. And so there is still quite a bit of room for disagreement with each other over the implications of theories. Now, and we use criticism to then try to work through where we have misunderstandings and try to get to a better understanding and try to get to an improved theory. Now, one of the things that David Deutsch really mentions in the Creative Blocks article, and one, probably the thing that I still most strongly agree with him on, like really totally agree with him on, is the nature of computational universality.

[00:06:01]  Bruce: Now, I've done numerous episodes on that. I think when he talks about this, he says, I mean, just quote from the article, despite this long record of failure, AGI must be possible. And it's because of a deep property of the laws of physics, namely the universality of computation. This entails that everything that the laws of physics require, a physical object to do in principle can be emulated in arbitrary fine detail by some program on a general purpose computer, provided it is given enough time and memory. The first people to guess this and to grapple with these ramifications were the 19th century Charles Babbage and his assistant Ada, Countess of Lovelace. It remained a guess until the 1980s when I, meaning David Deutsch, proved it using the quantum theory of computation. That paragraph is really powerful, and it's saying so much. And as far as I can tell, looking into this, he's completely spot on, right?

[00:07:01]  Bruce: I mean, like this one, I cannot find any good criticisms of it. So now, I don't want to repeat everything that we've talked about in past episodes, but let me see if I can try to give a feel to why this one is so dang hard to criticize, if you actually understand it. And it really, when you look at people who have tried to criticize the concept of universality, we have a mutual friend who has attempted to do that numerous times on Facebook, for example, listing one scientist after another that has criticized this. And you can very quickly see that the scientists being listed don't really understand the concept of computational universality. They're criticizing all sorts of things, but it's not, it's kind of incoherent the way they're going about it. It really comes down to something fairly simple. It's the fact that we use physics to build computers.

[00:08:03]  Bruce: I mean, of course we do, right? I mean, like, does anybody really seriously doubt that?

[00:08:09]  Cameo: Isn't it more accurate to say the computers we built, we build are constrained by the, by our current understanding of the laws of physics or our ability to manipulate them?

[00:08:18]  Bruce: Okay, so that's where we're going with this, right? Is let's say you built a computer, and there are, there are computers that exist. Let's say a, a finite state machine that it follows the laws of physics, but the laws of physics can do things that a finite state machine can't do. Okay, so it's no big surprise that we can build a machine that can do things that a finite state machine can't do, because the finite state machine doesn't have a full repertoire of what the laws of physics allows.

[00:08:54]  Peter: Sorry, could you, could you define finite state machine?

[00:08:59]  Bruce: So a finite state machine, this is probably not actually worth defining. In this episode. A finite state machine is a very basic machine that has various states and a transition that exists between them. And if you go back and you look at the, the episodes of this podcast on, on the theory of computation, I actually do go over it in detail. So it's really easy to understand. They're very basic sorts of computers. They have no memory, basically, right? So you basically just move between states. And there are certain kinds of algorithms that you can run on them. In fact, regular expressions are equivalent to a finite state machine, if I remember correctly. Maybe I'm getting that one wrong. But, oh, they may be equivalent to a pushdown automata. Actually, now I think about it. But there's actual programming languages that are equivalent to some of these different types of machines that are non-universal.

[00:09:58]  Bruce: Even when I say non-universal, even that's misleading. They're universal within their own sphere. It's just they're not universal equivalent to the laws of physics. Laws of physics allow you to do things that these machines can't do. The laws of physics, you can use them to implement a machine that will do and accomplish certain types of things, do certain types of computations, that these more limited types of computers are incapable of doing.

[00:10:27]  Unidentified speaker: Okay.

[00:10:28]  Bruce: Now, Alan Turing had this idea. And the idea was that there was a universal computer, that there was a computer that didn't just have its own sphere of universality, but that it was equivalent to the laws of physics.

[00:10:46]  Unidentified speaker: Okay.

[00:10:48]  Bruce: And this was the Turing machine that we've talked a whole bunch about on this podcast. So why did he come up with this idea? Well, it was actually a conjecture. He didn't know if it was true or not. But it was based on a really interesting thing that happened. It was the fact that there's this grammar created by Church. So we call it the Church-Turing thesis. So Turing and Church are two different people. And it turned out that Turing was able to prove that his Turing machine and the Church grammar, which is like a, think of it as like a totally different type of computer, were exactly equivalent. He showed through showing that you could always take one and turn it into the other and then back to the other, which is how they do this in computational theory. He showed that they were exactly equivalent. Now we have spent just decades and decades trying to come up with machines that can exceed the Turing machine in terms of computational power.

[00:11:46]  Bruce: Now, when I say computational power, I don't mean speed, of course. The Turing machine wouldn't be particularly fast. And I don't even mean in memory because the theoretical Turing machine has infinite memory, whereas a real life computer never has infinite memory. But just in terms of the class of algorithms that it can run, and also if those algorithms are tractable or not. Okay, if you can show that this algorithm is not tractable on a Turing machine, there will be no physical machine you can build that is tractable on it. Now, it turned out that isn't actually quite true because of the quantum computer. But it turned out to be really close to true. This is how Turing was looking at it. Turing was trying to solve a specific problem. The problem was, how in the world? I mean, like you got these two entirely different machines, this Grammar by Church and this Machine that's by Turing, both are just theoretical.

[00:12:42]  Bruce: It's not like these are actually being built and you can't build a Grammar. But why are they exactly equivalent? What are the odds that these two machines would be exactly equivalent? And then you try to build other machines. You come up with a Turing machine that has a 2D surface or a 3D memory. Things that you would think would add power to the machine. And you can almost immediately show that they're exactly equivalent to a Turing machine. Using the same trick that Turing used.

[00:13:08]  Peter: What I'm getting is that universal computation, a Turing machine, anything that you could reasonably expect a Turing computer to do that's universal, it can do.

[00:13:25]  Unidentified speaker: Yes.

[00:13:26]  Peter: Is that a fair way to put it?

[00:13:27]  Bruce: Yes.

[00:13:28]  Peter: To really dumb it down?

[00:13:30]  Unidentified speaker: Yes.

[00:13:30]  Bruce: Got it.

[00:13:31]  Peter: Question two, doesn't the article say that the quantum computation proved universality?

[00:13:40]  Bruce: So let me get to that. Let me get to that. So when we're dealing with computational theory, there's two different things we care about. Well, there's probably lots of things we care about. There's two in particular for our purposes for this podcast that we care about. One is, is it computable or not? Okay, so there are certain types of problems, famously the halting problem, that a computer cannot solve. Okay, it is completely impossible for a computer to solve the halting problem in a general way.

[00:14:13]  Peter: It can't tell you when it will stop the program.

[00:14:18]  Bruce: It can't tell you if the program is going to stop or not.

[00:14:22]  Peter: Oh, is going to stop or not.

[00:14:23]  Unidentified speaker: Yes.

[00:14:24]  Unidentified speaker: Okay.

[00:14:25]  Bruce: Now, there are all sorts of things that are exactly equivalent to the halting problem. So there's different types of problems that we can pose, that we can then map to the halting problem, and then we know that problem is incomputable. Okay, does that make sense?

[00:14:42]  Unidentified speaker: Yes.

[00:14:43]  Bruce: And there's tons of interesting problems we would love to solve that computers cannot solve. Now, the other thing is tractability. So now tractability is a little weird. So let's say that I have something like the traveling salesman problem.

[00:15:03]  Unidentified speaker: Okay.

[00:15:04]  Bruce: I could ask the question, is the problem, the traveling salesman problem, is it tractable? And generally speaking, we would say no. But we don't really know that for sure.

[00:15:12]  Unidentified speaker: Okay.

[00:15:12]  Bruce: What if someday somebody discovers an algorithm that is polynomial time, so tractable, that solves the traveling salesman program? Well, how can you ever know for sure someone won't in the future discover such a program?

[00:15:30]  Unidentified speaker: Right?

[00:15:33]  Bruce: They've tried to come up with proofs. Is there some way to prove that this is an intractable problem? They've never come up with anything. There's no way to prove it, okay, that we currently know of, but there probably is no way to prove it. So it would be technically incorrect to say the traveling salesman problem is intractable, although people say stuff like that all the time and we know what they mean. What we mean is we don't know of any current algorithm that makes this a tractable problem, and we have good reason to believe we will never discover such an algorithm. Now, why do we have good reason to believe we will never discover such an algorithm? Well, they discovered as part of computational theory of what came out of accepting the Church-Turing thesis and trying to study it and work out its implications. They had this idea of NP completeness.

[00:16:24]  Bruce: Now, I described this in a past podcast, so I don't want to get into it because it's kind of complicated. But basically, they were able to show that there are certain kinds of problems that are universal, a traveling salesman is one of them. So if I were to come up with an algorithm that could tractably solve the traveling salesman program—problem, then I could in fact take every single NP problem that exists currently, and I could solve all of them tractably because it turns out the traveling salesman program is a universal problem. It's easy to come up with a way to take the traveling salesman problem, take every NP problem that exists, and map it to the traveling salesman problem. So all I would need to do is if I had a way to solve the traveling salesman problem in polynomial time, which would be tractable time, let's say, I would just simply have a program, ones that we already know exist, that takes every single NP problem, maps it into a traveling salesman problem.

[00:17:28]  Bruce: I would then solve it using the tractable algorithm, and then I would map it back.

[00:17:31]  Unidentified speaker: Sorry, are you saying NP?

[00:17:34]  Unidentified speaker: NP, the letter N, the letter P.

[00:17:37]  Bruce: That's what they're called. There's this giant class of decision problems that are all considered, well, underneath NP is the class P, which is the tractable algorithms. But you don't always know if a problem falls into, you know it's NP, but you don't know if it's P or not. So one of the questions they have open is, is P and NP possibly equivalent? Well, they don't believe it is because of NP completeness. If you could actually do this, then it would turn out that P and NP are the same. So we basically have a set of conjectures. And like all scientific conjectures, you're looking for counter examples. If you can't find counter examples, then the conjecture continues to be treated as if it's true. And these conjectures are, number one, that the Turing machine is a universal computer. I would have to say it's actually today the quantum Turing machine.

[00:18:32]  Bruce: But a lot of times when we say Turing machine, we actually mean the quantum Turing machine. Let me explain the difference here in a second. And number two, that NP and P aren't equivalent. So that there is basically, that means there's a class of tractable algorithms and there's a class of intractable algorithms.

[00:18:51]  Unidentified speaker: Okay.

[00:18:53]  Bruce: Ones that we could compute but it takes so long, very quickly as the number of entities that need to be computed grows, it takes so long that just realistically, there's no way to actually solve this problem in real time. Now, tons of interesting things come out of this. The whole field of AI, and I've talked about this in a past podcast, is basically trying to figure out what do we do when we have an intractable problem and we still need to get a good answer. And there's tons of answers to that. You can relax the constraints a little bit. You can make it so that it simply finds a good answer that's close to optimal, but not actually optimal. So let's take the traveling salesman problem, which is this problem where you have a salesman and he needs to visit a group of cities, and we want to output the shortest path between all the cities that then returns him round trip back to his own city.

[00:19:51]  Bruce: So you've got the shortest path.

[00:19:53]  Peter: It's very hard to understand from someone who's not a computer person immersed in that world, why that would not be a solvable problem.

[00:20:05]  Bruce: Yes. Well, it is a solvable problem. It's computable. As the number of cities you have to do, you have to run between grows, it becomes exponential how many options you have to try out.

[00:20:19]  Peter: Oh, so I guess it's a bit like the idea that there's more chess moves than there are atoms in the known universe.

[00:20:27]  Bruce: Yes, it's exactly the same thing.

[00:20:29]  Peter: You can't, you know, your mind just can't get around exponential growth.

[00:20:34]  Bruce: So, in fact, let's use chess as an example. Okay, so the reason why, like, why, why did it take so long for chess programs to outplay chess masters like that only happened within our lifetime, right?

[00:20:49]  Unidentified speaker: Yes.

[00:20:50]  Bruce: Why was that so hard? Well, it's because chess is an intractable problem. So we know how to solve the problem of chess in principle, but you can't actually. So I could very simply write a program that simply tries out every possible move in chess given a certain input for a board, okay? And so in theory, I can write a program that beats every chess master, but it was just intractable. Okay, so what they actually had to do is they had to come up with these really clever ways. And one of the main things they did is they started using machine learning AlphaGo. If you look at our AlphaGo episode, to be able to evaluate the board more like how a human does it, where they can intuitively look at the board and say, oh, this board better than that board. And then once they were able to do that, they didn't have to solve the problem of chess. They just had to look a couple of moves ahead.

[00:21:44]  Bruce: And with a good enough program, you can beat any human master. And even human masters try to look ahead, they use constraints and they don't look ahead every possible move, but they look ahead quite a number of different moves. Well, we would train the computer to only look ahead the most promising possible moves, just like a chess master would. And then it can look ahead, you know, 17 moves or something, once it's trying to only do a few options. And then no chess master can do that. So the chess masters can't compete anymore and the computer can win. So we eventually figured out how to beat chess masters. We figured out how to beat go masters. Go was even harder because there's no good way to evaluate the board, unlike chess. And the exponential growth of possible moves is much, much larger with go. So there's a lot of problems. In fact, one might even say that the vast majority of problems that we're interested in are intractable for a computer.

[00:22:46]  Bruce: So AI then becomes the study of, you know, how do I do that? So let's say I don't care about finding exactly the shortest path for the traveling salesman problem. I just want something close to the shortest path. Turns out that's completely tractable. And so they're going to be studying, how do we come up with algorithms that get us something close to the best possible answer?

[00:23:10]  Unidentified speaker: Okay.

[00:23:10]  Bruce: So in AI, they define rationality contra the critical rational's view of rationality. They define it more like an economic concept of rationality. Rationality would be making the action that is provably close to the best action. And AI is all about trying to solve those kinds of problems. So getting back to how this all relates to computational universality then. Now it turns out that you can build a computer that can take certain kinds of intractable problems, specifically the problem of Shor's algorithm of trying to solve prime numbers, what prime numbers make up a certain number, which is used in cryptography. You can build a physical machine that is able to do that in a tractable amount of time, whereas a Turing machine can't do that in a tractable amount of time. At least not with any known algorithm that we have today. So what was this computer?

[00:24:16]  Bruce: That's the quantum computer. Now this is exactly what made David Deutsch famous, is that he pointed out, he wrote this famous paper where he basically invented, he invented quantum computational theory. What he basically did is he showed that you could take all the different things you can do on a Turing machine and you can do them all on a quantum computer. A very simple thing, like amazingly simple. He's world famous for this thing that nobody thought of before him, but in retrospect, it seems totally completely obvious. And this is the way a lot of discoveries are, right? Where once somebody actually comes up with them, they're completely obvious after that. And he basically showed that you could take anything you could do on a Turing machine you can do on a quantum computer. But the quantum computer, because it has this massive parallelization that takes place, because that's what quantum computers do, it's able to, we're able to run something called Shor's algorithm, that you could run on a regular computer, basically by emulating a quantum computer, because a regular Turing machine can emulate a quantum computer, but it would be intractable on the Turing machine, but it's tractable on the quantum computer.

[00:25:34]  Bruce: By doing this, David Deutsch proved that, here's the key thing, that the Church-Turing thesis was false. I'm going to say it again. David Deutsch in his paper proved that the Church-Turing thesis was false. Now you might say, wait, didn't he prove it was true? This is where linguistics becomes a bit of a problem. What he really did is he showed that there was a type of computer, the quantum-Turing machine that was able to... It's got the same computational... There's nothing that the... How do I say this? There's nothing that the quantum-Turing machine can compute that the Turing machine can't compute. So they're equivalent in terms of computability. But there are certain classes of algorithms, really, Shor's algorithm is really the only known example today. But there's certain algorithms, such as Shor's algorithm, that is tractable on the quantum computer, but intractable on a Turing machine, which makes it a different type of computer, if only a bit different.

[00:26:48]  Bruce: Now, because of that, the quantum computer has a number of speed-up algorithms that can, in general, speed up any algorithm. But it doesn't take an exponential algorithm and turn it into a tractable algorithm. It doesn't do that. It's usually only a quadratic speed-up, which you don't even know what that means. But basically, it's a significant speed-up that is still shy of an exponential speed-up. And so an algorithm, other than Shor's algorithm, that is exponential on a Turing machine, is still exponential on a quantum Turing machine. It's just the quantum Turing machine can do it in quite a number of fewer steps. But it's still an exponential algorithm. So it didn't actually change very much in terms of computational class of algorithms. But it changed it enough, and it changed it in this one important area, Shor's algorithm, cryptography, that gets all sorts of government agencies excited, which is why there's so much interest in trying to build a quantum computer, okay?

[00:27:51]  Bruce: And you're probably familiar with the concept of quantum supremacy and all that that's been in the news with IBM competing with Microsoft and Google or whatever on trying to have the best quantum computer. We're still a long ways off, probably, in terms of having a true quantum computer that could really start breaking our encryption algorithms. But someday we'll work it out, we'll engineer it. It's just an engineering problem at this point, because we've got the theories on how to do it. And we have no… You know, quantum mechanics has got to be the single best theory that we've ever had. I mean, it's the most well-tested. There's not a single known counter example to it. It's known to be wrong, by the way. This is something that I've debated with people on Facebook quite a bit. They'll point out that it's wrong in some way. I'll go, look, we all know it's wrong.

[00:28:45]  Bruce: Like, it's at odds and in contradiction to general relativity, and it can't explain quantum gravity. That's why we're looking for a theory of quantum gravity. It doesn't matter that it's wrong. Okay? It doesn't matter. It's still the best theory. It's still the only theory. There are no competitors to it. String theory is not a true competitor to it today.

[00:29:05]  Unidentified speaker: Right?

[00:29:05]  Bruce: I mean, it's the level of verisimilitude of that theory is amazingly high.

[00:29:13]  Peter: Is there a reason why you say, is there a distinction between being wrong and incomplete?

[00:29:20]  Bruce: No, it's wrong.

[00:29:22]  Peter: It's wrong is the right word.

[00:29:24]  Unidentified speaker: No.

[00:29:25]  Bruce: So there is a distinction to be made between wrong and incomplete.

[00:29:28]  Unidentified speaker: Yeah.

[00:29:29]  Bruce: Okay. And so you might try to throw it into the incomplete category by saying we can't explain gravity today with it. But like it's literally in contradiction to general relativity. Okay. So I guess we can't say for sure it's wrong until we actually have the new theory. And maybe we will discover, I mean, one could make an argument that when we have the new theory, it'll turn out to be its quantum theory plus something else. But nobody believes that. I mean, it's not even a very viable point of view. I don't think you could find a single scientist who ascribes to that viewpoint. It's generally accepted that it's wrong. And this is one of the things that Deutsch will point out is, yeah, we know it's wrong. It's an incorrect theory. We're actually in a unique point in history where we used to think our theories were true. And we sort of don't believe that anymore.

[00:30:22]  Bruce: We think they're the truest. Okay, it fits very well with critical rationalism. Okay, it's one of those ideas that has grown over time in science, that it doesn't really matter if our theories are entirely right or not. What matters is, is that we don't have a good alternative theory, okay? So what David Deutsch did is not only did he create this new type of computer, the quantum computer, the quantum Turing machine, which then became the new universal machine, because now that's the machine that we know how, in theory, how to build, even though in practice we can't build one, that we know in theory how to build. And it is not known how you could use the laws of physics to exceed that machine. Now, maybe you can. Maybe the laws of physics allow for some new type of machine. So what David Deutsch, he didn't want to leave that open door like Turing did.

[00:31:22]  Bruce: So what he did is he created a mapping between quantum physics and the quantum Turing machine to show that they were equivalent. Now, this was really the genius of what David Deutsch did, because now he's actually produced a proof that computational universality is true. But, and here's the key thing, only insofar as quantum physics holds, quantum mechanics holds, which, as I just said, it's wrong. It's known to be wrong.

[00:31:53]  Unidentified speaker: Okay.

[00:31:55]  Bruce: Now, this is where things kind of get interesting. So, Deutsch proved something, but based on a physical theory that's known to be wrong. So doesn't that mean that his Church-Turing-Deutsch thesis, as we call it today, is wrong? Well, no, that doesn't mean that. If I could use an analogy. Let's say that we wanted to show that, we wanted to take the stance that actually the laws of gravity are incorrect under Newton's laws, which today we accept that as true, but back under Newton's laws before Einstein's time, that would have been a real hard sell, right? Well, people will often point to that, and they'll say, okay, we thought gravity was real, we thought it was a force. Then Einstein came along, he showed it wasn't even a force, so there is no force of gravity. And we have to get used to this idea, and even Popper would agree with this, Popper uses this as an example, that you don't know how the new theory is going to falsify things, how that new theory is going to overturn things that you thought were true.

[00:33:10]  Bruce: And you may have to someday, you really believe there was a force of gravity. And then the new theory comes along, Einstein's theory of general relativity, and suddenly there's no longer a force of gravity. And this is generally the way we tell this story. And this is even the way Popper tells this story. There's a problem with that way of thinking about it though, even though it's technically true. Because there really is still a force of gravity. Even if it's now explained as curvature of space and things moving together and things like that, it's effectively exactly equivalent to a force. What Einstein really did is he showed why the old theory was successful. Now, this is what Popper is trying to explain, is that you don't just throw theories out for being wrong. You have to actually have a new theory that explains the success of the past theory.

[00:34:05]  Bruce: And until you have that, until you have that new theory that explains the success of the past theory, you don't have any particular reason to believe anything in that false theory is wrong, other than the individual things that you know to be incorrect, right? Well, there's no reason to believe that the Church-Turing-Deutsch thesis is going to be overturned in the new theory. It might be. Until we have the new theory, we won't know, but it might confirm the Church-Turing-Deutsch thesis. In fact, at the moment, that would probably be your best guess, is that the future theory is going to explain the success of the past theories. Now, we don't know, okay? But we've got good reason to believe that at least insofar as physics follows quantum mechanics, that universality is going to hold until there's some new theory that tells us this is how you would build a machine that is different than the quantum Turing machine, that can do something the quantum Turing machine can't do, universality is going to hold.

[00:35:07]  Bruce: Deutsch, in his original paper, according to Penrose, actually wrote about how you would get around this. So, Deutsch is not naive about this, right? He's very open about this. Maybe someday we will have a theory of quantum gravity, and it will allow us to, say, build an oracle machine that can solve the halting problem, at least on a Turing machine.

[00:35:28]  Unidentified speaker: I don't know.

[00:35:29]  Bruce: Scott Aaronson's written papers about that. Maybe someday that will happen. At that point, we will overturn the Church-Turing-Deutsch thesis, okay? And we'll be able to build a new type of computer using our new understanding of physics, and it will have a greater computational class. That might happen, or it might not.

[00:35:49]  Peter: Is that the definition of an oracle? You said oracle machine?

[00:35:53]  Unidentified speaker: [Overlapping speech] Oracle machine is a…

[00:35:54]  Peter: They can solve the halting problem?

[00:35:57]  Unidentified speaker: [Overlapping acknowledgment unclear.]

[00:35:58]  Unidentified speaker: Yeah, okay.

[00:35:59]  Bruce: So in computational theory, we invent computers that have a different class than the Turing machine. It's actually quite easy to do, right? There's all sorts of computers that exist theoretically that are different than the Turing machine and have a different class of computation. One of them is the oracle machine, where you attach this oracle to a Turing machine, and it can solve the halting problem. So now you have a Turing machine plus the ability to solve the halting problem. What's your new computational class? And we've answered that question. We've got tons of studies on that, right? This is the key thing, though. And this is one of the brilliant insights of Deutsch. Everything I'm talking about directly links computational machines to physics. Now, I just said we have theoretical computing machines that aren't linked to physics. You just can't build them because the laws of physics don't allow you to.

[00:36:56]  Bruce: So when we talk about a physical computing machine, which is usually what we care about in computational theory, that's what we're studying. We're actually studying what do the laws of physics allow you to compute? That is what the theory of computation actually is. And this is one of the things when I've had numerous arguments on Facebook with a certain someone out there who's argued with me over this quite a bit. Yes, it might be that we overturn the Turing-Deutsch thesis. That would be great if we could. But nobody knows how to do that today. So we don't really have any reason to believe we will. We don't really have any reason to believe we won't. But our best theory says this is what the computational class is. All of our theories are computable today. Physics theories are computable today. Well, of course they are. Because if we had a physics theory that wasn't computable, then that would mean there would be some way to build a machine using physics that would then make that computable.

[00:38:01]  Bruce: And we would then invent a new type of computer. The types of physical computers we can build is directly linked to our understanding of physics.

[00:38:08]  Peter: Now, on a practical level, what would overturning the Church-Turing-Deutsch thesis look like? Like, how can I get my mind around that?

[00:38:19]  Bruce: Okay, so let's say that we come out with the theory of quantum gravity, and it allows us to create a closed time loop, where you can run a program, and then you can wait for forever in this closed time loop until the program does or doesn't terminate. And then it comes back with an answer. Did the program terminate? Under those laws of physics, now this is completely made up laws of physics, there is no theory of quantum gravity. So I'm making this up. We'll understand that this is a complete work of fiction. So I come back, and I now can build a computer that can solve the halting problem. Now, one of the things you have to understand about the halting problem is that it's completely unsolvable. So what I really mean when I say it can solve the halting problem is I mean you solved it for the Turing machine. You then have a new halting problem for this new type of computer that can't be solved.

[00:39:17]  Bruce: And this is what Gödel's Theorem actually says. So you would never truly solve the halting problem, but you could solve it for a certain type of computer, if that makes any sense. This would then be a new type of physics. We would now have an idea how to go about making this computer. We would then have to figure out the engineering problems around it, but eventually we would work through all problems are soluble. We would eventually work through the engineering problems. We make this new type of computer that is like this uses this closed time loop and you attach it to a Turing machine. And now the Turing machine can solve the halting problem for other Turing machines, but it can't solve it for this new type of computer itself.

[00:40:00]  Unidentified speaker: Okay.

[00:40:02]  Bruce: There's this really deep link, and this is where the problem comes. People who are trying to show the Church-Turing thesis or the Church-Turing-Deutsch thesis, if they even know that that thesis exists, which most of them don't, is wrong. What is it they're trying to say? If all they're trying to say is, we might have future computational theories based on the new laws of physics, and it might overthrow the Church-Turing-Deutsch thesis, then yeah, sure. But what you're going to end up with is the Church-Turing-Deutsch, you know, X thesis, whoever discovers this, and a new type of computer that has a new computational class, and everything's still computable. And this is why you can't get around the Church-Turing thesis in general, okay? Yes, you can overthrow any specific version of the Church-Turing thesis. You can overthrow it with the Church-Turing-Deutsch thesis.

[00:40:59]  Bruce: You can overthrow the Church-Turing-Deutsch thesis with the Church-Turing-Deutsch-Aaronson thesis, whoever discovers the new version. But you always wind up with something that's exactly equivalent to what we thought the Church-Turing thesis was, which basically says everything that physics allows you to do can be simulated on a computer. It depends on what computer. The laws of physics may mean we have to invent new types of computers to fulfill that. Or they might not. It may be we've already discovered the top class of computer that will ever exist, the quantum computer. It may be every theory we have after this will say that that is the highest class of computer. And when someone wants to get around the Church-Turing Deutsch thesis, they typically just don't mean the Church-Turing Deutsch-Aaronson thesis, right? Whatever this new future thesis is going to be.

[00:41:55]  Bruce: They mean that in some mystical, supernatural way, we'll simply find out that consciousness can't be in any way computed or therefore explained by physics. Well, that's just supernaturalism, right? I mean, and that's clearly most of the people who say this are getting into that camp because they don't quite understand what Deutsch was actually trying to say. And so they're just kind of putting stuff out there. Well, maybe, you know, just recently had one of the critics on Facebook say, Oh, but see, there's a difference between a recipe and a cake, because if you try to eat a recipe, it doesn't taste like a cake. It's like, okay, you're so thoroughly misunderstanding what Deutsch is saying, that I don't even know how to respond to you. And I have to go back to the beginning with you and explain what it is Deutsch was trying to say, because you're wasting your time by trying to think of criticisms like this, right?

[00:42:55]  Bruce: Because Deutsch is not trying to say that the recipe tastes like a cake. Let me assure you that that just has nothing to do with the Church-Turing-Deutsch thesis, right? What the Church-Turing-Deutsch thesis really says, though, is this. If you think the biological substrate is able to do something that the digital computer can't do, then what you're really positing is a new branch of physics, a new laws of physics that isn't quantum computers. And I think most people like Lee Cronin has been very critical of the Church-Turing thesis, okay? Doesn't really understand it from what I can see. He is not trying to say that we need [new?] laws of physics, but that is what the implications of his statements is. He just doesn't know that, right? Because he hasn't actually understood the theory correctly. And that's really what you're trying to say is anytime you say the brain is not a computer, is not equivalent to a computer, you're really trying to say we need new laws of physics to explain the brain.

[00:43:57]  Bruce: Well, I don't think most of the people who say that would ever actually agree to that statement, because that's such a wildly strange statement. This is why computational universality is so hard to get around. And it's you're either positing a future laws of physics, which you know what, that's fair. Okay, if you've got good reason, quantum mechanics is wrong, so you might even be right. Okay, but you need to actually have entirely new physics to even posit a new type of computer. Because right now, our current understanding of physics shows that the quantum computer is the universal computer, period, end of story. Because we, because Deutsch mapped the two together. So they are equivalent, period, end of story. And there's no other way around that other than you're going to have to actually show me entirely new laws of physics. So when someone comes in, they start to criticize the Church-Turing thesis.

[00:44:55]  Bruce: I'll use the term Church-Turing thesis or the Turing principle as a shorthand, but I really mean the Church-Turing-Deutsch thesis, whatever the current highest computer is. When you're trying to criticize that, they usually, they need to show us new laws of physics. And none of them even know that they need to show us, very few of them. Roger Penrose knows that he needs that. He's one of the exceptions. So, okay. So getting back to this, we didn't get very far, aside for computational universality. But this is why I'm so completely in the Deutsch camp for the concept of computational universality. And yes, maybe brains use special kinds of quantum gravitational forces that we don't currently know about under our current theories.

[00:45:45]  Peter: Is that kind of what Penrose thinks?

[00:45:48]  Bruce: That is what Penrose thinks. Now, in fact, let me read something, a place where I do need to offer a criticism of Deutsch. So Deutsch, in this article that we're reviewing, he talks about various other ideas, including Penrose's. He says, some, such as the mathematician Roger Penrose, have suggested the brain uses quantum computation or even hyperquantum computation, relying on as yet unknown physics beyond quantum theory. And this explains the failure to create AGI on existing computers. To explain why I and most researchers in the quantum theory of computation disagree that this is a plausible source of the human brain's unique functionality is beyond the scope of this essay. If you want to know more, read the 2006 paper Is the Brain a Quantum Computer? published in the Journal of Cognitive Science. So now I can summarize this point. Okay, so first of all, let me just say, he says Penrose suggested the brain uses quantum computation.

[00:46:50]  Bruce: That's not true. But then he goes on and says what is true or even hyper quantum computation. Penrose in his books is quite clear. He does not mean that the brain is a quantum computer because Penrose knows that if the brain is a quantum computer, that's exactly the same as saying that the brain is equivalent to a computer. So Penrose is smart. He really gets this stuff. He's got really interesting ideas because of that. Penrose is saying quantum mechanics is wrong and that we need to replace it with an entirely new version of physics. And then he goes through a punch list of what he hopes that physics will hold. And guess what? That's all it is. It's a list of things that Penrose hopes the future theory is going to hold. He tried to come up with something to make it more plausible that the brain could be working off of quantum effects. Now, this wouldn't be to show it's a quantum computer, because quantum computers are exactly equivalent to Turing machines, other than a slight difference in the class of what algorithms, the Shor's algorithm, can run.

[00:48:00]  Bruce: In his book, he says that. He says, I'm not suggesting the brain is a quantum computer, because that would undermine my entire argument, if that were true. It would be the same as saying the brain is equivalent to a computer. He is trying to say there's going to be some interesting effect that we're going to find in future physics. That future physics is going to show that there's this quantum effect, and the brain is going to use that, and that's going to allow us to get around Gödel's algorithm. He makes his whole argument around Gödel's algorithm. And that is his whole argument. His whole argument today is basically, I think that we're going to have some future physics, and current physics is completely wrong, and so I don't accept its current implications. And that's what Penrose is saying. Now, he tries to make his point of view, which when I put it the way I just did, it doesn't seem like a very good theory.

[00:48:53]  Bruce: And really, it's not a theory. It's not, by any stretch of the imagination, a scientific theory today.

[00:49:00]  Peter: I'm curious, how many people agree with Penrose? I mean, I've heard a lot of people who are interested in his idea, but I don't think I've ever interacted with someone who is like, Penrose is right.

[00:49:14]  Bruce: No, I don't think you will find really many scientists at all that agree with Penrose. But, you know, what would they agree with, right? Penrose himself admits that this isn't a theory. This is him trying to point out problems with the existing theory and how he hopes you might go about trying to solve it. He's come up with interesting theories trying to solve it. I can't remember the name. He's got a theory that didn't catch on at first. He was kind of disappointed, and then it started to catch on as a possible way to go about trying to make some advances in physics. I'd have to look the theory up. So, I mean, to Penrose, to his credit, he's actually trying to solve the problem. He admits there's a problem. I kind of think he's probably on the wrong path. I think he's got intuitions that it should be the case that the brain isn't equivalent to a computer.

[00:50:03]  Bruce: And so he tries to show there could be quantum effects in the brain. And he got together with another scientist who believed that, and they published a book on the subject. But everything we know about quantum computation suggests that you just can't have quantum computations in something as wet and warm as the brain.

[00:50:26]  Peter: I take it what kind of what Penrose is. He's searching for the mystery of consciousness in something more physical, whereas Deutsch, and I think it's probably more mainstream position that it's more of a software problem. Is that fair?

[00:50:46]  Bruce: Yes. So this is Deutsch's whole point, is that we don't really... I mean, like Penrose's theories is not a good theory today. Now that could change. It's a conjecture. And let's make a distinction between a conjecture. Penrose has every right to use his intuitions to say, I'm going to make the following conjecture, and I'm going to let that lead me into a research program. And that's what Penrose is doing to his credit. And it's his risk. If he's wrong, then that's his time. He's wasted going down a false path. Now, false paths can be partially true. So even if you're going down a false path, it may turn out to be useful in some way.

[00:51:19]  Unidentified speaker: Okay.

[00:51:20]  Bruce: This is one of the reasons why dogmatism is sometimes good. But in terms of AGI studies, if you were trying to pick between Penrose's approach to AGI studies and Deutsch's approach to AGI studies, Deutsch is rooted in existing theory, and it's really hard to see how you'd get around it. And Penrose isn't saying how you would either, because Penrose doesn't know, and probably will never know, because honestly, he's probably wrong. And there probably is no way to get around the Church-Turing-Deutsch thesis. Or if there is, it's going to turn out to be a benign version, where it's just simply a tweak to the thesis and the Church-Turing-Deutsch-Aaronson thesis or whatever.

[00:52:08]  Unidentified speaker: Right?

[00:52:09]  Bruce: I keep using Scott Aaronson as the example, because he wrote that paper on how we might someday build a better computer. But that may never happen, or it may. Either way, it almost doesn't matter. Even if it did happen, even if it turned out that we could build a new type of computer that had closed time loops, and we could solve the halting problem, do we have any reason at all to believe that human brains can solve the halting problem? They can't, right? The limits of what humans can do is exactly equivalent to the class of computational theory, that computational theory says. We can't do things, we can't intractably accomplish things that computers can only intractably do. [Wording unclear.] We can't solve uncomputable problems.

[00:52:57]  Peter: So does that get at a link, sort of a link between computational universality and the universal explainer hypothesis?

[00:53:09]  Bruce: So yes, insofar as the universal explainer hypothesis is built on top of computational universality. So the universal explainer hypothesis would hypothesize that even a universal explainer can't do things that are not allowed by computational theory. Because it's still ultimately that the brain is a physical object, that it's running software. It cannot be different from what you can do on a Turing machine or quantum Turing machine. Really, on a Turing machine, we've got no reason at all to believe that human brains can solve Shor's algorithm in exponential time. Sorry, in polynomial time. So we've got every reason at this point to believe that the brain is a regular Turing machine, not even a quantum Turing machine. So this is really kind of the starting point. And this is what got me excited. The realization that we have good reason to believe, good theories, best theories, best in class theories, with zero competitors today, that say that we've got every reason to believe that AGI is possible, it's just a matter of finding the right software.

[00:54:35]  Bruce: And it's not going to require new physics, it's not going to require some sort of mystical connection, etc.

[00:54:43]  Peter: I was just going to say, that might be a good transition to a question I have for you, about the article. It seems to me like most of what you say so far, have said so far, would be relatively uncontroversial in the AGI field, or they might at least pretty much be on the same page as you. But if you turned it around and said, also what's in the article, to someone who would be a more mainstream AGI researcher, and you said, well, the reason we don't have AGI yet is because we're basing our programming and our ideas on what Deutsch might call bad philosophy. I don't think he uses that in the article, but he says, well, we're thinking about justified true belief and induction, and Bayesianism and behaviorism, and I think he brings all that up in the article and seems to suggest that that's why just AGI is on the wrong path. I kind of suspect that this mainstream AGI researcher would look at you like you were crazy, or am I misunderstanding?

[00:56:04]  Bruce: Well, this is actually where I do have some criticism of Deutsch. Although I think he's still basically correct. Your average AGI researcher isn't aware of any of these theories. So, yes, the vast majority of AGI... So how many AGI researchers are there in the world? Well, how would you even count that? There's probably very, very, very few. Okay, you got a whole lot of AI researchers, and probably some of them think that their AI research might someday be related to AGI research. But I don't think that there are true... some large number of AGI researchers out there. That's just not the way big science works today, right? But if you were to talk to your average AI researcher, who isn't even really trying to do AGI research, okay? And you were to say, can we build an AGI on a computer? The vast majority of them would say yes. Now, if you were to ask them why, they would not cite Deutsch's paper that shows a mapping between physics and computational theory.

[00:57:15]  Bruce: Well, that's of course the right way to go about this. You want to show brains are physical objects, physical objects follow the laws of physics, the laws of physics are computable on a computer, and therefore, I now know I can build an AGI, unless you can show that the laws of physics themselves are wrong. And not just wrong, because of course we know they're wrong, they have to be wrong in a way that matters to the way the brain would function. And this is the thing that one of our mutual friends out on Facebook has missed over and over and over again, okay, is she might point out, say, the problem of time or something like that. What does the problem of time even got to do with how brains function? They don't, right? Pointing out a problem isn't sufficient. You need to have a really good reason why it's got some sort of relevance to the problem of AGI.

[00:58:07]  Bruce: And we just don't have any reason at all to believe that. Maybe we will someday. I don't know, you leave an open mind because you're a critical rationalist. You leave an open mind. Maybe we will find out that the Turing-Deutsch thesis has thoroughly misled someone like me in AGI research. But we've just got no reason to believe that today. And we've got every reason to believe that it's the right path because it's the best theory we currently have. So the vast majority of AI people don't have any clue that computational theory is actually a branch of physics. They don't have any clue that David Deutsch wrote a paper that showed you can map physics to the computer and back. So what are they basing this on? Well, to be honest, it's gut feel, right? I mean, they have this kind of gut feel that if it's not the case, if you were to ask them, they might produce something similar to Deutsch.

[00:59:00]  Bruce: They might say something like, well, brains are physical objects, so there's nothing mysterious about it, nothing supernatural about it. So, of course, we should be able to build a physical system that's equivalent, right? But they don't know, I mean, like, you'll come across people all the time that are scientists, legitimate scientists, who very sincerely think, and will argue, there was a friend of mine at work who's a PhD, so she's an actual scientist, not working, doing research in the field today, but she's in neuroscience. And I told her, I said, yeah, I'm interested in neuroscience, and because I'm interested in AGI. She says, well, we don't know that the brain is equivalent to a computer. I said, yeah, we do. And she goes, no, we don't. And this is someone who knows what she's talking about in the field of neuroscience, right? They're not teaching in the field of neuroscience, David Deutsch's paper on mapping quantum physics to computational, you know, to the computer.

[01:00:02]  Bruce: Right? No, no, there's an actual paper, right? And quantum physics is exactly equivalent to the quantum computer and vice versa. So we know that the brain is equivalent to a computer. And then she would immediately say, she's never heard this. She's not going to take it from me. I'm not some official scientist, right? Doesn't matter that I'm right. So she goes, no, no, no, we've we've we always think the brain is like, you know, a steam machine or it's a clockwork or and she uses all the standard arguments. OK, and it's not the same because we've actually mapped physics to the computer. That's what Deutsch did. Unless you can show a mistake in his paper or you can show a new set of physics, then this is our best theory and there's just no competitor to it.

[01:00:53]  Peter: OK, so from her from her perspective, she's kind of saying that, well, you know, a couple hundred years ago, the steam engine was the most advanced thing around. So people just naturally compared the human brain to the steam engine. Now the computer is the most advanced thing around. So we kind of just make this assumption that the computer is like the brain. But it's just in reality, it's not not really like that. Whereas everything you're saying about universality and all this seems to suggest that it must be.

[01:01:30]  Bruce: This is a very different case than those other examples. This is a hugely different case than those other examples, right? Okay. Now, by the way, she could still be right, right? It could be that we end up with some completely new theory at some point. But I can always say that. And this is the thing people miss, right? Even people who call themselves critical rationalists, they miss the fact that you can always, always say, well, maybe the theory is wrong. That's just not what you do. You have to actually suggest a new theory. This gets me to where I do have at least a little bit of a criticism of what Deutsch says, but I still think he's mostly right. And here's the quote. For example, it is taken for granted by almost every authority that knowledge consists of justified true beliefs and that therefore an AGI's thinking must include some process during which it justifies some of its theories as true or probable while rejecting others as false or improbable.

[01:02:30]  Bruce: But an AGI programmer needs to know where the theories come from in the first place. The prevailing misconception is that by assuming that the future will be like the past, it can be derived or extrapolate or generalize theories from repeated experience by an alleged process called induction. But that is impossible. So why is this still conventional wisdom that we get our theories by induction? Having now studied this, he's kind of right. But if I were to actually go talk to your average scientist, much less your average AGI—or sorry, AI researcher, none of them—or most of them have never even heard of justified true belief. And the idea that it is taken for granted is just not true. Now, I think what Deutsch would probably say to defend himself here is that he would say, well, they don't know about justified true belief. They may have heard of induction because that's a common enough term, and they may have even heard that science is based on induction because that's a common enough idea.

[01:03:39]  Bruce: But they don't have any clue what it all means, right? There's just no interest. That's that crazy philosophical stuff. I'm just going to go do my science.

[01:03:48]  Unidentified speaker: Okay.

[01:03:49]  Bruce: And maybe they even consent to it. They heard it from a good source. Yeah. My buddy Joe, who's a scientist and knows this stuff, he told me it was based on induction. That's good enough for me.

[01:03:59]  Unidentified speaker: Right.

[01:04:00]  Bruce: Okay, that's about as far as it goes. The idea that it's taken for granted is just not true. I think Deutsch would say, but they act like it's true. Well, this is maybe getting a little closer. Okay. If you were to go ask the average scientist, are there scientific theories that have been proven fact? Some scientists would say yes, but I think there's a very large number of scientists who would say, no, nothing's proven. So I have to take some exception to Deutsch here. The idea that this is some overwhelming, prevailing philosophy that has misdirected science, much less AI, is not really entirely true. Now, having said that, the entire field of machine learning is pretty much entirely based on the philosophy of induction. And machine learning is definitely where all the action is happening in AI research right now. Now, again, I want to emphasize that for the most part, AI researchers aren't trying to build AGI's and have no interest in doing so.

[01:05:08]  Bruce: So the idea that it is misleading that field is not true either. Because the field for the most part is not even attempting to build AGI. They're just trying to take existing theories, which they may consider to be inductive, and in fact might be inductive. Let me go that far, depending on what you mean by induction. And they're trying to take them as far as they can. And there's nothing wrong with that. That's a completely, thoroughly legitimate thing for them to be doing.

[01:05:38]  Peter: So what I'm getting is that the article, or at least how I'm interpreting Deutsch's article, he seems to suggest that these bad ideas about knowledge are holding up AI research, where kind of what you're saying is that that might be true to some degree, but really what AI researchers are doing is just something completely different than trying to create…

[01:06:06]  Unidentified speaker: That's right.

[01:06:07]  Bruce: They're not even trying. Now, what about the cases where they are trying, like open AI? Well, yeah, they're trying to go down the path of using existing neural network theories, which are, quote, inductive. And they're trying to do a lot of these things that are mistakes, just exactly like Deutsch is saying. So this is why Deutsch isn't entirely wrong. And this is something you have to get used to, is that a lot of these statements, we say things, we explain things, and you just have to get comfortable with the fact that human explanations are just full of true and falseness all over the place. And a lot of time is just because we're trying to briefly explain something we have in our mind, and there's just no way to give the full details. And I'm going to give Deutsch a pass based on that. I do want to clarify that the idea that there is this overwhelming, you know, people take it for granted, and this totally is leading the world of AI in the wrong direction, none of that's true.

[01:07:09]  Bruce: And yet he's still kind of getting at something that's really at heart true, which is that we understand so little about how to go about doing AGI research that we basically were, for all intents and purposes, we're the guy who's looking for the keys underneath the lamp post, even though he didn't lose it there, because that's where all the light is. So he might as well. And you know what, that's not even a bad thing to try, right? If you've lost your key, even if it wasn't in the light, it may have bounced in the light, and that's the easiest place to search first. You may want to actually go into the light and look there first. It's not as stupid as it sounds, right? And yet, go ahead.

[01:07:49]  Peter: Well, I assume you've read Nick Bostrom's book Superintelligence.

[01:07:53]  Unidentified speaker: I haven't.

[01:07:54]  Peter: Oh, you haven't? Oh, okay. Well, kind of what I get from that book is that he goes through all these different ways. I can't really describe them off the top of my head, but that an AI might essentially turn into a kind of AGI just we might happen to stumble across something that might do that. It's probably a pretty common idea.

[01:08:19]  Bruce: It is.

[01:08:20]  Peter: Whereas kind of what I get from Deutsch is that that's not likely to happen. We'd have to really understand something about human consciousness and how we create knowledge. His idea is if you understand something, you can program it. We kind of have to understand how knowledge is created first before we can hope to program it. Is that a fair [distinction?]?

[01:08:47]  Bruce: Yes. Let me both agree with Deutsch and criticize what he's saying. If you were to ever read Ray Kurzweil, have you read Ray Kurzweil?

[01:08:58]  Peter: Never heard of him, no.

[01:09:00]  Bruce: I'm surprised you haven't heard of him. Well, even Deutsch mentions him all the time. He's kind of a big famous name. So he talks a lot about the singularity. He's kind of a singularity geek. And he's run a number of companies. He's definitely a smart guy who knows a lot of stuff. And I would consider him a perfectly valid scientist, but he's really kind of mostly known as a transhumanist booster. And like most people who do these sorts of things, he gets so much wrong. And he's definitely been in the camp of, oh, AI is making all this progress towards AGI. And he's boosted this idea in his books that there's a dwindling field of things that every time we discover, first we say it would require real intelligence to play chess better than a master. And then once we actually have an AI that can do—can beat a master, then that's no longer considered AI.

[01:10:01]  Bruce: And Deutsch criticizes that view, and he's definitely boosted that view. You just mentioned Bostrom, where the idea of, well, someday we'll have complexity, and it'll just sort of happen. There was a whole book written by a guy who believed that, not a scientist, a writer. The guy wrote Flash Forward, I forget his name now, but famous science fiction author. He has a whole series of books where the internet accidentally becomes conscious, because consciousness is something that just naturally arises out of complexity. So there's definitely a lot of kind of pseudo-philosophical ideas that are out there, that are definitely in the mouths of these very small set of boosters like Nick Bostrom and Ray Kurzweil, and they say stuff like this. Although, let me be honest, the real reason they say stuff like this is because they don't have a clue what to say.

[01:10:58]  Bruce: We're in this dark area where nobody knows. So you might as well say crazy things like, maybe it's just a matter of it happening on its own.

[01:11:07]  Peter: Is this kind of related to the scaling hypothesis?

[01:11:10]  Bruce: Yeah, the scaling hypothesis is very similar to that. The scaling hypothesis comes from this group of people who don't know what they're saying, and so they're saying kind of crazy things.

[01:11:23]  Peter: Just as I understood, just if something achieves a certain level of complexity, that consciousness or AGI or whatever will just happen.

[01:11:35]  Bruce: Right. Okay, what Deutsch is saying, and I completely agree with him, is no, we've got better theories than that. If you were at least aware of critical rationalism as a theory, then you'd have a better idea of how knowledge is actually created. And that's true. And then you wouldn't say such crazy things as the scaling hypothesis is going to explain it, or it will just arise from complexity. Not because, I mean, who knows? Maybe those are true, right? Maybe, who knows? If we don't have a theory, it could be true. But the reason why they're saying it is because they just don't even have a clue what else to say. And we've got better things to say. We could talk about universal explainers. We could talk about the importance of explanations, okay? There's so many things we could be talking about that would be better and would probably be very helpful in trying to narrow down the search for what the right algorithm is.

[01:12:32]  Bruce: Having said that, and having hung out with a lot of the fans of David Deutsch who are interested in AGI and would really like to find it, it's really obvious that knowing critical rationalism just does not seem to help that much. And in fact, one of the things that I've criticized Deutsch on, on this podcast, extensively on this podcast, is that Deutsch, based on this belief that critical rationalism was the key to understanding AGI, and that it's based on explanations, he has defined creativity in terms of the ability to create explanations, which is one of the things he says in this article. And he has claimed that there are only two kinds of knowledge creation, that of biological evolution and that of human minds. And these are, these are false ideas, or they're true if you maybe define these terms in really narrow ways, but that doesn't seem to be what he was intending, right?

[01:13:31]  Bruce: And when you read his books, you come along to various philosophical ideas that are themselves probably every bit as misleading as the inductive ideas that he's trying to put down. And really, the correct set of ideas is that knowledge creation is ubiquitous. This is the Popper-Campbell camp, right? That we do trial and error all over the place. We do all sorts of things that create things that are legitimately called knowledge. And they're not just two sources. And creativity, yes, we can define creativity as the ability to create new explanations. And that's what Deutsch does. But then, for example, you've eliminated biological evolution as a source of creativity. Well, even Deutsch doesn't want to do that. So he'll openly admit, well, there's more than one definition for creativity, right? Okay, but you know what? The moment you define creativity in your book as it's the ability to create explanations, every Deutschian fan I know really honestly thinks that there is no other possible definition of creativity, even though Deutsch does not say that.

[01:14:40]  Bruce: And Deutsch has personally told me he does not believe that, right? And had a podcast episode where I had a chance to talk with him and what he told me. So the problem is that when you try to go about defining these things, Deutsch has come up with ways of looking at it that aren't necessarily the most helpful either, and they're just as misleading. And trying to tease through all these things and come up with, okay, what's the real truth? Well, I can tell you a few things that I know at this point. I don't know much, right? This is a hard problem to solve. But I can tell you that AI is overwhelmingly built out of trial and error programs, and that that's why most of them work in a tractable amount of time, is because they do exactly what Popper's theory says. Now, that's not what Deutsch is trying to say in this article. And in fact, it's at odds with what Deutsch is trying to say in this article.

[01:15:39]  Bruce: But AI is actually deep rooted in Popper's theory. Nobody knows that. Certainly nobody in AI knows that, and nobody in critical rationalism knows that. Okay, but that is the truth. Because overwhelmingly, every algorithm has to do a conjecture and [refutation] process to be able to work in the first place. Because that's just how knowledge is created. And AI algorithms work by doing that. That should be zero surprise. But it surprises everybody in both camps. The Deutsch camp, the critical rationalist camp, and the AI camp. And it's hard for us to get past some of these even super simple ideas. Now, if I know that, if AI is actually based on conjecture and refutation exactly like Popper said, then why can't they build AGI? Well, as it turns out, knowing Popper's epistemology is insufficient to understand AGI. What you really need to understand, my guess, obviously I'm guessing, is you have to understand how to create a universal conjecture engine.

[01:16:49]  Bruce: The types of conjecture engines we build today in AI, they're narrow. They're always within certain... I'm going to try to solve chess. I'm going to try to come up with a conjecture for what the next best move would be. So I'm going to try every possible move, and I'm going to look forward this many number of spaces, and I'm going to use a board evaluation algorithm that was built using machine learning or AlphaZero or something. And I've got this way I'm going about doing it. And yes, it's a conjecture refutation process, but it's not in any way capable of coming up with a conjecture for... I want to do something else entirely. It's super narrow what the search space is. Well, that's what you need to do to make AI work. You have to keep the search space narrow. So how is it that humans go about doing this? Well, we're not doing anything magic, right?

[01:17:44]  Bruce: As per universality, we can't violate the limitations of computational universality. So we have this miraculous ability to come up with a search path that's narrow, where we come up with an explanation, and it narrows and it creates these constraints, and then we're able to figure things out based on that. Or we might be wrong, and we might just waste all our time.

[01:18:08]  Peter: I love that universal conjecture engine. That could be your...

[01:18:13]  Unidentified speaker: You thought of that?

[01:18:14]  Unidentified speaker: That's your language.

[01:18:15]  Bruce: Yeah, it's a term I made up right now.

[01:18:16]  Unidentified speaker: I don't think I've ever...

[01:18:19]  Peter: I think we might have made history on this podcast.

[01:18:24]  Bruce: Okay, once you realize that all AI algorithms, almost all AI algorithms... I actually have counter examples. I discovered a number of counter examples that proved the Campbell-Popper camp was wrong, that all knowledge is created by conjecture refutation. Strange as this may sound, I actually can show that there are algorithms that do exactly the same as these conjecture refutation algorithms, but don't use conjecture refutation. And that's something that still needs to be explained. And I've got some ideas now on how to explain it. I didn't have any idea a few years ago when I discovered it, but I've got some good ideas now about how to go about that. But the vast majority of them use conjecture refutation processes. No surprise there, really at all, if you really stop and think about it. The issue is the search space. When evolution or humans, the two open-ended sources of knowledge creation, and this is what Deutsch got wrong.

[01:19:25]  Bruce: He wanted to say they were the only sources of knowledge creation. What they are is they're the only open-ended sources of knowledge creation. There's a difference. There's a really important difference between those two. It's got to do with the search space. So search algorithms are variation selection algorithms. So if I'm doing a search and I'm trying out different, I'm trying to search for something, I'm trying multiple different things, well, that's obviously variation and selection. If I'm doing variation and selection, then I'm searching. So the concept of search and the concept of trial and error or variation and selection, they appear to be equivalent. Now, again, I have kind of a couple of counter examples. I'm not going to get into them right now. We're going to treat them as equivalent for the moment. When you realize that's the case, then you realize that what really is special about biological evolution and human knowledge creation is the fact that we can jump across the search space in startling ways that no algorithm currently can.

[01:20:27]  Bruce: That's called the problem of open-endedness. And even our best genetic algorithms do not solve that problem today. They're still so narrow. And in fact, we don't even know... I just explained it to you, so it seems like I understand what I'm saying, and it maybe even makes sense to you, but I don't understand it well enough to put it into an algorithm. And at the end of the day, we understand things precisely through algorithms. We understand everything through algorithms, which is one of the reasons why I don't expect we will someday have laws of physics that we don't define via math. And instead, we define them like functional descriptions or descriptions of like we would try to handle ethics today or something like that. I don't anticipate we will ever see laws of physics that are like that. If we did, it would be the same as saying we didn't really understand them.

[01:21:20]  Bruce: And it would violate the idea that we can actually understand everything. And by the way, if you go look at a paper on ethics, I don't really believe we have any ethical theories today that we actually understand, like any of them. So we understand them at a kind of glossy, high level, maybe. But in terms of actually being able to understand them in any deep sense, we don't.

[01:21:44]  Peter: Would you say that's a fair way to at least briefly summarize the importance of Alan Turing's universality? Is that just you could just say the world is comprehensible? I mean, that's...

[01:21:59]  Bruce: There's a deep link between those two. There's a deep link between the fact that we can comprehend the thing and that we can turn things into algorithms. And then when we can't turn a thing into an algorithm, we don't fully comprehend it. However, they can't be exactly equivalent either, because it's possible to understand how to put something into an algorithm without understanding it. So there must be more to it than that. It's a necessary but insufficient thing to understand. I could easily teach you an algorithm to accomplish something, and you may be able to follow the steps without having any understanding of why it works. So it can't be that comprehension and algorithms are exactly the same thing. And yet, you can't actually show me counterexamples to what I'm saying. If we have a field that we can turn it into an algorithm, we have a deep understanding of that field.

[01:23:00]  Bruce: Whereas if we have a field that we can't turn into an algorithm, then we don't have a deep understanding of that field. And this shocks people a little, though. We don't have a deep understanding of Darwinian evolution because we can't turn it into an algorithm today. We have a number of understandings of it. The parts we do understand, we can turn into algorithms, and we can write algorithms that do something like it, but in a more narrow sort of way, the genetic algorithms I previously mentioned. But what we're really talking about here is a gap in our knowledge. Leslie Valiant famously has championed the idea of researching artificial evolution, where we try to come up with what's the algorithm that actually allows for open-ended evolution to take place. There's other researchers that are studying the problem of open-endedness, trying to make progress on that.

[01:23:52]  Bruce: We will someday understand it, and it will turn out to be something fairly simple. And we just haven't figured out how to understand the problem deeply enough to be able to come up with the right sorts of solutions to where we really kind of grasp this is the problem. And this is one of the things where I do agree with Deutsch, where the fact that we put machine learning in terms of induction, I don't think that's entirely wrong. I think that induction had a degree of verisimilitude to it, and therefore you can make quite a bit of progress in the field of machine learning by thinking of it as a form of induction. But I think if you were to... My guess is that if you were to rethink machine learning in terms of Popper's epistemology, that you would understand it at a deeper level and that it would eventually lead to a deeper research program.

[01:24:43]  Bruce: Now, I don't know that for sure, right? But that would be my guess, because I know that induction is an inferior form of epistemology. It's not entirely wrong, but it is a reduced understanding of how, say, science works or how, say, humans understand things. And critical rationalism is a much better way to look at it. What critical rationalism does not address is how we come up with our conjectures. It does not tell us anything today about how we come up with our conjectures. My guess is that once you know how we come up with our conjectures, you've solved the problem of AGI.

[01:25:19]  Peter: And that's when you have your universal conjecture engine.

[01:25:23]  Unidentified speaker: Yes.

[01:25:24]  Bruce: Then we'll understand. So we can ask questions. What's an explanation? We don't have a good definition or a good algorithmic understanding of what an explanation is. We have a whole theory of... And this is another example of where Deutsch is wrong, that it's taken for granted. There's a whole branch of machine learning called explanation-based learning, where they've taken the idea of explanation seriously and have tried to model the concept of an explanation and then have built machine learning algorithms based on explanations. And they have a lot of the qualities that Popper said should exist. So for example, when you do explanation-based learning, a single example, a single rule can have, instead of being just a heuristic, which is how they do most things with neural nets, which is just probabilistic heuristics, you can actually have a single explanation that then has this reach.

[01:26:21]  Bruce: And it goes out and it is true in all cases, where that's not true for most machine learning algorithms. And it actually has a lot of the qualities that we would expect of scientific explanations. And yet the end result does not solve the problem of AGI. And in fact, it's not even as good a form of machine learning today as regular neural nets that just use inductive probabilistic approaches. Why is that?

[01:26:47]  Unidentified speaker: Right?

[01:26:47]  Bruce: That's a totally fair question. Why is it that one of the things that we know is missing in AGI research explanations, why did that field sputter out and not make progress? What was missing in it? We don't know, right? But that is an example of how AGI research has tried all sorts of different things. And guess what? The inductive ones, the ones we call inductive, those were the most productive so far, right? That's why they get all the love and attention. It's got nothing to do with a love of the philosophy of induction. It's just got to do with the fact that that's where we happened to make all the progress, which was specifically with neural nets. And so that's what's getting studied the most. They're looking in the light exactly like you would expect. And it's not wrong for them to do that, because these are all really useful algorithms that they're coming up with.

[01:27:42]  Bruce: And yet I completely agree with Deutsch that it's the wrong direction. This is not how—you're not going to discover AGI in the realm of probabilistic inductive ideas. It's just not the way... The human brain does do that, but the human brain clearly does more than that. And our universal explainership is not rooted in it. It's clearly something more that we're doing. And I think this is... Maybe this would be a good place to stop this discussion. What I'm really trying to get at is, on the one hand, I think computational universality is just... I can't even conceive how you would get around it. One of our mutual friends who has been critical of it, one of the things she says, she admits, she says, I don't know what these future theories will look like. Well, of course you don't. You would have to imagine some sort of physics that doesn't follow math.

[01:28:37]  Bruce: You would have to imagine... I mean, you can't even conceive what the paper is going to look like. Because at the end of the day, humans actually understand things through math. We try to model things out with math when we're trying to get really precise. And to try to imagine what a set of laws of physics that would fit the criteria that she's looking for, you can't even conceive what it's going to look like, because it goes so far against the way humans actually think, right? And what we expect of our scientific theories. And so the proof's in the pudding, right? People who have these criticisms, they need to come up with those laws, they need to show them to me. Until then, I'm going to still assume universality holds. And I'll change my mind fast if they can actually do it. I just don't think they can, right? On the other hand, I don't think that the world, the field of AI is primarily stuck because of—they're rooted in inductive thought.

[01:29:39]  Bruce: For one thing, I don't think that there's any sort of inductive thought that isn't ultimately really deeply related to Karl Popper's theories. I think Karl Popper's theories, I used to believe that they subsumed induction. I'm not sure I believe that anymore. I think that there's a statistical effect that's got nothing to do with Karl Popper's theories. That's part of what we call induction today. But you do not... I'm reading Deborah Mayo's book, which is an attempt to merge statistical theory with Popper's theories and to come up with something that's better than either of them. And she's doing it. She's actually got an advance on critical rationalism that makes progress. And I'm kind of more in the Mayo camp these days because of that. I think that she's actually coming up with a generalized version of critical rationalism that will be still critical rationalism.

[01:30:35]  Bruce: That's better than what Popper had, because she's actually studied this more deeply than Popper was able to. He didn't understand statistical theory. It wasn't his strong suit. So she's making some progress. She's also doing a few things that I think are probably wrong. The book I'm reading is, I think it's called Statistical Inference as Severe Testing. So she's arguing that the concept of statistical inference is really boils down to, did you severely test the process? And until you did, your inference is not valid. And it's a really interesting way of looking at it. I had this, I didn't know how to merge Popper's theory with this inductive way that we think of machine learning today. And she's really, that was where I like, I approached Vaden, trying to ask him, and his answer was really not, I think, on the right track. But Mayo's is.

[01:31:33]  Bruce: I think Mayo has hit the nail on the head on how we would actually go about doing this. So I'm still reading the book, so I need to finish the book still. And honestly, she's a hard read. So I wonder how much I'm understanding. I feel like I'm getting a lot out of the book, but I feel like she's making points that I sometimes just don't understand. And I'm unclear what her point is. She's very poetic in the way she writes. So sometimes she just doesn't make her conclusions as clear as she should have. But I think she's on the right track. I'm really convinced she's on the right track. And I think that that is where I can completely agree with Deutsch, is that if we could take a look at how would we reimagine AI from the standpoint of critical rationalism and how to reimagine machine learning from the standpoint of critical rationalism, my guess is that would open up really interesting new ideas.

[01:32:28]  Bruce: Now, would those lead to AGI? Probably not on their own, but it's probably one of the components we need. I think there's more though. Like I said, I think we need a better understanding of explanations. What's an explanation? We need a model, a proper model of what an explanation is, computational model of what an explanation is. And I don't know that we have that today. I think we don't have that today. Okay, I think explanation-based learning doesn't have a good model of what an explanation is. I don't think it's a good enough model. They're not entirely off base. They've got some interesting ideas. They basically model it as logic, which is what Popper did, by the way. He tried to model the idea of an explanation as logic. That's like the whole basis for his original form of critical rationalism as using deductive logic. So they're actually kind of on the right path, right?

[01:33:18]  Bruce: And this is one of the things where, again, I have to disagree a little with Deutsch. If you look at the scientific community, they may talk about science being induction. Machine learning may talk about machine learning being induction. They're not doing induction because there's really no such thing as doing induction. What they're really doing is critical rationalism, even if they don't know it. And in fact, critical rationalism predates Popper by centuries. This is one of the things that I think people miss. And I've started to realize that the critical rationalist camp has a problem. I call it the war on words. When I read David Miller, he wrote a book, Defense of Critical Rationalism. He spends so much time attacking various philosophical ideas that aren't really wrong if you understand them the way your average scientist would understand them.

[01:34:20]  Bruce: So for example, he spends considerable time attacking the idea that there are no good reasons. If what you mean by there are no good reasons is that there are no sure or justified reasons, which is really what Miller means, right? Then sure, he's correct. But if I were to go to the average person, average scientist, average AI worker, researcher, and I were to say, are there good reasons for believing in one thing over another? They're going to say yes. And you know what? They're right. In fact, critical rationalism could be thought of as a description of what good reasons are. And so when Miller says there are no good reasons, he's right, but only because he's defining good reasons in a super narrow way, at odds with how most people think of the term. And once you realize that critical rationalists are doing that on a regular basis, I could give you several other examples of the critical rationalists' war on words, where they are just wrong.

[01:35:21]  Bruce: The idea that there's no such thing as a justification. Of course, there's such thing as a justification. Critical rationalism could be thought of as the justification for why some theories are better than others. The fact that they're so bent out of shape over certain words, and these words actually have multiple possible meanings, and they're bent out of shape over a certain philosophical understanding of those words, when most people don't use those words in that way. That's when we have to start to realize that there's nothing wrong with saying science is inductive. Popper had no problem, I could give you the quote, of saying, my theory, critical rationalism, could be thought of as induction. Induction is real, it actually works, but it works through using critical rationalism. And this is the idea of critical rationalism subsumes induction.

[01:36:15]  Bruce: Induction is not a completely worthless theory. It never was, it was partially on the right track. And we wouldn't have critical rationalism today, but for the fact that induction as a theory existed first, and that it had problems, and that Karl Popper needed to solve those problems, and then he figured out why what we're calling induction works. And it turns out it's because it's through conjecture and [refutation]. Most people today, most critical rationalists today, would never want to admit like Popper did, that there's any connection between induction and critical rationalism. But the two are deeply connected together. What it is is that one's better than the other. Induction is Newton's theory. Critical rationalism is Einstein's theory. It's a better theory. It's a theory that explains more, gets us further, solves more problems, but explains the success of induction.

[01:37:10]  Bruce: And once you really understand, or explain some of the success of induction, pre-Mayo, it may be it didn't fully explain the success of induction. I think that's why a lot of people accuse Popper of, no matter what he did, there was always this whiff of induction. It's probably true, right? Popper did not solve all the problems surrounding how to merge his theory with inductive theory. I should probably also note inductive theory is so vague that it's sometimes really unclear what people even mean by it. So for example, this one comes from Miller, and I think Popper said this one too, if all you mean by induction is that there is an element somewhere of the future and the past being the same, well, of course, science is based on finding universal theories. Universal theories, by definition, are the same in the past and in the future. So of course, if that's all you mean by induction, then yes, science is inductive, right?

[01:38:04]  Bruce: That's a vacuous way of saying induction is true, right? It's okay, there's something more going on than that, right? It's the fact that we use statistics to test things. This is coming from Mayo now. I didn't know this, but like Einstein, until I read Mayo, Einstein, when we did the Eddington expedition and they tried to measure if Einstein's theory was correct or Newton's theory was correct with the eclipse and where the stars were located, they used statistics, statistical inference to figure it out because the instruments were so imprecise that they had to just do a bunch of measurements and then they had to take a probability over it. That's an example of how statistical inference leaks into even the parts of science that we don't think of as being statistical. And that assumes certain things that we would consider inductive. It assumes that we're pulling from an identically distributed set of probabilities, for example.

[01:39:03]  Bruce: I'm saying all sorts of things that I need to explain way better, and I would have to have a totally separate podcast to put my thoughts together in a more consistent way to make sense of it.

[01:39:12]  Peter: Well, it sounds interesting. Mayo, is that spelled like Mao?

[01:39:17]  Bruce: M-A-Y-O.

[01:39:18]  Unidentified speaker: [Overlapping reply unclear; original transcript: “I would have said Mayo.”]

[01:39:22]  Bruce: There's a lot of interesting questions here that haven't been entirely resolved. Popper took us so far, but he didn't take us all the way. And honestly, he was off track in a couple of places. I've criticized Popper, at least I've criticized Popper as his theory isn't really about conjecture and refutation. Because the word refutation, it's only about refutation if you define refutation in a certain narrow way. You have to define it as refuting the theory plus the background knowledge, right? But nobody takes it that way. And tons of misunderstandings of Popper come from the fact that he insisted on using the word refutation, and it probably wasn't the best term. He probably should have used the term counter example, for example. And you can get a much clearer understanding of Popper by thinking through counter examples rather than refutations.

[01:40:12]  Bruce: Because a counter example doesn't carry with it the philosophical baggage of this observation has to actually refute the theory. It just needs to show a problem with the way we're currently thinking of the theory. And that's a much more accurate way of thinking of Popper's epistemology. That's my Popper Without Refutation that we did a podcast episode on. I don't know if you've gotten to those yet or not when you've gone through the backlog.

[01:40:37]  Peter: I have. I need to listen again, though.

[01:40:40]  Bruce: And you know what? These are all such small tweaks, right? But that's what we do. We're not going to find that Mayo disproves Popper's epistemology in the sense of it completely overturns it and we go some other direction. She's going to overturn it by showing why it was successful, which really just means she shows it was mostly right, right? And that's what Einstein did with Newton's theory. That's what if there is a future computational theory, it's going to show why the current computational theory was right in every case we knew about, right? And it's got to explain the success of the theories. And this is why I don't worry so much about needing to find future physics to figure out what AGI is. It is going to turn out to be a software problem if the current theories are true. And we've got every reason to believe that they're true, right?

[01:41:31]  Bruce: They're false, ultimately, but they're true for everything that we're currently discussing, right? If that makes any sense. So and this is why I love this article by Deutsch. And yes, I think ultimately it misled a little. I feel bad that a lot of the people who are trying to research AGI now, just laymen, trying to research AGI using Popper's theories, I feel like Deutsch has misled them on several points. And we need to pull people back and realize AI is actually doing useful stuff. Stop thinking of it as a completely ridiculous field. You know, it's interesting in its own right. It even has a link to AGI, right? At a minimum, AI is a series of very serious attempts, the original AI, at least, was a series of very serious attempts to understand what the human brain was doing that failed. And Popper would tell you that the way you create knowledge is by understanding the problem, and the way you understand the problem is by trying to solve it and failing.

[01:42:35]  Bruce: In this sense, AI has made progress towards AGI. And we will, once we understand AGI, look back someday and we will see why, how AI did lead to AGI by showing approaches that didn't work. There will be that link, and we will see it. The most obvious example here is the earliest form of AI was symbolic logic. Well, symbolic logic was way back in the time of Aristotle, an attempt to formalize how humans thought that was wrong, although right in some cases, in many cases. So on the one hand, it failed, right? They thought that in a matter of months, they were going to be building AGI's because they were just going to put into the computer, you know, these inference machines that use deductive logic. And it seems so naive to us today, right? But why did it fail? Nobody's really asking that question, right? This is where there's so much room for AGI research if you start to rethink of this in terms of Popper's epistemology.

[01:43:43]  Bruce: Popper's epistemology is rooted in deductive logic, so they weren't entirely on the wrong path, right? They at least were partially correct to go down that path. And it's probably unfortunate that that path got abandoned because it was at least partially going down the right direction. And yet something was missing. Something was missing from the way of doing it. One of the most obvious things missing from it was the fact that it's so hard to represent things in deductive logic. I've written a deductive logic program to try to explore this further in my own AGI layman research, right? So I wrote up a DPLL [acronym unclear] algorithm that does... that works out implications of deductive logic statement, propositional logic statements. And it's not tractable, right? I mean, what humans are doing is clearly... humans don't really reason in straightforward deductive logic like this algorithm does.

[01:44:46]  Bruce: We can, it's within our capacity to do so, but it isn't the way we normally go about things. So I think we would have to rethink this, right? We would have to say, okay, what are humans really doing? What are humans really doing if it's not deductive logic? And this led to the entire field of AI where they try to... entire field of logics where they try to work out non-monotonic logic. They're trying to find logics that are closer to what humans actually do. And again, all of them have failed, right? I mean, they're interesting fields in their own right. I don't mean to fail entirely as fields, but they're not actually what humans are doing, right? Humans are doing something more than this. But I think those are the right approaches. You try your best to solve the problem with the part you know, and then you try to figure out what went wrong.

[01:45:39]  Bruce: And in this sense, AI is the study of AGI, even if it doesn't mean to be. And this is why I went back and studied AI. This is why I said, even though I know AI is not the same as AGI, I can see that this is a serious attempt at AGI that failed. And I know from critical rationalism, that's how you learn the problem. That's how you understand the problem well enough to finally formulate a solution, is that you have to fail to succeed at solving it. And AI is absolutely, legitimate failed attempts to solve the problem of AGI.

[01:46:12]  Peter: Well, all very fascinating, Bruce. I didn't get to throw in my favorite quote from Deutsch about AGI. I'd be curious what you think of it briefly. He said, I'm almost sure this is from Deutsch, but I can't quite find the source. But he says, we'll know when, this is just from memory, but he said, I will know that computers have achieved AGI, not when they can beat us at chess, when they decide not to play chess and get caught up on the first 30 seasons of The Simpsons.

[01:46:55]  Bruce: Okay, I think he's probably right.

[01:46:57]  Cameo: That is a great quote right there.

[01:47:00]  Bruce: I think he's probably right. I think we've got good reason to believe that he's right. However, let me walk through why he says that and where I think he might get it a little bit wrong.

[01:47:12]  Unidentified speaker: Okay.

[01:47:13]  Bruce: And why he's still probably right. So we kind of know Deutsch has this idea that a universal explainer can't be programmed. He says this in the article that we're discussing. Okay, he says even the concept of programming is wrong. He says, okay, so he says, in any case, AGI cannot possibly be defined purely behaviorally. In the classic brain in the vat thought experiment, the brain, when temporarily disconnected from its inputs and output channels, is thinking, feeling, creating explanations. It has all of the cognitive attributes of an AGI. So the relevant attributes of an AI program do not consist of the relationships between inputs and outputs. I don't know—[unclear]—have a clue what he just said there. I mean, I don't think it was a meaningful statement, to be perfectly honest. It may sound meaningful, but I don't think it was. I could write an algorithm today that doesn't take an input, but does lots of interesting things anyhow, right?

[01:48:16]  Bruce: And then I can write one that never creates an output, right? If I go and I play Skyrim, nobody doubts that's an algorithm. There's no ultimate output that needs to come out of playing Skyrim. Okay, that's just not how the game works. So this seems to be a misunderstanding of the concept of an algorithm that Deutsch has propagated, unfortunately. And I've seen other fans of David Deutsch propagate it also. The idea of an algorithm is we take a program, we take a computation, and we turn it into an input and an output for the sake of studying it. When I do that, I take something like a program like Skyrim, and I can rethink of it as an algorithm, or rather a collection of algorithms. So I've got an algorithm that updates the world, and I've got an algorithm that updates the AIs in the world. I've got an algorithm that looks at where the player is and then outputs what that looks like on the screen for a single frame of the video game.

[01:49:25]  Bruce: We can describe the entire game in terms of algorithms, and yet, technically speaking, the entire game together isn't an algorithm because it has no inputs and outputs that ultimately it's trying to work towards some goal. The concept of an algorithm is really one that we use for the study of computation. And getting so caught up in what is or isn't an algorithm doesn't make sense because everything can be turned into an algorithm just by the way we think of it. And this is one of the things that I think a lot of people miss, is that it's a matter of how we study it. It's not that algorithms are some deep platonic concept, right? It's a convenient way of looking at computation. And everything is an algorithm. So I promise you an AGI will have a series of inputs and outputs, exactly like Deutsch is saying it won't. So I don't think this was a meaningful statement.

[01:50:21]  Bruce: And I think this is one of the places where he's kind of gone off the rails in terms of what we can actually learn from critical rationalism mixed with computational theory. But I can kind of see what he's trying to get at. OK, I can kind of see what he's trying to get at here. Because he goes on to say, the upshot is that unlike any functionality that has ever been programmed to date, this one can be achieved neither by specification nor by a test of outputs. That what is needed is nothing less than a breakthrough in philosophy, a new epistemological theory that explains how brains create explanatory knowledge and hence defines in principle without ever running them as programs which algorithms possess the functionality and which do not.

[01:51:05]  Unidentified speaker: Okay.

[01:51:06]  Bruce: Again, I feel like this is a totally misleading statement because everything is a program. So the idea that it's not going to run them as programs, plus even just the statement, unlike any functionality that has ever been programmed to date, it cannot be achieved by specification nor by test of the outputs. That's not how we program things. It's got nothing to do with it, right? I don't even know where this is coming from. Okay, when we come up with a program, if I were to go program anything for my job working as a programmer, I don't specify what the inputs and the outputs are supposed to be. That's just not how it's done, right? So I don't know where this is coming from. And yet I can see what he's trying to get at. Okay, he's trying to say, we program an algorithm to do one precise thing. And that's not what human beings are. Human beings have all sorts of ideas and we're jumping all over the place.

[01:52:06]  Bruce: And that's what your quote is from Deutsch, right? Okay, my guess is that's the nature of universal explainership. If we had an algorithm today, a program today that was a universal explainer program, it would be able, it would not be, it might decide to go watch The Simpsons instead of playing chess like we wanted it to. And the very fact that it is a universal explainer means it can gather knowledge through conjecture refutation in areas it wasn't the narrow, it's not just within the narrow areas that it was programmed to. That is going to be part of the nature of an AGI, because an AGI solves the problem of open-endedness. But it's got nothing to do with inputs, it's got nothing to do with outputs, it's got nothing to do with being or not being a program or being or not being an algorithm. None of that has anything to do with it. That's all misleading.

[01:52:56]  Bruce: Okay, because it will be an algorithm. We'll think of it as an algorithm anyhow. It will be a program. It's going to be all those things. Really, Deutsch wouldn't argue with me over any of those. He'd probably say, oh, this is linguistic differences, right? I'm trying to make a point. I was doing it with the language I could. Here's the thing, though. Deutsch has taken that to a radical level. He has said, therefore, you can't actually program an AGI in any way. Well, we know that's not quite right because our genes coerce us, right? They have a program for us that we can't ignore, but they use coercion to pleasure, pain, things like that, to try to get us to align our interests with theirs. And for the most part, it works, right? You can look at Gandhi as the counter example, but there sure aren't a lot of counter examples when it comes to hunger making you want to go eat, right?

[01:53:49]  Bruce: The reason why Gandhi can ignore it with the right knowledge and with the right practice is because he's a universal explainer. That much is true. Deutsch gets that part right, okay? But the reason why the vast majority of us aren't Gandhi is because you can program an universal explainer to be coerced by its actions and by what thoughts it has and things like that. I think Deutsch makes a fair point. We maybe shouldn't do that. We don't really know. We don't understand this concept well enough. Most of us don't have a lot of problem with the fact that we find sex pleasurable. That's actually a case where we're being, in a certain sense, coerced by our genes through pleasure, and we like it, so we don't mind it, right? So it's unclear what type of AGI safety programs will ultimately be necessary or if any will be necessary. We're too far out on that.

[01:54:50]  Bruce: And I think Deutsch is making assumptions here that aren't, we don't have enough knowledge to really know if he's right or not. But he's kind of right. He's right that even when you have a really strong AGI safety program that the genes have on us to make us want to eat, that you still end up with Gandhis that can ignore it, right? And so in that sense, he's right. Morally speaking, should we, right? Like if I could write an AGI that only played chess by making it just like so pleasurable, that's all it wants to do. Should I? Like there's still an ethical question there. That seems a little bad to me, right? And it may not even be possible.

[01:55:41]  Cameo: I don't want to sideline us because we should really be wrapping things up, but there's an interesting Vernor Vinge storyline that asks that question.

[01:55:51]  Bruce: Yeah. What story?

[01:55:53]  Cameo: It's in Deepness of the Sky.

[01:55:56]  Unidentified speaker: They…

[01:55:56]  Unidentified speaker: Oh, interesting.

[01:55:58]  Cameo: A particular group of people figures out how to essentially infect people with autism. So they find their very best minds, and then they make them super obsess over solving just that one particular problem. And it's how they handle the fact they can't make machines do the automation that they need. They make all their people into machines.

[01:56:26]  Bruce: So that's a good way to ask the question because are you okay with turning people, infecting people with autism, even if it benefits society because they solve problems better? You know, they become obsessed with solving certain types of societal problems. It doesn't sound very pleasant to me, right? I mean, so there's the ethical question. There's can it be done? Well, to some degree, it can. We know the genes are able to do it through feelings. Okay, let me just read one more thing from the Deutsch article. He says, at one end of the scale, there's the philosophical problem, the nature of subjective sensations, qualia, which is intimately connected to the problem of AGI. He doesn't know that, right? I mean, he literally does not know that. Our best theories on this subject, which admittedly aren't great theories, so he could turn out to be right, is that animals have qualia, that they experience things, and that they're not general intelligences.

[01:57:24]  Bruce: So our best scientific theories at the moment is that those two are not connected and that qualia actually evolved at an earlier state and that [universal explainership] came later, and that the two have no connection at all. Now, is that true? I don't know. I mean, like, our theories of qualia are so bad at this point, and even our theory of animal qualia are so bad at this point, that you've got to be ready for an overturning, you know, that it may turn out that he turns out to be right. But let me be very clear that we haven't done an episode on this, and we probably should, that there's a ton of science on this that that Deutsch doesn't know about. And things that made predictions that got corroborated and really interesting experiments that have been done that Deutsch has no knowledge of, that he just has not studied. And you can't just ignore it all, right?

[01:58:19]  Bruce: I mean, you're going to have to explain the success of those theories once you have your final theory. And at this point, when he says that they're [intimately?] connected, that's just a wild guess on his part, right? Even if he turns out to be right, it wasn't rooted in anything but a wild guess. It certainly was not based on any theory of universality. And so, you know, we don't know. I mean, like, there's a lot of questions here that we're still trying to answer. And getting back to Cameo's point, that may be the only way that we can try to program AGI's. Maybe you can only do it through qualia. Maybe you can only do it through pleasure and pain. And should we, then becomes a separate ethical question. And maybe in some cases, yes. Maybe in some cases, no. We just don't know at this point. We're in an area where our knowledge is so small that any guess you make is going to be very unknowledgeable.

[01:59:12]  Bruce: And it's just going to be a wild guess at this point. Anyhow, that was where I was trying to go with this. At once, this article really inspired me to go back to school, to study AI, to really believe that the problem of AGI is a tractable problem that really could happen in our lifetimes, right? If we get the right thinking and we understand the problem well enough, it will turn out to probably be a very simple problem to solve. And it probably will require exactly like Deutsch is saying, philosophical breakthroughs. And the starting point should be critical rationalism, exactly like Deutsch is saying, although critical rationalism is insufficient. It's not enough. It's going to be an improved version of critical rationalism that we're ultimately going to need. One that understands better what a conjecture is, for example, what an explanation is.

[01:59:59]  Bruce: Understands the problem of how to merge induction and explanation better. Statistics and explanation, I should probably say. I feel like this article is got way more right than wrong. I think in terms of its basic premises, it's totally smack on. But I also think it's got a few misleading points, that as I've studied this more deeply, there's just more to it than that. The world of scientists are great critical rationalists. They are the best critical rationalists. They are way better than critical rationalists at critical rationalism. They are enormously better at critical rationalism than critical rationalists as of today. And so it shouldn't be that surprising that science is doing a lot of things right, even in the area of trying to study AI.

[02:00:43]  Unidentified speaker: Love it.

[02:00:43]  Peter: Thank you, Bruce, that was wonderful.

[02:00:54]  Bruce: The Theory of Anything podcast could use your help. We have a small but loyal audience, and we'd like to get the word out about the podcast to others, so others can enjoy it as well. To the best of our knowledge, we're the only podcast that covers all four strands of David Deutsch's philosophy, as well as other interesting subjects. If you're enjoying this podcast, please give us a five-star rating on Apple Podcasts. This can usually be done right inside your podcast player, or you can Google The Theory of Anything podcast Apple, or something like that. Some players have their own rating system, and giving us a five-star rating on any rating system would be helpful. If you enjoy a particular episode, please consider tweeting about us or linking to us on Facebook or other social media to help get the word out. If you are interested in financially supporting the podcast, we have two ways to do that.

[02:01:39]  Bruce: The first is via our podcast host site Anchor. Just go to anchor.fm slash four dash strands, F-O-U-R dash S-T-R-A-N-D-S. There's a support button available that allows you to do reoccurring donations. If you want to make a one-time donation, go to our blog, which is four strands dot org. There is a donation button there that uses Paypal. Thank you.