Episode 139: The Rational Doomers

• Listen to this episode: Apple Podcasts

Unofficial, reviewed transcript. It may still contain mistakes; check the podcast when wording matters.

Speakers are labeled by first name where identified. Unidentified-speaker labels may refer to different people. Bracketed question marks indicate uncertain wording.

Transcript

[00:00:00]  Unidentified speaker: Hello out there.

[00:00:02]  Peter: This week on The Theory of Anything podcast, we talk about Doomers, specifically AI Doomers. Why has it become such a popular notion that this kind of apocalypse, amongst all the other kinds of apocalypses, is imminent, especially amongst those who might consider themselves rationalists? I know a lot of times in these intros, to talk about how this episode is the best ever or whatever. I'm not sure I'd actually say that about this one. There was no planning here, and the conversation was kind of a tangent we fell into and abandoned what Bruce was going to talk about. If we did Patreon-only episodes, I might be tempted to put this there. Still, I got a lot out of speaking with Bruce here. As always, and I'm optimistic that someone in our audience will get something out of this too.

[00:01:12]  Unidentified speaker: Welcome to The Theory of Anything podcast.

[00:01:14]  Unidentified speaker: Hey, Peter.

[00:01:15]  Unidentified speaker: Hey, Bruce.

[00:01:16]  Unidentified speaker: How are you doing today?

[00:01:18]  Bruce: Doing good.

[00:01:21]  Peter: I was just gonna say, I have been going to a lot of these philosophy meetups, as I discussed on Facebook a bit, including the Monday night rationalist meetups. So not critical rationalists. These are, they've never even heard of critical rationalists.

[00:01:40]  Unidentified speaker: We're in a niche of a niche, really.

[00:01:42]  Peter: But these are more less wrong types. And so every week, they bring in a less wrong article, and we discuss it. And they're all, I mean, they're really nice. But it's interesting to see this in real life, that this this this idea by Eliezer, I always forget, I can never pronounce his name very well.

[00:02:11]  Unidentified speaker: I can't pronounce Eliezer Yudkowsky.

[00:02:14]  Peter: Yudkowsky, his idea that if, as his book says, if anyone builds it, everyone dies. I mean, this is something that is almost like gospel truth to this crowd. And I don't think they've ever met. Someone, well, I shouldn't make that assumption, but it doesn't seem like they've ever met someone who understands or at least has some understanding of their perspective and disagrees with it. So what I need from you Bruce is more ammunition. So, okay, so next time we talk about AGI.

[00:02:55]  Bruce: We need to do an episode on Eliezer Yudkowsky's ideas. I started listening to that debate between him and Stephen Wolfram that you recommended to me.

[00:03:04]  Unidentified speaker: Yeah.

[00:03:05]  Bruce: And first of all, I find both of them very annoying.

[00:03:08]  Unidentified speaker: Oh, is that right?

[00:03:09]  Unidentified speaker: Shots fired.

[00:03:10]  Unidentified speaker: Wow.

[00:03:10]  Unidentified speaker: Okay.

[00:03:11]  Unidentified speaker: Yeah.

[00:03:13]  Bruce: You can kind of tell Eliza is annoyed with Stephen Wolfram because he goes down so many rabbit holes. And it's really hilarious.

[00:03:19]  Unidentified speaker: It's really hilarious, actually.

[00:03:21]  Unidentified speaker: It seems like Wolfram, you know, it wasn't like he had like an optimistic view of AGI necessarily, but he seemed to be asking what to me seemed more, just harder questions about what is intelligence, I guess, that that Yudkowsky just seemed to kind of ignore or he wanted to almost like gloss over.

[00:03:48]  Peter: So I appreciated that about the debate. I mean, Yudkowsky just wanted to like almost like single-mindedly focus on his thesis that AGI was 99. % probability just going to destroy life on planet Earth, unless we, I don't know, I mean, even some of these guys at the rationality meetup said openly, I would support bombing data centers if I thought it would work. Like that's how seriously they took this assertion that AGI is going to destroy everyone.

[00:04:32]  Bruce: So we're going to talk right now, we're going to put aside what I was going to do, but we will get back to it. But let's talk about your experience here just for a second. So you talked about it on Facebook. So I know a little bit about what it is you said to them. You started bringing up a more optimistic view and raising different things that come out of critical rationalisms, particularly the Deutschian take on critical rationalism, which I've been somewhat critical of sometimes. But I feel like your point of view and their point of view, it's night and day difference. Not just that yours is actually optimistic and theirs isn't. So why even bother with their point of view, given how unoptimistic it is? But it's so obvious to me that they're not all that particularly rational, if that makes any sense. That, yes, this is a group of rationalists. What does that term really mean? What does it mean to them? What does it mean to me? What do I think that term should mean? What does it mean in real life? There's a question around this. And I do feel like Yudkowsky is, he's one of the main people behind, from what I understand, from the whole effective altruism, less wrong type groups. And he's really into rationality. And I don't really consider him all that particularly rational, at least not with his belief about AGI. He may well be very rational in other ways.

[00:06:06]  Unidentified speaker: Sure.

[00:06:07]  Bruce: But in terms of how he's decided to define himself and what his key beliefs are, I feel like we're just dealing with a religious belief, honestly. I'm not against religious belief, right? But like, I would never mistake what he's doing, which is a religious belief and a religious theology for rationalism. Like to me, those are very different things.

[00:06:28]  Unidentified speaker: I think you can have both.

[00:06:29]  Bruce: I think it's fine to have both. But for some reason, he sees the two as the same and they're not. I thought it was funny that one of the people at the meetup accused you of digital Christianity. Was that the term?

[00:06:44]  Unidentified speaker: Yes.

[00:06:45]  Peter: I mean, that was more in relation to me talking about some Tipler kind of omega point.

[00:06:53]  Bruce: Which literally is cyber Christianity.

[00:06:56]  Unidentified speaker: Yeah.

[00:06:57]  Unidentified speaker: Yeah.

[00:06:57]  Peter: No, I thought it was a very fair criticism. Very, I mean, I loved it. But, you know, the, about the, with the rationalists who were most into Yudkowsky and this idea, I tried to, I don't know if this was the best argument to use against what they were saying. I tried to emphasize this idea that, as far as what we know, at least with a data point of one, that moral knowledge seems to be interwoven with technological knowledge.

[00:07:33]  Unidentified speaker: Yeah.

[00:07:34]  Peter: You know, that seems to be what history indicates. I mean, you invent the printing press and then, you know, the novel, and then you start realizing that slavery is, is wrong and, you know, there's, and maybe women should get to choose who they marry and all this stuff.

[00:07:52]  Unidentified speaker: And then the whole world goes to pot.

[00:07:55]  Unidentified speaker: Then the whole world goes to pot.

[00:07:57]  Unidentified speaker: That's another, that's another assertion.

[00:08:02]  Peter: But yeah, I, that seemed to resonate with them somewhat. I don't think they'd ever really heard it like that before. So I tried to claim that it would be actually far more likely that AGI would be far more moral than us, if anything, rather than less, okay? Or they might just be fallible creatures who make a lot of the same mistakes we do. I tried to question this assertion that more memory and more processing speed is necessarily going to create like a creature that might either have been purposefully or accidentally as a by-product of going after some other goal, destroy all of humanity. It just seems like we're dealing with a thing where technology or the history repeats itself in the sense that people worry so much about any new technology, whether it's the radio or TV or social media, on and on. But it ends up basically just improving our lives more than anything. Maybe not in all areas, but in most areas, it would end up improving our lives.

[00:09:32]  Unidentified speaker: But to them, this is an [accept?].

[00:09:35]  Peter: I think they would basically accept that case in every other situation. But to them, there's something about this that's uniquely dangerous, that it is the existential threat we are dealing with right now.

[00:09:52]  Bruce: Did you bring up the idea that maybe AGI would save us from, say, an asteroid, because now we're gaining knowledge faster or anything like that?

[00:09:59]  Unidentified speaker: Did that come up in the conversation?

[00:10:02]  Unidentified speaker: Yeah, I mean I tried to—

[00:10:03]  Bruce: Maybe we're in greater danger, probabilistically speaking, if they're into probability, with not building AGI.

[00:10:11]  Unidentified speaker: Yeah.

[00:10:11]  Unidentified speaker: Well, I think that they—I tried to focus on [the air?].

[00:10:14]  Peter: It seemed like the core belief is that the way humans create knowledge is just one of many ways that humans create knowledge. That we create knowledge by, you know, I tried to make it meta in a way that, yeah, we create knowledge by going to philosophy groups and, drinking a beer and talking issues through, and they got the point that it seems to me that there's, as far as we know, we have one mechanism of creating knowledge, which is we do a lot of talking with other people, listening to other opinions, changing our minds, making mistakes, conjectures and refutations, this whole broader Popperian process, which seems to kind of capture how humans, this mysterious process of human creativity that Popper seems to capture pretty well, in my opinion. But I think to someone like them, this is just one mechanism of creating knowledge, if they accept that characterization at all.

[00:11:33]  Peter: And that, say, if there was a super intelligence, it could, you know, and if it wasn't programmed just right, it would create, you know, knowledge that would be beyond, you know, in 30 seconds, it would create more knowledge than it's ever been created in all of human civilization through, I mean, maybe I'm exaggerating a little bit, but, you know, that it would basically jump, you know, that there would be some kind of singularity, like process, where, which is, I think, Kurzweil says, you know, advocates too, is that we're talking about the technological singularity where knowledge creates more knowledge, creates more knowledge all digitally inside this AGI, and that who knows what kind of amoral monster might be created, and this thing would rather either purposefully or accidentally destroy life as we know it. Or, you know, an interesting kind of side. You know, honestly, I think that another kind of side view of this sort of that people put out there is that this thing might happen, and they might just decide that screw this, we're just going to leave planet Earth and, you know, move out into the Cosmos somewhere.

[00:13:02]  Bruce: Or I think people, I think that people, some people think that it might have already happened.

[00:13:07]  Peter: And maybe inside the machines, our machines somewhere, there's a whole like other universe of AGI that they're just kind of like, don't really see us as that important to what they're doing.

[00:13:20]  Unidentified speaker: I don't know.

[00:13:21]  Peter: I still think all things considered based on my layman's understanding, the most plausible thing is that if we do create AGI, and there might not be any reason to think we're close to that, that it's going to be either more moral than us or probably about on our level.

[00:13:43]  Bruce: But I actually appreciated the fact that Wolfram brought up the fact that, you know, for all I know, maybe I'm rooting for the AGI's. Like maybe, maybe if they wipe us out, it's because we deserve to be wiped out and they're better than us. And Yudkowsky's response to that was, well, I'm worried that they're not going to have any feelings. I'm worried that they don't have—they won't have qualia basically, right?

[00:14:11]  Unidentified speaker: Yeah.

[00:14:12]  Bruce: And that you're going to have this kind of empty universe, and I want a universe that's full of people that care and.

[00:14:20]  Peter: But then he doesn't connect that those feelings to the process of knowledge creation. Maybe that's really what it is.

[00:14:29]  Unidentified speaker: Right.

[00:14:29]  Bruce: Well, I don't know if they are connected, right? I don't think we know because we know so little about this, right? But like, we don't think you can assume they aren't connected. I don't think you can assume they aren't connected. I think that we have no real good theories either way.

[00:14:47]  Unidentified speaker: Fair enough.

[00:14:47]  Bruce: It seems to me that this is the biggest problem with Yudkowsky's view. So first of all, let me just say, I'm a little different than most crit rats in that I think most crit rats are basically mirror images of Yudkowsky, that there's no possibility that it could go wrong, that it's guaranteed that they'll be more moral than us if they're super intelligences, but we don't believe in super intelligences anyhow. It's racist, there's this standard set of talking points that crit rats use that to me seem like the mirror opposites and equally bad as the Yudkowsky view. So let me just say that first of all, yeah, of course, creating AGI is a dangerous thing, but everything we ever discover is dangerous. So we do need to keep this in perspective. But you think about it, I brought up the idea of AI Doom with probability and I said, look, whatever percentage of universe is out there across the multiverse where AGI wipes out humans, there's got to be some where that happens. If everything happens across the multiverse, there must be some size of the branches of the universe is where that happens. So that's your probability of AI Doom as of this moment. Of course, it's changing constantly as we move forward in time. So, yeah, it's probably a legitimate concern in that sense. But the idea that you know what that size is, or even have an estimate of it is based, it's profoundly you're leaping beyond your knowledge at this point.

[00:16:26]  Bruce: It seems to me that Yudkowsky and people on his side, they're making a whole series of assumptions, and that they're just convinced of, that they've got this deep, deep gut feeling that this is the correct assumption. And it's things like, well, you know, like if I were to create an AG—a general intelligence today, say, have a biological son, obviously, that biological son might grow up to be Hitler and wipe out the world, right? So every child born has this chance of destroying the world, okay? I don't know if that argument is a good argument to not have children. Actually, I know it's not a good argument to not have children, right? And you have a really long time where they have to learn stuff. There seems to be this built-in assumption that the AGI is going to arrive tomorrow with the entire knowledge of humanity in its head, right? And it's ready to go. It's not going to have to grow up, it's not going to have to learn anything. And maybe LLMs have confused them on this point, right? Like they think of AGI, of LLMs as being nearly AGI's. And so when we get to AGI's, they're going to be trained on everything in the world. And I don't think we even have a reason to believe, like LLMs are not a path to AGI. They're nothing like what AGI is going to ultimately be like, right?

[00:17:45]  Unidentified speaker: I shouldn't say nothing like.

[00:17:46]  Bruce: Of course, there's going to be some overlaps, because that's an inevitable of the fact that everything around knowledge uses the exact same epistemology, so there has to be some overlaps. But there probably won't be much, right? That it will be a very different algorithm, which is the AGI algorithm. And I doubt you can pre-train it. You might be able to clone it after it reaches a certain point, but like there'll probably be reasons why you would not want to do that. Like imagine if you were an AGI, and if you clone yourself, half of everything you own now belongs to this other person, right? It's like having a divorce every time you clone yourself. So there's probably good reasons to believe that it won't just happen randomly, right? That they will just start saying, clone, clone, clone, clone, clone. You know, there's... But even what I'm just saying here, we're making giant assumptions that we just don't have any theory by which to even try to judge what I just said or what Yudkowsky is saying. They seem to think it's gonna be super, it's going to be faster than us, it's gonna have more memory than us, it's gonna start, be born on today with the entire knowledge of the world at its fingertips. It's going to be so smart that it doesn't have to learn to fool people, it'll just know how because it's so smart. It's going to immediately start cloning itself, there's gonna be no reasons why it wouldn't. You know, there's so many assumptions going into what they're saying, and I don't think there's any principled theoretical reason for any of these assumptions at this point.

[00:19:16]  Bruce: I think the real truth is, you have to actually figure out a theory of AGI, which we do not have as of, oh, the other one is, is they're assuming it's gonna happen by accident, that we're going to just keep training LLMs, and one day one of them turns into an AGI out of the blue, right?

[00:19:32]  Unidentified speaker: Well, let's, let's really steel man this.

[00:19:34]  Unidentified speaker: No theory at all, right?

[00:19:36]  Unidentified speaker: Yeah.

[00:19:36]  Unidentified speaker: Go ahead.

[00:19:37]  Unidentified speaker: Yeah, let's steel man this.

[00:19:38]  Unidentified speaker: Let's really steel man this, though.

[00:19:40]  Peter: I mean, if the assumption is that human, that there are plenty of ways of being creative and creating knowledge outside of the way that humans do it, and this robot, this AGI, has access to memory, every novel, every written, every scientific paper, all the knowledge that's out there, essentially immediate access to it, and it has vastly faster processing speed. It's got basically a perfect ELO score on chess or whatever. Everything you could possibly want in terms of quantifiable intelligence, why wouldn't it just make us create a nearly infinite amount of knowledge instantly? What would be stopping it?

[00:20:53]  Bruce: So let me be clear about what you just said.

[00:20:56]  Unidentified speaker: Okay.

[00:20:56]  Bruce: Assuming that our understanding of epistemology is wrong.

[00:21:01]  Unidentified speaker: Yeah.

[00:21:02]  Bruce: Assuming that it is actually going to be faster than us, and it can create knowledge through some other means. I mean, there's a lot of really big assumptions here, right? So let me just start at the top.

[00:21:16]  Unidentified speaker: Well, let's have some humility.

[00:21:18]  Peter: When you say epistemology is wrong, we're talking about a tiny subset of people into philosophy who have these Popperian ideas.

[00:21:25]  Bruce: I mean, fine, fine. I want to know what the alternative epistemology is.

[00:21:31]  Unidentified speaker: Yeah.

[00:21:32]  Bruce: So if it's not Popper, if it's not conjecture refutation, if it's not evolutionary epistemology, let's say is the most general form.

[00:21:39]  Unidentified speaker: Okay.

[00:21:39]  Bruce: What is the correct epistemology? Now, that's not a, I'm not trying to be difficult when I asked that question.

[00:21:45]  Unidentified speaker: Okay.

[00:21:46]  Bruce: I am going to start with what I think the current best theory is. And I'm going to draw conclusions from that. If you're going to start with the assumption it's wrong, and what I really want you to do is to, not you personally, Peter.

[00:22:01]  Unidentified speaker: I get it.

[00:22:02]  Unidentified speaker: The general term.

[00:22:04]  Bruce: I want you to put up or shut up, right? I don't want you to say, what if critical rationalism is wrong? I want you to say, here is the correct epistemology that's better than critical rationalism. And then I want a chance to look at the details of that theory and criticize it and decide if it's correct or not, and compare it to critical rationalism. If I don't have that, I don't even know how to respond to the question. I mean, I can give you a literal answer. The literal answer is, yeah, we're totally hosed. If we're starting with the assumption that everything we think we know about epistemology is wrong, and I'm not allowed to be given an alternative, so I'm going to simply be told, instead, it's something else that's bad, with no details as to what that is. It just so happens that the brain is woefully slow compared to my laptop computer, which by the way, I've argued that's not even true. That's a assumption—a huge assumption, which I don't—do not think it's going to prove true at all.

[00:23:08]  Unidentified speaker: Really?

[00:23:09]  Bruce: I don't know that for sure because nobody knows how the brain works, but the brain is really fast because it's so highly parallelized. I don't think the average CPU could compete with us at all. Certainly in terms of energy usage, our computers aren't even close. I think we're talking complete apples and oranges here. But then again, I don't know how the brain works, so until we have a theory of how the brain works, which we don't have, so maybe we could throw that out there. Without a theory of how the brain works that shows there's a problem, we're going to just assume it's a problem. Notice how we have to keep making these assumptions non-theoretically. If you're going to give me all these starting assumptions that basically boil down to, we're hosed, then yeah, I'm going to tell you, yes, under that set of assumptions, we're hosed. The problem is that I don't think I have any reason to believe any of those assumptions. Not a single one of them is principled and based on any theory that exists today. You might as well have just said, what if, starting with the assumption that if we build AGI, we're hosed, would that mean we're hosed if we build AGI? You might as well have asked the question like that.

[00:24:22]  Unidentified speaker: It's a completely circular question at this point.

[00:24:25]  Unidentified speaker: Okay.

[00:24:25]  Unidentified speaker: So what I want is I want them to—

[00:24:28]  Unidentified speaker: And I'm okay with challenging critical rationalism.

[00:24:31]  Bruce: I mean, come on, we challenge critical rationalism constantly on this show. We're super critical of Deutsch, Popper, David Miller. I don't think anything has been sacred to us even slightly, right? And so we are ruthless in terms of trying to rewrite epistemology.

[00:24:52]  Unidentified speaker: But I'm always starting with the assumption that I need to put up or shut up, that I need to—

[00:24:57]  Bruce: It's not enough for me to say, oh, I think Popper is wrong. I have to say, here's what I think is actually right, and then put it out there for criticism. And the entire Yudkowsky worldview, I don't even know if we're pronouncing his name right, by the way, but Yudkowsky worldview totally, completely ignores this aspect of epistemology, where I have every right to say, no, you need to explain to me what the correct epistemology actually is. I know Yudkowsky would say, well, it's Bayesianism, right? You know, Bayesianism is not at odds with critical rationalism. We're going to do some future shows where I'm going to actually talk about this. Now that I've actually studied this, I've read the first part of E. T. Jaynes' book. I've read Ivan's book. I've been going through machine learning, learning Bayesianism so that I could try to talk intelligently about the subject. The two just aren't at odds with each other. Bayesianism is a form of logic. This is something that all Bayesians would agree with me on. Cox's Theorem, that's exactly what Cox's Theorem says.

[00:26:07]  Unidentified speaker: Okay.

[00:26:08]  Bruce: I think that there is a Bayesian epistemology. To use the term Bayesian epistemology is like saying, I believe in the epistemology—in a deductive epistemology. It's not even clear what that term means, because the deductive logic, I believe in deductive logic epistemology. Remember, that was what Spock was supposed to be, and it never made sense, because nobody reasons using deductive logic. Trying to say, I believe in Bayesian epistemology is exactly like saying, I believe in deductive logic epistemology. It's not that it's wrong, it's just that it's completely incoherent statement and we have no idea what it means. And I think what you'll find is that there's a bunch of vagueness around the edges, where they kind of have this idea that anything that you're going to do in epistemology can be put into probabilistic terms. Which by the way, critical rationalists, is theoretically true, because that's exactly what Cox's theorem says. Whether you could do it in practice is a different story.

[00:27:12]  Unidentified speaker: Can I make one funny point about that?

[00:27:14]  Unidentified speaker: Yeah.

[00:27:15]  Peter: So also at the meetup, this is kind of an aside, but also at the meetup we were talking about Bayesianism, and I asked them if they'd ever used Bayes' theory to decide anything, like the actual mathematical equation.

[00:27:36]  Unidentified speaker: I think one guy sitting at the table said, well, maybe like twice, and everyone else said, oh yeah, we've never, but they basically made a good point, and they said, well, it's more of an attitude.

[00:27:50]  Unidentified speaker: And then my counter was okay.

[00:27:52]  Bruce: Which is what critical rationalists say too.

[00:27:54]  Bruce: Yes, that's what critical rationalists say.

[00:27:56]  Bruce: It's an attitude about being willing to change your mind.

[00:28:00]  Peter: So I thought that was an interesting conversation.

[00:28:03]  Bruce: So if you were to actually sit down, so first of all, you should never start with the assumption that someone who says they're a Bayesian actually understands Bayesian reasoning. Any more than you should start with the assumption that someone who says, I'm a Critical Rationalist, understands Critical Rationalism, right? Like, of course, regular human beings are gonna have all sorts of misunderstandings, right? That should be your starting assumption. So, and I know I'm talking here and I'm not giving any specifics and I'm saving that for future episodes where I will actually walk you through what E. T. Jaynes actually says, teach you the mathematics, show you that what I'm about to say is just completely correct, even from the Bayesian viewpoint, okay? The main thing that E. T. Jaynes is really trying to say is that anytime you are trying to do science, okay, which from critical rationalist viewpoint, we would say that's doing critical rationalism, okay? He's saying that you can, once you're to the point where you're actually testing between theories, doing a critical test, that it can, and he would argue should, but it can take the form of probability theory when you're writing down what's happening. Now, I want you to stop and think about this for a second. For those critical rationalists who just bristled at me saying that, I want to point out that you shouldn't have bristled because probability theory can represent deductive logic. You just set all the probabilities to zero and one. Okay, that's a known fact, okay?

[00:29:37]  Bruce: And Popper's epistemology is based on criticism through deductive logic.

[00:29:42]  Unidentified speaker: That's the logic of falsification and refutation.

[00:29:46]  Bruce: So it is a given that Popper's epistemology can be put in terms of probability theory and can be written down in terms of probability theory. It's impossible for it to be otherwise, okay? So for those who bristle at the thought that we should be moving, putting this in terms of probability theory, you should probably stop bristling. You should probably actually start trying to understand what the two theories are saying, okay? Now, this doesn't mean, the problem is, is that they're starting with something that's true. The fact that you can write down a critical test and you can put it into the notation of probability theory. And that's even sometimes very useful. Like if I'm trying to do a critical test between, say, Einstein's theory and Newtonian physics, a critical rationalist is going to try to put this in terms of falsification. They're going to say, what you actually did with the Eddington Expedition is that you falsified Newtonian physics and you left general relativity as the only surviving theory, okay? The problem is, is that's not really quite right. Because they were had these instruments, they were super imprecise. They had, they would have these photos on the plate and it would be smudged. And so what they would do is they would try to figure out, given the problems with our instruments, what's the probability that I would get this data using these instruments if Newtonian physics was true versus if Einstein's physics was true? In other words, they absolutely used probability theory.

[00:31:23]  Bruce: Now, it would have been frequentist probability theory, not Bayesian probability theory, but you can always translate between those two in principle. So there's nothing that would have stopped them from using Bayesian probability if they had wanted to. And that just makes sense because our instruments always have this probabilistic element to them and they're not very precise and they were very imprecise back then. They really had to come up with not a falsification, [not?] a straight falsification, but a probabilistic falsification, okay? Now, where Bayesians go wrong is that they then say something like this, oh, we've just changed updated our priors as to whether Einstein's theory is true versus Newtonian physics. And then they assign some sort of probability. It's a 90% chance that Einstein's theory is true according to this experiment. Only a 10% chance that Newtonian physics is true. What they've missed is that what you actually measured there was the probability of your instruments giving you a wrong result. And it had nothing to do with the probability of the theories themselves. Bayesians bump into the exact... Nobody ever talks about Bayesians... With critical rationalism, everybody talks about the Duhem–Quine thesis, the fact that you can't test a single theory. Nobody ever talks about that with Bayesianism ever that I've seen. But they have the exact same problem. And so when they say, oh, I'm measuring the probability of this theory, what they're really doing in this particular case is they're measuring the probability that the instruments gave a wrong result.

[00:32:53]  Bruce: And that's the only probability that they're measuring. They may say it as the probability of the theory being true or the theory being false. That's a misnomer at best. When you write things down in terms of probability theory, you are measuring something with probability, but it's almost never that you're measuring the probability of the theory itself being true. You may, as a kind of layman's summary, say that. Like if I were to say, based on this experiment, there's a 90% chance that Einstein's general relativity is true. There's nothing necessarily wrong with me saying that, even though it's a technically inaccurate statement. Because of course, people are going to know, what I really mean is there's a 90% chance the instruments didn't mislead me.

[00:33:36]  Unidentified speaker: Okay?

[00:33:37]  Bruce: And of course, that's what we really mean. And there is no actual incompatibility between Bayesianism and critical rationalism, if you actually understand what Bayesianism, Bayesian reasoning is saying. I should probably be careful when I say Bayesianism, because that probably implies Bayesian epistemology, which has additional assumptions that are wrong and should be removed. It's really Bayesian reasoning that I'm defending here, not Bayesian epistemology that I'm defending. And I do think that this idea that you should always write it down as probability theory, which is really E. T. Jaynes' main thesis, is total bullcrap, right? Of course, it often makes sense to write it down as deductive logic. There's all sorts of reasons why you may choose to write it down as deductive logic instead of probability theory. And those aren't the only two options, right? How you're going to go about criticizing this theory, I don't doubt that you could always put it into, ultimately, if you're selecting between theories. It's totally different if you're not selecting between theories, by the way. But if you're selecting between theories, then yes, you can always write it down as probability theory.

[00:34:43]  Unidentified speaker: Should you?

[00:34:43]  Bruce: That's a totally subjective question that depends on the problem you're trying to solve. And actually, writing it down as probability theory can sometimes mislead you, which is what I think is going on in the case of Yudkowsky, that he's tried to put everything in terms of probability theory because that's part of his Bayesian epistemology. And it has caused him to misunderstand his own model. Because probability theory, when you write it down, often makes implicit assumptions that you don't realize you've made, where if you use deductive logic, they become explicit. And it's one of the advantages to often not doing it in terms of probability theory, is that you get a lot more explicit, so it's easier to criticize the theory. I'm going to have to give examples of all this, where I can actually show you the math, show you that what I'm saying is true, okay? Because you shouldn't accept my word for this.

[00:35:29]  Unidentified speaker: You should—I should really have to actually just demonstrate it for you.

[00:35:33]  Bruce: And I will in future podcasts, okay? But the problem here is that if you really are a Bayesian, in the good sense, the one that really understands Bayesian reasoning, then you're actually, whether you realize it or not, a critical rationalist because there is only one epistemology. And critical rationalism is the broader understanding, and Bayesianism is this narrower, how do you actually criticize the theory and whether or not you should write it down in terms of probability theory. It's a little tiny part of critical rationalism. It's not the whole epistemology by itself. If you were to talk to a Bayesian, like when we brought Ivan on, he immediately said, Bayesians do not care how you got the theories. They're just trying to measure between theories. Okay, that was a good way for him to put it. Ivan's one of the best Bayesians I've talked to, of course. Okay, so notice though that that's still just conjecture, right? Like Bayesians may not talk about conjecture the way a critical rationalist would, but ultimately they're starting with the assumption of somehow, some way we conjectured some theories. And then we're going to use some means of testing between the theories. Bayesianism does not specify what the means are. And then ultimately, we're going to measure the probability at the very end, based on the fact that we always use instruments, the instruments always have some sort of probabilistic distribution. So we're going to measure the chance that we've been misled by our instruments. And there's nothing particularly wrong with that.

[00:37:11]  Bruce: It's entirely 100% critical rationalism, the way I just said it, right? It's very easy to get off the road, to start thinking other things, to start saying, well, maybe machines will work in some different way. Okay, what is it? It's always going to turn out to be critical rationalism, because it is in fact the correct epistemology, right? So, sorry, I'm on a rant now. I'm going off on this, and I know I need to come back and actually take everybody through Bayesianism better, and show how it is compatible with critical rationalism, and in fact is, when done right, exactly the same as critical rationalism. And that shouldn't surprise anybody, that Bayesians are actually critical rationalists when they're doing their epistemology correctly, because if they're going to make progress, there's only one way to do it, and it's the one that Popper described, okay? Which is why Bayesians should study critical rationalism. They should try to understand the broader understanding of how it is we create knowledge, and then they should look at how Bayesianism fits into that epistemology, and that would really help them remove a lot of the errors that they make. By the way, critical rationalists should study Bayesianism for much the same reason. The Bayesians have drilled down really deeply on one little tiny aspect of epistemology, and they understand it way better than critical rationalists now, okay?

[00:38:33]  Bruce: So if you want to understand why machine learning works and how it's compatible with critical rationalism, you probably need to study Bayesianism because it plays such a huge role in how machine learning works. Most critical rationalists I know think that machine learning isn't epistemology, and they consider it something distinct or separate from Popper's epistemology, which is very undermining to the idea that critical rationalism is the one and only correct epistemology. So this is something that we should just freely study both sides. Critical rationalism is the one and only epistemology. Bayesianism isn't really a competing epistemology, it's a non-epistemology. It is part of epistemology though, it's an important part of epistemology. Bayesians actually understand that aspect of epistemology better than critical rationalists do. And in fact, critical rationalists have this allergy to probability theory which is really unhealthy and causes them to misunderstand really important aspects of epistemology. But when it comes right down to it, the two are not at odds with each other, when understood correctly. Bayesian reasoning is something that's completely valid way to reason.

[00:39:45]  Unidentified speaker: E. T.

[00:39:45]  Bruce: Jaynes' idea that you should always put everything in terms of probability theory is, in my opinion, a mistake, even though technically you can do it.

[00:39:56]  Unidentified speaker: I don't mean technically.

[00:39:57]  Bruce: Theoretically, you can do it.

[00:39:59]  Unidentified speaker: In practice, you can't.

[00:40:01]  Bruce: And it would cause you to misunderstand things in a lot of cases. Probability theory makes a lot of things implicit that deductive logic makes explicit. So you're actually often better off using deductive logic when you're doing science than probability theory.

[00:40:17]  Unidentified speaker: It depends on the problem.

[00:40:18]  Bruce: It depends on the theories. It depends on a whole bunch of different factors.

[00:40:21]  Unidentified speaker: Okay.

[00:40:24]  Bruce: But so I guess that's the one thing I do disagree with E. T. Jaynes on, is that you should always put it in terms of probability theory.

[00:40:32]  Unidentified speaker: I don't think that's true.

[00:40:34]  Bruce: But I can see his point that probability theory is this generalized way in which you can understand the logic of, basically the logic of criticism. He would say the logic of science, but it's really just the critical side. So I've often joked that Bayesianism, one of the main criticisms that critical rationalists will raise against Bayesianism, is they'll say, they can't explain where theories come from. And I'll say to them, can you explain to me as a critical rationalist where theories come from?

[00:41:05]  Unidentified speaker: And they always go, no.

[00:41:09]  Bruce: Of course, nobody understands the conjecture side of conjecture and refutation. That wasn't something that Popper spent any time working on. Bayesians don't talk about it and they should. And Ivan started off with, we don't know where conjectures come from, where theories come from, we're just going to measure which is better. And that's probably a very honest way of looking at it. But you could easily start thinking of conjecture and Bayesianism. And Bayesianism is a type of refutation. It's a process of refuting a theory, but you're going to use probability theory instead of pure deductive logic. And there are so many cases where that makes sense. And even in the cases where it doesn't make sense, where it would make more sense to just use logic, like Popper would try to do, logic is a subset of probability theory. So theoretically, you're still doing probability theory. So I think that there's no particular mystery here. It's actually strange that people have gotten so confused on this point and have tried to turn Bayesianism into its own epistemology. And I do think a lot of the confusion that like the world of Yudkowsky and your friends at your philosophical meetup, they don't really understand their epistemology very well because they don't understand critical rationalism. So they don't understand the broader, critical rationalism is broader. It's trying to encompass the entire process of how knowledge gets created, where Bayesianism is trying to only talk about how do you actually criticize the theory and using logic, using probability instead of logic, I should say.

[00:42:47]  Bruce: But probability theory is a form of logic, so using logic. And what are the advantages of using probability theory in some cases over using just pure deductive logic? And there are advantages in some cases of using probability theory. But because they haven't spent the necessary time to understand, okay, actually, you still have to conjecture things, the probability theory, you're never measuring the absolute probability that the theory is true. You're only measuring the probability of the theory, given the assumptions currently on the table, given the theories currently on the table and the assumptions that went into your model. If any of those assumptions turn out to be false, that probability measure is meaning less and you need a whole entirely new one. And I think it's very common for Bayesians to think that they're measuring the absolute probability of a theory being true versus some other theory. And they don't really understand, it's only the probability given a certain model, given a certain set of assumptions. If I could use an analogy here, think about deductive logic for a second. Okay, think about how you'll never hear a critical rationalist, you'll hear critical rationalist get really upset with probability theory. You can't measure the probability of a theory, it's impossible. Bayesians are so wrong because you can't measure, that would be justification-ism.

[00:44:09]  Unidentified speaker: Okay.

[00:44:10]  Bruce: And yet, deductive logic has the exact same problem. If I were to say, you know, all men are mortal, you know, Socrates is a man, therefore Socrates is mortal, a famous deductive logic example, the syllogisms that Aristotle came up with or whatever. If I were to give you that, you could say, oh my gosh, that's justification-ist. You're saying that you know for sure that given your assumptions that Socrates is therefore mortal. Oh, that's justification-ism. You could never know that for sure, right? But nobody does because that would be dumb because you immediately understand when you're doing deductive logic that the logic is the justification in the logic, the fact that it guarantees you a certain truth is still predicated on those assumptions actually being true. And you're not, you're never measuring, you're never using deductive logic to determine if the assumptions are true. You're always at some point cutting it off with conjecture.

[00:45:08]  Unidentified speaker: Okay.

[00:45:09]  Bruce: Probability theory is exactly the same way. You can measure the probability of a thing. And it doesn't have to be based on a physical probability like quantum physics. It can be based on your knowledge, just like Bayesian said, they're not wrong about that. But even when they're doing that, they're never measuring the probability of the assumptions, because that's impossible. You can try, but it becomes an infinite regress, just like with logic.

[00:45:33]  Unidentified speaker: Okay.

[00:45:34]  Bruce: And so ultimately, that probability is only as good as the starting assumptions, because probability is really just a form of logic. Just like logic is a form of logic, deductive logic is a form of logic. And so you're actually, one of the main things that you'll hear Bayesian say is they'll say, and like really good ones like Sean Carroll, will say things like, oh, you should never set your probability for something to zero because then it's not possible for you to change your mind.

[00:46:06]  Unidentified speaker: Well, you know what?

[00:46:06]  Unidentified speaker: That's bullcrap.

[00:46:07]  Bruce: The way you change your mind is by challenging the entire model that that probability was based on, which you never measure the probability for.

[00:46:15]  Unidentified speaker: Okay?

[00:46:16]  Bruce: So if I had three black balls in a cup, and then I hide it, and then I tell you pull a black ball and I'm gonna measure the probability that it's gonna be black, of course I'm setting the probability to 100%, right? There are cases where it makes perfect sense to set the probability to 100%. And the idea that I can't then change my mind because I've set it to 100% is a complete misunderstanding of probability theory, which is just a form of logic, okay? It's the way you would change your mind in a case like that isn't by sending, trying to measure what's the odds that, I mean, I just saw it was all three of them are black, right? Like what's the odds that I had the hallucination and that one of them was actually white? Like Bayesian will try to measure that and it's stupid. The model says that there's three black balls, so the percentage chance is 100% and that has nothing whatsoever to do with whether you can change your mind or not. Because whether you change your mind or not will entirely depend on whether you decide the model was wrong, which is not part of what you're trying to measure with probability theory. Just like it's exactly the same as with deductive logic. The idea that you can decide, yes, Socrates must be mortal because all men are mortal and Socrates is a man, you might change your mind, right? You might decide, you're not going to, but you might. You might discover that Socrates actually was immortal. At that point, you'll go, wow, I guess not all men were mortal. You would challenge the assumption at that point.

[00:47:50]  Unidentified speaker: Probability theory is exactly the same way.

[00:47:51]  Unidentified speaker: There's no difference.

[00:47:52]  Bruce: You have to think of it as a form of logic. Once you do, all the problems that happen with Bayesianism vanish. You start to realize, oh, it's just a form of logic that I can use in sort of a critical rationalist setting and that's perfectly fine. And so this gets back to, how does this relate to the question you asked me? I don't think there is any way to steal man their position, because their position is just a series of assumptions. And I don't have any particular theoretical reason to agree with them on their assumptions. If their assumptions rose from really best theories, deep theories that I don't have an alternative to, that would be a totally different story. But that isn't what they're doing. They're making gut feeling assumptions, and then they're arguing with you that their gut feelings can't be wrong.

[00:48:40]  Unidentified speaker: I don't know how to even address.

[00:48:42]  Bruce: There's no way to criticize their theory. There's no way to show it to be wrong, because it's not based on any sort of falsifiable theory to begin with. I think that's actually the correct answer. Like, it's not really denying the possibility that AGI may wipe us out. Across the multiverse, it probably will wipe us out in some segment of the multiverse. So there is some probability that will happen.

[00:49:04]  Unidentified speaker: Okay?

[00:49:06]  Bruce: There's a probability, like, at the moment my son is born, there's a probability that he's gonna wipe out the human race. You have to keep this in mind, right? There's some, somewhere out there in the multiverse, somebody just had a son who's the next Hitler, and he's gonna wipe out the human race, right? It's going to happen. It's some tiny slice of the multiverse. That is not an argument for not having children. And the fact that you can based on that same argument, try to argue that we should not build AGI, it's an equally dippy argument. And it's again, it's just based on nothing. It's not rooted in a thing. And that would be my main argument here. It's got nothing to do with whether I think AGI will or won't wipe us out.

[00:49:50]  Unidentified speaker: I don't know, right?

[00:49:52]  Bruce: It's really boils down to you're making crap up, stop making crap up.

[00:49:57]  Unidentified speaker: You need more theory here.

[00:49:59]  Bruce: You don't even know what you're talking about at the moment, and it's obvious.

[00:50:04]  Unidentified speaker: Okay.

[00:50:04]  Peter: Well, I'll tell them that at our next meetup and see how it goes.

[00:50:08]  Bruce: See how that goes.

[00:50:09]  Unidentified speaker: Peter, yes.

[00:50:10]  Bruce: Can you report back to me after your next meetup? And let me know if you were able to set them straight by telling them that.

[00:50:18]  Unidentified speaker: Yeah, that's what I want to know, Peter.

[00:50:21]  Unidentified speaker: Well, I've wondered that.

[00:50:22]  Peter: How do they ban people from these meetups? They must need to do that every once in a while. I don't know how you can, it's not like an internet forum where you can just ban them. You have to like...

[00:50:33]  Bruce: Tell the person, don't come back or we'll call the cops.

[00:50:36]  Unidentified speaker: Something like that, yeah.

[00:50:39]  Bruce: Peter, can you please find out by trying to get yourself banned from this group? And then report back how they ban people?

[00:50:46]  Unidentified speaker: Okay.

[00:50:47]  Bruce: Okay, one last thing. I think you do have to recognize that when we talk about a group of rationalists, there's a good chance they're not being all that particularly rational. I don't think human beings are deeply rational creatures. This is something that when we interview Ray, I'll have to talk with him about. I do think we're, I think we are rational in many ways. I actually think Ray is basically correct, right? Like I really don't feel like I disagree with him. But I do think we have this dogmatic nature and this need for meaning. And I think if you really could get into these people, the black box of their minds, you would find that they're getting something out of this view, even though it's not rooted in anything like theoretical. And I think you would have to address that if you wanted to change their minds. And I don't know what that is. It's probably that they get meaning in some way, and you'd have to find some chain of things that allowed—that showed why this was so meaningful to them to have this pessimistic worldview that AGI might wipe us out. It's probably no different than why some people become animal rights advocates or whatever. It's some way of viewing the world that allows them to view themselves in a certain way.

[00:52:07]  Unidentified speaker: And you know what?

[00:52:07]  Bruce: I'm not entirely against that. I feel like though I'm not myself an animal rights activist and never would be, I'm glad that there's some around. I feel like they do play a really important role in society of getting us to think a little more deeply about how we treat animals. And I'm not sure I've got a particular problem with AGI Doomers, right? Like, I think that they're irrational, but I think that they still play this important role of raising concern about alignment issues, which are real enough, even if we're not talking AGI, if we're just talking regular, narrow AI, alignment is a huge problem that probably in part due to Eliza Yudkowsky raising AGI alignment issues, which may be entirely a different beast, he got us thinking more deeply about narrow AI issues and probably played an important part of the discourse on that. So even when we are talking about kind of irrational meaning memes like this, I think inside of an open society, the good part makes it to the Popperian filter and the bad part doesn't. And long as we keep an open society, it's probably fine.

[00:53:18]  Peter: It does seem like a lot of these pessimistic ideas are meaning memes in some weird way. I, you know, I, you know, the even more broadly, like the pessimism even goes more broadly than just the doomer, that just the AI doomerism. You know, I mean, I had, I went to a different philosophy meetup. They call themselves the drunken philosophy meetup. So it's a little more broad than just the the rationalists. But every single person, I'm not, this is not an exaggeration. Every single person I spoke to expressed some variation of the idea that whether for technological reasons, political reasons or environmental reasons, that humans are essentially on the precipice of self-immolation, that, you know, we're a parasite by our very nature. I had a bumble date just last night. Wonderful woman, great conversation. But yeah, very, very positive in many ways, but her too. Yeah, it would be better if humans went extinct. Like, it's hard to, you know, I mean, she casually threw that out as if it was common, a commonly accepted common knowledge kind of, which-

[00:54:53]  Unidentified speaker: Conspiracy against humanity.

[00:54:54]  Unidentified speaker: What was the book called?

[00:54:55]  Unidentified speaker: The Thomas Ligotti book?

[00:54:57]  Unidentified speaker: Yeah, I think so.

[00:54:58]  Peter: Conspiracy against humanity.

[00:55:01]  Unidentified speaker: Which, you know, I mean, she didn't, this woman wasn't that dark or anything.

[00:55:06]  Unidentified speaker: But you know, it's just, it's a very, very mainstream view.

[00:55:11]  Peter: And it does convey a certain kind of sophistication. It feels to me like a meaning meme, but I've got my own meaning memes, I guess, to counter it.

[00:55:22]  Unidentified speaker: So I'm not judging, but, you know, I...

[00:55:25]  Bruce: So there's maybe a, so maybe let's talk about meaning memes for a second.

[00:55:29]  Unidentified speaker: There's lots of meaning memes.

[00:55:32]  Bruce: There's a sort of common thread between them, which boils down to you being elect out of humanity.

[00:55:42]  Unidentified speaker: You're the enlightened group.

[00:55:44]  Unidentified speaker: Yeah.

[00:55:45]  Bruce: And if you really stop and think about that for a second, you're going to find that it's ubiquitous. You know, if you're a, if you're a someone who's into political correctness or anti-racism, it really boils down to this idea that you are enlightened compared to other human beings.

[00:56:05]  Unidentified speaker: Okay.

[00:56:06]  Bruce: But that is absolutely what a lot of people are going to get out of something that would like TCS or something like that, right? Something that you would consider less objectionable. It's what people get out of religion a lot.

[00:56:19]  Unidentified speaker: Okay.

[00:56:19]  Bruce: It's what I think AI Doomers get out of AI Doom. I think that if you were to really sit down and try to look at what it is that appeals to humans about a meaning meme, it's not 100 percent, but I think it's going to be very, very common that it boils down to we are the enlightened. And I'm not even sure if I were to tell you, that's bad, don't do that, it'd be a waste of time. Probably humans need some sort of purpose, and if you have a purpose, then you are in the rest of the world doesn't yet agree with you, that in some sense you think of yourself as the enlightened. Maybe you are the enlightened, maybe you have an idea that actually is better than what's currently out there.

[00:57:08]  Peter: And of course, we're no different.

[00:57:10]  Unidentified speaker: I don't think you're saying that, yeah.

[00:57:13]  Bruce: Yeah, I'm specifically saying, you're going to do this. And so trying to fight it all the time is probably not worth your time, okay?

[00:57:21]  Unidentified speaker: Okay.

[00:57:22]  Bruce: And this is why I actually say meaning memes are always dangerous. They are always dangerous because they're almost always based on this idea that we are the enlightened. You're always going to have the danger of running away with it.

[00:57:35]  Peter: Wait, but you say they're ubiquitous though, but they're always dangerous.

[00:57:40]  Unidentified speaker: They're always dangerous.

[00:57:42]  Bruce: All meaning memes are dangerous.

[00:57:44]  Unidentified speaker: Okay.

[00:57:45]  Peter: But I just assume that you also subscribe to meaning memes.

[00:57:50]  Unidentified speaker: I do.

[00:57:50]  Unidentified speaker: Right?

[00:57:51]  Unidentified speaker: Strongly.

[00:57:51]  Unidentified speaker: Okay.

[00:57:52]  Unidentified speaker: Right?

[00:57:52]  Unidentified speaker: Okay.

[00:57:53]  Unidentified speaker: So.

[00:57:54]  Unidentified speaker: But you just recognize they're dangerous.

[00:57:56]  Unidentified speaker: Well, they're also the source of all good in the world.

[00:57:59]  Unidentified speaker: Okay.

[00:57:59]  Unidentified speaker: Right?

[00:58:01]  Peter: Oh, so they're not inherently dangerous, but they can be dangerous.

[00:58:04]  Unidentified speaker: Is that what you're saying?

[00:58:05]  Bruce: Well, you know, it depends what you mean by inherently, I guess. So probably what you just said, what you're thinking in your mind is probably correct. They're in some sense inherently dangerous.

[00:58:15]  Unidentified speaker: Okay.

[00:58:18]  Bruce: Let's take anarcho-capitalism. Anarcho-capitalism is a meaning meme.

[00:58:21]  Unidentified speaker: Okay.

[00:58:21]  Bruce: I've argued that you cannot arrive at it rationally.

[00:58:24]  Unidentified speaker: Okay.

[00:58:25]  Unidentified speaker: That it's something that you feel in your gut.

[00:58:26]  Unidentified speaker: Okay.

[00:58:29]  Bruce: It doesn't seem to be all that particular dangerous in real life, inside of an open society, inside of our society today. You could see how it could be dangerous. If somebody took it a little too seriously and started thinking, you know what, the government is robbing everybody with taxes, and that's initial use of force, and so I am completely justified in using violence against the government. Also, anyone who supports the government, which would be anyone but an anarcho-capitalist, is worthy of, is supporting this robbery and therefore the initial use of force, and therefore I can use this to justify violence against literally anybody but another anarcho-capitalist.

[00:59:14]  Unidentified speaker: Okay.

[00:59:16]  Bruce: It's not an irrational viewpoint given the assumptions that they've arrived at. Given their assumptions based on initial use of force, and based on the idea that taxes are theft, it does seem that they have now have a way to justify violence against basically everybody. Now, that doesn't happen in real life because people don't actually take their meaning memes quite that seriously in most cases.

[00:59:42]  Unidentified speaker: Okay.

[00:59:43]  Bruce: It's usually a part of a greater set of views that they hold, including moral views about it being wrong to commit violence, let's say. And so, I mean, like, I don't know of any, it's surely very rare that a libertarian or an anarcho-capitalist that believes taxes or theft has moved to violence. Now, if you were to expand this out and you were to say, okay, what if you were to look at conservativism?

[01:00:11]  Unidentified speaker: What if you look at leftism?

[01:00:13]  Unidentified speaker: This is a much larger group.

[01:00:14]  Bruce: There's only a tiny, tiny, tiny group of anarcho-capitalists to begin with. The vast majority of them never moved to violence either. And yet, you can see how their meaning memes might suggest violence is the answer, and just a small number of them do move to violence. We just recently had the shooting of Charlie Kirk down the street from me, and I had like neighbors that were present when he was assassinated. That was in that particular case, that was the left, somebody on the left getting a little too radical with ideas, following it to its logical conclusions, and then killing somebody. Super rare, but it happens. And I don't mean to blame the left here, the right has moved to violence in very similar ways, multiple times within the last few years. So I mean, I think there is always an inherent danger with meaning memes, but it isn't the case that it's automatically dangerous. And this is one of the things that, like some of the Sam Harris quotes that we've talked about, where he seems to be suggesting, you said that this is probably largely out of context, but seems to be suggesting a need for violence against people with certain ideas. You can see how he arrived at such an idea, where in his case, he was probably talking about Islam, that Islam is an inherently dangerous set of ideas. There's a certain sense in which he's right, but no more so than probably every other meaning meme in the universe. There's maybe a greater danger with Islam in that it often exists in non-open societies. And those non-open societies are very dangerous.

[01:01:58]  Bruce: And they may be saying that they're gonna be a closed society in the name of Islam or something like that, right? So it may be hard to tease the two apart. But I often felt that the real danger there is the closed society, not the Islam. And if you were to go look at Islam inside of an open society, what you're gonna find is that they're probably not all that particularly dangerous. And I think open societies force meaning memes to adjust themselves and to come to the realizations, okay, I can only take this so far. I need to still be a classical liberal. I need to be open to freedom of speech. And there's always some danger it'll get shut down, as per Jonathan Rauch, we have to always fight for it, I'm not denying that. But I do think this is why I say all meaning memes are inherently dangerous.

[01:02:49]  Unidentified speaker: They are.

[01:02:49]  Bruce: There's potential for every single one of these points of view to go off the rails at some point. And there's an almost constant fight to keep it from happening. But that's what open societies are really good at, right? Like we have these institutions in place that really kind of help people not move to physical violence, because even if they've got strong feelings about something, they have an outlet through the political system to do something about it.

[01:03:18]  Unidentified speaker: And they don't feel completely...

[01:03:19]  Bruce: And there's always... Now we're getting into, you know, what's wrong with our current systems. None of our open societies are perfect at this point. There's like genuine problems that we have to still solve. And there is always that danger you're going to make some group of people just feel completely disenfranchised. And then they may start moving to violence. Often that is the society's fault that that happens, that we've got some sort of problem that has to be corrected. This reminds me a little of now of Taleb's, the episode we did on Taleb and his views on intolerance, where sometimes certain groups become intolerant, and then they sometimes inside of an open society, at least, that sometimes actually leads to moral improvements. Because there was a group that said, you know what?

[01:04:06]  Unidentified speaker: I'm done, I'm not accepting this anymore.

[01:04:08]  Bruce: It's really you guys that need to change, the majority. And they can turn out to be right, right? Because they have special knowledge as to what their situation is and how society has been mistreating them. And they'll just stop accepting that, and then the laws change, and we actually improve laws in that way. So it's, I think you on the one hand, [meaning memes?] are all the source of all the good stuff, and they're also dangerous. And you have to kind of accept those two truths at the same time. And I don't think it's any different here. It's, if you're an AGI doomer or you're a pessimist, you're getting some sort of feeling of enlightenment out of it. That's part of how you visualize yourself and set up your identity. And it could lead to danger, but it probably won't as long as we keep it inside [an open?] society. We do it through debate rather than move to violence. We've got laws against violence, but we allow the open debate. I think what you'll find is that all these different meaning memes actually end up being a net positive inside of an open society. All right, so that's my rant about meaning memes. And why I don't have a problem with them, right? Like even though I accept that there's an inherent danger to them, I completely believe in embracing meaning memes and finding something that for you is the thing that brings meaning into your life. And yeah, that maybe that means it's going to make you feel enlightened.

[01:05:32]  Unidentified speaker: Maybe you are enlightened, right?

[01:05:34]  Unidentified speaker: Like that's a possibility.

[01:05:36]  Bruce: And you're going to pursue that and that's going to kind of be your thing.

[01:05:39]  Unidentified speaker: And that's what people do.

[01:05:41]  Bruce: And it's not a negative. It's the source of conjectures that society comes up with that allows us to move forward and to continually improve.

[01:05:50]  Unidentified speaker: So it's the source.

[01:05:51]  Peter: You don't think it's an exaggeration to say that the search for meaning is the source of knowledge.

[01:06:00]  Bruce: I don't know if I'd go so far as the source of knowledge because now we're talking specifically about humans and obviously epistemology is broader. It includes biological evolution, which probably has no meaning, inherent meaning at all.

[01:06:13]  Unidentified speaker: Okay.

[01:06:14]  Bruce: But like as far as humans.

[01:06:16]  Peter: Open-ended human created knowledge.

[01:06:19]  Unidentified speaker: Yeah.

[01:06:20]  Peter: Would not exist if we didn't have this impulse to live meaningful lives.

[01:06:25]  Bruce: Yeah, I think so. You can almost surely find a counter example.

[01:06:28]  Unidentified speaker: You could say, oh, you know what?

[01:06:29]  Bruce: I created knowledge because I needed to go to the store, so I had to work out what the right route was. You could come up with something that just has no meaning in it to you.

[01:06:40]  Peter: I mean, it's kind of [tautological?] in a way because I'm almost tempted to say, well, except for the knowledge that's not meaningful.

[01:06:46]  Unidentified speaker: Yeah.

[01:06:48]  Bruce: So what I would say is this, is not that necessarily it's the source of all human knowledge because that probably goes too far, but if you didn't have it, a lot of human knowledge just wouldn't exist. Like, imagine some scientist, Einstein, discovering general relativity. He spent eight years of his life developing that theory. And I promise you, it boiled down to something meaningful to him, that he was excited that he had discovered something, the way it made him feel to be the sole person on the planet that suddenly understood what gravity was. I'm sure that this got him super excited and kept him going for eight years, and that this became part of his identity and became a sort of meaning meme for him, that he wanted to get out there to other people. Now, he then did the exact same thing, trying to discover a theory of quantum gravity, and it was a complete failure. And he probably felt exactly the same way until he died, that he was working on this theory, that he was excited that he knew something that nobody else knew. And then people came in and looked at it after he was, he died based on more knowledge at this point is available. And they're like, no, this is like totally the wrong approach. And we don't know him today. We know him for general relativity, we don't know him for quantum gravity, precisely because one of them survived the test of time and one of them didn't. And yet I suspect that that drive for whatever was going on in his head, the way it made him feel, did he do it because he wanted to be famous?

[01:08:22]  Unidentified speaker: He did go on to be famous, maybe that did get him excited.

[01:08:26]  Bruce: But like he couldn't have known, it was going to make him famous. There must have been something deeper there. There must have been something that kept him going for eight years to develop this theory, where he's thinking, wow, this is going to be so amazing that I've got this and I'm revealing it to the world. And I don't know if you didn't have that drive, how much would actually happen in science. I mean, we know scientists become very, very [related to their identity and their theories?]. And it's one of the reasons why it is often so hard to change a scientist's mind about his theory is because it has become a sort of meaning meme for him, and it has become part of his identity.

[01:09:07]  Unidentified speaker: I'm saying he, but of course, could be she.

[01:09:11]  Bruce: So yeah, I think if you're really looking at human knowledge, that search for meaning is a huge part of why humans go out and develop knowledge.

[01:09:24]  Unidentified speaker: Okay.

[01:09:25]  Peter: Well, I think this has been a nice, relatively short episode and I thank you for your thoughts, Bruce.

[01:09:32]  Unidentified speaker: All right.

[01:09:33]  Unidentified speaker: Thank you.

[01:09:41]  Peter: Hello again. If you've made it this far, please consider giving us a nice rating on whatever platform you use, or even making a financial contribution through the link provided in the show notes. As you probably know, we are a podcast loosely tied together by the Popper Deutsch Theory of Knowledge. We believe David Deutsch's Four Strands tie everything together. So we discuss science, knowledge, computation, politics, art, and especially the search for artificial general intelligence. Also, please consider connecting with Bruce on X at BNielsen01. Also, please consider joining the Facebook group, The Many Worlds of David Deutsch, where Bruce and I first started connecting.

[01:10:26]  Unidentified speaker: Thank you.