Episode 14: Theories of Artificial General Intelligence
- Links to this episode: Spotify / Apple Podcasts
Unofficial, reviewed transcript. It may still contain mistakes; check the podcast when wording matters.
Speakers are labeled by first name where identified. Unidentified-speaker labels may refer to different people. Bracketed question marks indicate uncertain wording.
Transcript
[00:00:10] Bruce: Okay, we probably need to start wrapping this up, but I want to talk about actual theories of AGI and kind of what our guests are working on as far as their own personal theories and where they're trying to take their research into AGI. So maybe Ella, let's start with you.
[00:00:26] Unidentified speaker: Yeah, sure.
[00:00:28] Ella: So I call sort of my thoughts on AGI CTP theory. And I think I said at the start of the podcast, that is an acronym that sort of used to make sense, but the theory has changed a bit and it doesn't really make sense anymore. But the basic idea of CTP theory is that it provides a way of representing ideas computationally. And it does so in a way that allows the mind to compute the consequences of an idea, which I think is very important. And it also allows the mind to determine when two ideas are contradictory. And so I think that that is something that any AGI algorithm is going to have to do because that is sort of that right there is kind of the heart of critical rationalist epistemology, in my view. Sort of being able to explore the consequences of ideas, the logical consequences and searching for contradictions, problems between those ideas.
[00:01:26] Ella: And so that's the basic framework that CTP theory allows for. And it also has some stuff to say about how, once a problem has been identified, how it could be solved. Though that's sort of the more active area of research. I think that there are some missing details and some errors in my current way of thinking about how the program would actually go about solving problems. And so that's kind of my current area of research in CTP theory.
[00:01:55] Bruce: Thank you.
[00:01:56] Dennis: Any progress in that vein?
[00:01:57] Ella: Yes. So I should have a new article coming out relatively soon. I sort of introduced the idea of the article a while ago. Which is that I think that what CTP theory is lacking in some sense is the ability to represent sort of desires in the mind. And desire is the term that I used when describing this idea a while ago. But I've sort of realized that that might have too many anthropomorphic implications. And I'm now thinking of them more as requirements. In the sense that CTP theory right now can find contradictions between ideas. And in order to resolve a contradiction between an idea and another idea, you could just remove one of the ideas, right? That'd be a very simple way to solve a problem. Just get rid of one of the problematic ideas. But if that's all there is to it, then the way the mind works would just be completely trivial. And a system like that certainly wouldn't be generally intelligent.
[00:02:54] Ella: But according to sort of the basic CTP theory, that's the only thing that's fully described is sort of a reason to get rid of ideas. And the idea of the requirement system is that it will hopefully provide a reason, a force that makes the mind want to keep ideas around. And so I'm hoping that between sort of the desire to not have any contradictions, but also the desire to have sort of requirements be fulfilled, which is the basic idea of the requirement system that I'm currently working on. I'm hoping that those two forces will be able to balance each other out and sort of be what would be necessary for the more sophisticated cognition that you would see in a true AGI.
[00:03:37] Bruce: Right. Thank you. Dennis, why don't you explain about your own, your book and your own thoughts on AGI at this point?
[00:03:44] Unidentified speaker: Sure.
[00:03:45] Dennis: Well, the book generally is basically one of the underlying themes is that we need to perform a sort of unification of software engineering and philosophy, because I think there are many problems in both fields that we are going to have a hard time solving unless we perform such unification, and AGI is just one of them, but it is the one that I'm most interested in by far. The specific theory that I'm currently entertaining is that at some point in our evolution, something occurred which must have been, I think, very similar to the origin of life on Earth. The reason I think that is because although I should say from the start, that there are differences between genetic knowledge and human knowledge, I still think that the mechanisms by which they are created and change are the same. This goes back to Popper who discovered that there's a very tight analogy between evolution of genes and evolution of human knowledge, because both are about evolution in the broad sense.
[00:04:54] Dennis: If we think of evolution as being a theory about abstract replicators that mutate and that are selected. So to illustrate what I mean, to illustrate what I think happens in a mind right around birth is what I think, one of the theories of how life on Earth got started is basically, billions of years ago when the planet was still forming and the oceans had just cooled down enough, they were highly active chemically, and their molecules were forming spontaneously all over the place, and there were some molecules that acted as catalysts. A catalyst is just something in chemistry that can cause a net change somewhere without undergoing any change itself. So then it can perform that change again and again. It just so happened that some of these catalysts created components, molecular components, of which they themselves were made. So they were floating in the water and they created this in the primordial soup, as it were, and they created these components of which they themselves were made.
[00:06:00] Dennis: And if, through lucky circumstances, some of these components rearranged and created ever more of the same components, then over time, if this process gets more targeted, it gets targeted enough that you can speak of replication. And so that's how the first replicators came on the scene. And I think replication is really the key ingredient of evolution. It's one of the three key ingredients, replication, mutation, and selection, or variation, instead of mutation. And basically, what happened from that point on, these were targeted enough to be considered replicators. They instantiated more molecules of the same kind, and occasionally they make mistakes when they do this. So, what you get is you get pockets of the population that look a little bit different from the original. And then, once you get this top-down force, so to speak, acting on this pool of replicators, that's when you have natural selection happening.
[00:06:55] Dennis: And that's how you get this appearance of design. That's when it comes on the scene. It looks like it is purpose-driven, even though it is not. And so you get, at that point, you can really speak of these replicators encoding knowledge, in the sense that they are adapted to replicating. And sometimes in order to replicate, they then incorporate knowledge about their environment. And sometimes, like David Deutsch points out, they are such good replicators that they even incorporate approximations to the laws of physics and aerodynamics, for example, where a bird knows how to fly. Now, the reason I say all this is because I think something very similar happens, like I said, in a human mind right around birth. And that is, I think, when a baby is born or maybe it still happens in the womb, at some point, the brain is more or less fully formed, but it is fully formed in the sense that it is a universal computer.
[00:07:47] Dennis: And babies are born, just like all organisms, with inborn knowledge. And so they will contain ideas about how to, let's say, breathe, how to chew with their gums, you know, that sort of stuff. And I'm not actually sure if these are a true example, but they will have some inborn knowledge, just like animals do, you know, a puppy will have inborn knowledge of how to walk, how to bark, all these sorts of things. But there comes a point where I think one of these ideas that people are born with begins to replicate. It creates some of the building blocks of which it itself is made. And once this starts happening, basically the same thing that happened in the primordial soup happens, but it happens in a computer that is the brain. And so over time, because as this idea replicates, it makes mistakes, it then morphs, it changes into different ideas.
[00:08:43] Dennis: And that is, I think, how we explain how it is that people come up with ideas that aren't genetically encoded. So I think that creativity as a program, intelligence as a program is fully genetically given. So I differ here from David Deutsch, for example, who says that it's partly memetic. I think it is fully genetically encoded, but no particular piece of knowledge that is created is genetically encoded. So that also means that, for example, evolutionary psychology, which Bruce, you and I and others have spoken about, I think is not correct. I think it's not true. There are some inborn ideas that may determine or inform, I should say, our behavior, but they can easily be overwritten. Basically, what's missing from this theory is it still doesn't explain, for example, what consciousness is. I think any good explanation of AGI will contain an explanation of consciousness because it seems that there are good explanations that I think suggest that a universal explainer is automatically conscious.
[00:09:44] Dennis: That ability just comes along for the ride, but it might be hard to build an AGI without a good explanation of what that is. It might be the case that you could simply focus on the evolutionary part and simply build this idea pool with self-replicating ideas that then change over time, so for evolution to take its course. But I have an inkling that there's something missing there. There must be another component to this. I write about this and I talk about it as the meta-algorithm that exerts selection pressure on it, on the evolving idea pool. But yeah, that is the thing that I'm working on now and thinking about now is, what are the missing pieces of the theory so that you could actually build the thing?
[00:10:27] Bruce: I think it's interesting that Ella's theory basically starts with Popperian epistemology and tries to, this is my take, so feel free to disagree Ella, but tries to take that into an algorithm. How do I instantiate Popperian epistemology into an algorithm? Dennis is really starting with, or at least again, this is my take, feel free to disagree Dennis, starting with neo-Darwinism, with how did evolution work, and how could that be used to explain the creation of ideas? Now, obviously, there's a tight tie between Popperian epistemology and neo-Darwinism, but it is interesting how it seems like each of those is the starting point for each of your theories. Is that fair or am I reading too much in?
[00:11:11] Ella: I think that that's a fair description. I can't speak for Dennis, but that sounds fair to me.
[00:11:16] Dennis: Yeah, I'm obviously hugely inspired by Popper. Much of the epistemology that I lay out in the book is inspired by him. But yeah, I do think Popper's epistemology misses the notion of a replicator. I think Ella disagrees with this, but I think that replication is a crucial ingredient of evolution. I don't really know why Popper didn't write about this. I haven't read all of this, but maybe he did and I just missed it. But yeah, I think this idea that there is replication going on in the mind is crucial to understanding how the mind works, and it also allows us to explain other things that are seemingly unrelated like memory, so-called neuroplasticity, and so forth. The theory has some reach to explain more things.
[00:11:56] Bruce: Thank you.
[00:11:56] Ella: Sorry. I was just going to say, yeah, Dennis and I definitely, probably the biggest disagreement between Dennis and I is that, as Dennis said, he sees replication as being essential to knowledge creation, broadly speaking. Whereas, as I explained earlier, I take the view of Donald Campbell, and that I think any process of blind variation and selective retention has the potential to create knowledge. Now, the neo-Darwinian process of having replicators competing for dominance in a population, that's certainly one kind of blind variation and selective retention, but I don't think it's the only kind. I don't think it's the kind that fits best with what we know from Popperian epistemology and so a process in which there are no replicators. Ideas don't create copies of themselves, but they simply exist and the implications of them are explored and compared to other ideas.
[00:12:51] Ella: You look for conflicts between them. That is a system which involves blind variation and selective retention, and thus could create knowledge in theory, which I think has much more in common with Popper and Deutsch's description of how knowledge is created. I think that's the biggest disagreement between Dennis and I right now.
[00:13:14] Dennis: I think that's true, and I should also add that it's very possible that Popper entertained the idea and rejected it, that he just didn't think it was worthwhile or didn't think there was much to the idea. I shouldn't just assume that he didn't think of it. But yeah, I think you summarized the differences between our theories nicely.
[00:13:30] Bruce: Thank you. Then Thatchaphol, I don't think you have necessarily an AGI theory, but you've been researching into related areas. Maybe just talk about anything that you're currently researching. In particular, I'm hoping you will bring up algorithmic evolution, which you and I have talked about and you even gave me a paper you were very excited about recently.
[00:13:53] Thatchaphol: Right. There are some small ideas I'm trying to think about, but actually let me get to the core thing. I actually think that one of the most important things I really want to understand is that I feel that I don't really understand Popperian and Deutschian enough in the following sense. I cannot understand many concepts of their work in the sense that I don't know how to program it. There are so many concepts like the problem in the sense of conflicting between ideas, like the conflict between ideas, and to solve a problem, which means to give good explanation, not just explanation, but good explanation, and how to compare between bad and good explanation. If you want to program this thing, what does it mean to decide which one is a good one, which one is bad? If there is some perfect program for designing this, what would that program look like?
[00:15:07] Thatchaphol: The point is, I think we can try to maybe make this concept more mathematical, make it more very precise, and then see if it still fits with the concept that Deutsch and Popper still wanted to mean, and then compare them. Once they are very precise and mathematical, then you know exactly how to make it as a program, computer program. So I want to do that kind of thing. Basically, let's say once I have this definition of program, very precisely, then we can try to prove theorems about it: whether a universal explainer can really exist with respect to this definition. And then once we have this set precisely, the next step I would like to do is maybe to compare it with the main framework of evolution in theory that people look at like Leslie Valiant’s evolution that is related to PAC learning, that we say that it's not universal at all. I want to prove that their framework is just a limited process with respect to the Popperian framework once we make it mathematical.
[00:16:45] Thatchaphol: Or like to say that the current genetic algorithm right now, in what precise sense, why it is not universal yet. I really want to understand this kind of limitation and make the Deutsch and Popper theory right now be more precise and programmable. That's the thing, I think.
[00:17:14] Bruce: I have similar aspirations as you on that. I would actually like to see a lot of these ideas more programmable. Deutsch does talk about how you don't really understand an idea until you can program it in a computer. And I definitely think that's a limitation of a lot of the ways we think about both biological evolution and Popperian epistemology today, is that a lot of the ideas, we kind of intuitively understand them, but we don't understand them so well that they can be easily programmed. Is that kind of what...
[00:17:43] Thatchaphol: Yeah, so I kind of feel that, like, trying to understand the theory itself right now is still, it's maybe a bit fruitful. Instead of, I mean, trying to build the AGI right now is actually quite, it's already nice, but I think we can try to understand the theory, the Popperian theory itself as well. That's another direction, I think.
[00:18:10] Ella: Yeah, I just wanted to echo, like, I'm kind of right on the same page with you and Thatchaphol and Bruce. So something that I've been writing about in my new article is this idea that really any theory of AGI is implicitly going to be a theory of epistemology, right? And it's in the sense that it's saying, you know, this program would be able to create knowledge, right? And any theory of epistemology is at least implicitly a partial theory of AGI in the sense that it says, well, an AGI would have to work like this in order to be able to create knowledge. And so what we're, the current sort of task of AGI, if you're starting from Popperian epistemology, is to try to nail down the details of Popperian epistemology and make it programmable. Right now we have, you know, good descriptions at a high level. We kind of have, you know, a lot of good explanations of, you know, what it should look like at a high level and some low level details.
[00:19:08] Ella: But what we need to be doing right now, and this is sort of what I'm trying to do with CTP theory, is nail down a lot of the details about Popperian and Deutschian epistemology, which aren't yet specified. And, you know, just, I'm sure we're short on time, but just briefly, Thatchaphol, you mentioned that you don't really see a way to make the notion of a problem a conflict between ideas sort of computable. You don't know how to program that. And I agree that that's a very central thing that needs to be programmed. But I actually think that CTP theory, what I'm working on, has a pretty good solution to that. That's something that I think is solved pretty nicely by the theory right now. The theory doesn't solve everything, but I think that problem specifically is quite well understood in CTP theory. And briefly, the way that CTP theory views this is that, so the mind has a set of ideas.
[00:19:58] Ella: You could call it the idea pool to steal a term from Dennis. And basically, the mind has a built-in way of detecting a direct contradiction between two ideas, which is to say that just by the computational form of the ideas themselves, the mind has a rule that says, okay, this idea directly contradicts this other idea. But importantly, and the reason why this system isn't just trivial, is that a direct contradiction isn't the only kind of contradiction. So it might be that, you know, an idea A has a consequence idea B, and then that idea B contradicts something else in the mind. And in that case, idea A would also be implicated in that contradiction, because it's what led to B existing in the first place. And so in that sense, you could say that A is sort of an indirect contradiction, or it's in indirect contradiction with something else.
[00:20:48] Ella: And so that's sort of a brief look into how CTP theory views contradiction in the mind and how it could be expressed in a computational format.
[00:20:56] Bruce: I actually wanted to explain one thing Thatchaphol brought up that maybe not everybody would understand. He mentioned Leslie Valiant, and he mentioned a couple times PAC learning. So PAC stands for Probably Approximately Correct. And it's the basis, it's the theory, theoretical basis for everything that's in machine learning today. And Leslie Valiant also, though, in his book Probably Approximately Correct, he raised the fact that just like Ella is talking about, and Thatchaphol is talking about, that we don't, we don't really know entirely how to pin down Popperian epistemology into an algorithm. He points out that we also don't know how to pin down biological evolution into an algorithm, or that's his claim in any case. And so algorithmic evolution is the study of how to try to pin that down into an algorithm. Thatchaphol, he sent me a paper recently that made some interesting advances in that area.
[00:21:52] Bruce: And unfortunately, a lot of it went over my head. And I would like to ask questions about, not in the podcast, but just separately to ask questions about it. But it's kind of a related study that's just as you can try to study, how do we make Popper's ideas more computable? You can also study how do we make theory of evolution more computable? And how do you come up—specifically, what Leslie Valiant is looking for is: how do you make an algorithm that is tractable? Because obviously the process has to be tractable for it to work, where it actually comes up with the new adaptations—obviously over billions of years, but we would consider that tractable in this case. So, and I'm actually very fascinated with a lot of your ideas on this, Thatchaphol. I would really like it if you could share more ideas on what you're doing with this.
[00:22:46] Thatchaphol: I can talk a bit if you want. So, right now, Leslie Valiant actually formalized some notion of evolution, and it goes like this. It's a process that kind of just in each step, you replicate yourself into variation, and then into many copies of yourself, but each of the variations may not have the same fitness. And now, the guy with good fitness, with better probability, will survive for the next iteration. And the process keeps going on like this. And he proved that with this framework, with this specific algorithm, there are many types of function or theory that this process can evolve to or can converge to solve some explanation or to get to that explanation. But this class of explanation or function is actually so limited with respect to this algorithm. And why is that? The high-level idea that I kind of see is that the reason for this is because the notion of fitness is fixed.
[00:24:23] Thatchaphol: So that is the selection process is just fixed to be something, and it never changes. And this is really different from Popperian or Deutsch, the way that we understand how knowledge progresses. Because for us, we kind of see that it's very important that the selection process also evolves. That is, we have more and better and better ways to select the variation of your idea. And the new paper, the paper that I sent to Bruce, actually has some of this flavor. It kind of says that, okay, it kind of studies this Valiant evolution process, but it says that, okay, what if the way you select your children or the next generation of population, the selection process actually changes with respect to, they call it the ecology. That is, the selection process actually can depend on the current set of idea pool or whatever, like the set of population right now.
[00:25:44] Thatchaphol: If that selection process can depend on this, that is, the selection process evolves in some sense. In the paper, the way the selection process evolves is very limited. But anyway, this selection process evolves a bit. But he actually shows that just by allowing this, although his way of evolution of selection process is very limited, the class of theories that you can get to is significantly widened. So this is very interesting to me. Some people try to do something that is formal, mathematical, and it has some flavor similar to Popperian philosophy. So yeah, I like that paper.
[00:26:36] Bruce: Thank you for that explanation. So basically, he had come up with a way to use ecology to add that into the evolutionary process and that allowed it to actually explore more paths and to therefore solve certain types of problems that the previous Leslie Valiant approach to algorithmic evolution wasn't able to solve. Very fascinating paper. The math is a struggle for me. I need to probably read it a few more times and try to tease that out. But yeah, it was a fascinating paper, and I could see exactly why you were so interested in it. All right, we've probably gone way over what we would normally do for a podcast episode. So I just want to say thank you to our guests for coming and joining us. Maybe we can invite you guys back some other time for another episode. But you've all been wonderful and you've shared great ideas, and I really appreciate you coming.
[00:27:28] Bruce: Thank you, everybody.
[00:27:29] Unidentified speaker(s): That was fun. Thank you. Yeah, it was great to be here.
[00:27:36] Bruce: The Theory of Anything podcast could use your help. We have a small but loyal audience, and we'd like to get the word out about the podcast to others, so others can enjoy it as well. To the best of our knowledge, we're the only podcast that covers all four strands of David Deutsch's philosophy, as well as other interesting subjects. If you're enjoying this podcast, please give us a 5-Star rating on Apple Podcasts. This can usually be done right inside your podcast player, or you can Google The Theory of Anything Podcast Apple or something like that. Some players have their own rating system, and giving us a 5-Star rating on any rating system would be helpful. If you enjoy a particular episode, please consider tweeting about us or linking to us on Facebook or other social media to help get the word out. If you are interested in financially supporting the podcast, we have two ways to do that.
[00:28:21] Bruce: The first is via our podcast host site, Anchor. Just go to anchor.fm slash four dash strands, F-O-U-R dash S-T-R-A-N-D-S. There's a support button available that allows you to do recurring donations. If you want to make a one-time donation, go to our blog, which is fourstrands.org. There is a donation button there that uses PayPal. Thank you.