Episode 128: Induction’s Immunizing Strategy

• Links to this episode: Spotify / Apple Podcasts

Unofficial, reviewed transcript. It may still contain mistakes; check the podcast when wording matters.

Speakers are labeled by first name where identified. Unidentified-speaker labels may refer to different people. Bracketed question marks indicate uncertain wording.

Transcript

[00:00:00]  Peter: Hello out there. This week on the Theory of Anything Podcast, Bruce takes another deep dive into induction. Why is induction such a confusing and multidimensional concept? What is this debate between critical rationalists and inductivists really about? I enjoyed listening to Bruce here, and I hope someone else out there does too.

[00:00:32]  Bruce: Welcome to The Theory of Anything podcast.

[00:00:34]  Unidentified speaker: Hey, Peter. Hello, Bruce.

[00:00:37]  Peter: How are you doing today?

[00:00:38]  Unidentified speaker: Doing good.

[00:00:39]  Bruce: We've got an interesting episode today. We're gonna talk about induction again, or kind of. The episode's not really about induction. I'm just going to be using a discussion I had with a self-proclaimed inductivist to make some points. But it's also a setup for later, where I am planning to do a podcast about induction and try to explore the concept of induction better and why it is that I don't think there seems to be a lot of agreement as to what it means. So, but I think you'll see that the conversation I had with this inductivist helps explain why it's sort of a fuzzy, confusing concept. So, all right.

[00:01:22]  Peter: Well, I am getting the strangest university level course in epistemology and machine learning that I think anyone has ever gotten. This life is a strange place, but I'm very, very happy to continue listening to you, Bruce.

[00:01:47]  Unidentified speaker: All right.

[00:01:49]  Bruce: Okay, so in previous podcasts, we talked first about Popper's critique of concepts and how he found theories to be a hundred times more valuable than concepts. But I then critiqued Popper's critique and found room for some strong disagreement. Concepts are quite a bit more valuable than Popper thought, but only if you consider the conjecture side of his epistemology, which I don't think he really had in mind. However, I also accept that Popper was correct, that theories are a hundred times more valuable than concepts if we are talking about his critical methodology. Indeed, it's impossible to falsify a concept, so it's impossible to hold a critical discussion about concepts. Concepts by their very nature, built on analogies, at least according to Hofstadter, can't be falsified because they are intentionally fuzzy and extensible. So I also argued that there is no clear-cut demarcation between a concept and a theory. A concept can often be treated like a theory. You lock it down long enough that you can test it. And a theory can be fuzzed out until it becomes a concept, free from any testable content. And as Popper put it, we sometimes use a concept as a summary for a theory or a shorthand for a theory, and sometimes concepts hide theories inside of them. So I'm going to argue today, using a real-life example, that when people vague-man their theories to immunize their theories from criticism, that often what they are actually doing is they're treating their theories less like a theory and more like a concept, okay?

[00:03:23]  Bruce: Now there are several ways in which this can be done to treat a theory like a concept. We'll look at one way to do it today through use of analogy, okay?

[00:03:34]  Unidentified speaker: Hofstadter's point of view here.

[00:03:36]  Bruce: The result of doing this is exactly what you'd expect. The theory stops being falsifiable and loses all its content and it becomes immune to criticism. So many of you may have met a guy on Twitter named Kieren. I don't know if I'm pronouncing that correct. That's presumably just a handle, not his real name. He seems to be a philosophy student, I think, or just maybe someone really interested in philosophy. I think probably a philosophy student because, anyhow, he is a self-proclaimed inductivist and he wrote an article years ago about how Deutsch's hard-to-vary criteria is based on induction. So in his opinion, it doesn't really refute induction. So in the show notes, I'll have to include a link to the actual article that kicked off what we're going to be talking about in this episode. Now, here is a quick summary of Kieren's argument.

[00:04:29]  Unidentified speaker: This is a quote now.

[00:04:32]  Bruce: Induction implied by hard-to-vary criteria is deduced as follows. A hard-to-vary theory is one in which each of its concepts is constrained by some past observed phenomena. When we reflect back on past observations, we see how constrained our concepts are. It is the same as reflecting back to see how well confirmed our concepts are. Therefore, every concept involved in a hard-to-vary theory must have support from past observations. So, in Kieren's mind, this proves induction is real, and that critical rationalism, including, in this case, Deutsch's hard-to-vary criteria, are really just inductivism, or at least relying on inductivism. Therefore, he concludes, From this conclusion, we see that the hard-to-vary criteria is really a requirement of support for the concepts employed by our theories. The harder to vary, the more support. The better the theory is for our acceptance.

[00:05:30]  Unidentified speaker: This is induction, says Kieren.

[00:05:34]  Bruce: Now, sure, a problem is often an observation. That doesn't match our current theory. So there is a sense in which Kieren must be correct. But what about if a theory has a problem by contradicting another theory? So I asked Kieren about this, and Kieren sees this is still fitting his theory of induction because, quote, These other theories are themselves constrained by observations, unquote. So, quote, any theory that says something new about the world, that is, makes claims that must either be explained by other theories or were accepted as knowledge via induction. Whereas we saw he specifically defines induction as constrained by past observations. So, for example, this is a quote now from Kieren, You might rely on a theory of human behavior related relating to trade, but this theory itself is constrained by our observations of how humans behave in such situations. So he claims he is, quote, just taking an additional step than me, as a critical rationalist, to see where our theory has been constrained by observations. Okay, so Kieren is essentially arguing that induction is correct, because apparently induction just means that our theories are constrained by past observations. Now, as a critical rationalist, this argument seems true but uninteresting to me. So that is my lens, and I am going to make that argument to him, and he's not going to accept it.

[00:07:08]  Unidentified speaker: Okay.

[00:07:09]  Bruce: So, now I, as a critical rationalist, fully accept Popper's concept of degrees of corroboration. And Kieren's supposed theory of induction seems to me to be undifferentiatable from Popper's theory of corroboration, at least so far. In fact, let me rewrite Kieren's argument against Popperian terminology—again using Popperian terminology, just to prove that so far the two are the same. A hard-to-vary theory is one in which each of its concepts is constrained by past observations, particularly tests. When we reflect back on past testing to see how constrained our concepts are, it is the same as reflecting back on how well corroborated our concepts are. Therefore, every concept involved in a hard-to-vary theory must have corroboration from past observations. The way I've reworded that, that sounds just like critical rationalism to me now. I don't know that I'm saying something conceptually different, or at least it's not obvious, that I'm saying something conceptually different than what Kieren was saying.

[00:08:11]  Unidentified speaker: So this really does sound exactly right to me.

[00:08:14]  Bruce: So if induction just means constrained by past observations, then okay, I guess critical rationalism involves what Kieren happens to be calling by the label I-N-D-U-C-T-I-O-N, Induction, with no real relationship to what the term originally meant or implied. We'll see later, it's a bit more complicated than that, when Kieren starts to ad hoc add to his definition in various attempts to exclude critical rationalism. But before we go there, let's talk about a common crit-rat response to someone like Kieren, or even to Kieren, there was numerous people that responded to him, that is, in my opinion, a really unhelpful response. A crit-rat might argue to Kieren something like this, no, that's not induction. Induction isn't merely being constrained by past observations, that's corroboration. No, induction, not induction. No, no, you've misunderstood induction, Kieren. Induction is all about having only observations with no background theories at all, and then forming a theory about it. That's logically impossible, so induction is impossible. Or they might even add, and induction is about justifying a theory as true. That's impossible, so induction is impossible. I can't tell you how often I've seen crit-rats make exactly these arguments. The problem is that this response is 100% pure essentialism. It's trying to argue over the meaning of the word induction, which is of course a total waste of time because that's an argument over a word.

[00:09:58]  Bruce: It simply isn't true that every person that claims they believed in induction or had some concept of induction that they believed in, believed induction justified truth in 100% certain way. In fact, really no one ever believed that. Okay, nor is it the case that every person that believes in induction believed you could induce using observations absent any background theory. We should always accept all definitions. If to Kieren induction really and truly only means constrained by past observations, as at the time he was claiming, then that's fine. Of course, we'll quickly discover he doesn't accept this definition either. When accepting his definition to induction forces him to admit his induction is just a new name for critical rationalism. Then he'll, on his own accord, ad hoc adjust his definition to avoid that fate. But until then, we should accept whatever definition he cares to give to us for induction, but also hold him accountable for that stated definition. However, it's okay to point out that he's not exactly using a historical definition of induction. If we can put it that way, then the CritRat response that I just parodied might actually make more sense. So imagine instead of arguing to Kieren that CritRat response, let's argue something like this to him instead. Well, Kieren, to be honest, I'm not familiar with the idea of defining induction as meaning something as innocuous as constrained by past observations.

[00:11:29]  Bruce: Now, if you want to define induction that way, I'm happy to refer to this as Kieren's induction to make it distinct from how I would normally understand that term different than what we'd call, say, Baconian induction, as maybe for the original form of induction. And I'll be happy to continue the conversation forward referring to this as Kieren's induction. Notice how this gives you exactly the same result that the crit rats are trying to get at, but does so without turning to essentialism and while staying a word nominalist. Now, of course, Kieren is not about to accept that his induction is merely a word for critical rationalism. So, he is also unsatisfied with defining induction as merely constrained by past observations too. Interestingly, he admits to me that his form of induction is not the induction of Bacon. I was very pleased to hear him say that. In fact, this is an actual quote from him. So, you are right that the induction I'm considering is not Bacon's inductivism, he says. He takes issue with the induction of Bacon because it is supposedly used to induce new theories and he feels that this is impossible through induction. Now, I want to note here something important for crit rats. They tend to believe that induction has a single essential definition that includes the following. Things like induction is about coming up with conjectures using solely observations without any background knowledge at all. Or induction is about proving a theory as true or at least probably proving it as true. Note that Kieren is outright denying the first of these.

[00:13:04]  Bruce: Instead, he's going to claim he's arguing only for what he calls inductive inference, which he sees as distinct from the first claim, because it's got nothing to do with how you make conjectures. It is unclear to me if Kieren believes the second claim or not. I do happen to know that Ivan the Bayesian, who we've had on this podcast number of times, is also an inductivist and he does not accept either claim as correct, a correct view of induction. So, I am often left with the impression that crit-rats are straw-manning inductivists and don't really understand the claims of inductivists. They haven't taken the time to understand the claims of inductivists. To clarify his view, Kieren offers the following definition of induction from Wikipedia, quote, my understanding of induction can be summarized by what you will find on Wikipedia or Stanford Encyclopedia of Philosophy, quote, In a good inductive argument, the truth of the premises provides some degree of support for the truth of the conclusion, where this degree of support might be measured via some numerical scale. Now we need to break this argument down a bit. A critical rationalist that agrees with Popper on corroboration — there are so few of us today, it would seem, that agree with Popper on corroboration — probably has no problem with the idea that observations do quote support a theory, if by support you mean degrees of corroboration. I think most critical rationalists would struggle a bit with this part of what was in the quote, however, where this degree of support might be measured via some numerical scale.

[00:14:40]  Bruce: What numerical scale would that be? Turns out again, no surprise, Kieren means probability calculus. Now, if we're talking specifically about a statistical inference problem, then sure, even as a critical rationalist, I agree, I see no problem with that. I even personally, following the Popperian Deborah Mayo's usage, might don't mind referring to statistical inference as statistical induction, and I see those as synonymous terms. Kieren later repeated to me multiple times an example of using what he calls induction to determine if a certain medicine works or not by doing a double blind study. Now, his chosen example of induction, in this example of a medical double blind study, really does utilize probability calculus to indirectly, quote, measure the degree of support or corroboration of the theory that, say, the medicine works, and it's not just the placebo effect. But is this in general how we do science?

[00:15:43]  Unidentified speaker: Well, not that I can see.

[00:15:45]  Bruce: In fact, except for a pure statistical problem like Kieren's much-repeated medical example, I'm not even sure what measured via some numerical scale would contextually mean without some additional explanation or additional context. Let me see if I can explain what I mean here. So, for example, Ivan the Bayesian, in episode 90, I pointed out to him that every single physics theory is presumably wrong. He claimed he only meant, well, the theory is true enough for our purposes right now, which I thought was actually maybe not such a bad answer because that would be capturing the same idea under critical rationalism as verisimilitude. That answer, while I still consider it to be ultimately technically incorrect — that is a discussion for another time — I can make sense of what Ivan's saying. So, Ivan's view is that induction isn't even trying to measure that a theory is quote probably correct per se, but only quote correct enough for our current purposes. Probably correct enough for our current purposes? Given that view, what does measured via some numerical scale even mean? It almost must imply something like this.

[00:16:55]  Bruce: I'm doing this experiment at the moment using this particular instrument and I'm comparing the theory to the only known competitor and I'm making the false, probably false assumption that one of these two theories is correct and I'm coming up with a probability output given the assumption of equal priors and the end result is going to only be used to determine not if the theory is probably correct, but if given our experimental setup and assuming all our auxiliary hypotheses are correct, which they may not be, which of the two theories is probably better? Whew, that was quite the mouthful. With all those caveats in place, sure, I guess I can agree that we can and do measure a degree of support via some numerical scale, even under critical rationalism in science. And we may even do that using probability calculus. So I guess under this still pretty lame definition of induction, sure, critical rationalism is induction.

[00:17:53]  Unidentified speaker: But so what?

[00:17:54]  Bruce: It seems like an unremarkable comment at this point. It doesn't change the fact that critical rationalism is not induction in the more generally used understanding of the term, say, by Bacon. Nor does it change the fact that when I say we do not measure the probability of the truth of a theory, that we do not in fact measure the truth of a theory in science, other than in that extremely caveated sense that I just suggested. And really, if we're being honest, not even then, since we probably forgot several caveats, because they were unstated assumptions that we never thought to make explicit. So really never, we never measure the probability of any theory being true. That just isn't something science does. To use an analogy from episode 122, think about how people argue over logic when what they really, really what they're arguing over are premises. There seems to be a similar mistake being made here by so-called inductivists, or possibly Bayesians too. You never measure the probability of a theory as true or false in any absolute sense. You only measure the probability given a set of specific assumptions, each of which may be wrong, and you have no ability to measure the probability of the assumption being wrong. So if accuracy is required, the claim that we measure the degree of support on a numerical scale, is really only true if you throw in all those additional caveats that I just mentioned. And if you do throw them in, then your argument starts to feel as lame as it actually is, because you've now exposed the problem with what you're saying.

[00:19:30]  Bruce: This seems to be one of the most important mistakes inductivists and Bayesians make. They don't see themselves as measuring the probability of something given a set of assumptions. They mistakenly think they're measuring the absolute probability of something. What makes this all worse is that the actual statement Kieren quoted to me from Wikipedia was, and I quote, where this degree of support might be measured via some numerical scale. So apparently, you only sometimes need to measure degree of support on a numerical scale to be induction. If you don't, well, it's still induction, whether you measure it on a numerical scale or not. Okay, if that's what we mean, then let's drop the optional clause. Quote, in a good inductive argument, the truth of the premise provides some degree of support, which I take to mean corroboration for the truth of the conclusion. Oops, we're back to corroboration now. Induction apparently is just now another word for critical rationalism. At this point, Kieren jumped to a new tactic with me. He said, quote, my view is that we do use a form of inductive reasoning, meaning inductive inference, especially when it comes to our general claims about the world. The general form of such an induction is to infer the truth or falsity of a general claim from a random sample of its predictions. I see induction as one aspect of the scientific method, where its role is primarily for testing hypotheses, not for generating them like Bacon wanted.

[00:21:05]  Bruce: I have trouble accepting critical rationalism, critical rationalists, when they choose to outright reject all forms of induction. He later clarifies, as part of his description of the scientific method, Quote, induction is defined as, test a random sample of the predictions to infer the truth-falsity of your hypothesis. The proportion of successful predictions corresponds to the proportion of support we reveal for our hypothesis. Ah, so induction now means using a random sample of predictions to infer the truth and falsity of a claim. Which sounds to me like another term for statistical inference. The beauty of this claim, the way Kieren just put that, is that it is now clearly false. We absolutely do not use statistical inference with random samples when doing all of science. In fact, I would say, in most cases, we just don't do it other than maybe measuring, say, the quality of the instrument output. Kieren, much repeated of a medical example aside, where is the random sample in how the Eddington Expedition went about corroborating general relativity? They didn't randomly take positions of stars in the sky. They targeted specific stars at a specific time during an eclipse.

[00:22:22]  Unidentified speaker: This is about as non-random as you can imagine.

[00:22:25]  Bruce: This was nothing like a random sample. It's like they were intentionally making it not a random sample.

[00:22:31]  Unidentified speaker: Yes. Just quick question.

[00:22:33]  Peter: So am I, is this this this this person, Kieren or whatever you're arguing with, debating with, is he a inductivist but not a Bayesian?

[00:22:45]  Unidentified speaker: Is he a Bayesian or not?

[00:22:47]  Bruce: He only talked to me about inductivism.

[00:22:48]  Unidentified speaker: He did not bring up Bayesianism, so I don't actually know. Okay. Okay.

[00:22:52]  Peter: I was just trying to read in between the lines here and thinking, maybe he was making sort of a unique case for induction, that I just wanted to kind of get my mind around because I kind of associate the two.

[00:23:05]  Unidentified speaker: You know, it just seems like most people—

[00:23:07]  Bruce: Bayesians do accept induction, so there is a relationship between Bayesianism and induction, but they're not specifically the same thing.

[00:23:15]  Peter: But there's plenty of people who argue for inductivism who aren't Bayesian.

[00:23:20]  Bruce: Yes, and even for a Bayesian, inductivism is like one small aspect of their overall theory.

[00:23:26]  Unidentified speaker: Oh, is that right? Okay. Okay. I'm just trying to place this, get my mind around this. Okay.

[00:23:33]  Bruce: So with this additional clarification from Kieren now in place, apparently Popper was right, induction is false. Let's just drop the whole embarrassing theory and embrace critical rationalism. So here is how Kieren responded to this problem. Quote, what I mean is they, the observations gathered by the Eddington Expedition, were not chosen with any sort of bias except for the fact that they were all that the universe provided us, and so in an important sense, they were random. You don't need to get hung up on the requirement for a random sample anyway. We still proceed with our inductions even when there are severe limitations on the sample, but we do so with degraded confidence in our conclusion. There are degrees of strong and weak inductions. Okay, so apparently by random sample, Kieren actually means unbiased sample, even if it's non-random. And why are we even using the word sample here? What exactly is the observations of the Eddington Expedition a sample of? Is it a sample of possible outcomes of doing the Eddington Expedition experiment? Why are we calling this a sample? It seems to me that he's saying random sample, but he's stretching, or if you prefer clarifying, that the term really means we test it by doing an experiment. Okay, let's plug this new definition back into the original statement. Quote, the general form of such an induction is to infer the truth or falsity of a general claim by testing it. Some crit rats might take issue with the word truth there and want to say that we only infer the falsity of a general claim by testing it.

[00:25:14]  Bruce: But even those crit rats would admit that if this is your best surviving theory, we're going to embrace it as tentatively true. And since neither Kieren nor Ivan are claiming that we prove a theory definitively true, it's unclear there's any actual disagreement here between the crit rats and the inductivists at this point. Okay, so apparently, induction is another word for critical rationalism.

[00:25:35]  Unidentified speaker: Yay! Are we done yet?

[00:25:37]  Bruce: Kieren then makes another argument to me. He says that it doesn't have to be random, that's true, but it is preferred to be random, because otherwise, quote, otherwise it, quote, confidence in our inference will be degraded. Honestly, I'm not seeing why this would be true. The Eddington Expedition was convincing precisely because it was non-random. So apparently, we've again disproven induction.

[00:26:01]  Unidentified speaker: Yay!

[00:26:02]  Bruce: Let's all become critical rationalists. Also, I'm not seeing how to turn this into easy probability-calculus-based analysis. We have no idea how many corroborating tests of Newton came over the centuries. For the sake of argument, let's assume it's like, say, 100,000. So Newton has 100,000 corroborating tests and GR has 100,001 corroborating tests. I doubt there is any way to have the probability calculus, if you're using it correctly, show anything but a difference of maybe out to the sixth digit at this point. Okay, like literally, if we're treating this as random samples like Kieren says we're supposed to be doing, then there is something drastically off over the fact that the Eddington Expedition suddenly caused cataclysm through the scientific community because it suddenly showed that there was something wrong with Newton. We should be treating it as, oh, it's just one out of 100,000. So we're going to adjust our proportions by one out of 100,000. If you were to do that, you would find that both are, what, 50-50 or something at this point, or [49.999999?] and [50.0001?]. That's the most you would actually adjust things using probability calculus, if you're going to treat them as random samples. Surely, we do not use the probability calculus for something like this, or at least not in the way Kieren is trying to position it as every test is to be treated like a random sample. What really happened with the Eddington Expedition fits poorly when treated as just one more random sample of an observation.

[00:27:38]  Bruce: But it makes perfect sense under critical rationalism as a critical test between two theories where we falsified Newton's theory and left GR as the sole standing theory. So again, induction, as currently defined by Kieren himself, is definitely NOT how science works in real life. The moment there is a counterexample to Newton, immediately everyone knows that you've either got to show there was something off with the experiment, or you have to accept GR is now the current best theory. It isn't treated like, oh, that's just one observation out of 100,000, because it's just one more sample.

[00:28:14]  Unidentified speaker: Okay, so good.

[00:28:15]  Bruce: We have now disproved induction.

[00:28:17]  Unidentified speaker: Yay! Are we done yet?

[00:28:20]  Bruce: Given this criticism, Kieren again, quote, clarifies, The way induction applies to GR is through testing its consequences. We derive consequences, predictions from GR, such as orbital precession, light reflection, redshift, GPS signaling, etc.,

[00:28:37]  Unidentified speaker: and then we test them. Okay!

[00:28:39]  Bruce: So apparently induction just means we do critical tests and doesn't require any sampling at all. Okay, looks like induction is, again, just a word for critical rationalism.

[00:28:50]  Unidentified speaker: Yay! Wait!

[00:28:52]  Bruce: Kieren clarifies further, quote, it might not be as easy to quantify this sort of induction compared to the quantification of induction related to medicine, the medical example, but the form of reasoning is still the same. We infer the probable future success of our theory based on the proportion of successful tests we found in the past. Okay, now we can disprove induction for sure if we're going to accept that statement as correct. It is not—not only—not just not easy to quantify this sort of induction done by the Eddington Expedition. It's completely impossible because we have no idea how many past observations there were. But if we did try to do it, again, maybe we'll just accept there's 100,000 for the sake of argument.

[00:29:39]  Unidentified speaker: This basically tells us nothing.

[00:29:40]  Bruce: For the reason I just explained, if you treat it like it's just one more sample, that's all you don't see that this is a critical test. You don't see why it's a falsification. And instead, you think of it as just a random sample, then absolutely the Eddington Expedition showed nothing but maybe a sixth digit.

[00:29:58]  Unidentified speaker: Okay.

[00:29:59]  Bruce: So clearly, we did not in the Eddington Expedition, quote, infer the probable success of our theory based on the proportion of successful tests we have found in the past. In fact, it had nothing to do with that at all. For that matter, science doesn't really take this idea of, quote, inferring the probable future success of our theory based on the proportion of successful tests in the past seriously anyhow. Even when there was, what we now know, a misprediction for Newton, say, on the perihelion of Mercury, we didn't say, oh, I'm now going to assume that Newton will occasionally make mispredictions and I'm going to adjust my confidence of successful future predictions accordingly. That would literally be a silly thing to assume. Instead, there was an assumption that one of the auxiliary theories must be wrong, and we started putting to work trying to figure out what it was, in this case, looking for a missed planet called Vulcan. It was only after we failed to find Vulcan that it started to occur to us, oh, something might be wrong with Newton's theory.

[00:31:05]  Unidentified speaker: So, woo, Kieren's induction has again been disproven.

[00:31:09]  Bruce: We can drop it as a theory and we can all become critical rationalists. Then Kieren clarifies again, quote, what I'm trying to say is that we can't be as precise when presenting our inductive support for GR in comparison to medicine. One of the reasons is because in a medical study, the samples are recorded precisely and therefore we can present the precise results of a statistical induction. When it comes to Newton and Einstein's theories, we can reflect back on countless observations of objects moving about under gravity and see that they confirm these theories, but we can't easily give exact figures. However, we know that Einstein's theories are confirmed in more cases than Newton's. Ah, so what he really means is that GR explains all of what Newton does plus something Newton can't. So we favor GR because it is the sole surviving theory.

[00:32:02]  Unidentified speaker: Okay, looks like induction is now just a term for critical rationalism. Are we done yet?

[00:32:08]  Bruce: This back and forth continued for a while. I would alternate between saying, okay, I can refute your theory of induction easily, only to have Kieren quote clarify his theory of induction in such a way that it was so vague it could easily be mapped to critical rationalism. By doing this, he immunized his theory of induction from criticism. With a vague enough understanding of induction, it's basically impossible to ever refute Kieren's theory of induction. Every time you think you've got a way to refute his theory, he simply takes whatever he said and extends it via analogy, so that the refutation is no longer valid. A few examples of this from what we just quoted. He wants to say science is about taking a random sample.

[00:32:54]  Unidentified speaker: Now, that is clearly false.

[00:32:56]  Bruce: But it is not clearly false if, by analogy, you decide that random sample includes any test or observation, even if it was neither random nor a sample. Likewise, induction is clearly false if you claim that we utilize the proportion of past successful tests to determine our confidence in future tests. But if you can just reimagine the phrase, portion of past successful tests, to merely mean you go with the theory that wasn't refuted because it passed more tests, then clearly that phrase, initially clearly false, now might be understood as true. I have a term for this rational fallacy. I call it playing with words or word games. It always surprises me that people don't seem to see how easy it is to start with a well-stated and fully testable theory and fuzz it out by extending the meaning of the word through use of analogy into an untestable theory. Words and concepts are meant to be stretched, so this is a trivially easy thing to do. It can always be done. I'm going to say it again. It can always be done. There is never a case where your theory can't be stretched via analogy to where it becomes unfalsifiable.

[00:34:12]  Unidentified speaker: Okay.

[00:34:13]  Bruce: Notice also that Hofstadter is vindicated here. Kieren's stretching his terms specifically requires use of analogy. A random sample is analogous to a test. So in his mind, all tests are now just random samples. Once you realize this, that's the trick. Of course, it becomes impossible to refute him, because science really is about testing theories severely. But notice that each time Kieren fuzz out his theory using these language tricks, that he can't help but turn induction into something so muddled that now critical rationalism counts as a form of induction, where all the true aspects of his theory of induction are only true because they happen to match critical rationalism. So far, the argument loop goes like this. Kieren would claim that critical rationalism relies on induction. He'd quote prove this by defining induction such that critical rationalism was relying on induction. Typically, by defining induction is merely meaning something like our theories are constrained by past observations. I then say to Kieren, okay, if you're going to define induction that way, then I agree critical rationalism is a form of induction. Indeed, critical rationalism is the only true form of induction. Now, I'm supposedly giving Kieren exactly what he just said he wanted. I'm a critical rationalist and I'm admitting that critical rationalism relies on induction, though only in the way he defined it vaguely. At this point, Kieren would then offer an attack on critical rationalism, attempting to prove that critical rationalism is not induction after all.

[00:35:45]  Bruce: Kieren never saw the irony of the fact that he himself was just a moment ago claiming that critical rationalism was induction, or at least relying on induction. And I was now agreeing with him. And that made him, when I would agree with him, that would make him want to argue, no, critical rationalism is not induction. Let me give a few examples of his attacks on critical rationalism. And allow me to show how I responded to each attack. And let's make a distinction between the responses that I give him from Popper and the way he was clarifying through stretching terms via analogy. For example, Kieren argued to me, quote, My claim is not just that we use observations to test our theories. If that was, if it was, then as you suggest, I would be in agreement with Popper. Okay, we agree on that much. But then he says, That leaves out the important claim of inductive inference that we can provide some rational justification slash support for our general claims through our particular observations. Popper allows us to falsify our general claims, but he denies that we can rationally justify them. I wonder where you stand on this, he says. Do you believe you can justify or provide a degree of support for our universal claims about the world? So notice that Kieren is inviting me to respond and to explain my own stance, so I do. By the way, he probably picked the worst critical rationalist to have this argument with, because I believe in corroboration, and a lot of critical rationalists don't.

[00:37:11]  Bruce: If he had happened to be arguing with Brett Hall, or one of the critical rationalists that doesn't believe in corroboration, his argument might have had some teeth, but against me, it's almost pointless. Note that Kieren is making an attack on Popper's epistemology over the fact that, famously, it rules out the idea of verifying a theory, but allows one to falsify a theory. Now, of course, I could just play the very same word game Kieren is playing. I might simply vaguely define falsification in such a way that everything counts as a falsification. And, in fact, I've noticed that some crit-rats do exactly this, and I'm planning to do a podcast in the future to criticize crit-rats over how they sometimes do this. But this would make critical rationalism no better than Kieren's induction.

[00:37:55]  Unidentified speaker: Can we do better? I think we can.

[00:37:58]  Bruce: Put another way, is it possible to respond to this attack from Kieren without relying on his fuzz-out trick of extending words and concepts via analogy until they are so loose that his theory has no possibility of turning out to be wrong?

[00:38:12]  Unidentified speaker: Again, I think we can, I think we should.

[00:38:15]  Bruce: We do not want to indulge in simply stretching our terms and associated concepts until they mean anything imaginable. What we want is to respond to Kieren's criticism, not by fuzzing out our explanations until they become vague content-less concepts like he's doing, but by focusing in and making our explanations increasingly explicit. So let's attempt to respond to Kieren's attack using that approach. Note first that support and verification are not the same thing. Kieren has missed this and is acting like Popper said there was no such thing as support, when what Popper really said is there's no such thing as verification. Verifying a theory as true is impossible.

[00:38:58]  Unidentified speaker: Period. End of story.

[00:38:59]  Bruce: Unless, of course, I guess you could stretch the term verified by analogy to just mean corroborate. And people may do that. But in general, when we say verify a theory, we mean verify it is true.

[00:39:10]  Unidentified speaker: Okay.

[00:39:12]  Bruce: Supporting a theory can mean many different things. Might include the concept of verification, but it has a much broader set of meanings. But surely, supporting a theory could be used as a synonym for corroborating a theory. So Kieren is strictly wrong that Popper doesn't allow for any concept of support. I need to clarify, by which I mean make explicit, not fuzz out, what support or corroboration means for Popper. For us, Popperians, corroboration means this. You propose a theory that gives an explanation, and it is not an ad hoc explanation.

[00:39:51]  Unidentified speaker: That is, it makes independently testable predictions.

[00:39:53]  Bruce: Implications that put the theory at risk, meaning the theory is genuinely testable. To avoid being ad hoc, at least some of those predictions must be things we did not already know. A good theory yields a surprising previously unknown outcome, for example that starlight bends around an eclipsed sun.

[00:40:12]  Unidentified speaker: We then test that implication.

[00:40:14]  Bruce: If the test fails, the theory faces a serious problem because the theory was formulated as an explanation with consequences and not as a vague stretchable concept. If the test succeeds, and the older theory can't explain the result without adding its own independently testable auxiliary theories, then our new theory becomes the best available and the old one must be abandoned unless it can introduce its own non-adhoc independently testable auxiliary theories that also explain the outcome. But surviving such tests offers a form of support for a theory, and more tests in more critical circumstances increases the degree of corroboration, or the degree of support, if you will. Now, notice how my clarification differs from Kieren's. When Kieren clarifies, he fuzzes things out, stretching his terms by analogy until no counter-examples are possible. When I, or rather, Popper clarifies, Popper gets more explicit and specifies what he offers, and Popper gets more explicit and specific, and what he offers is always an explanation, not a debate over terminology. That means the checkability of his theory increases rather than dropping to zero, as in the case of Kieren's approach. I also let Kieren use whatever terms he wants while holding him to the definition he chose. This is exactly what I call Popper's ratchet, and I argued in past episodes it applies to philosophical theories just as much as to empirical ones. Kieren avoids [refutation?] by vague-manning, turning his theories into vague analogies, while Popper keeps his fully checkable explanations.

[00:41:56]  Bruce: How did Kieren respond to me doing this? Here is a sample response. Quote, So in your view, one theory outperforming another in a particular case, in the past, is a reason to prefer using this theory in a similar future case. This is essentially the same solution as the one I provided.

[00:42:13]  Unidentified speaker: It is inductive.

[00:42:15]  Bruce: When we infer the future success of our theory based on its past success, can you give me an answer that clearly doesn't invoke induction? So we're back to the original word game. To be clear, Kieren's question to me was about how Popper didn't allow for any idea of support. And when I responded by showing that Popper does allow a specific kind of support, Kieren then disresorted back to defining induction in such a way that my response now must of necessity count as induction. Which, as I have already admitted to Kieren, if you are going to define induction this way, sure, critical rationalism is a form of, and the only true form of, induction. Kieren then doubles down like this, quote, Popper wasn't just arguing against 100% justification. Even partial probable justification support was a problem for Popper. Now, Popper rather explicitly talks about degrees of corroboration, a choice of words that for sure implies increasing support for a theory based on how severely tested the theory is compared to its competitors. So right off the bat, we know Kieren has a problem with his argument. Of necessity, he must be misreading Popper.

[00:43:28]  Bruce: It might be helpful here to let Popper speak for himself, quote, By the degree of corroboration of a theory, I mean a concise report evaluating the state at a certain time t, of the critical discussion of a theory, which with respect to the way it solves its problems, its degree of testability, the severity of tests it has undergone, and the way it has stood up to these tests, corroboration or degree of corroboration is thus an evaluating report of past performance.

[00:43:57]  Unidentified speaker: That's Objective Knowledge, page 18.

[00:44:00]  Bruce: Popper even imagines a degree of corroboration as being roughly measurable amount, though he warns there's no way to actually assign a value to it. Quote, Can we assess the degree to which a theory has stood up to tests? In particular, can we compare two theories, say Newton's and Einstein's theory of gravitation, and say precisely why Einstein's is better tested or better corroborated and therefore tentatively more acceptable? Quote, in its simplest form, that was a quote too, sorry, this is a continuing quote. In its simplest form, this problem may be put as, how well is Theory T tested? And can we ascribe to it a number, a measure of degree, summing up the severity and number of tests and the manner in which the theory has stood up to them, awarding to it something like an examination mark to be called its degree of corroboration? That's from Realism and the Aim of Science, page 230. So Popper literally has no problem with even attempting to put it into a numerical mark of some sort, okay, at least in principle. So far, Popper's degree of corroboration easily fits within Kieren's fuzzed out view of induction, which only requires that past observations, including tests, narrow down which theories have been falsified. But Popper does not treat degrees of corroboration as equivalent to probability calculus. And this is probably where we're starting to see the actual difference between the two theories. Among other reasons Popper does this is theories rarely overlap in a way that would let you directly compare how much content one has relative to another.

[00:45:29]  Bruce: Popper argues that theories and their content form a partial order. To be a partial order means some of the elements in your set can be ordered, but others can't. So here think of like imaginary numbers where some can be ordered, but others can't because there's two values you have to order. The opposite of a partial order is a total order, which means every element can be ordered.

[00:45:48]  Unidentified speaker: Think of integers here.

[00:45:50]  Bruce: So to Popper, content of a theory is a partial order, not a total order. So, quote, contents of theories are sometimes logically comparable and sometimes not. They form a partially ordered system, objective knowledge, page 50. Because of this, among other reasons, it would be impossible to force fit degrees of corroboration into the probability calculus. So Popper's degrees of corroboration is not equivalent to a probability. It may be something that you could in context even give a number to maybe, based on numbers of tests or something like that. He's not against that. But it would never fit into the probability calculus. I shouldn't say never. It would—it would not usually fit into the probability calculus.

[00:46:34]  Unidentified speaker: If you had the—

[00:46:37]  Bruce: If the theories were set up such that they one encompassed the other, that may be a different story.

[00:46:42]  Unidentified speaker: That's something we'll have to discuss in a future podcast. I just earmark it for the moment.

[00:46:46]  Bruce: Even when two theories content can be directly compared though, say, Newton's and general relativity, it's still wrong to treat corroboration as a probability. Here I differ slightly from Popper. We could treat it probabilistically, but only if we reinterpret it as something like the probability that our instruments didn't mislead us given certain assumptions, not the probability that this theory is true. This is because a new theory can always emerge, and the probability calculus usually assumes—must assume that the full set of possible theories is known and mutually exclusive. An assumption that almost never holds in science, because there's always new theories that could come up, that might become competitors in the future. So even when using probability calculus in science, we're never measuring the probability that a theory is correct. We're measuring something else. With that something else, what that something else is depends on the problem being solved. For this reason, Popper is right that degrees of corroboration are not probable support in the Kieren sense. But corroboration is a legitimate concept of support. It just isn't a kind of support that follows the probability calculus. Despite this difference, Popper often uses the term support and corroboration interchangeably. As at one point he writes, if X is confirmed or corroborated or supported by Y, clearly in that quote, he is using all three terms in this context as synonyms.

[00:48:16]  Bruce: Popper even originally called degrees of corroboration his degrees of confirmation, before realizing that that wording misled people and switching back to degrees of corroboration. So Popper clearly held that science provides a kind of support, namely corroboration, not an inductive probabilistic justification of a theory's truth.

[00:48:38]  Unidentified speaker: What about justification?

[00:48:39]  Bruce: Kieren claims that Popper rejects all forms of justification. Now many modern crit rats agree with Kieren, the inductivist here, but let's let Popper speak for himself. Quote, we can never rationally justify a theory, that is claim a theory is claimed to know its truth. But we can, if we're lucky, rationally justify a preference for one theory out of a set of competing theories for the time being, that is with respect to the present state of the discussion. And our justification, though not a claim that the theory is true, can be the claim that there is every indication at this stage of the discussion that the theory is a better approximation to the truth than any competing theory so far proposed.

[00:49:20]  Unidentified speaker: That's Objective Knowledge, page 82.

[00:49:22]  Bruce: So Popper does allow a kind of justification, just not the justification of a theory as true, or even probably true. He says that's impossible. But Popper does allow justification as a preference for one theory over another, and he treats that as entirely legitimate. So Kieren is, again, strictly incorrect. Popper isn't against all forms of justification. Note that, unlike Kieren, Popper clarifies by being more explicit, not by fuzzing out, and widening his terms over until they cover everything.

[00:49:58]  Unidentified speaker: This matters.

[00:49:59]  Bruce: It's how you distinguish a good explanation from a bad one. Watch who clarifies by expanding meanings into vagueness, and who clarifies by tightening them into an explicit and checkable explanation. I proposed this as a rule—proposed this in past podcasts as a rule. Don't vague man your theories. In this context, that means, don't stretch your concepts until the theory becomes unfalsifiable. Only clarify by making your explanations more explicit and more checkable. Popper has now made it clear that he accepts a narrow form of support and a narrow form of justification, and he rejects all other forms, keeping his theory highly checkable because it has real content. So Popper is following Popper's ratchet, surprise, while Kieren is vague manning his theory. My exchanges with Kieren went on much longer, but they quickly fell into a loop. He'd say, What you just said uses past observations. That's induction. See, Popper is really induction. I'd agree that under his extremely expansive definition of induction, that he was right. Then Kieren would insist, No, Popper rejected induction and attacked Popper again, usually with what I would say is an out of context quote. I'd explain what Popper actually said.

[00:51:18]  Unidentified speaker: Then Kieren would repeat, You use past observations. See, that's induction.

[00:51:23]  Bruce: I'd again grant the point that under his definition, it is. This cycle would repeat three or four times before we both gave up. It's literally a perfect immunization strategy because it's a circular argument. I honestly wanted to shout, Make up your mind. Are you claiming Popper's epistemology really is induction?

[00:51:42]  Unidentified speaker: Or is it that it's not induction? It can't be both.

[00:51:45]  Bruce: But because he would never, anytime I would clarify something as to what Popper actually meant, he would simply look for where in that answer it mentioned past observations, and then he would say, see, that's induction.

[00:51:57]  Unidentified speaker: Then we would go back to the beginning. I would say, yes, it is.

[00:52:01]  Bruce: There was no way out of this loop because it was a circular argument. It was an immunization strategy. To be fair to Kieren, let me just say that I have not presented his full arguments. And I'm actually going to present his full arguments in a future podcast. I'm saving his full arguments for a future podcast, the one I'm going to do on corroboration. Where, in all honesty, I may even agree with Kieren on a few things that he says here and may end up being more siding with him than the CritRat community as it is today. Instead, but for now, I'm not really trying to catalog every issue with his view of induction. Not in this podcast. I'm not really even talking about induction at all. I'm examining what makes a good rational argument or explanation versus a bad explanation. My claim is that Kieren's rational mistake is this. Whenever you show him a problem with his theory, he fuzzes it out into a vague concept, often using Hofstadter-style analogy extension. Popper, by contrast, never does this. When challenged, he drills down, makes the theory more explicit and therefore more checkable. To put simply, when people want to immunize a theory, they turn it into a concept, and concepts, as Popper noted, are unfalsifiable. Because this is a podcast by a critical rationalist, for critical rationalists, most of you, except maybe Ivan the Bayesian, are probably nodding, yep, Kieren fuzzed out his theory into concepts, and Popper focused in clarifying them by explanation. But would you feel the same if it weren't a theory you already disagreed with, like Induction?

[00:53:42]  Bruce: Let's take an example from Episode 121, where I discuss the problem of the crit-rat disobedience criteria. Those using the disobedience criteria fall into the same rational fallacy as Kieren. When you ask, in what sense is AGI disobedient, they often start with an analogy. A program follows its code and an AGI disobeys its original programming.

[00:54:05]  Unidentified speaker: But this is problematic.

[00:54:06]  Bruce: General intelligences follow their code just like any software. Physics won't allow it to be otherwise. When faced like this, they try to quote clarify disobedience to save the theory. For example, they might claim that variation and selection is really quote disobeying discarded variants. A huge stretch from the original meaning of the term disobedience. Even so, it's still problematic. The immune system would also count as disobedient because it too follows variation and selection. Therefore, it is disobeying the variants that didn't work.

[00:54:39]  Unidentified speaker: No problem!

[00:54:39]  Bruce: CritRats then arbitrarily restrict the definition. Discarded variants only count as disobedience if a general intelligence does it open-endedly and keeps asking for explanation. Now disobedience, scare quotes intended, is defined so it necessarily only applies to general intelligences. How comfortable are you with my proposed rule now? Do you still think my criticism of Kieren is fair when we apply the same rule to a favored CritRat theory? If you have to decide, you have to decide, is it a real rule that universally applies to explanations or is it flawed? And what I mean here is, is it a real rule that you should never fuzz out your clarifications and turn your theory into a concept, and should only clarify by making your theory more explicit and more specific? Okay, I'm proposing that as a rule. Is that a universally true rule for good explanations or is it flawed? If it is a universally good rule, then the disobedience criteria is a bad explanation for exactly the same reason that induction is.

[00:55:52]  Unidentified speaker: Okay, Kieren's induction is.

[00:55:54]  Bruce: If it is a flawed rule, does it contain a grain of truth that could be refined into a better rule? And if so, what is the better rule? Or are you simply opposed to trying to define rationality and content to leave it as, I'll know it when I see it? What you shouldn't do is decide it's a good rule when applied to Kieren's induction, but not a good rule when applied to CritRat pet theories. All right, that is the end of my first part here. So I've introduced my argument with Kieren over induction. I've explained the problem I had with the overall tactics he was using, but I have not gone over his actual full arguments against Popper's corroboration, which I think do deserve more of a look, and I'm going to do that in a future podcast.

[00:56:42]  Unidentified speaker: Any questions, Peter?

[00:56:44]  Peter: Well, my only comment, Bruce, is that you seem to have a limitless well of thoughts about epistemology inside of you, and I'm happy to listen and be a part of this. It's such a unique position to be in, and I hope that I'm pretty sure it's pronounced Kieren.

[00:57:05]  Unidentified speaker: Kieren. There's a lot of variation on the name.

[00:57:08]  Bruce: Yeah, Kieren or Kieren.

[00:57:09]  Peter: I think every iteration of possibility was-

[00:57:16]  Unidentified speaker: Attempted. Got out there.

[00:57:19]  Peter: If it's not Kieren, then I guess I'll be-

[00:57:23]  Bruce: I've always pronounced it in my mind as Kieren.

[00:57:26]  Unidentified speaker: I know.

[00:57:26]  Bruce: Because there [is an E-N?] at the end.

[00:57:29]  Unidentified speaker: I'm a teacher.

[00:57:30]  Peter: I've got hundreds of names to memorize all the time. I know it's really hard when you get it in your mind that a name is pronounced one way and then you figure out it's pronounced a different way and then you can't remember what it is. So no judgment from me and I hope our audience and Kieren himself will forgive you.

[00:57:54]  Unidentified speaker: Sounds good. Okay. Thank you, Bruce. All right. Thank you. Bye-bye.

[00:58:05]  Unidentified speaker: Hello again.

[00:58:07]  Peter: If you've made it this far, please consider giving us a nice rating on whatever platform you use, or even making a financial contribution through the link provided in the show notes. As you probably know, we are a podcast loosely tied together by the Popper Deutsch Theory of Knowledge. We believe David Deutsch's Four Strands tie everything together. So we discuss science, knowledge, computation, politics, art, and especially the search for artificial general intelligence. Also, please consider connecting with Bruce on X at BNielson01. Also, please consider joining the Facebook group, The Many Worlds of David Deutsch, where Bruce and I first started connecting.

[00:58:50]  Unidentified speaker: Thank you.