Essay talk:Bayesian Inference and the Power of Skepticism

Can't wait for more of this - it's all new to me but it's fascinating so far Keep ^{me in the DARK} 14:31, 9 August 2007 (CDT)

How to present the introduction[edit]

I've been asked to comment on this; probability is (with probability 0.845392) the area of math that I am weakest on. I don't really speak the language. I finally (sort of) understood what was going on when I realized the following principle: Probability theory can only go from fundamental (usually unseen) reality to observations that we make. I guess what I call "fundamental reality" what you call "hypothesis", or "h", and what I call "observation" you call "data", or "d". The similarity of "observation" and "data" is trivial, but the I think the notion of "fundamental reality" is, well, fundamental. Neither the observation that someone is drunk nor the observation that they had 10 beers is fundamental reality.

Here's what I mean. We can't see the fundamental reality; we just know that it controls the observations. The math that we study is about manipulating various combinations of reality, and seeing how it affects the observations, but you can never really go the other way. Example: If we know (fundamental reality; don't ask how we know it) that a coin has a 1/2 probability of coming up heads, and we know that a die has a 1/6 probability of coming up "3", we can work out, on solid mathematical grounds, what the probabilities are, of such things as getting 7 consecutive heads, or tails+3 when we flip the coin and roll the die, etc. And we can work out all the Pascal's triangle stuff about binomial distributions etc. etc. when we flip the coin or roll the die many times. But, in most cases, we don't really know the probability. In the case of the coin or the die, we can use physical symmetry to argue that we really do know the probability, and hence use that to illustrate the things we want to illustrate, but in the real world, we don't know the probability, say, that drug X will cure disease Y. And, if we don't know the shape of the die (it's just someone behind a curtain calling out numbers), figuring out its probability is a tricky, but important, task.

The thing we want to do, of course, is to go the other way, observation to underlying reality, but we can't. All we can do is calculate "If the probability that drug X cures disease Y is P or less, what is the probability of the outcome we observed in our clinical trial?" And we find a value of P such that, based on sound mathematics, the probability of observing what we observed is very low. When it is low enough, we use our human intuition to say that "It follows that the probability that X cures Y must be 70% plus or minus whatever."

Anyway, I hope my insights can be helpful, though you clearly know a lot about the subject. SJIHAS 15:18, 19 August 2007 (CDT)

I think you have described the standard statistical model fairly clearly. The idea that we calculate the likelihood of our data given our hypothesis (sorry I only speak probability language in the science dialect) and if its sufficiently high while the "null" hypothesis is sufficiently low we can make some sort of intuitive claim about the our hypothesis. Or in the popperian sense we have failed to falsify our alternative hypothesis.

But Bayesian inference attempts to link the likelihood probability directly with the probability of our hypothesis with out having to make a link to "intuition" or "significance" cut offs. Bayes equation is:

With P(Ai | A) being our posterior or our probability of our hypothesis given our data (our goal) and P(Ai) being a prior probability and then P(A | Ai) being the likelihood.

So the likelihood probability is what you just described and its where frequentist stop the proccess. And you are right that we can't really move much further with just the likelihood probability, but if we have a prior we can mathematically relate that likelihood probability to a posterior probability or our "gold standard" of an actual probability for our hypothesis.

Makes sense right? tmtoulouse ^pester 15:37, 19 August 2007 (CDT)

Very difficult (for liberal arts types, maybe)[edit]

“”This is often a very difficult subject for statistical novices to grasp intuitively.

Not really. It is difficult for people who are not good in math. Everybody that was in my probability class "got" this the day it was explained. They were mostly physics, computer science, and math majors though. Quick Comment 15:06, 27 August 2007 (CDT)

*shrug* it depends how its presented, but I couldn't keep track of the number of undergrads I have had in lab classes that can't figure this stuff out, but thats psychology and biology. Also I am attempting to write this for an audience with very little statistics at all, not sure how that working out. tmtoulouse ^pester 15:10, 27 August 2007 (CDT)

Simply describe it as "initially counterintuitive". Most intuitively fail to predict the Monty Hall Problem (and the Birthday paradox) until these are specifically taught. Quick Comment 15:13, 27 August 2007 (CDT)

Works for me. tmtoulouse ^pester 15:22, 27 August 2007 (CDT)

Ugh[edit]

“”Essentially this axiom stats that the probability for X and the probability for not X must sum to 1. If this axiom is obeyed the hypothesis that predicts everything is prima facie in violation and is not considered a valid hypothesis.

If P(X)=1, and P(~X)=0, P(X)+P(~X)=1. In any event, the wording here is awkward (and the idea you're trying to convey may be incorrect--but I can't say I understand the idea you're trying to convey). I would use "premise" "condition is met" and show how P(X)+P(~X) /= 1 in your example. Quick Comment 15:25, 27 August 2007 (CDT)

I agree i muddled several ideas into one here, I will think about how to make it clearer. tmtoulouse ^pester 15:59, 27 August 2007 (CDT)

Karl Popper's bright line[edit]

You should be careful here; Popper considered the theory of evolution as non-scientific under falsifiability criteria (though he said many of the underlying theories were scientific). In any event, my understanding is the anti-Darwinians quoted Popper first, and argued that ToE alternatives are equally non-scientific (specifically, metaphysical). Popper of course was used against them in some trial. And other philosophers of science disagree with Popper and his bright line (to the advantage of ToE proponents). Quick Comment 15:50, 27 August 2007 (CDT)

Well, Popper of course later retracted that statement completely, and one can accept Popper's ideas about demarcation without needing Popper to say where individual theories fall under the demarcation. tmtoulouse ^pester 16:01, 27 August 2007 (CDT)

Also I am not really putting forth Popper as the sole authority on demarcation, only bringing in how his ideas fit into Bayesian inference and side on the skeptic and scientist perspective. There are other themes in the philosophy of science I have worked in here too. tmtoulouse ^pester 16:06, 27 August 2007 (CDT)

I would like to see where "Popper...later retracted that statement completely". In my recollection of a reading of the retraction, he hedged. Maybe there was another retraction that was "complete." Quick Comment 16:16, 27 August 2007 (CDT)

Skeptical Inquire addressed this a few years back tmtoulouse ^pester 16:27, 27 August 2007 (CDT)

The (without quote marks in your link) "quite a bit more to the theory than he had understood from a cursory examination of the subject" is a more accurate portrayal of what Karl Popper wrote, if my memory is correct. In 1980 he wrote "their hypotheses can in many cases be tested"--Popper, Karl (1980), "Letter to the Editor", New Scientist 87, page 611--as cited by Numbers, Ronald L. (1992), Creationists: The Evolution of Scientific Creationism page 247. Ronald Numbers does not seem to take the stance of "complete" retraction, and the creationists do not either (with some merit, it seems). I'll have to get the Dialectica 32:344-346 to see if evolutionists are quote mining again. Primary sources are better in my opinion. Quick Comment 17:09, 27 August 2007 (CDT)

Context[edit]

Here it is in as much context as need I believe:

“”

The fact that the theory of natural selection is difficult to test has led some people, anti-Darwinists and even some great Darwinists, to claim that it is a tautology. A tautology like “All tables are tables” is not, of course, testable; nor has it any explanatory power. It is therefore most surprising to hear that some of the greatest contemporary Darwinists themselves formulate the theory in such a way that it amounts to the tautology that those organisms that leave most offspring leave most offspring. And C. H. Waddington even says somewhere (and he defends this view in other places) that “Natural selection . . . turns out . . . to be a tautology”. However, he attributes at the same place to the theory an “enormous power . . . of explanation”. Since the explanatory power of a tautology is obviously zero, something must be wrong here.

Yet similar passages can be found in the works of such great Darwinists as Ronald Fisher, J. B. S. Haldane, and George Gaylord Simpson; and others. I mention this problem because I too belong among the culprits. Influenced by what these authorities say, I have in the past described the theory as “almost tautological”, and I have tried to explain how the theory of natural selection could be untestable (as is a tautology) and yet of great scientific interest. My solution was that the doctrine of natural selection is a most successful metaphysical research programme. It raises detailed problems in many fields, and it tells us what we would expect of an acceptable solution of these problems.

I still believe that natural selection works this way as a research programme. Nevertheless, I have changed my mind about the testability and the logical status of the theory of natural selection; and I am glad to have an opportunity to make a recantation. My recantation may, I hope, contribute a little to the understanding of the status of natural selection. What is important is to realize the explanatory task of natural selection; and especially to realize what can be explained without the theory of natural selection.

We may start from the remark that, for sufficiently small and reproductively isolated populations, the Mendelian theory of genes and the theory of mutation and recombination together suffice to predict, without natural selection, what has been called “genetic drift”. If you isolate a small number of individuals from the main population and prevent them from interbreeding with the main population, then, after a time, the distribution of genes in the gene pool of the new population will differ somewhat from that of the original population. This wilL happen even if selection pressures are completely absent. Moritz Wagner, a contemporary of Darwin, and of course a pre-Mendelian, was aware of this situation. He therefore introduced a theory of evolution by genetic drift, made possible by reproductive isolation through geographical separation. In order to understand the task of natural selection, it is good to remember Darwin’s reply to Moritz Wagner. Darwin’s main reply to Wagner was: if you have no natural selection, you cannot explain the evolution of the apparently designed organs, like the eye. Or in other words, without natural selection, you cannot solve Paley’s problem.

In its most daring and sweeping form, the theory of natural selection would assert that all organisms, and especially all those highly complex organs whose existence might be interpreted as evidence of design and, in addition, all forms of animal behaviour, have evolved as the result of natural selection; that is, as the result of chance-like inheritable variations, of which the useless ones are weeded out, so that only the useful ones remain. If formulated in this sweeping way, the theory is not only refutable, but actually refuted. For not all organs serve a useful purpose: as Darwin himself points out, there are organs like the tail of the peacock, and behavioural programmes like the peacock’s display of his tail, which cannot be explained by their utility, and therefore not by natural selection. Darwin explained them by the preference of the other sex, that is, by sexual selection. Of course one can get round this refutation by some verbal manoeuvre: one can get round any refutation of any theory. But then one gets near to rendering the theory tautological. It seems far preferable to admit that not everything that evolves is useful, though it is astonishing how many things are; and that in conjecturing what is the use of an organ or a behavioural programme, we conjecture a possible explanation by natural selection: of why it evolved in the way it has, and perhaps even of how it evolved. In other words, it seems to me that like so many theories in biology, evolution by natural selection is not strictly universal, though it seems to hold for a vast number of important cases. According to Darwin’s theory, sufficiently invariant selection pressures may turn the otherwise random genetic drift into a drift that has the appearance of being purposefully directed. In this way, the selection pressures, if there are any, will leave their imprint upon the genetic material. (It may be mentioned, however, that there are selection pressures that can operate successfully over very short periods: one severe epidemic may leave alive only those who are genetically immune.) I may now briefly sum up what I have said so far about Darwin’s theory of natural selection.

The theory of natural selection may be so formulated that it is far from tautological. In this case it is not only testable, but it turns out to be not strictly universally true. There seem to be exceptions, as with so many biological theories; and considering the random character of the variations on which natural selection operates, the occurrence of exceptions is not surprising. Thus not all phenomena of evolution are explained by natural selection alone. Yet in every particular case it is a challenging research programme to show how far natural selection can possibly be held responsible for the evolution of a particular organ or behavioural programme.

I do not see how any of that can remotely support the creationist position. And again I stand by the point that regardless of what Popper says the tautological nature of natural selection or the scientific status thereof is not defined by his approval or disapproval. tmtoulouse ^pester 17:37, 27 August 2007 (CDT)

That evolutionism is metaphysics? Quick Comment 11:12, 28 August 2007 (CDT)

Uh, no it's not. tmtoulouse ^pester 13:16, 28 August 2007 (CDT)

My comment was an explanation of how ID/creationists might use what Popper wrote against the scientific status of theory of evolution (ism). When you say "it's not," I assume you mean that evolutionism is not metaphysics, and not that Popper did not classify it as such. Quick Comment 13:40, 28 August 2007 (CDT)

I don't think Popper says that no. tmtoulouse ^pester 00:48, 29 August 2007 (CDT)

Another recommendation.[edit]

Assume some prior knowledge on the part of your readers, and/or point them to other peviously written articles for primers on probability. Then delete the tutorial, and focus on the point of your paper. Quick Comment 15:52, 27 August 2007 (CDT)

I will think about it, but want to finish the essay first. tmtoulouse ^pester 16:02, 27 August 2007 (CDT)

Yeah, I was guessing it might help you finish to essay faster. But then again, if it helps organize your thoughts, it's probably worthwhile. Quick Comment 16:17, 27 August 2007 (CDT)

Some other explanations of Bayes' theorem[edit]

The best explanation[edit]

http://lesswrong.com/lw/2b0/bayes_theorem_illustrated_my_way/ (Visual)