## How counter-intuitive can you be?

Posted: April 25, 2014 in Educational Material
Tags: , ,

More often than not science (and statistics especially) is counter-intuitive. Since the human world is built on and maintained by science (and to a larger and larger extent, on and by statistics), this should tell us something about the value of “common sense” (of which I am not particularly fond of). I was just reminded of that by an excellent paper by Zhu and Lu (2004), where the authors present, with students in mind, the strongly counter-intuitive/confusing/hard-to-believe, case of the uninformative prior distribution in the context of Bayesian estimation related to the apparently simple case of a Bernoulli random variable where we want to estimate the probability that the variable will take the value 1.

Now almost everybody (myself included), using -what else- common sense, view a Uniform prior (ranging in (0,1) in our case), as the bona fide uninformative one (which goes back to The Principle of Insufficient Reason). In our case, using such a prior distribution reflects a prior belief that the probability we want to estimate can take any value in (0,1) with equal probability –how more uninformative can you be?

Oh, but you can. In fact Zhu and Lu paper shows clearly that for the case at hand, such a prior  influences rather distinctly the posterior results -and so it is not-uninformative at all. They also derive the actual uninformative prior for this case.