Some Bayesian statistics

Mathematics isn’t always counterintuitive.

Among our consultants, we are pretty much agreed that more nonsense has been written by intelligent people on probability than on any other subject.  (Actually, more nonsense has been written on quantum mechanics, but most of it by people with no actual knowledge of the field.)  We do not exclude such venerated peers as Sir Karl Popper and Jacob Bronowski, though we’re not going to talk about either one here.  For the moment, we’ll take one problem from an Advanced Placement Statistics prep book, in principle written by an expert on the subject.  We summarize and modify the numbers a bit, to make them more manageable, but we’ve not distorted anything in the process.

We are given that a manufacturing process produces five defective products out of every hundred.  We monitor the start of the next run.  The first product is not defective.  What is our estimate of the probability that the next one is defective?

For all practical purposes, we can retain our 5% estimate.  Most of the products will be fine, and the fact that one comes out well should be no surprise, and not require a change in thinking.  But our author contends that there is a small change: since five of every hundred come out defective, and there are now only 99 left, there is a slightly larger chance that the next will contain a flaw: so our estimate must now be 5/99.

This is the Gambler’s Ruin fallacy, one with which the author should have been familiar.  It tacitly assumes some deterministic process that somehow unfailingly mars five out of every hundred; as the ruined Gambler assumes that, having had five bad hands, the next one must be good to restore the average.  The author notes that his answer implies that the manufacturing process is not random, apparently unaware of the fact that it’s his error than has done it.

What is the right approach?  Well, we have one new piece of information, a good product.  We might expect that our probability estimate of a defect should decrease, though only slightly.  The important question is: where did our original estimate of 5% come from?  It makes a real difference.  If we had noted five defective products in the previous hundred, we can go through the math and say our best estimate is now 5 out of 101, a slightly smaller chance.  If we had noted fifty in the previous thousand, we come to 50/1001; less of a difference, as we expect if our knowledge base is larger.

But our tutor sees a chance to bring in Bayesian Statistics.  We’re not going to give a full exposition of that field here, but consider turning the question around: instead of being given the probability of defective parts and asked to calculate data, start with the data and calculate the probability of a defective part.  If we’ve had five out of a hundred, going through a lot more math we find the answer peaks at 5%, which should not be surprising.  But a probability of 4% is almost as certain, as is one of 6%; either will give 5/100 almost as often as 5%.  We find we don’t really know our starting number as well as a textbook would imply.  That may be why the AP Statistics course does not include Bayes.