The Majesty of Bayes' Theorem
Math when applied to real life possesses a beautiful way of opening up the mind to a different way of seeing the world or the situation facing you. Bayes' Theorem provides a logical way to look at a situation and the available facts and to put things into a clearer perspective. I am by no means a mathematician, but recently, I have found myself developing a crush on Bayes' Theorem. It may sound strange to say I have feelings for a theorem. I will explain what I mean, and don’t be surprised if you too may find your thoughts and attentions wandering back to Bayes' Theorem.
The theorem derives its name from the Presbyterian minister, philosopher and statistician Reverend Thomas Bayes'. Thomas was born into a nonconformist family in England around 1701. After studying logic at the University of Edinburgh he served at the family chapel in London before taking up the ministry at Mount Sion Chapel in Kent where he would serve for nearly 20 years. During his life, he published two works. One was a philosophical work titled “Divine Benevolence, or an Attempt to Prove That the Principal End of the Divine Providence and Government is the Happiness of His Creatures.” The other was a defense of calculus, a branch of mathematics developed by Isaac Newton. Later in life he pursued his interest in probability and developed his theorem. Interestingly, Bayes' never published his theorem. After his death at the age of 59, Bayes’ writings were given to Richard Price. Price summarized Bayes' work and published it posthumously.
Bayes' theorem in short combines normal probability with conditional probability to calculate the chance of something being true. Normal probability is the chance of something happening if it is done over and over again. For example, flipping a two-sided coin has a 50:50 or 1 in 2 chance of coming up heads. It is expressed mathematically at P(A). Conditional probability on the other hand is the chance that some event happening given an observation that some other event has happened. For example, what is the probability that the red Camaro you see going down the road is the same red Camaro that was stolen from you. This is expressed as P(A|D) or the likelihood of A being true given the observation of D. Bayes' Theorem uses these notations and is written as follows.
The genius of Bayes' Theorem is that it finds the likelihood of A happening given the observation of D by tuning over the conditional probability and using inverse of what you are looking for. In other words, the probability of D given A.
Equations can be intimidating, so I will walk through an example. Say it is November first, early in the morning, and your mom is driving you to school. The night before you and you brother had a great night of trick or treating and afterward came home and compared the candy you both received. Amazingly, you both came home with a good haul of candy, but oddly you had 29 Snickers and only 2 Reese’s Peanut Butter Cups and your brother received 15 Snickers and 13 Reese’s Peanut butter cups. Your parents wouldn’t let you have any candy before going to bed and had you put your candy in two identical jars. Just before arriving at school, your brother peeks into his lunch and says, “Mmmm great, a Reese’s.” Knowing how few Reese’s you have in your jar at home and that it is your favorite candy, you ask your mom which jar she took the Reese’s from. She says she does not know. Panicked that you may have even fewer Reese’s in your jar back home, you rush to Bayes' Theorem to calm your nerves.
Here is what you know—there are two jars. So, just like the coin flip normal probability applies. There is a 1:2 chance the candy came from your jar “D”. You also know that your jar has only 2 Reese's in it so the likelihood of your mom pulling a Reese’s from your jar is a 2 in 31 chance. This is your conditional probability. Now all you need to figure is the chance of getting a Reese’s from your brother’s jar which is 13 out of 28 pieces of candy. Using this information you can calculate using Bayes' Theorem there is only an 13% chance that Reese’s came from your candy collection. A 13 out of 100 chance is low. That would be a very long shot at the race track. You can relax about your candy collection. When faced with unknowns sometimes a little reflection and some statistical analysis can really make the situation more bearable.
Dr. Smith’s career in scientific and information research spans the areas of bioinformatics, artificial intelligence, toxicology, and chemistry. He has published a number of peer-reviewed scientific papers. He has worked over the past seventeen years developing advanced analytics, machine learning, and knowledge management tools to enable research and support high level decision making. Tim completed his Ph.D. in Toxicology at Cornell University and a Bachelor of Science in chemistry from the University of Washington.
You can buy his book on Amazon in paperback here and in kindle format here.