Visualizing Bayes Theorem
I recently came up with what I think is an intuitive way to explain Bayes’ Theorem. I searched in google for a while and could not find any article that explains it in this particular way.
Of course there’s the wikipedia page, that long article by Yudkowsky, and a bunch of other explanations and tutorials. But none of them have any pictures. So without further ado, and with all the chutzpah I can gather, here goes my explanation.
Probabilities
One of the easiest ways to understand probabilities is to think of
them in terms of Venn
Diagrams. You basically have a Universe with all the possible
outcomes (of an experiment for instance), and you are interested in
some subset of them, namely some event. Say we are studying cancer,
so we observe people and see whether they have cancer or not. If we
take as our Universe all people participating in our study, then there
are two possible outcomes for any particular individual, either he has
cancer or not. We can then split our universe in two events: the event
“people with cancer” (designated as
So what is the probability that a randomly chosen person has cancer?
It is just the number of elements in
Since
Good so far? Okay, let’s add another event. Let’s say there is a new screening test that is supposed to measure something. That test will be “positive” for some people, and “negative” for some other people. If we take the event B to mean “people for which the test is positive”. We can create another diagram:
So what is the probability that the test will be “positive” for a
randomly selected person? It would be the number of elements of
Note that so far, we have treated the two events in isolation. What happens if we put them together?
We can compute the probability of both events occurring (
But this is where it starts to get interesting. What can we read from the diagram above?
We are dealing with an entire Universe (all people), the event
Now, the question we’d like answered is “given that the test is
positive for a randomly selected individual, what is the probability
that said individual has cancer?”. In terms of our Venn diagram, that
translates to “given that we are in region
So what is it? Well, it should be
And if we divide both the numerator and the denominator by
we can rewrite it using the previously derived equations as
What we’ve effectively done is change the Universe from
Now let’s ask the converse question “given that a randomly selected
individual has cancer (event
Now we have everything we need to derive Bayes' theorem, putting those two equations together we get
which is to say
Which is Bayes' theorem. I have found that this Venn diagram method lets me re-derive Bayes' theorem at any time without needing to memorize it. It also makes it easier to apply it.
Example
Take the following example from Yudowsky:
1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get positive mammograms. 9.6% of women without breast cancer will also get positive mammograms. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer?
First of all, let’s consider the women with cancer
Now add the women with positive mammograms, note that we need to cover
80% of the area of event
It is clear from the diagram that if we restrict our universe to
Note that the efficacy of the test is given from the context of
Even without an exact Venn diagram, visualizing the diagram can help us apply Bayes' theorem:
-
1% of women in the group have breast cancer.
P(A)=0.01 -
80% of those women get a positive mammogram, and 9.6% of the women without breast cancer get a positive mammogram too.
P(B)=0.8P(A)+0.096(1−P(A))P(B)=0.008+0.09504P(B)=0.10304 -
we can get
P(B|A) straight from the problem statement, remember 80% of women with breast cancer get a positive mammogram.P(B|A)=0.8
Now let’s plug those values into Bayes' theorem
which is 0.0776 or about a 7.8% chance of actually having breast cancer given a positive mammogram.