To arrive at an understanding of Bayes Theorem, we begin with the
Definition of a conditional probability
P(A|B) = translates to “The probability of A happening, given that event B has occurred”.
P(A|B) can be rewritten like
P(A | B) = P(A ∩ B)
Also, P(A ∩ B) = P(A | B) × P(B) (i.e. the probability of A and B).
And because P(A ∩ B) = can also be expressed as P(B ∩ A)
We can say P(B ∩ A) = P(B | A) × P(A)
Therefore, we can also express “the probability of A happening given event B” as
P(A | B) = P(B | A) × P(A)
We can further expand the probability of B by writing it as
P(B) = P(A) × P(B | A) + P(Al) × P(B | Al )
(in normal language, the probability of B is equal to the probability of A multiplied by the probability of A given B, plus the probability of the complement of A multiplied by the probability of the complement of A given B)
Thus resulting in the Law of Total Probability.
This law enables us to find the total probability of a particular event based on conditional probabilities. Also, our new expression can be substituted into our formula for “the probability of A happening given event B” as the denominator to give us Bayes Theorem.
Bayes Theorem provides a means of finding reverse conditional probabilities when you don’t know every probability up front.
P(A | B) = P(A) × P(B | A)
P(A) × P(B | A) + P(Al) × P(B | Al)