Independent Events - Data 89 Course Notes

In the Section 1.5 we learned how to compute conditional probabilities. In this section we’ll use those definitions to introduce the last fundamental idea in this chapter: independence. We will conclude by deriving the last probability rule for the course.

What do we mean when we say two events are independent?

When we say two events are independent, we mean that they are entirely unrelated. They do not influence one another. Knowing something about one teaches you nothing about the other.

That last sentence is suggestive. In the last section we saw that conditional probabilities provide a natural model for information. Conditioning on an event changes the probabilities of other events. Those changes represent what we learn when we condition. For example, we saw that, knowing the second card drawn in a pair was an ace would change the chance the first card drawn was an ace.

So, here’s an informal definition of independence: two events are independent if, knowing the outcome of one, no matter the outcome, would not change the distribution of outcomes for the first event.

Here’s the same definition stated formally:

Generalization

We can extend the same definition to other partitions of $\Omega$ . Suppose that $\{A_1,A_2,..., A_n\}$ and $\{B_1,B_2,...,B_m\}$ both partition $\Omega$ . These could represent different categories, for example, you pick a word at random from the dictionary. The first category corresponds to the first letter of the word. The second category corresponds to the second letter of the word. For example, if $\omega \in A_1 \cap B_4$ then the word starts “ad”. Suppose that you get to observe the second letter, e.g. we know whether $\omega \in B_1$ or $B_2$ , etc. Then, the assignment to categories $\{A_j\}_{j=1}^n$ is independent of the assignment to categories $\{B_i\}_{i=1}^m$ if:

\text{Pr}(A_j|B_i) = \text{Pr}(A_j|B_k) \text{for all } j \text{ and all } i \neq k

(2)

That is, knowing the second category tells us nothing about the first category. This is obviously not true for succesive letters in English, since, for instance, a word starting in “q” almost always starts “qu”.

There is a simpler way to express this same idea. If observing one event $B$ teaches us nothing about a different event $A$ , then the conditional probability of $A$ given $B$ must be the same as the marginal probability of $A$ . Since information flows both ways between related events, the same must hold for $B$ regarding $A$ . Therefore, we say two events are independent if:

Multiplication Rule for Independent Events¶

When two events are independent the multiplication rule simplifies. It is always true that:

\text{Pr}(A,B) = \text{Pr}(A) \text{Pr}(B|A)

(4)

However, if $A$ and $B$ are independent then $\text{Pr}(B|A) = \text{Pr}(B)$ so:

\text{Pr}(A,B) = \text{Pr}(A) \text{Pr}(B)

(5)

This is our last probability rule. Its really a special case of an existing rule, but, it’s so useful we’ll highlight it:

This is an extremely useful result since it makes calculating joint probabilities straightforward; just multiply the marginals. It is often used to check for independence:

Joint Probability Tables¶

Suppose that $\{A_j\}_{j=1}^3$ , $\{B_i\}_{i=1}^2$ are two different partitions of the space of possible outcomes, and membership in the $A$ categories is independent of membership in the $B$ categories. Then, all the joint probabilities are products of the marginal probabilities. Therefore, the associated joint probability table will take the form:

Event	$A_1$	$A_2$	$A_3$	Marginals
$B_1$	$p_{A_1} \times p_{B_1}$	$p_{A_2} \times p_{B_1}$	$p_{A_3} \times p_{B_1}$	$p_{B_1}$
$B_2$	$p_{A_1} \times p_{B_2}$	$p_{A_2} \times p_{B_2}$	$p_{A_3} \times p_{B_2}$	$p_{B_2}$
Marginals	$p_{A_1}$	$p_{A_2}$	$p_{A_3}$	1

We can use this rule to fill in the joint entries knowing only the marginals. For instance, given:

Event	$A_1$	$A_2$	$A_3$	Marginals
$B_1$	3/8	?	?	3/4
$B_2$	?	?	?	1/4
Marginals	1/2	1/3	1/6	1

Try filling in a missing joint probability.

Solution

The joint entries must be:

Event	$A_1$	$A_2$	$A_3$	Marginals
$B_1$	3/8	1/4	1/8	3/4
$B_2$	1/8	1/12	1/24	1/4
Marginals	1/2	1/3	1/6	1

Be careful: ⚠️ multiplying marginals only works if the events are independent!

Examples¶

We’ve already seen one example of independent events. In Section 1.2 we calculated the probability that the first two draws from a thoroughly shuffled deck are an ace, then a spade. We found that:

\text{Pr}(AS) = \frac{12 + 3 \times 13}{52 \times 51} = \frac{51}{52 \times 51} = \frac{1}{52}.

(8)

Just as we saw for the chance of two successive aces, we can expand this calculation by thinking sequentially. On the first draw we either get an ace, or we don’t. On the second draw we either get a spade, or we don’t. Since there is one ace in every suit, and the suits match in size, knowing whether or not we drew an ace will not change the probability that we draw a spade. Therefore, the events are independent. It follows that:

\text{Pr}(AS) = \text{Pr}(A) \text{Pr}(S|A) = \text{Pr}(A) \text{Pr}(S) = \frac{4}{52} \times \frac{13}{52} = \frac{4}{52} \times \frac{1}{4} = \frac{1}{52}.

(9)

1.6 Independent Events

Multiplication Rule for Independent Events¶

Joint Probability Tables¶

Examples¶