Indicator Random Variables and the Fundamental Bridge

This section is devoted to indicator random variables, which we have encountered previously but will treat in much greater detail here.

In particular, we will show that indicator r.v.s are an extremely useful tool for calculating expected values. Recall from the previous chapter that the indicator r.v. $I_A$ (or $I(A)$) for an event $A$ is defined to be 1 if $A$ occurs and 0 otherwise. So $I_A$ is a Bernoulli random variable, where success is defined as ''$A$ occurs'' and failure is defined as ''$A$ does not occur''. Some useful properties of indicator r.v.s are summarized below.

Theorem: Indicator r.v. Properties

Let $A$ and $B$ be events. Then the following properties hold.

1. $(I_A)^k = I_A$ for any positive integer $k$.
2. $I_{A^c} = 1 - I_A$.
3. $I_{A \cap B} = I_A I_B$.
4. $I_{A \cup B} = I_A + I_B - I_A I_B$

Proof:

1. Property 1 holds since $0^k=0$ and for any positive integer $k$.

2. Property 2 holds since $1-I_A$ is $1$ if $A$ does not occur and 0 if $A$ occurs.

3. Property 3 holds since $I_AI_B$ is $1$ if both $I_A$ and $I_B$ are 1, and 0 otherwise.

4. Property 4 holds since

   $$I_{A \cup B} = 1 - I_{A^c \cap B^c} = 1 - I_{A^c}I_{B^c} = 1 - (1-I_A)(1-I_B) = I_A+I_B-I_AI_B.$$

Indicator r.v.s provide a link between probability and expectation; we call this fact the fundamental bridge.

Theorem: Fundamental Bridge between Probability and Expectation

There is a one-to-one correspondence between events and indicator r.v.s, and the probability of an event $A$ is the expected value of its indicator r.v. $I_A$:

$$P(A)=E(I_A).$$

Proof:

The fundamental bridge connects events to their indicator r.v.s, and allows us to express any probability as an expectation.

Conversely, the fundamental bridge is also extremely useful in many expected value problems. We can often express a complicated discrete r.v. whose distribution we don't know as a sum of indicator r.v.s, which are extremely simple. The fundamental bridge lets us find the expectation of the indicators; then, using linearity, we obtain the expectation of our original r.v.

Recognizing problems that are amenable to this strategy and then defining the indicator r.v.s takes practice, so it is important to study a lot of examples and solve a lot of problems. In applying the strategy to a random variable that counts the number of [noun]s, we should have an indicator for each potential [noun]. This [noun] could be a person, place, or thing; we will see examples of all three types.

Example Putnam Problem

Solution:

Law of the Unconscious Statistician (LOTUS)

As we saw in the St. Petersburg paradox, $E(g(X))$ does not equal $g(E(X))$ in general if $g$ is not linear. So how do we correctly calculate $E(g(X))$?

Since $g(x)$ is an r.v., one way is to first find the distribution of $g(x)$ and then use the definition of expectation. Perhaps surprisingly, it turns out that it is possible to find $E(g(X))$ directly using the distribution of $X$, without first having to find the distribution of $g(X)$. This is done using the law of the unconscious statistician (LOTUS).

Theorem: LOTUS

If $X$ is a discrete r.v. and $g$ is a function from $\mathbb{R}$ to $\mathbb{R}$, then

$$E(g(X)) = \sum_x g(x) P(X=x),$$

where the sum is taken over all possible values of $X$.

This means that we can get the expected value of $g(X)$ knowing only $P(X=x)$, the PMF of $X$; we don't need to know the PMF of $g(X)$.

The name comes from the fact that in going from $E(X)$ to $E(g(X))$ it is tempting just to change $x$ to $g(X)$ in the definition, which can be done very easily and mechanically, perhaps in a state of unconsciousness. On second thought, it may sound too good to be true that finding the distribution of $g(X) is not needed for this calculation, but LOTUS says it is true. We will omit a general proof of LOTUS, but let's see why it is true in some special cases.

Let $X$ have support $0,1,2,\dots$ with probabilities $p_0,p_1,p_2,\dots$, so the PMF is $P(X=n)=p_n$. Then $X^3$ has support $0^3,1^3,2^3,\dots$ with probabilities $p_0,p_1,p_2,\dots,$ so

As claimed by LOTUS, to edit the expression for $E(X)$ into an expression for $E(X^3)$, we can just change the $n$ in front of the $p_n$ to an $n^3$; the $p_n$ is unchanged, and we can still use the PMF of $X$.