Section 4

Expectations, Variance, and the Fundamental Bridge (BH Chapter 4)

The Expected Value (or expectation, mean) of a random variable can be thought of as the "weighted average" of the possible outcomes of the random variable. Mathematically, if x1,x2,x3,x_1, x_2, x_3,\cdots, are all of the possible values that X can take, the expected value of X can be calculated as follows:

E(XA)=ixiP(X=xiA)E(X | A) = \sum\limits_{i}x_iP(X=x_i | A)

Linearity of Expectation

The most important property of expected value is Linearity of Expectation. For any two random variables X and Y, a and b scaling coefficients and c is our constant, the following property of holds:

E(aX+bY+c)=aE(X)+bE(Y)+cE(aX + bY + c) = aE(X) + bE(Y) + c

The above is true regardless of whether X and Y are independent.

Conditional Expectations

Conditional distributions are still distributions. Treating them as a whole and applying the definition of expectation gives:

E(XA)=ixiP(X=xiA)E(X | A) = \sum\limits_{i}x_iP(X=x_i | A)

Variance

Variance tells us how spread out the distribution of a random variable is. It is defined as

Var(X)=E(XE(X))2=E(X2)(E(X))2Var(X) = E(X - E(X))^2 = E(X^2) - (E(X))^2

Properties of Variance

  • Var(cX)=c2Var(X)Var(cX) = c^2 Var(X)

  • Var(X±Y)=Var(X)+Var(Y)Var(X \pm Y) = Var(X) + Var(Y) if XX and YY are independent

Indicator Random Variables

Indicator Random Variables are random variables whose value is 1 when a particular event happens, or 0 when it does not. Let IAI_A be an indicator random variable for the event AA. Then, we have:

IA={1A occurs0A does not occurI_A = \begin{cases} 1 & \text{$$A$$ occurs} \\ 0 & \text{$$A$$ does not occur} \end{cases}

Suppose P(A)=pP(A) = p. Then, IBern(p)I \sim Bern(p) because II has a pp chance of being 1, and a 1p1-p chance of being 0.

Properties of Indicators

  • (IA)2=IA(I_A)^2 = I_A, and (IA)k=IA(I_A)^k = I_A for any power kk.

  • IAc=1IAI_{A^c} = 1 - I_A

  • IAB=IAIBI_{A \cap B} = I_A I_B is the indicator for the event ABA \cap B (that is, IAIB=1I_A I_B = 1 if and only if AA and BB occur, and 0 otherwise)

  • IAB=IA+IBIAIBI_{A \cup B} = I_A + I_B - I_A I_B

The Fundamental Bridge

The fundamental bridge is the idea that E(IA)=P(A)E(I_A) = P(A). When we want to calculate the expected value of a complicated event, sometimes we can break it down into many indicator random variables, and then apply linearity of expectation on that. For example, if X=I1+I2++InX = I_1 + I_2 + \ldots + I_n, then:

E(X)=E(I1)+E(I2)++E(In)=P(I1)+P(I2)++P(In)E(X) = E(I_1) + E(I_2) + \ldots + E(I_n) = P(I_1) + P(I_2) + \ldots + P(I_n)