---
description: Conditional Expectation, LLN, and CLT (BH Chapter 9)
---

# Section 9

**Conditioning on an Event** - We can find the expected value of _Y_ given that event _A_ or _X_ = _x_ has occurred.

This would be finding the values of _E_(_Y_|_A_) and _E_(_Y_|_X_ = _x_). Note that conditioning in an event results in a _number_. Note the similarities between regularly finding expectation and finding the conditional expectation. The expected value of a dice roll given that it is prime is $$\frac{1}{3}2 + \frac{1}{3}3 + \frac{1}{3}5 = 3\frac{1}{3}$$. We are still taking an average.

**Conditioning on a Random Variable** - We can also find the expected value of _Y_ given the random variable _X_. The resulting expectation, _E_(_Y_|_X_) is _not a number but a function of the random variable X_. For an easy way to find _E_(_Y_|_X_), find _E_(_Y_|_X_ = _x_) and then plug in _X_ for all _x_. This changes the conditional expectation of _Y_ from a function of a number _x_, to a function of the random variable _X_.

## Law of Total Expectation

This is an extension of the _Law of Total Probability_. For any set of events $$B_1, B_2, B_3, ... B_n$$ that partition the sample space (simplest case being {$$B$$, $$B^c$$}):

$$
E(X) = \sum_{x} x P(X=x)= \sum_{x} x \sum_{i} P(X =x| B_i) P(B_i) = \sum_i \sum_x xP(X=x|B_i) P(B_i) = \sum_i E(X|B_i)P(B_i)
$$

## Properties of Conditional Independence

**Independence** - if _X_ and _Y_ are independent, then $$Y | X \sim X$$ as conditioning on $$X$$ gives us no additional information about $$Y$$:

$$
E(Y | X) = E(Y)
$$

**Taking out what’s Known** - If we are finding the expectation that involves some function of $$h(X)$$ and we're conditioning on $$X$$, then we can treat $$h(X)$$ as a constant because $$X$$ is known. $$E(h(X) Y | X) = h(X) E(Y | X)$$

**Linearity** - We have linearity in the first term:

$$E(aY_1 + bY_2 | X) = a E(Y_1 | X) + bE(Y_2 | X)$$

**Law of Iterated Expectation (Adam’s Law)** - For any two random variables _X_, _Y_,

$$E(E(Y | X)) = E(Y)$$

We can prove the law of iterated expectation with the law of total expectation. Let $$g(X) = E(Y|X)$$ since $$E(Y|X)$$ is a random variable. Then,

$$
E(E(Y|X)) = E(g(X)) = \sum_{x} g(X) \cdot P(X = x) = \sum_x E(Y | X=x) \cdot P(X = x) = E(Y)
$$

You can also do Adam's Law with extra conditioning: $$E(Y |X) = E(E(Y |X, Z)|X)$$

**Eve’s Law** - For any two random variables _X_, _Y_:

$$Var{(Y)} = E_X( Var{(Y | X)}) + Var_X{(E(Y | X))}$$

Both $$Var{(Y | X)}$$ and $$E(Y | X)$$ are **functions** of the random variable $$X$$.

## Law of Large Numbers (LLN)

Let us have $$X_1, X_2, X_3 \dots$$ be i.i.d.. We define $$\bar{X}_n = \frac{X_1 + X_2 + X_3 + \dots + X_n}{n}$$ The Law of Large Numbers states that as $$n \longrightarrow \infty$$, $$\bar{X}_n \longrightarrow E(X)$$.

## Central Limit Theorem (CLT)

**Approximation using CLT**: We use $$\dot{\,\sim\,}$$ to denote _is approximately distributed_. We can use the central limit theorem when we have a random variable, _Y_ that is a sum of _n_ i.i.d. random variables with _n_ large. Let us say that $$E(Y) = \mu_Y$$ and $$Var(Y) = \sigma^2_Y$$. Then,

$$
Y \dot{\,\sim\,} N(\mu_Y, \sigma^2_Y)
$$

When we use central limit theorem to estimate $$Y$$, we usually have $$Y = X_1 + X_2 + \dots + X_n$$ or $$Y = \bar{X}_n= \frac{1}{n}(X_1 + X_2 + \dots + X_n)$$. Specifically, if we say that each of the $$X_i$$ have mean $$\mu_X$$ and $$\sigma^2_X$$, then we have the following approximations.

$$
X_1 + X_2 + \dots + X_n \dot{\,\sim\,} N(n\mu_X, n\sigma^2_X)
$$

$$
\bar{X}_n = \frac{1}{n}(X_1 + X_2 + \dots + X_n) \dot{\,\sim\,} N(\mu_X, \frac{\sigma^2_X}{n})
$$

**Asymptotic Distributions using CLT**: We use $$\xrightarrow{d}$$ to denote _converges in distribution to_ as $$n \longrightarrow \infty$$. These are the same results as the previous section, only letting $$n \longrightarrow \infty$$ and not letting our normal distribution have sany $$n$$ terms. $$\frac{1}{\sigma\sqrt{n}} (X_1 + \dots + X_n - n\mu_X) \xrightarrow{d} N(0, 1)$$

$$\frac{\bar{X}_n - \mu_X}{\frac{\sigma}{\sqrt{n}}} \xrightarrow{d} N(0, 1)$$
