> For the complete documentation index, see [llms.txt](https://justinthezhu.gitbook.io/stat110/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://justinthezhu.gitbook.io/stat110/untitled-2.md).

# Section 2

All Probability is Conditional Probability (BH Chapter 2)

**"|" is a given** - We read "|" as "given", that is, $$A|B$$ means A occurring given that B occurred.

**Conditional Probability** - Suppose we observe event *B* and are interested in the probability of event *A* occurring given this information. Then,

$$
P(A|B) = \frac{P(A \cap B)}{P(B)}
$$

**Bayes’ Rule** - This is arguably one of the most important concepts and tools you will learn in this course.

$$
P(A|B) = \frac{P(B|A)P(A)}{P(B)}
$$

**Bridging Conditional Probability and Sets**

An intuitive way to visualize conditional probability is to think about the intersection of sets. In order to find the intersection of two different sets *A* and *B*, we establish one of these sets to be our sample space and find the likely occurrence of the other set within this established sample space.

$$P(A\cap B) = P(B|A)P(A) = P(A|B)P(B)$$

$$P(A\_1\cap A\_2\cap A\_3\cap\cdots A\_n) = P(A\_1|A\_2\cap A\_3\cap\cdots A\_n)P(A\_2| A\_3\cap\cdots A\_n)\cdots P(A\_{n-1}|A\_n)P(A\_n)$$

**Law of Total Probability (LOTP)**

A common theme in this course is that it is far easier to solve a problem by breaking it down into smaller, simpler components than tackling it head-on. LOTP is one such tool. Suppose you want to find the probability of some event *B*, and you can partition the sample space into disjoint events $$A\_1,A\_2,\cdots,A\_n$$. Then,

$$P(B) = \sum\_{i=1}^n P(B | A\_i) P(A\_i)\ = \sum\_{i=1}^n P(B \cap A\_i)$$

We often use LOTP with Bayes' rule! Specifically, the denominator of Bayes' rule, $$P(B)$$, is often difficult to calculate outright, so we will instead calculate it with LOTP.

## Extra Conditioning

Incorporating extra information $$C$$ is a simple extension of Bayes' rule and LOTP:

$$
P(A|B, C) = \frac{P(A \cap B | C)}{P(B|C)}
\= \frac{P(B|A,C)P(A|C)}{P(B|C)}
$$

$$P(B|C) = \sum\_{i=1}^n P(B | A\_i, C) P (A\_i |C) = \sum\_{i=1}^n P(B \cap A\_i |C)$$

## Disjoint vs. Independent

**Disjoint**, or mutually exclusive, events are events that cannot occur simultaneously. That is, observing event *A* precludes the possibility of also observing event *B*. We can state this equivalently as $$P(A \cap B) = 0$$.

**Independent** events are events such that observing event *B* yields no information about the possibility of also observing event *A*. That is, conditioning on observing event *B*, the probability of observing event *A* is unchanged:

$$P(A|B) = P(A)$$

We can apply this result to Bayes’ rule and quickly demonstrate an alternative definition of independence:

$$P(A \cap B) = P(A)P(B)$$

Another form of independence is conditional independence. Two events *A* and *B* are said to be conditionally independent given *C* if $$P(A \cap B | C) = P(A|C) P(B|C)$$

However, just as pairwise independence does not imply conditional independence, conditional independence does not imply pairwise independence.