Section 1

The Language of Probability (BH Chapter 1)

Like the English language, the language of probability has its own nouns, verbs, and adjectives. Confusing these parts of speech will result in "category errors." Let's familiarize ourselves with these parts of speech.

• Probability will always be a value between 0 and 1 inclusive that expresses how likely a particular event will occur out of all possible outcomes in the sample space. Moreover, $P(\Omega) = 1$.

• Experiment: An experiment is a process of obtaining outcomes about an uncertain phenomenon.

• Sample space ($\Omega$): A sample space contains all the possible experimental outcomes that could happen.

• Event: An event is a certain subset of the sample space.

• Naive Definition - If all outcomes are equally likely, the probability of event {A} happening is

$$P_{{naive}}(A) = \frac{{number\ of\ outcomes\ favorable\ to\ {A}}}{{number\ of\ outcomes}}$$

Set Theory

Intersection - Given two events A and B, A ∩ B means A and B.

Union -Given two events A and B, A ∪ B means A or B.

Complement - Given an event A, $A^C$ is called A’s complement, and means when "A does not occur", everything that’s not in A.

Subevent - Given two events A and B, A ⊆ B means "B includes everything in A". We can write all valid events A as a subevent of the total sample space Ω: A ⊆ Ω.

De Morgan’s Laws - A useful identity that can make calculations easier by relating unions to intersections. Analogous results hold with more than two sets.

$$\begin{array}{l}{(A \cup B)^{c}=A^{c} \cap B^{c}} \end{array}$$

$$\begin{array}{l}{ (A \cap B)^{c}=A^{c} \cup B^{c}}\end{array}$$

Principle of Inclusion-Exclusion - For any events $A_1 ,\dots , A_n$,

$$P\left(\bigcup_{i=1}^{n} A_{i}\right)=\sum_{i} P\left(A_{i}\right)-\sum_{i<j} P\left(A_{i} \cap A_{j}\right)+\cdots+(-1)^{n+1} P\left(A_{1} \cap \cdots \cap A_{n}\right)$$

Counting

Multiplication Rule - If we have n decisions to make and the j-th decision has $r_j$outcomes, then the total number of potential outcomes is $r_1\cdot r_2\cdot\dots\cdot r_{n-1}\cdot r_n$

Factorial - The number of ways to order n objects is given as $n! = n\cdot(n-1)\cdot\dots\cdot2\cdot1$

Binomial Coefficient Formula - For k ≤ n, we have

$\binom{n}{k} = \frac{n(n-1)\dots(n-k+1)}{k!} = \frac{n!}{k!(n-k)!}$

$\binom{n}{k} = \binom{n}{n-k}$

Binomial Theorem - $(x+y)^n=\sum_{k=0}^n \binom{n}{k}x^k y^{n-k}$

The sampling table gives the number of possible samples of size $k$ out of a population of size $n$, under various assumptions about how the sample is collected.

$$\begin{array}{c|cc} & \text { Order Matters } & \text { Order Doesn't Matter } \\ \hline \text { With Replacement } & n^{k} & \left(\begin{array}{c} n+k-1 \\ k \end{array}\right) \\ \text { Without Replacement } & \frac{n !}{(n-k) !} & \left(\begin{array}{l} n \\ k \end{array}\right) \end{array}$$