Let $\bar{X}$ be the sample mean, which is equal to $\frac{X_1 + \ldots + X_n}{n}$. Saying that there is at least a 99% chance that $|\bar{X}_n - \mu| < 2 \sigma$ is the same as saying that there is at most a 1% chance for $|\bar{X}_n - \mu| > 2 \sigma$. Thus, we want to calculate an $n$ such that:

Applying Chebyshev's inequality, we get the following:

Therefore, if we choose $n = 25$, we get the desired inequality.

Let $X_n = Y_1 + Y_2 + \ldots + Y_n$. Then, we can have $X_n \sim Gamma(n, \lambda)$ and $Y_i$'s be i.i.d. $Expo(\lambda)$. By the central limit theorem, since $X_n$ is the sum of i.i.d. random variables, it converges to a normal distribution as $n \to \infty$.

Using the parameters of the Gamma distribution, the central limit theorem also tells us that

In order to convert this normal distribution to a standard normal ($N(0,1)$), all we need to do is subtract the mean and divide by the standard deviation. Thus:

as $n \to \infty$.

By solving $\vec{s} Q = \vec{s}$, we have that

And by solving this system of linear equations, it follows that

To verify the validity of a stationary distribution for a chain, we just need to show that $s_i q_{ij} = s_j q_{ji}$, which is done if we can show that $s_0 q_{01} = s_1 q_{10}$. We have that

which satisfies our reversibility condition and verifies our stationary distribution from the previous question.

Yes, $Z_n$ is a Markov Chain because conditional on $Z_n$, $Z_{n+1}$ is independent to $Z_{n-1}$. This is because the components of $Z_{n+1}$ and $Z_{n-1}$ are either constants conditioned in $Z_n$ or independent of each other given that $X_n$ is a Markov Chain.

The states are given as $\{ (0, 0), (0, 1), (1, 0), (1, 1) \}$. The transition matrix is given as