14.2 Distributions Reference

A random variable $X$ is drawn from a discrete uniform distribution over $S = \{s_1,s_2,...,s_n\}$ if:

1. Support: $X \in S$

2. Distribution Function:

   $$\text{PMF}(x) = \begin{cases} \frac{1}{n} & \text{ if } x \in S \\ 0 & \text{ else } \end{cases}.$$

   (1)

Then we say $X \sim \sim \text{Uniform}(S)$.

Properties:

1. Expected Value:

   $$\mathbb{E}[X] = \frac{1}{n} \sum_{j=1}^n s_j$$

   (2)

2. Variance:

   $$\text{Var}[X] = \frac{1}{n} \sum_{j=1}^n (s_j - \mathbb{E}[X])^2$$

   (3)

A random variable $X$ is an indicator for an event $E$ if $X = 1$ when the event occurs, and is zero otherwise. Indicator random variables are drawn from Bernoulli distributions.

A random variable $X$ is drawn from a Bernoulli distribution with success probability $p$ if:

1. Support: $X \in \{0,1\}$

2. Distribution Function:

   $$\text{PMF}(x) = \begin{cases} p & \text{ if } x = 1 \\ 1 - p & \text{ if } x = 0 \end{cases}.$$

   (4)

Then we say $X \sim \text{Bernoulli}(p)$. If $X$ is an indicator for an event, $E$, then $p = \text{Pr}(E)$.

Properties:

1. Expected Value:

   $$\mathbb{E}[X] = p$$

   (5)

2. Variance:

   $$\text{Var}[X] = p(1 - p)$$

   (6)

If $X$ counts the total number of trials, up to and including, the first success in a string of independent, identical, binary trails then $X$ is a geometric random variable.

A random variable $X$ is drawn from a geometric distribution with success probability $p$ if:

1. Support: $X \in \{1,2,3,...\}$

2. Distribution Function:

   $$\text{PMF}(x) = (1 - p)^{x-1} p$$

   (7)

Then we say $X \sim \text{Geometric}(p)$. The parameter $p$ is the chance each trial ends in a success.

Properties:

1. Expected Value:

   $$\mathbb{E}[X] = \frac{1}{p}$$

   (8)

2. Variance:

   $$\text{Var}[X] = \frac{(1 - p)}{p^2}$$

   (9)

If $X$ counts the total number of successes in a string of $n$ of independent, identical, binary trails then $X$ is a binomial random variable.

A random variable $X$ is drawn from a binomial distribution with success probability $p$ if:

1. Support: $X \in \{1,2,3,..., n\}$

2. Distribution Function:

   $$\text{PMF}(x) = {n \choose x} p^x (1 - p)^{n - x}$$

   (10)

Then we say $X \sim \text{Binomial}(n,p)$. The parameter $p$ is the chance each trial ends in a success.

Properties:

1. Expected Value:

   $$\mathbb{E}[X] = n p$$

   (11)

2. Variance:

   $$\text{Var}[X] = n p (1 - p)$$

   (12)

If $X$ counts the total number of successes in a sample of size $n$ drawn without replacement from a finite population of size $N$ containing $p N$ success cases, then $X$ is a hypergeometric random variable.

A random variable $X$ is drawn from a hypergeometric distribution with parameters $N$, $K = p N$, and $n$ if:

1. Support: $X \in \{\max(0, n + K - N), ..., \min(n, K)\}$

2. Distribution Function:

   $$\text{PMF}(x) = \frac{{K \choose x} {N - K \choose n - x}}{{N \choose n}}$$

   (13)

Then we say $X \sim \text{Hypergeometric}(N,K,n)$. The parameter $N$ is the population size, $K = p N$ is the number of successes in the population, and $n$ is the sample size. The proportion $p$ is the fraction of all cases that are successes in the full population.

Properties:

1. Expected Value:

   $$\mathbb{E}[X] = n p$$

   (14)

2. Variance:

   $$\text{Var}[X] = \left( \frac{N - n}{N - 1} \right) n p (1 - p)$$

   (15)

A random variable $X$ is drawn uniformly on an interval $[a,b]$ if:

1. Support: $X \in [a,b]$

2. Distribution Function:

   $$\text{PDF}(x) = \begin{cases} \frac{1}{|b - a|} & \text{ if } x \in [a,b] \\ 0 & \text{ else } \end{cases}$$

   (16)

Then we say $X \sim \text{Uniform}([a,b])$.

Properties:

1. Expected Value:

   $$\mathbb{E}[X] = \frac{a + b}{2}$$

   (17)

2. Variance:

   $$\text{Var}[X] = \frac{(b - a)^2}{12}.$$

   (18)

The beta distribution is a flexible distribution on $[0,1]$, often used to model random probabilities or proportions. It also models the value of the $k^{th}$ largest sample of $n$ independent, uniform random variables on $[0,1]$.

A random variable $X$ is drawn from a beta distribution with shape parameters $\alpha > 0$ and $\beta > 0$ if:

1. Support: $X \in [0,1]$

2. Distribution Function:

   $$\text{PDF}(x) \propto x^{\alpha - 1} (1 - x)^{\beta - 1}$$

   (19)

   The normalization constant is $c = \frac{\Gamma(\alpha + \beta)}{\Gamma(\alpha) \Gamma(\beta)}$ where $\Gamma(s) = (s - 1)!$ when $s$ is integer valued.

Then we say $X \sim \text{Beta}(\alpha, \beta)$.

Properties:

1. Expected Value:

   $$\mathbb{E}[X] = \frac{\alpha}{\alpha + \beta}$$

   (20)

2. Variance:

   $$\text{Var}[X] = \frac{\alpha \beta}{(\alpha + \beta)^2 (\alpha + \beta + 1)}$$

   (21)

If $X$ measures the waiting time until the first arrival in a memoryless arrival process with rate $\lambda$, then $X$ is an exponential random variable.

A random variable $X$ is drawn from an exponential distribution with rate $\lambda > 0$ if:

1. Support: $X \in [0, \infty)$

2. Distribution Function:

   $$\text{PDF}(x) \propto e^{-\lambda x}$$

   (22)

   The normalization constant is $c = \lambda$.

Then we say $X \sim \text{Exponential}(\lambda)$. The parameter $\lambda$ is the rate at which arrivals occur.

Properties:

1. Expected Value:

   $$\mathbb{E}[X] = \frac{1}{\lambda}$$

   (23)

2. Variance:

   $$\text{Var}[X] = \frac{1}{\lambda^2}$$

   (24)

The Pareto distribution is a heavy-tailed distribution often used to model quantities such as wealth, city sizes, or other power-law phenomena.

A random variable $X$ is drawn from a Pareto distribution with scale parameter $x_m > 0$ and shape parameter $\alpha > 0$ if:

1. Support: $X \in [x_m, \infty)$

2. Distribution Function:

   $$\text{PDF}(x) \propto \begin{cases} x^{-(\alpha + 1)} & \text{ if } x \geq x_m \\ 0 & \text{ else } \end{cases}$$

   (25)

   The normalization constant is $c = \alpha x_m^{\alpha}$.

Then we say $X \sim \text{Pareto}(x_m, \alpha)$. The parameter $x_m$ is the minimum possible value, and $\alpha$ controls the heaviness of the tail.

Properties:

1. Expected Value:

   $$\mathbb{E}[X] = \begin{cases} \frac{\alpha x_m}{\alpha - 1} & \text{ if } \alpha > 1 \\ \infty & \text{ else } \end{cases}$$

   (26)

2. Variance:

   $$\text{Var}[X] = \begin{cases} \frac{x_m^2 \alpha}{(\alpha - 1)^2 (\alpha - 2)} & \text{ if } \alpha > 2 \\ \infty & \text{ else } \end{cases}$$

   (27)

If $X$ is a sum of $n$ independent, identical, exponential random variables with rate $\lambda$ then $X$ is a gamma random variable.

A random variable $X$ is drawn from a gamma distribution with shape parameter $n > 0$ and rate parameter $\lambda > 0$ if:

1. Support: $X \in [0, \infty)$

2. Distribution Function:

   $$\text{PDF}(x) \propto x^{n - 1} e^{-\lambda x}$$

   (28)

   The normalization constant $c = \frac{\lambda^{n}}{(n - 1)!}$

Then we say $X \sim \text{Gamma}(\alpha, \beta)$.

Properties:

1. Expected Value:

   $$\mathbb{E}[X] = \frac{n}{\lambda}$$

   (29)

2. Variance:

   $$\text{Var}[X] = \frac{n}{\lambda^2}$$

   (30)

The normal distribution (or Gaussian distribution) is the most ubiquitous continuous distribution. It is the limit of sums of many independent, identical random variables (Central Limit Theorem).

A random variable $X$ is drawn from a normal distribution with mean $\mu$ and standard deviation $\sigma > 0$ if:

1. Support: $X \in (-\infty, \infty)$

2. Distribution Function:

   $$\text{PDF}(x) \propto \exp\left( -\frac{1}{2} \left( \frac{x - \mu}{\sigma} \right)^2 \right)$$

   (31)

   The normalization constant is $c = \frac{1}{\sqrt{2 \pi} \sigma}$

Then we say $X \sim \text{Normal}(\mu, \sigma^2)$.

Properties:

1. Expected Value:

   $$\mathbb{E}[X] = \mu$$

   (32)

2. Variance:

   $$\text{Var}[X] = \sigma^2$$

   (33)

A random variable $X$ is drawn from a $t$ distribution with mean $\mu$ and scale $\sigma > 0$, and shape $\nu > 0$ if:

1. Support: $X \in (-\infty, \infty)$

2. Distribution Function:

   $$\text{PDF}(x) \propto g(x) = \left(1 + \frac{1}{\nu} \left(\frac{x - \mu}{\sigma}\right)^2 \right)^{-\frac{\nu + 1}{2}}$$

   (34)

Properties:

1. Expected Value:

   $$\mathbb{E}[X] = \mu$$

   (35)

2. Variance:

   $$\text{Var}[X] = \begin{cases} \frac{\nu}{\nu - 2} \sigma^2 & \text{ if } \nu > 2 \\ \infty & \text{ else} \end{cases}$$

   (36)