14.5 Index

An alphabetized index of the key concepts, definitions, distributions, theorems, and techniques introduced in Chapters 1–13. Each entry links to the section(s) where the term is defined or discussed.

A¶

• Additivity of expectation — §4.2, §4.4, §A.3

• Algebraic properties of covariance — §11.1

• And statements — §1.4

• Approximation, exponential — §6.2

• Approximation, factorial — §6.3

• Approximation, function — §6.0

• Approximation, linear — §6.1, §9.1

• Approximation, polynomial — §6.1

• Approximation, quadratic — §6.1

• Asymptotic rates — §5.2

• Asymptotically unbiased estimator — §12.2

• Axioms, probability — §1.3

• Axis of symmetry — §3.1, §3.3

B¶

• Base rate — §1.5

• Bayes’ rule — §1.5, §1.7, §12.1

• Bernoulli distribution — §2.2, §2.5, §A.2

• Best-fit line (linear regression) — §11.2, §11.3

• Beta distribution — §A.2

• Bias (of an estimator) — §12.0, §12.2, §12.3

• Biased estimator — §12.2, §13.1

• Big-O notation — §5.2, §5.3, §5.5

• Binomial coefficient — §A.1

• Binomial distribution — §2.2, §2.5, §A.2

• Binomial random variable, variance of — §13.1

• Birthday problem — §6.2

• Bounded above / below — §3.1

• Bounded function — §3.1

• Bounding the chance of a union — §1.3

C¶

• Cartesian coordinates — §8.1

• Categorical distribution — §1.3

• Center of mass — §4.1, §A.3

• Centered random variable — §4.3, §11.1

• Central Limit Theorem (CLT) — §13.0, §13.4, §13.5

• Chain rule of expectation — §10.0, §10.2

• Change of density — §7.2, §7.3

• Change of variables — §7.2, §7.3, §10.3

• Chebyshev’s inequality — §13.2, §13.3, §13.5

• Chernoff’s inequality — §13.2

• Combinatorics — §A.1

• Combining events — §1.1

• Comparison test (direct, limit) — §5.3, §5.5

• Comparison test (integral) — §5.3, §5.5

• Complement rule — §1.3

• Complement, set — §1.1

• Composition of functions — §3.2, §3.4

• Concave function — §3.1, §3.4

• Concentration — §13.0

• Concentration inequality — §13.2

• Conditional density — §8.3, §10.2

• Conditional distribution — §1.5

• Conditional expectation — §10.0, §10.2, §A.3

• Conditional probability — §1.5, §1.7

• Consistency (of an estimator) — §12.0, §12.2, §13.5

• Constrained optimization — §9.4

• Containment, set — §1.1

• Continuity (of a function) — §3.1

• Continuity in measure — §2.3

• Continuous distributions (reference) — §A.2

• Continuous models — §2.3

• Continuous random variable — §2.3, §2.5

• Contour plot — §8.2

• Convergence tests for series — §5.3, §5.5

• Convex function — §3.1, §3.4

• Convolution — §10.3, §10.4, §13.4

• Coordinate systems — §8.1

• Correlation — §11.1, §11.3, §A.3

• Count variable — §4.2

• Countable / countably infinite — §2.2

• Covariance — §11.1, §11.3, §A.3

• Covariance matrix — §13.1, §A.3

• Critical point — §3.3, §9.3

• Cross sections — §8.2

• Cumulative distribution function (CDF) — §2.1, §2.5

D¶

• Data generating process — §12.1

• Density, change of — §7.2

• Density, probability — §2.4

• Dependent variables — §13.1

• Derivatives, directional — §9.2

• Derivatives, partial — §9.1

• Differentiable — §3.1, §3.3

• Dilation of a function — §3.2

• Direct comparison test — §5.3, §5.5

• Direction, of a vector — §8.1

• Directional derivative — §9.2

• Dirichlet distribution — §9.3

• Discrete distributions (reference) — §A.2

• Discrete models — §2.2

• Discrete random variable — §2.2, §2.5

• Discrete uniform distribution — §A.2

• Disjoint events / sets — §1.1, §1.2

• Distribution function — §2.0, §2.1

• Distribution of estimates — §12.2

• Distributions reference — §A.2

• Dot (inner) product — §8.1

• Double integral — §10.0, §10.1

E¶

• Empirical distribution — §11.2, §12.0, §12.1

• Equally likely outcomes — §1.2, §1.5

• Estimation problem — §12.0, §12.3

• Estimator — §12.0, §12.1

• Estimator, properties of — §12.2

• Even function — §3.1

• Event — §1.1, §1.7

• Expectation, additivity — §4.2

• Expectation, conditional — §10.2

• Expectation, iterated — §10.2, §A.3

• Expectation, joint — §10.0

• Expectation, linearity of — §4.2, §7.1, §10.2

• Expectation, rules of — §4.2, §A.3

• Expected value — §4.0, §4.1, §A.3

• Expected value of a function of a random variable — §4.1

• Exponential approximation — §6.2

• Exponential distribution — §A.2, §6.0

• Exponential tails — §5.4

• Exponential, limiting definition of — §6.2

• Extrema (multivariate) — §9.3, §9.4

F¶

• Factorial approximation (Stirling’s) — §6.3

• Failure probability — §13.2

• Frequency measures chance — §1.2

• Fubini’s theorem — §10.0, §10.1

• Function addition — §3.2

• Function approximation — §6.0

• Function combinations — §3.2, §3.4

• Function composition — §3.2

• Function inversion — §3.2

• Function multiplication — §3.2

• Function operations — §3.2

• Function properties, global — §3.1

• Function properties, local — §3.3

• Functions of multiple variables — §8.2

• Functions of random variables, expectation of — §4.1

G¶

• Gamma distribution — §A.2

• Gaussian distribution — see Normal distribution

• Geometric distribution — §2.2, §A.2

• Geometric series — §5.1

• Global function properties — §3.1

• Gradient — §9.2

• Gradient ascent — §9.3

• Gradient, level sets and — §9.2

H¶

• Harmonic series — §5.1

• Heatmap — §8.2

• Heavy tails — §5.4, §5.5

• Hypergeometric distribution — §A.2, §4.2

I¶

• i.i.d. random variables — §12.1

• If statements — §1.5

• Independent events — §1.6, §1.7

• Independent products — §10.2

• Indicator (Bernoulli) — §A.2, §2.2

• Inflection point — §3.1, §3.3

• Inner (dot) product — §8.1

• Integral comparison test — §5.3, §5.5

• Integration by parts — §7.1

• Integration by substitution — §7.2

• Integration on manifolds — §10.3

• Integration techniques — §7.0

• Integration, iterated — §10.1

• Integration, polar coordinates — §10.3

• Intersection of sets — §1.1

• Inverse function — §3.2, §3.4

• Iterated convolution — §13.1

• Iterated expectation — §10.2, §A.3

• Iterated integration — §10.1

J¶

• Jensen’s inequality — §4.1, §4.2

• Joint density function — §8.3

• Joint distribution — §8.3, §8.4

• Joint distribution table — §8.3

• Joint expectation — §10.0

• Joint probability — §1.4, §1.7

• Joint probability mass function — §8.3

• Joint probability table — §1.4

L¶

• L’Hôpital’s rule — §3.3, §5.2

• Lagrange multipliers — §9.4

• Laplace distribution — §3.1

• Law of Large Numbers (Weak) — §13.3, §13.5

• Law of Small Numbers — §6.4

• Learning rate — §9.3

• Least squares regression — §9.3, §11.2

• Length / magnitude (of a vector) — §8.1

• Level set — §8.2, §9.2

• Light tails — §5.4, §5.5

• Limit comparison test — §5.3, §5.5

• Limiting definition of $e$ — §6.2

• Limiting distributions — §6.4

• Linear approximation — §6.1, §9.1

• Linear change of density formula — §7.2

• Linear combination — §3.2, §8.1

• Linear regression — §11.2, §11.3

• Linear transformation — §3.2, §3.4

• Linearity of expectation — §4.2, §7.1, §10.2, §A.3

• Linearization — §9.1

• Local function properties — §3.3

• Log-log plot — §5.2, §5.4

M¶

• Magnitude (vector length) — §8.1

• Manifold — §10.3

• Marginal density — §8.3, §10.2

• Marginal probability — §1.4

• Marginal probability mass function — §8.3

• Markov’s inequality — §13.2, §13.5

• Maxima / minima (multivariate) — §9.3, §9.4

• Maximum Likelihood Estimator (MLE) — §12.0, §12.1

• Mean absolute deviation (MAD) — §4.3

• Mean squared error (MSE) — §9.3, §12.3

• Mean — see Expected value

• Median — §4.1, §A.3

• Mode — §4.1, §A.3

• Models of chance — §2.0

• Moments — §4.3

• Monotone composition — §3.3

• Monotonic function — §3.1

• Multimodal — §4.1

• Multiplication rule — §1.5

• Multiplication rule for independent events — §1.6

N¶

• Nondifferentiable — §3.3

• Nonnegativity (probability axiom) — §1.3

• Normal (Gaussian) distribution — §5.4, §6.4, §A.2

• Normal random variables, sums of (independent) — §10.3

• Normalization — §1.5, §2.4

• Normalization (probability axiom) — §1.3

• Normalize a vector — §8.1

• Norms — §8.1

O¶

• Objective function — §9.3

• Odd function — §3.1

• Odds — §1.5

• Optimization, constrained — §9.4

• Optimization, unconstrained — §9.3

• Ordering tail decay rates — §5.2

• Origin (vectors) — §8.1

• Orthogonal vectors — §8.1

• Outcome — §1.1

• Outcome space — §1.1

• Outcome tree — §1.5

P¶

• Parameter — §2.2

• Pareto distribution — §5.1, §A.2

• Partial derivative — §9.1

• Partial geometric sum — §5.1

• Partition — §1.1

• Permutation — §1.1, §1.2, §A.1

• Poisson distribution — §5.4, §6.4

• Polar coordinates — §8.1, §10.3

• Polynomial approximation — §6.1

• Power law — §5.1, §5.4

• Probability as proportion — §1.2

• Probability axioms — §1.3

• Probability density function (PDF) — §2.4, §2.5

• Probability fundamentals — §1.0

• Probability histogram — §1.3, §2.1

• Probability mass function (PMF) — §2.1, §2.2

• Probability measure — §1.3

• Probability model — §1.3

• Probability rules — §1.3, §1.7

• Properties of estimators — §12.2

• Properties of expectation — §4.2, §A.3

• Properties of variance — §4.3, §A.3

• Proportion, probability as — §1.2

Q¶

• Quadratic approximation — §6.1

R¶

• Random process — §1.1

• Random variable — §2.1, §2.5

• Random variable, continuous — §2.3

• Random variable, discrete — §2.2

• Random vector — §8.0, §8.1

• Rate parameter — §A.2

• Rates, asymptotic — §5.2

• Ratio test — §5.3, §5.5

• Reasoning with sequences — §1.5

• Regression, linear — §11.2

• Roots (of a function) — §3.3

• Rules of chance — §1.3

• Rules of expectation — §4.2, §A.3

• Rules of variance — §4.3, §A.3

S¶

• Saddle / extrema — §9.3

• Sample average — §4.1, §12.1, §13.1

• Sample covariance — §11.2

• Sample variance — §11.2

• Sampling with / without replacement — §6.2

• Scalar (vs. vector) — §8.1

• Scaling of a function — §3.2

• Scaling, expectation under — §4.2

• Scaling, variance under — §4.3

• Sequences — §5.2

• Series convergence — §5.3

• Sets and subsets — §1.1

• Shape parameter — §A.2

• Smoothness — §3.1

• Standard deviation (SD) — §4.3, §A.3

• Standard normal distribution — §6.4, §13.4

• Standard units / standardization — §4.3, §11.1

• Steepest ascent — §9.2

• Stirling’s approximation — §6.3

• Student’s t distribution — §5.4, §5.5

• Sub-additivity — §1.3, §1.7

• Subexponential tails — §5.4

• Subset — §1.1

• Substitution (integration) — §7.2

• Success probability — §2.2

• Summation by parts — §7.1

• Sums of independent random variables — §10.3, §13.1

• Sums, integrals of — §7.1

• Sums, iterated — §10.1

• Superexponential (heavy) tails — §5.4

• Support — §2.1, §2.4

• Surface, visualizing — §8.2

• Survival function — §5.1

• Symmetric distribution — §4.2

• Symmetric function — §3.1

T¶

• Tail bounds — §13.2

• Tail decay rate — §5.2

• Tail integrals / sums — §7.3, §A.3

• Tails and rare events — §5.1

• Tangent plane — §9.1

• Taylor series — §6.1

• Theorem, Central Limit — §13.4

• Translation, horizontal — §3.2

• Translation, vertical — §3.2

• Translations, expectation under — §4.2

• Two-sided / one-sided bounds — §13.2

U¶

• Unbiased estimator — §12.2

• Unconstrained optimization — §9.3

• Uncorrelated but dependent — §11.1

• Uncountably infinite — §2.2

• Uniform distribution (continuous) — §A.2

• Uniform distribution (discrete) — §A.2

• Union bound — §1.3

• Union of sets — §1.1

• Unit vector — §8.1

V¶

• Variance — §4.3, §4.4, §A.3

• Variance of binomial random variables — §13.1

• Variance of sums and averages — §13.1, §A.3

• Variance, properties of — §4.3

• Variance, rules of — §4.3

• Vector — §8.1

• Vector addition — §8.1

• Vector field — §9.2

• Vector geometry — §8.1

• Vector operations — §8.1

• Vector scaling — §8.1

• Vector-valued function — §9.2

• Visualizing functions and models — §3.0

• Volume beneath a surface — §10.1

W¶

• Weak Law of Large Numbers — §13.3, §13.5

• Weighted average — §4.0, §4.1

Z¶

• Zero vector — §8.1