3.1 Global Function Properties

Before visualizing a function in detail, it often helps to look for some global properties that constrain its behavior. The more you can identify, the easier it is to visualize the function.

This chapter provides a basic checklist of properties. Many of these should be reviews. I would skim the main terms first, then check the ones you are less familiar with, are new to you, or where your knowledge is rusty.

Smoothness¶

First, you should check whether the function is well-behaved. Look for locations where it might jump suddenly, or where its slope changes instanteneously.

1. Continuity: A function $f$ is continuous if $\lim_{x \rightarrow x_*} f(x)$ is well defined for all $x_*$ and equals $f(x_*)$.

   • In other words, the function has no jumps, or isolated points. It is smooth.

   • For example, a random variable is continuous if and only if its CDF is a continuous function (see Section 2.3).

   

   • The CDF of a discrete random variable always includes jump discontinuities at each $x$ where $\text{PMF}(x) \neq 0$. The example function provided below could be the CDF of a discrete random variable. It is discontinuous:

   

   • If you can write down your function, and it doesn’t come in many separate pieces, then it is usually continuous.

2. Differentiable: A function $f$ is differentiable if $\frac{d}{dx} f(x)$ exists for all $x$. That is, if it has a well-defined slope everywhere.

   • Functions with kinks, or corners, (e.g. $|x|$) may not be differentiable everywhere.

   • A function is second differentiable if its slope is differentiable (second derivatives exist everywhere).

   • Almost every function we study in this class will be infinitely differentiable, i.e. all derivatives exist for all relevant $x$.

   • Not all density functions are differentiable at all $x$. For instance, the density function $\text{PDF}(x) \propto e^{-|x|}$ is a popular model for sparse regression.

   

3. Symmetric: A function is even if $f(-x) = f(x)$. A function is odd if $f(-x) = -f(x)$.

   • The names even and odd point to the simplest examples; polynomials. A polynomial is even if all its terms are $x$ to an even power. It is odd if all its terms are $x$ to an odd power. If all the terms in a polynomial are even powers then the polynomial is an even function, for instance $x^2$. If all ther terms are odd powers, then the function is odd, for instance $x$ or $x^3$.

   • A function is symmetric about some axis of symmetry if there exists an $x_*$ such that $f(-(x - x_*)) = f(x - x_*)$ or $f(-(x - x_*)) = - f(x - x_*)$. That is, if $f$ is even or odd with respect to reflections about $x_*$.

   • For example, $(x-2)^4$ is an even function with an axis of symmetry at $x_* = 2$. Often we can recognize axes of symmetry by finding translations of the function that move the axis of symmetry to the origin.

   • Since distribution functions are nonnegative they are never odd. Mass functions and density functions may be even. Some of the most famous density functions are even. For instance, $\text{PDF}(x) \propto (1 + \frac{1}{a} x^2)^{-(a +1)/2}$ for $a > 0$ is an important model for the distribution of test statistics. Notice, this function is even since it can be expressed as a composition of two functions, an outer function $g(x) = x^{-(a+1)/2}$, and an inner function $h(x) = 1 + \frac{1}{a} x^2$. Then $\text{PDF}(-x) = g(h(-x)) = g(h(x)) = \text{PDF}(x)$.

   

   • Anytime a function can be expressed as a composition of functions, and the innermost function is even, then the whole function is even.

   • Symmetries are very useful tools for visualization. If you know that a function is symmetric about an axis, then you only need to visualize half of it. The other half is implied by reflection (even), or reflection and negation (odd). The better you get at recognizing symmetries, the faster you will be at visualizing functions.

4. Bounded: A function is bounded if it lies below some finite upper bound, or above some finite lower bound.

   • A function is bounded from below if $f(x) \geq m$ for some $m$. The most important examples in probability are distribution functions, which are bounded below by 0 (are nonnegative).

   • A function is bounded from above if $f(x) \leq M$ for some $M$. For example, every PMF and CDF is bounded above by one because PMF’s and CDF’s return chances. PDF’s, in contrast, return densities, and need only integrate to one. So PDF’s need not be bounded above.

5. Monotonic: A function $f$ is monotonic or monotone if it never changes direction.

   • It may be monotonically increasing ($f(a) > f(b)$ if $a > b$), nondecreasing ($f(a) \geq f(b)$ is $a > b$), nonincreasing ($f(a) \leq f(b)$ if $a > b$), or decreasing ($f(a) < f(b)$ if $a > b$). For example, every CDF is monotonically nondecreasing.

   • A differentiable function is monotonic if its derivative never changes sign.

   • Every CDF is a monotonically nonincreasing function. The CDF of a continuous random variable is increasing wherever the variable is supported, e.g. has nonzero density.

6. Convex/Concave: A function is convex if it ``bends upwwards." It is concave if it “bends downwards.”

   • Formally, a function is convex if the chord from $(a,f(a))$ to $(b,f(b))$ is never beneath $f(x)$ for any $x$ between $a$ and $b$. It is concave if the chord lies beneath the function between $a$ and $b$. In the example shown below, the function $f$ is displayed in blue and the chord for arbitrarily selected $a$ and $b$ is in red.

   

   • If the function is second differentiable, it is convex if its second derivative is nonnegative. It is concave if its second derivative is nonpositive. It is strictly convex/concave if its second derivative never changes sign and does not equal zero on any open interval.

   • In essence, convex functions always curve up so that they lie above their tangent lines, and beneath any chord.

   • Because density functions, mass functions, and cumulative distribution functions are usually bounded above and below, they are rarely convex or concave for all $x$. Some simple examples, like the Exponential and Pareto distributions from Section 2.4 are convex, but it is more common to see distribution functions that are convex on some intervals and concave on others. The famous “bell curve” is concave about its peak, and convex in its tails. Locations where the curvature of a function changes are called inflection points.

   

   • It is very common to see distribution functions whose logarithm is concave, most famously, the normal distributions whose simplest form is $\text{PDF}(x) \propto e^{-\frac{1}{2} x^2}$. Taking a log returns $-\frac{1}{2} x^2$, whose second derivative is -1 everywhere. More generally, it is common to build distribution functions as a composition of two functions, a concave or convex function on the inside, then a monotonic function that is nonnegative and converges to zero given infinite inputs on the inside. For example, $\text{PDF}(x) \propto (1 + \frac{1}{a} x^2)^{-(a +1)/2}$.

   • Concavity and convexity are important for visualizing functions and are essential properties in optimization. It is much easier to find the peak of a concave function, or the valley of a convex function, than it is to find the peak or valley of a function that is neither concave nor convex. Functions that are strictly concave have, at most, one unique maximizer. As a result, distributions that are constructed as a composition of a concave/convex function inside a monotone function have a single peak. We call distributions with a single peak unimodal since they have a unique mode.

Interactive Function Properties Quiz¶

Run the code cell below to test your knowledge. You should be able to quickly categorize every elementary function listed according to the properties defined above. Practice until this categorization is instinctive.