3.3 Local Properties

So far we’ve discussed two methods for visualizing functions. These were, check its characteristics (Section 3.1), then check its composition (Section 3.2). The first helps narrow down the ways in which the function can behave. The second breaks it into simpler parts.

Both of those strategies are global. They provide information about $f(x)$ at all possible inputs $x$.

This section is about local strategies. The simplest local strategy is the oldest. Just pick a bunch of inputs, $\{x_1,x_2,x_3,...\}$, plug them each in to find $\{f(x_1),f(x_2),f(x_3),...\}$, plot each, then trace a curve through your plot. This is the strategy you used in HW 2 to guess how the mode of the binomial depended on $n$ and $p$.

That strategy always works, but it is slow and laborious. It’s also not very insightful. You usually won’t learn much about the function by doing it, so if we change the problem set up slightly, say by varying a free parameter, then you’ll have to do all of your work over again.

That said, evaluating a function at some specific input points is an essential strategy. The key is to pick those points wisely. Try to pick as few points as are sufficient to describe the function. In essence, select a set of reference points, learn what you can about $f$ near those references, then constrain your plot using what you’ve learned.

Choosing Reference Points¶

Make a list of reference points. In rough order of work to evaluate, check:

1. Any inputs where $f$ is easy to evaluate (often, $x = 0$ and $x = \pm 1$).

   • Be greedy. Always do the easiest thing first.

2. The smallest and largest possible inputs. If necessary, take limits.

   • If $X$ is a random variable with support equal to some interval $[a,b]$, then you should always include $a$ and $b$ in your list of reference points.

   • Some of these limits are easy. For example:

     • If you want to plot the CDF of $X$, and $X \in [a,b]$, then $\text{CDF}(x) = 0$ for all $x < a$ and $\text{CDF}(x) = 1$ for all $x > b$.

     • If you want to plot a PMF or a PDF, and $X$ is unbounded in some direction (e.g. $X$ can be arbitrarily large), then your PMF/PDF must converge to zero as $x$ diverges. Otherwise, it couldn’t add/integrate to one so would not be normalized. This is why, whenever we’ve drawn a distribution for a random variable that is unbounded above, the distribution tailed off to zero for sufficiently large inputs.

3. An axis of symmetry if it exists.

   • For instance, if you’re given a density $\text{PDF}(x) \propto g(x)$ where $g(x) = (1 + \tfrac{1}{4}(x + 3)^2)^{-5/2}$, notice that this is a composition of functions, $x^{-5/2}$ and $1 + \tfrac{1}{4}(x + 3)^2$. The inner function is a linear transformation (vertical translation, scale, horizontal translation) of our friend, $x^2$, so must have some symmetry. The function $x^2$ is even about zero. So, if we shift it left by 3, then the shifted function must be even about $x_* = -3$.

4. Any roots (locations where $f(x) = 0$) or poles (locations where $f(x)$ diverges).

   • Sometimes roots are obvious. For instance, the function $g(x) = x^{3.1}(1 - x)^{1.2}$ will have roots at $x = 0$ and $x = 1$. Roots are obvious when our functions are factored for us.

   • Sometimes roots are very hard to recover. For instance, try to find the roots of $g(x) = 3x^4 + 2 x^3 - 10 x - 6$. Not so easy. Add a power of $x^5$ and $x^6$ and there is no mathematical formula that could find the roots.

   • So, be strategic. Only look for roots if they jump out at you from the page.

   • To find poles, look for roots in the denominator. For instance, the function $g(x) = (1 + x^2)/(1 - x^2)$ will have poles at $x = \pm 1$ since the denominator equals zero there.

   • So, apply the same logic about roots to poles. Only look for poles if they are obvious. It is worth putting in a bit more elbow-grease when looking for poles, since adding a vertical asymptote to a plot is more important for organizing the plot than adding a root.

   • Be careful with poles. Remember that, if the numerator and denominator are both zero at some $x$, then $g(x)$ may be zero, infinity, or some other finite number. In this case, you have to take a limit. We’ll review l’Hopital’s rule in our chapter on asymptotics and tail behavior.

5. Any points of discontinuity or nondifferentiability.

   • These are usually easy to spot. Look for a piecewise definition, or an absolute value in the function definition.

6. All maxima and minima of the function.

   • These are the most important reference points for drawing distribution functions accurately. Always look for maxima and minima.

   • There are a couple strategies for finding maxima and minima. If your function is smooth (continuous and differentiable), then you should check for places where the derivative equals zero.

   • Once you find the roots of the derivative, evaluate its sign between the roots. This creates a partition of the number line into intervals where your function is increasing or decreasing. Anywhere a function switches from increasing to decreasing is a maximum. Anywhere that it switches from decreasing to increasing is a minimum.

   • The same arguments work for discrete functions, like PMF’s. We have to do a bit more work since a PMF is a bar chart, and doesn’t have a well defined slope. To see an example, look ahead to “Optimization”.

7. Any inflection points of the function.

   • Check for places where the second derivative changes sign. Follow the same procedure we used for slope. First set $\frac{d^2}{dx^2} f(x) = 0$, then check the sign of the second derivative in between its roots

   • Inflection points of the CDF correspond to maxima and minima of the PMF/PDF since the second derivative of the CDF is the first derivative of the PDF.

   • Inflection points are the least important features on this list unless you are drawing a CDF. They are also often the most work to find. So, save them for last. See if you have enough information to draw your function well before finding its inflection points.

That’s a long list, but you rarely need all of it. The more you practice, the faster you’ll get at recognizing functions, and the fewer reference points you’ll need. Most functions don’t have all the references on this list, so you can usually skip a bunch with little effort. For many distributions it is enough to check the largest and smallest possible inputs, check for symmetry, and to identify maxima and minima.

Evaluating $f$ Locally¶

Then:

1. Evaluate $f$ at each reference.

   • Add a dot on your plot at each $(x,f(x))$ pair.

2. Evaluate the slope of $f$ at each reference.

   • Add a small tangent line to your plot at $(x,f(x))$ with slope $\frac{d}{dx} f(x)$. Make sure your drawing is tangent to the tangent line (has the correct slope).

3. Evaluate the slope of $f$ at some arbitrary point between each reference.

   • Make sure your function is increasing and decreasing on the correct intervals.

   • I usually draw a quick number line reference, mark the roots of the derivative, and add a plus sign on the intervals where $f$ is increasing, and a minus where it is decreasing.

   • If you’ve already determined which of the roots of $\frac{d}{dx} f(x)$ are maxima and minima, then you can fill in the intervals without evaluating the slope again.

4. Evaluate the sign of the second derivative of $f$ at each reference.

   • This can be slow. So, check whether you know the sign of the second derivative before computing a derivative.

Steps (1) and (2) are essential. If you’ve done (1) and (2) you usually get (3) for free. Step (4) is slow. It helps for accurate plots, especially of CDF’s, but is the least important of the three. Do it last and only when necessary.

Optimization Refresher¶

Let’s remind ourselves how to find maxima and minima of a function, $f(x)$.

We’ll go through four techniques. The first two are the easiest. Always do them first. They mostly require looking at your function, and identifying its structure. The last two are the slowest, but they are failsafe. They are largely mechanical. Do them last as back-ups in case the first two fail.

Symmetry¶

As usual, start by looking for symmetries. If:

1. If $f(x)$ is even about an axis of symmetry, $x_*$, then $x_*$ must be a maximum or a minimum of the function.

   • If this isn’t clear to you, try to draw a counterexample. You’ll see that, if you draw $f(x)$ increasing as $x$ leaves $x_*$ to the right, then, by symmetry, $f(x)$ must also increase as $x$ leaves $x_*$ to the left.

   • Picture $x^2$. You come into zero descending, then, by symmetry, leave zero ascending.

   • The same is true for concave functions. If, as $x$ approaches $x_*$ from below, $f(x)$ is increasing, then by symmetry $f(x)$ must be decreasing as $x$ moves away from $x_*$ to the right.

   • The only counterexample is the constant function.

   • It follows that if $f$ is differentiable, then $\frac{d}{dx}f(x_*) = 0$ at any even axis of symmetry. So, when you add an even axis of symmetry, you never need to evaluate the derivative there. Just add a horizontal tangent.

2. If $f(x)$ is odd about an axis of symmetry, $x_*$, then $x_*$ cannot be a maximum or a minimum.

   • If this isn’t clear to you, try to draw a counterexample. You’ll find that, if $f(x)$ is increasing as $x$ approaches $x_*$ from below, then $f(x)$ is also increasing as $x$ leaves $x_*$ to the right.

   • Picture $x^3$.

Monotone Compositions¶

Suppose that $f(x) = g(h(x))$ for some outer function $g$ and some inner function $h$. Then, suppose that $g$ is a monotonic function. For instance, we’ll see lots of examples where $g$ is either $e^{x}$, $e^{-x}$, or $x^{-\alpha}$ for some $\alpha > 0$.

Suppose that $g(x)$ is monotonically increasing or non-decreasing. Then, $h_* \geq h$ implies $g(h_*) \geq g(h)$. So, if $h(x_*) \geq h(x)$, then $g(h(x_*)) \geq g(h(x))$. So, to maximize $g$ it is enough to maximize $h$.

The same logic works in reverse for monotonically decreasing, or non-increasing, functions. If $g$ is monotonically decreasing or non-increasing, then making $h$ smaller makes $g(h)$ larger. So, to maximize $g(h(x))$ minimize $h(x)$.

This strategy works whenever $h$ is easier to optimize than $f$. It is espeically useful for distributions that are defined using an exponential. For example, we will see lots of distributions that look like:

for some $h(x)$. In this case, if you’re asked to optimize $f$, just optimize $h$.

Some functions don’t look like $e^{h(x)}$ at first glance, but are still easier to optimize if we rewrite them this way. In particular, if your function involves a product of terms, then, since exponentials exchange products and sums, we can write:

Now, optimize the argument of the exponential, which is the log of $f$:

This trick is important for random variables generated by processes that involve a bunch of and statements. Those and statements produce products, so the corresponding distributions are often expressed as a product of terms. For instance, the binomial PMF is a product of three terms:

and the geometric PMF is a product of two:

In both cases, if we want to find the choice of $p$ that maximizes the PMF at a given $x$, we should start by taking a logarithm.

By Direction¶

In general, $x_*$ is a maximizer if $f(x)$ is increasing to the left of $x_*$ and decreasing to the right. It is a minimizer if $f(x)$ is decreasing to the left and increasing to the right. So, $x_*$ is a critical point (maximizer or minimizer) if $f$ changes direction at $x_*$.

Differentiable Functions¶

If $f(x)$ is smooth, then we can use its derivative to check whether it is increasing or decreasing. If the derivative is positive then $f$ is increasing. If the derivative is negative, then it is decreasing. So, $f$ has critical points where the derivative changes sign.

If $f(x)$ is continuously differentiabe (that is, its slope is continuous), then its slope cannot change sign without crossing zero. So, to find critical points, we rely on the old calculus trick:

1. Take the derivative

2. Set it to zero.

Then, evaluate the sign of the derivative on either side of its roots to classiy critical points as maxima, minima, or neither.

Non-differentiable Functions¶

What if $f(x)$ is not a smooth function of $x$? Then it doesn’t have a well defined slope, so our friendly “set derivative to zero” method doesn’t apply.

We can still find maxima and minima by looking for locations where $f(x)$ switches from increasing to decreasing, or from decreasing to increasing. Here’s an example.