3.14 Direction of the Fastest Increase

Let $f$ be a differebtiable function of $x, y$ and $z$. As the directional derivative $D_{\mathbf{v}}f(x_0,y_0,z_0)$ gives the rate of change of $f$ at $(x_0,y_0,z_0)$ (of course when $\mathbf{v}$ is a unit vector), to find the direction in which $f$ increases most rapidly, we have to maximize $D_{\mathbf{v}}f(x_0,y_0,z_0)$. Because $f$ is differentiable, we have:
\begin{align}
D_{\mathbf{v}}f(x_0,y_0,z_0)&=\overrightarrow{\nabla} f(x_0,y_0,z_0)\boldsymbol{\cdot} \mathbf{v}\\
&=|\overrightarrow{\nabla} f(x_0,y_0,z_0)| \underbrace{|\mathbf{v}|}_{=1} \cos\alpha \tag{$\mathbf{v}$ is a unit vector}\\
&=|\overrightarrow{\nabla} f(x_0,y_0,z_0)|\ \cos\alpha
\end{align}
where $\alpha$ is the angle between the gradient $\overrightarrow{\nabla} f(x_0,y_0,z_0)$ and the direction vector $\mathbf{v}$. Therefore:

The maximum value of $D_{\mathbf{v}}f(x_0,y_0,z_0)$ occurs when $\cos\alpha=1$ or $\alpha=0$. This means $D_{\mathbf{v}}f(x_0,y_0,z_0)$ reaches its maximum value when $\mathbf{v}$ has the same direction as $\overrightarrow{\nabla} f(x_0,y_0,z_0)$ (that is when $\mathbf{v}=\frac{\overrightarrow{\nabla} f(x_0,y_0,z_0)}{|\overrightarrow{\nabla} f(x_0,y_0,z_0)|}$), and the largest value of $D_{\mathbf{v}}f(x_0,y_0,z_0)$ is $\left|\overrightarrow{\nabla} f(x_0,y_0,z_0)\right|$.

When $\mathbf{v}$ has the opposite direction as $\overrightarrow{\nabla} f(x_0,y_0,z_0)$; that is when $\mathbf{v}=-\frac{\overrightarrow{\nabla} f(x_0,y_0,z_0,z_0)}{|\overrightarrow{\nabla} f(x_0,y_0,z_0)|}$, we have $\alpha=\pi$ or $\cos\alpha = -1$. This means moving in the opposite direction of $\overrightarrow{\nabla} f(x_0,y_0,z_0)$, the function decreases most rapidly.

If we move in a direction that is normal to $\overrightarrow{\nabla} f(x_0,y_0,z_0)$ ($\mathbf{v}\bot\overrightarrow{\nabla} f(x_0,y_0,z_0)$), there is zero change in $f$.

When $\overrightarrow{\nabla} f(x_0,y_0,z_0)=(0,0,0)$, the rate of change of $f$ is zero in all directions (because $D_{\mathbf{v}}f(x_0,y_0,z_0)=0$ for every $\mathbf{v}$).

Figure 1. A function $f(x,y)$ increases most rapidly if we move in the direction of its gradient and decreases most rapidly if we move in the opposite direction of its gradient.

This way we could prove the following theorem.

Theorem 1. Suppose a function $f:U\subseteq\mathbb{R}^n\to\mathbb{R}$ is differentiable at $\mathbf{x_0}$. Then

If $\overrightarrow{\nabla} f(\mathbf{x}_0)\neq\mathbf{0}$, $f$ increases most rapidly if we move in the direction of $\overrightarrow{\nabla} f(\mathbf{x}_0)$. The maximum value of $D_{\mathbf{v}}f(\mathbf{x}_0)$ is $|\overrightarrow{\nabla} f(\mathbf{x}_0)|$.
If $\overrightarrow{\nabla} f(\mathbf{x}_0)\neq\mathbf{0}$, $f$ decreases most rapidly if we move in the opposite direction of $\overrightarrow{\nabla} f(\mathbf{x}_0)$. The minimum value of $D_{\mathbf{v}}f(\mathbf{x}_0)$ is $-|\overrightarrow{\nabla} f(\mathbf{x}_0)|$.
Suppose $\overrightarrow{\nabla} f(\mathbf{x}_0)\neq\mathbf{0}$. If we move in a direction that is normal to $\overrightarrow{\nabla} f(\mathbf{x}_0)$, there will be no change in $f$.
If $\overrightarrow{\nabla} f(\mathbf{x}_0)=\mathbf{0}$, the rate of change of $f$ is zero in all directions. For every $\mathbf{v}$, we have $D_{\mathbf{v}}f(\mathbf{x}_0)=0$.

Example 1

Suppose the temperature (measured in degree Fahrenheit) in the space when
$-\frac{\pi}{5}\leq z\leq \frac{\pi}{5}$ is given by $T(x,y,z)=1000 e^{-3x} e^{-4y} \cos 5z$, where $x,y$ and $z$ are measured in inches. If you are at $(-4,3,0)$, which way should you move to cool down fastest? What is the rate of temperature decrease?

Solution

Temperature decreases most rapidly if we move in the opposite direction of
$\overrightarrow{\nabla} T$. So we need to find $\overrightarrow{\nabla} T(-4,3,0)$:
\[\begin{align} \overrightarrow{\nabla} T(x,y,z)&=\left(\frac{\partial T}{\partial x},\frac{\partial T}{\partial y},\frac{\partial T}{\partial z}\right)\\ &=(-3000 e^{-3x-4y} \cos 5z, -4000 e^{-3x-4y} \cos 5z,-5000 e^{-3x-4y} \sin 5z)\\ \Rightarrow \quad \overrightarrow{\nabla} T(-4,3,0)&=(-3000,-4000,0)\end{align}\]
Therefore, to cool down most rapidly we need to move in the direction of
$-\overrightarrow{\nabla} T(-4,3,0)$ and our unit vector is
\[\overrightarrow{\nabla}{v}=-\frac{\overrightarrow{\nabla} T(-4,3,0)}{|\overrightarrow{\nabla} T(-4,3,0)|}=\frac{(3000,4000,0)}{5000}=(3/5,4/5,0)\]

The rate of change in temperature if we move along $(3/5,4/5,0)$ would be
\[-|\overrightarrow{\nabla} T(-4,3,0)|=-\sqrt{3000^2+4000^2}=-5000\ ^\circ{\rm F}\text{ per inch.}\]

Example 2

Suppose you are climbing a mountain. Further suppose you are equipped with a map showing the height of each point of the mountain is given by $h(x,y)=4-x^2-4y^2$ (all measured in kilometers). If you want to start climbing from $(1,-\frac{1}{2})$, find the path to the peak of the mountain if you always move in the direction of maximum height increase.

Solution

If you want to always climb in the direction of maximum increase in height, you have to move in the direction of $\overrightarrow{\nabla} h$. If $\mathbf{r}(t)=(x(t),y(t))$ is the parametrization of the path, we should have:
\[\mathbf{r}'(t)\parallel \overrightarrow{\nabla} h(x,y) \Rightarrow \left(\frac{dx}{dt},\frac{dy}{dt}\right)=k\left(\frac{\partial h}{\partial x},\frac{\partial h}{\partial y}\right)\]
for some $k>0$ which may vary along the path. In other words, at each point of the path we have a different $k$. Because $\left(\frac{\partial h}{\partial x},\frac{\partial h}{\partial y}\right)=(-2x,-8y)$, we have:
\[\frac{dx}{dt}=-2kx,\quad \frac{dy}{dt}=-8ky, \quad \Rightarrow\quad \frac{dx}{2x}=-kdt, \quad \frac{dy}{8y}=-kdt\]
Therefore we have:
\[\frac{dx}{2x}=\frac{dy}{8y}\quad 4\frac{dx}{x}=\frac{dy}{y}\] By integrating both sides we have:
\[\int 4\frac{dx}{x}=\int\frac{dy}{y}\Rightarrow 4\ln |x|=\ln |y|+C \Rightarrow \ln x^4=\ln |y|+C\]
Thus: \[y=A x^4\] where $A=1/e^C$. We can find $A$ such that $(1,-\frac{1}{2})$ lies on the path: \[-\frac{1}{2}=A \times 1^4\Rightarrow A=-\frac{1}{2}\] and our path to the peak is $y=-\frac{1}{2} x^4$. This path is shown in thick black in the following figure. As we enforced, the path is along the gradient vectors. We also see that the path is perpendicular to level curves at the points of intersections. This is not a coincidence; gradient at each point is perpendicular to level curves (or level surfaces).