3.10 Differentiability

Alternative Definition for the Differentiability of Single-Variable Functions 

 

Show the alternative definition for differentiability of single-variable functions

Let’s start with a function \(f\) of single variable \(x\). The derivative of \(f\) at the point \(x\) is \[\lim_{h\rightarrow 0}\frac{f(x+h)-f(x)}{h}=f'(x)\]
if this limit exists. Let’s define \(\phi(h)\) as
\[\phi(h)=\frac{f(x+h)-f(x)}{h}-f'(x).\]
Although \(\phi(h)\) is not defined at \(h=0\), but*

\[\lim_{h\rightarrow 0} \phi(h)=0.\]

* In general, for a function \(g(x)\) suppose  \(\lim_{x\rightarrow a}g(x)=L\). Because \(\lim_{x\rightarrow a}(g(x)-L)=0\) if we define \(\phi(x)\) as \(\phi(x)=g(x)-L\), then \(\lim_{x\to 0}\phi(x)=0\).

We can write (if \(h\neq 0\)):
\[\phi(h)=\frac{f(x+h)-f(x)}{h}-f'(x)\Rightarrow f(x+h)-f(x)=f'(x) h+h \phi(h)\]
or
\[\begin{aligned} f(x+h)=f(x)+f'(x) h+h\phi(h)\end{aligned}\]

Let’s define \(\varepsilon(h)\) by:
\[\varepsilon(h)=\left\{\begin{array}{ll} \phi(h) & \text{if } h>0\\ -\phi(h) & \text{if } h<0 \end{array} \right.\]
Therefore, if \(f(x)\) is differentiable, there exists a function \(\varepsilon(h)\) such that
\[f(x+h)=f(x)+f'(x)h+|h|\varepsilon(h),\]
and \(\varepsilon(h)\rightarrow0\) as \(h\rightarrow0\).

Recall that \(f(x)+f'(x)h\) is the linearization of \(f\) at the point \(x\). Therefore \(|h|\varepsilon(h)\) is the error in this approximation. Eq. (i) means that if \(f'(x)\) exists, the function \(f\) can be approximated by its linear approximation and the growth of the error is nothing compared to \(|h|\), that is error\(|h|\rightarrow 0\) as \(h\to0\) [1], where \(|h|\) is the distance of the point \((x+h)\) from the point \(x\).

Conversely, suppose there exists a number \(a\) and a function \(\varepsilon(h)\) such that
\[\label{Eq:Diff-1D} f(x+h)=f(x)+ah+|h|\varepsilon(h), \quad \text{and}\quad \lim_{h\to 0}\varepsilon(h)=0.\tag{i}\]
Dividing both sides by \(h\neq 0\):
\[\frac{f(x+h)-f(x)}{h}=a+\frac{|h|}{h}\varepsilon(h)\]
and taking the limit as \(h\to 0\):
\[\lim_{h\to 0}\frac{f(x+h)-f(x)}{h}=a+\underbrace{\lim_{h\to 0} \frac{|h|}{h}\varepsilon(h)}_{=0} \Rightarrow \lim_{h\to 0}\frac{f(x+h)-f(x)}{h}=a.\]
This means the derivative of \(f\) at the point \(x\) exists and is equal to \(a\):
\[f'(x)=a.\]

Therefore, we can define the differentiability of a function \(f\) at the point \(x\) as the existence of a number \(a\) and a function \(\varepsilon\) that satisfy Eq. (i). We can extend the definition for functions of two or more variables.

 

Differentiability of Two-Variable Functions

 

Let \(z=f(x,y)\). We say \(f\) is differentiable at the point \((x,y)\) if there exist two numbers \(a\) and \(b\), and a function \(\varepsilon(h,k)\) such that:
\[f(x+h,y+k)=f(x,y)+ah+bk+\underbrace{\left|(h,k)\right|}_{=\sqrt{h^2+h^2}}\varepsilon(h,k)\]
and**

\[\lim_{(h,k)\rightarrow (0,0)}\varepsilon(h,k)=0.\]

** Note that the magnitude (or the absolute value) of a vector \((h,k)\) is: \(|(h,k)|=\sqrt{h^2+k^2}\)

If such an approximation is valid, let \(k=0\), divide both sides by \(h\neq 0\) and take the limit \(h\to 0\), then we will have:
\[\underbrace{\lim_{h\to 0} \frac{f(x+h,y)-f(x,y)}{h}}_{=f_x(x,y)}=a+\underbrace{\lim_{h\to 0}\frac{\sqrt{h^2}}{h}\varepsilon(h,0)}_{=0}\Rightarrow f_x(x,y)=a.\]
Similarly, we can show: \[f_y(x,y)=b\]

Definition 1. We say a function \(f\) is differentiable at the point \((x,y)\) if its partial derivatives \(f_x(x,y)\) and \(f_y(x,y)\) exist and there exists a function \(\varepsilon\) such that:
\[f(x+h,y+k)=f(x,y)+f_x(x,y)h+f_y(x,y)k+\sqrt{h^2+k^2}\varepsilon(h,k)\]
and
\[\lim_{(h,k)\rightarrow (0,0)}\varepsilon(h,k)=0.\]

The above definition means that if \(f(x,y)\) is differentiable, it can be approximated by its linearization and the growth of the error in this approximation, \(\sqrt{h^2+k^2}\varepsilon(h,k)\), is nothing compared to the growth of \(\rho\), where \(\rho=\sqrt{h^2+k^2}\) is the distance of the point \((x+h,y+k)\) from the point \((x,y)\).

Example 1
Show that \(f\) is differentiable at each point \((x,y)\) if \[f(x,y)=2x+3y^2\]
Solution
First let’s compute \(f_x(x,y)\) and \(f_y(x,y)\)
\[f_x(x,y)=2,\quad f_y(x,y)=6y\]

Now let’s form \(f(x+h,y+k)\):
\[\begin{aligned} f(x+h,y+k)&=2(x+h)+3(y+k)^2\\ &=2x+2h+3(y^2+2yk+k^2)\\ &=\underbrace{2x+3y^2}_{f(x,y)}+\underbrace{2}_{f_x(x,y)}h+\underbrace{6y}_{f_y(x,y)}k+3k^2\end{aligned}\]
If we call \(\varepsilon(h,k)=\frac{3k^2}{\sqrt{h^2+k^2}}\), then we have shown
\[f(x+h,y+k)=f(x,y)+f_x(x,y)h+f_y(x,y)k+\sqrt{h^2+k^2}\varepsilon(h,k).\]
The last step is to show \(\lim_{(h,k)\rightarrow (0,0)}\varepsilon(h,k)=0\). To this end, we can use polar coordinates
\[h=r\cos\theta,\quad k=r\sin\theta\]
Now \((h,k)\to(0,0)\) is equivalent to \(r\to 0\). Therefore:
\[\begin{aligned} \lim_{(h,k)\rightarrow (0,0)}\varepsilon(h,k)&=\lim_{(h,k)\rightarrow (0,0)}\frac{3k^2}{\sqrt{h^2+k^2}}\\ &=\lim_{r\rightarrow 0}\frac{3r\sin^2\theta}{r\underbrace{\sqrt{\cos^2\theta+\sin^2\theta}}_{=1}}\\ &=\lim_{r\rightarrow 0} 3r\sin\theta=0.\end{aligned}\]
[Because \(-r\leq r\sin\theta\leq r\) and \(\lim_{r\to 0} r=0\), if we use the squeeze theorem, we could conclude \(\lim_{r\rightarrow 0} 3r\sin\theta=0\).]

Therefore, we have proved that \(f(x,y)\) is differentiable at each \((x,y)\).

From the definition of differentiability it is clear† that if a function is differentiable at \((x,y)\) then it is continuous there. Therefore, if a function is not continuous, it cannot be differentiable.

†Take the limit \((h,k)\to(0,0)\) from both sides, and remember that \(f_x(x,y)\) and \(f_y(x,y)\) are the values of \(f_x\) and \(f_y\) at \((x,y)\) not two functions of \(h\) or \(k\), so they are just two constants. 

differentiability \(\Rightarrow\) continuity

Also according to Definition 1, if a function is differentiable, its first partial derivatives exist. Therefore, if any of the first partial derivatives of a function does not exist at a point, the function is not differentiable at that point.

differentiability \(\Rightarrow\) existence of first partial derivatives

Example 2

Show that \(f(x,y)\) is not differentiable at \((0,0)\) if \[f(x,y)=\left\{\begin{array}{ll} \frac{xy}{x^2+y^2} & \text{if } (x,y)\neq (0,0)\\ 0 & \text{if } (x,y)= (0,0) \end{array} \right.\]

Solution

In Example 5 of the Section on Limits and Continuity, we showed that \(\lim_{(x,y)\to (0,0)}f(x,y)\) does not exist. Therefore, \(f\) is discontinuous at \((0,0)\). A discontinuous function cannot be differentiable.

Also in Example 2 of the Section on Linear Approximation, we showed that \(f_x(0,0)=f_y(0,0)=0\) and the linearization of \(f\) does not provide a good approximation for \(f\) at \((0,0)\). So the existence of the partial derivatives does not guarantee that the function is differentiable or even continuous.

Using Definition 1 to verify whether a function is differentiable is often hard. Here we introduce a theorem that can be applied to most functions to show that they are differentiable.

Theorem 1. If the first partial derivatives of a function \(f\) exist in some neighborhood of \(\mathbf{x}_0\) and are continuous at \(\mathbf{x}_0\), then \(f\) is differentiable at \(\mathbf{x}_0\).

Example 3

Use Theorem 1 to show that \(f\) is differentiable everywhere if \[f(x,y)=\left\{\begin{array}{ll} \frac{x^2y^2}{x^2+y^2} & \text{if } (x,y)\neq (0,0)\\ 0 & \text{if }(x,y)= (0,0) \end{array} \right.\]

Figure 1. Graph of \(z=\dfrac{x^2y^2}{x^2+y^2}\)
Solution

To find \(f_x\) and \(f_y\), we have to consider two cases: (1) \((x,y)=(0,0)\) and (2) \((x,y)\neq (0,0)\).

Case (1): To calculate \(f_x(0,0)\) and \(f_y(0,0)\), we need to use the definition of partial derivatives (similar to Example 2 of the Section on Linear Approximation):
\[f_x(0,0)=\lim_{h\to 0}\frac{f(0+h,0)-f(0,0)}{h}=\lim_{h\to 0}\frac{0-0}{h}=0,\]
\[f_y(0,0)=\lim_{k\to 0}\frac{f(0,0+k)-f(0,0)}{k}=\lim_{k\to 0}\frac{0-0}{k}=0,\]

Case (2): If \((x,y)\neq(0,0)\), we can differentiate \(f\) with respect to \(x\) while treating \(y\) as a constant to find \(f_x(x,y)\) and differentiate \(f(x,y)\) with respect to \(y\), while treating \(x\) as a constant to find \(f_y(x,y)\) (although because of symmetry of \(x\) and \(y\), to find \(f_y(x,y)\), we just need to replace \(x\) by \(y\) and \(y\) by \(x\) in the formula of \(f_x(x,y)\)). We have
\[\begin{aligned} f_x(x,y)&=\frac{2xy^2(x^2+y^2)-2x(x^2y^2)}{(x^2+y^2)^2}=\frac{2xy^4}{(x^2+y^2)^2},\\ f_y(x,y)&=\frac{2yx^4}{(x^2+y^2)^2}.\end{aligned}\]
Therefore:
\[f_x(x,y)=\left\{\begin{array}{ll} \frac{2xy^4}{(x^2+y^2)^2} & \text{if }(x,y)\neq (0,0)\\ 0 & \text{if }(x,y)= (0,0) \end{array} \right.\]
and
\[f_y(x,y)=\left\{\begin{array}{ll} \frac{2x^4y}{(x^2+y^2)^2} & \text{if }(x,y)\neq (0,0)\\ 0 & \text{if }(x,y)= (0,0) \end{array} \right.\]
It is clear that when \((x,y)\neq (0,0)\), \(f_x(x,y)\) and \(f_y(x,y)\) are continuous, because the numerators and denominators are continuous functions and the the ratio of two continuous functions is a continuous function if the denominator is nonzero (recall Theorem 3 in the Section on Limits and Continuity). We just need to show that \(f_x\) and \(f_y\) are continuous at \((0,0)\). To this end, again we can use polar coordinates:
\[x=r\cos \theta,\quad y=r\sin\theta.\]
In polar coordinates \((x,y)\to(0,0)\) is equivalent to \(r\to 0\):
\[\lim_{(x,y)\to(0,0)}\frac{2xy^4}{(x^2+y^2)^2}=\lim_{r\to 0}\frac{2r^5 \cos\theta\sin^4\theta}{r^4\underbrace{(\cos^2\theta+\sin^2\theta)^2}_{=1}}=\lim_{r\to 0} 2r\cos\theta\sin^4\theta=0.\]

So we have shown \(\lim_{(x,y)\to(0,0)}f_x(x,y)=f_x(0,0)=0\), that is \(f_x\) is continuous at \((0,0)\). In a similar way, you can show that \(\lim_{(x,y)\to(0,0)}f_y(x,y)=f_y(0,0)=0\). Therefore \(f\) has continuous first partial derivatives at each \((x,y)\). So according to Theorem 1, \(f\) is differentiable everywhere.

 

Differentiability of  Functions in n-Space

 

At the end of this section, we can extend the concept of differentiability of functions of several variables:

Definition 2. We say a function \(f:U\subseteq \mathbb{R}^n\to \mathbb{R}\) is differentiable at \((x_1,\cdots,x_n)\), if its first partial derivatives \(\frac{\partial f}{\partial x_i}(x_1,\cdots,x_n)\) (for \(i=1,\cdots,n\)) exist and there exists a function \(\varepsilon(h_1,\cdots,h_n)\) such that:
\begin{align}
f(x_1+h_1,\cdots,x_n+h_n)=f&(x_1,\dots,x_n)+\frac{\partial f}{\partial x_1}\cdot h_1+\cdots+\frac{\partial f}{\partial x_n}\cdot h_n\\
&+\sqrt{h_1^2+\cdots+h_n^2}\ \varepsilon(h_1,\cdots,h_n)\tag{i}
\end{align}
and
\[\lim_{(h_1,\cdots,h_n)\to(0,\cdots,0)} \varepsilon(h_1,\cdots,h_n)=0.\]
The partial derivatives are evaluated at \((x_1,\cdots,x_n)\).

  • Equation (i) can also be written as \[f(\mathbf{x}+\mathbf{h})=f(\mathbf{x})+\sum_{i=1}^n \frac{\partial f}{\partial x_i} h_i+|\mathbf{h}|\varepsilon(\mathbf{h}),\] with $\mathbf{x}=(x_1,\dots,x_n)$, $\mathbf{h}=(h_1,\dots,h_n)$, and \[\lim_{\mathbf{h}\to\mathbf{0}}\varepsilon(\mathbf{h})=0.\]

 

Continuous Differentiability and Functions of Class $C^p$

A function that has continuous first partial derivatives is called continuously differentiable or a function of class \(C^1\). A function that not only its first partial derivatives are continuous but also all its second partial derivatives are continuous is called twice continuously differentiable or a function of class \(C^2\). In the same manner we can define function of class \(C^3\), \(C^4\), and so on. A function that has continuous partial derivatives of all orders is called a \(C^\infty\) function. A function that is continuous is referred to a function of class \(C^0\).

Definition 3.Let $f$ be defined on an open set $U\subseteq \mathbb{R}^n$. We say $f$ is of class $C^p$ in $U$ if all the partial derivatives of $f$ of order $\leq p$ are continuous everywhere in $U$.

[1] Mathematically, this is often written as \(error=o(|h|)\)