3.19 Constrained Extrema and Lagrange Multipliers

Table of Contents

Constrained Extrema

In practice, we wish to optimize a function considering some existing constraints. In economics and engineering, the constrain may be due to limited funds, materials, or energy. If we wish to find the distance from a point $P = (x_{0}, y_{0})$ to a line $a x + b y = c$ , we should find the minimum value of $d (x, y) = \sqrt{(x - x_{0})^{2} + (y - y_{0})^{2}}$ while $(x, y)$ satisfies $a x + b y = c$ . Suppose $T (x, y, z)$ represents the temperature in space, and we want to find the maximum temperature on a surface given by $g (x, y, z) = 0$ . When the constraint (also called the side condition) equation is given, we can solve it for one of the variables, say $z = ϕ (x, y)$ , and replace it in the function $T$ . Then the problem would reduce to finding the extremum value of the function $T (x, y, ϕ (x, y))$ , which now depends only on two independent variables. We have already applied this method in this example, where we optimized the value of the function on the boundary of a region. To practice how to use this method, let’s consider the following example.

Example 1

We want to make a rectangular box without a top, and of given volume

V

. If the least amount of material is to be used, determine the design specifications.

Solution

Let

x =

length of the box

y =

width of the box

z =

height of the box
where

x, y

and

z

are in the interval

(0, \infty)

. The specified volume is

V = x y z

The amount of material for constructing this box is proportional to its surface area

S = x y + 2 x z + 2 y z .

So we want to minimize

S (x, y, z)

subject to

x y z = V

From $x y z = V$ we obtain $z = \frac{V}{x y}$ and then we plug it into the formula of $S$ : $S = x y + 2 \frac{V}{y} + 2 \frac{V}{x} .$ Now $S$ is expressed as a function of only two variables. To determine the relative extremum, we differentiate and set the partial derivatives equal to zero:
$\frac{\partial S}{\partial x} = y - \frac{2 V}{x^{2}} = 0, \frac{\partial S}{\partial y} = x - \frac{2 V}{y^{2}} = 0$
or equivalently:
${\begin{cases} x^{2} y = 2 V \\ x y^{2} = 2 V \end{cases}$ If we divide the first question by the second one, we obtain $x / y = 1$ . Therefore: $x^{2} y = x^{2} x = x^{3} = 2 V \Rightarrow x = y = (2 V)^{1 / 3}$ From these values of $x$ and $y$ , we get $z = \frac{V}{x y} = \frac{(2 V)^{1 / 3}}{2}$ Using the second derivative test, we can show that these values of $x, y$ and $z$ give a relative minimum of $S$ . Because $S \to \infty$ as either $x \to 0 +$ , $y \to 0 +$ , $x \to \infty$ , or $y \to \infty$ , we can conclude that the relative minimum is also the absolute minimum.

Lagrange Multipliers

When the constraint is given implicitly by $g (x, y, z) = c$ , it is not always possible or easy to solve the constraint equation for one of the variables (express $x, y$ or $z$ as a function of the remaining variables). The problem can be more complicated when there is more than one constraint. In such cases, an alternative procedure is a method called Lagrange multipliers.

To explain this method, let’s start with an example. Suppose we want to find the shortest and longest distance between the point $(1, - 1)$ and the curve $C$

$g (x, y) = \frac{(x - y - 2)^{2}}{18} + \frac{(x + y)^{2}}{32} = 1.$

The distance between a point $(x, y)$ and $(1, - 1)$ is given by $f (x, y) = \sqrt{(x - 1)^{2} + (y + 1)^{2}} .$ So we want to maximize and minimize $f (x, y)$ subject to $g (x, y) = 1$ . First let’s sketch the curve $C$ and some level curves of $f$ (Fig. 1).

Figure. 1: The orange lines show contour lines of

f

with different values. The function

f

takes on extreme values at four points

A, B, P

, and

Q

To extremize $f (x, y)$ subject to $g (x, y) = 1$ , we have to find the largest and smallest value of $k$ such that the level curve $f (x, y) = k$ intersects $g (x, y) = 1$ . Among the level curves that intersect $g (x, y) = 1$ , the minimum value of $f (x, y)$ occurs at the points $P$ and $Q$ where $f (x, y)$ has a value of 3 (see Fig. 1). At these points, the constraint curve $g (x, y) = 1$ and the level curve $f (x, y) = 3$ just touch each other; in other words, $f = 3$ and $g = 1$ have a common tangent line at $P$ and a common tangent at $Q$ .

Note that $g (x, y) = 1$ is the level curve of $z = g (x, y)$ . Because at each point $\vec{\nabla} f$ is perpendicular to the level curves of $f$ and similarly $\vec{\nabla} g$ is perpendicular to the level curves of $g$ , a common tangent at $P$ means that $\vec{\nabla} f (P)$ and $\vec{\nabla} g (P)$ are parallel. That is, there is a number $λ_{1}$ such that
$\vec{\nabla} f (P) = λ_{1} \vec{\nabla} g (P) .$
Similarly there is a number $λ_{1}$ such that
$\vec{\nabla} f (Q) = λ_{1} \vec{\nabla} g (Q) .$

We also observe that the maximum value of $f (x, y)$ subject to $g (x, y) = 1$ occurs where the constraint curve and a level curve (here $f = 4$ ) touch each other (In Fig. 1, they are denoted by $A$ and $B$ ). Thus $\vec{\nabla} f (A) = μ_{1} \vec{\nabla} g (A),$ and $\vec{\nabla} f (B) = μ_{2} \vec{\nabla} g (B),$ for some $μ_{1}$ and $μ_{2}$ . Therefore, to find the maximum or minimum of $f (x, y)$ subject to the constraint $g (x, y) = 1$ , we look for a point $x_{0}$ such that
$\vec{\nabla} f (x_{0}) = λ \vec{\nabla} g (x_{0})$
for some $λ$ . This is the method of Lagrange multiplier. But why this is true?

Suppose the constraint curve $C$ is parameterized by some functions:¹ $x = X (t), y = Y (t)$ If, in the equation of $f$ , $x$ and $y$ are replaced by $X (t)$ and $Y (t)$ , then the distance between $(1, - 1)$ and points on $C$ becomes a function of $t$ $F (t) = f (x (t), y (t)) .$ Therefore, the extreme values of the distance occur where $F^{'} (t) = 0$ . From the chain rule we know
$F^{'} (t) = \frac{d F}{d t} = \frac{\partial f}{\partial x} \frac{d X}{d t} + \frac{\partial f}{\partial y} \frac{d Y}{d x}$
We can write the equation $F^{'} (t) = 0$ as

$\begin{aligned} \frac{\partial R}{\partial x} \frac{d X}{d t} + \frac{\partial R}{\partial y} \frac{d Y}{d x} & = 0 \\ (\frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}) \cdot (\frac{d X}{d t}, \frac{d Y}{d t}) & = 0 \end{aligned}$
This means at the extreme points the gradient vector $(f_{x}, f_{y})$ is perpendicular to $(X^{'} (t), Y^{'} (t))$ . Recall that $(X^{'} (t), Y^{'} (t))$ is tangent to the curve $C$ .

On the other hand $C$ is a level curve of $g$ . Therefore, at each point $\vec{\nabla} g$ is perpendicular to $C$ . Therefore at the extreme points, $\vec{\nabla} g$ and $\vec{\nabla} f$ are parallel. The simplest version of the method of Lagrange multipliers is as follows.

Theorem 1. Suppose $U \subseteq R^{n}$ is an open set and $f : U \to R$ and $g : U \to R$ are two continuously differentiable functions². Let $S = {x \in U | g (x) = c}$ be the level set for $g$ with value $c$ . If $f | S$ denoting “ $f$ restricted to $S$ ,” has a relative extremum on $S$ at $x_{0} \in U$ , and $\vec{\nabla} g (x_{0}) \neq 0$ , then there exists a real number $λ$ such that
$\vec{\nabla} f (x_{0}) = λ \vec{\nabla} g (x_{0}) .$

Read the proof

Let $r : I \subseteq R \to R^{n}$ be a differentiable curve on the level set $S$ such that $r (t_{0}) = x_{0}$ , $r^{'} (t_{0}) \neq 0$ , and $r (t) \in S$ for every $t \in I$ . Then $g (r (t)) = c$ , so the chain rule gives
$\vec{\nabla} g (\underset{= x_{0}}{\underset{⏟}{r (t_{0})}}) \cdot r^{'} (t_{0}) = 0.$
Because $f | S$ attains a relative maximum or minimum at $x_{0}$ , the function $h = f \circ r : I \to R$ attains a relative maximum or minimum at $t_{0}$ . Hence $h^{'} (t_{0}) = 0$ and according to the chain rule we have:
$h^{'} (t_{0}) = \vec{\nabla} f (x_{0}) \cdot r^{'} (t_{0}) = 0.$

Thus the vectors $\vec{\nabla} f (x_{0})$ and $\vec{\nabla} g (x_{0}) \neq 0$ are both normal to the nonzero vector $r^{'} (t_{0})$ and are therefore parallel; that is, $\vec{\nabla} f (x_{0}) = λ \vec{\nabla} g (x_{0})$ for some $λ$ .

The number $λ$ in the above theorem is called a Lagrange multiplier.
$λ$ might be zero.

Note that to find the extremum of $f | S$ , we have $n + 1$ unknowns ( $n$ components of $x_{0}$ and $λ$ ) and $n + 1$ equations:
$\begin{matrix} (i) & \begin{aligned} \frac{\partial f}{\partial x_{1}} (x_{0}) & = λ \frac{\partial g}{\partial x_{1}} (x_{0}) \\ ⋮ \\ \frac{\partial f}{\partial x_{n}} (x_{0}) & = \frac{\partial g}{\partial x_{n}} (x_{0}) \\ g (x_{0}) & = c \end{aligned}} (n + 1) equations \end{matrix}$

Example 2

Find the extrema of the function

f (x, y) = x y

subject to the constraint

\frac{1}{4} x^{2} + \frac{1}{9} y^{2} = 1

Solution

Let

g (x, y) = \frac{1}{4} x^{2} + \frac{1}{9} y^{2}

, so

S

consists of all points

(x, y)

such that

g (x, y) = 1

. We have:

\vec{\nabla} f (x, y) = (y, x), \vec{\nabla} g (x, y) = (\frac{1}{2} x, \frac{2}{9} y)

Note that $\vec{\nabla} g = 0$ if and only if $(x, y) = (0, 0)$ . Thus if $f$ subject to the constraint has an extremum at $(x_{0}, y_{0})$ , we must have $\vec{\nabla} f (x_{0}, y_{0}) = λ \vec{\nabla} g (x_{0}, y_{0})$ . Equations i take on the form:
$y_{0} = \frac{1}{2} λ x_{0}, x_{0} = \frac{2}{9} λ y_{0}, \frac{1}{4} x_{0}^{2} + \frac{1}{9} y_{0}^{2} = 1.$
There are different ways to solve the above system of equations. One way is to replace $y_{0}$ from the first equation in the second equation
$x_{0} = \frac{2}{9} λ y_{0} = \frac{2 λ}{9} (\frac{λ}{2} x_{0}) = \frac{1}{9} λ^{2} x_{0}$
This leads to
$x_{0} - x_{0} \frac{λ^{2}}{9} = x_{0} (1 - \frac{λ^{2}}{9}) = 0 \Rightarrow x = 0 or λ = \pm 3.$
If $x = 0$ , then substituting in $g (x_{0}, y_{0}) = 1$ , we obtain $\frac{1}{4} 0^{2} + \frac{1}{9} y_{0}^{2} = 1 \Rightarrow y_{0} = \pm 3.$ If $λ = \pm 3$ , it follows from $y_{0} = \frac{1}{2} λ x_{0}$ that $y_{0} = \pm \frac{3}{2} x_{0}$ . Substituting into $g (x_{0}, y_{0})$ , we get:
$\frac{1}{4} x_{0}^{2} + \frac{1}{9} \frac{9}{4} x_{0}^{2} = 1 \Rightarrow \frac{1}{\sqrt{2}} x_{0}^{2} = 1 \Rightarrow x_{0} = \pm \sqrt{2} .$
If $x_{0} = \sqrt{2}$ , then $y_{0} = \pm \frac{3}{2} \sqrt{2}$ , and if $x_{0} = - \sqrt{2}$ , then $y_{0} = \pm \frac{3}{2} \sqrt{2}$ .

Now we list the points we found and the values of $f$ at these points:

$\large (x_0,y_0)$	$(0, 3)$	$(0, - 3)$	$(\sqrt{2}, \frac{3 \sqrt{2}}{2})$	$(\sqrt{2}, - \frac{3 \sqrt{2}}{2})$	$(\sqrt{2}, - \frac{3 \sqrt{2}}{2})$	$(- \sqrt{2}, - \frac{3 \sqrt{2}}{2})$
$f (x_{0}, y_{0}) = x_{0} y_{0}$	$0$	$0$	$3$	$- 3$	$- 3$	$3$

The function $f$ attains its maximum value 3 at $(\sqrt{2}, 3 \sqrt{2} / 2)$ and $(- \sqrt{2}, - 3 \sqrt{2} / 2)$ , and its minimum value -3 at $(- \sqrt{2}, 3 \sqrt{2} / 2)$ and $(- \sqrt{2}, 3 \sqrt{2} / 2)$ . Note that at these points the level curve $g = 1$ is tangent to one of the level curves of $f$ (see Fig. 2).


(a)	(b)
Figure 2. (a) The graph of $f (x, y)$ ; the blue curve shows $f$ restricted to the ellipse $g (x, y) = 1$ . (b) The level curves of $f$ and $g$ ; the extrema of $f$ subject to the constraint $g (x, y) = 1$ occur at points where the level curves of $f$ are tangent to the level curve $g (x, y) = 1$ .

Let’s go back to the example of making an open box and try to solve it using the method of Lagrange multipliers.

Example 3

We want to make a rectangular box without a top, and of given volume

V

. If the least amount of material is to be used, determine the design specifications using Lagrange multipliers.

Solution

Again Let

x, y

and

z

be the length, width, and height of the box, respectively. We want to minimize

S = x y + 2 x z + 2 y z

subject to

x y z = V

, where

V

is the given volume.

If we let $g (x, y, z) = x y z$ , then
$\vec{\nabla} S = (y + 2 z, x + 2 z, 2 x + 2 y), \vec{\nabla} g = (y z, x z, x y)$
Note that $\vec{\nabla} g = 0$ if and only if $(x, y, z) = (0, 0, 0)$ which is not on the level surface $g (x, y, z) = V$ . So if $S$ has an extremum at $(x_{0}, y_{0}, z_{0})$ , we have
$\vec{\nabla} S (x_{0}, y_{0}, z_{0}) = λ \vec{\nabla} g (x_{0}, y_{0}, z_{0}) .$
Hence our equations become
$\begin{array}{r} {\begin{array}{lcc} y + 2 z = λ y z & (i) \\ x + 2 z = λ x z & (i i) \\ 2 x + 2 y = λ x y & (i i i) \\ x y z = V & (i v) \end{array} \end{array}$
Because $x, y$ and $z$ are nonzero, if we divide the first equation by $y z$ , the second equation by $x z$ and the third one $x y$ , we get:
$\begin{array}{r} {\begin{array}{lcc} \frac{1}{z} + \frac{2}{y} = λ & (i^{'}) \\ \frac{1}{z} + \frac{2}{x} = λ & (i i^{'}) \\ \frac{2}{y} + \frac{2}{x} = λ & (i i i^{'}) \\ x y z = V & (i v^{'}) \end{array} \end{array}$
Then
$Eq. (i^{'}), Eq. (i i^{'}) \Rightarrow x = y$
$Eq. (i i^{'}), Eq (i i i^{'}), and x = y \Rightarrow z = \frac{1}{2} x$
$x = y = 2 z, Eq (4) \Rightarrow \frac{1}{2} x^{3} = V \Rightarrow x = y = 2 z = (2 V)^{1 / 3} .$
This is the result we previously obtained in Example 1.

The following example shows an application of Lagrange multipliers in economics and when there are many variables.

Example 4

Assume there are

n

commodities with prices per unit

p_{1}, \dots, p_{n}

. Assume we have

L

dollars to spend on these commodities. This means we have the constraint

p_{1} x_{1} + \dots p_{n} x_{n} = L

where

x_{1}, \dots, x_{n}

are the amounts of the commodities. Assume the utility is given by the Cobb-Douglas function

U = f (x_{1}, \dots, x_{n}) = K x_{1}^{α_{1}} \dots x_{n}^{α_{n}}

where

K, α_{1}, \dots, α_{n}

are positive constants. Find the maximum utility we can achieve.

Solution

Let

g (x_{1}, \dots, x_{n}) = p_{1} x_{1} + \dots + p_{n} x_{n}

. So we seek for the maximum value of

f (x_{1}, \dots, x_{n})

subject to

g (x_{1}, \dots, x_{n}) = L

. Because

x_{i} \geq 0

(for

i = 1, \dots, n

) the constraint is a part of the level set

g (x_{1}, \dots, x_{n}) = L

. We note that when the amount of a commodity

x_{i}

approaches zero, the utility approaches zero. Hence, the utility takes on its maximum value when

x_{i}

’s are positive. Because

\vec{\nabla} g (x_{1}, \dots, x_{n}) = (p_{1}, \dots, p_{n}) \neq 0

, if

f

attains its maximum at a point, we have

\vec{\nabla} f (x_{1}, \dots, x_{n}) = λ \vec{\nabla} g (x_{1}, \dots, x_{n})

; that is,

{\begin{cases} α_{1} K x_{1}^{α_{1} - 1} x_{2}^{α_{2}} \dots, x_{n}^{α_{n}} = λ p_{1} \\ ⋮ \\ α_{n} K x_{1}^{α_{1}} x_{2}^{α_{2}} \dots, x_{n}^{α_{n} - 1} = λ p_{n} \end{cases}

If we multiply both sides of the first equation by

x_{1}

, the second equation by

x_{2}

\dots

, and the

n

-th equation by

x_{n}

, we get:

{\begin{cases} α_{1} f (x_{1}, \dots, x_{n}) = λ p_{1} x_{1} \\ ⋮ \\ α_{n} f (x_{1}, \dots, x_{n}) = λ p_{n} x_{n} \end{cases}

Therefore:

\frac{p_{1}}{α_{1}} x_{1} = \dots = \frac{p_{n}}{α_{n}} x_{n}

This result makes sense because it says the amount of commodity

x_{i}

is proportional to its power and inversely proportional to its price,

p_{i}

Let $\frac{α_{i}}{p_{i}} x_{i} = C$ , then
$p_{1} x_{1} + \dots + p_{n} x_{n} = p_{1} \frac{α_{1}}{p_{1}} C + \dots + p_{n} \frac{α_{1}}{p_{n}} C = L$
or
$C (α_{1} + \dots + α_{n}) = C \sum_{i = 1}^{n} α_{i} = L \Rightarrow C = \frac{L}{\sum_{i = 1}^{n} α_{i}} .$
Consequently,
$x_{i} = \frac{α_{i}}{p_{i}} \frac{L}{\sum_{i = 1}^{n} α_{n}} .$ Finally we conclude the maximum value of $f$ is:
$K {(\frac{α_{i} L}{p_{i} \sum_{i = 1}^{n} α_{i}})}^{α_{1}} \dots {(\frac{α_{n} L}{p_{n} \sum_{i = 1}^{n} α_{i}})}^{α_{n}} = K {(\frac{L}{\sum_{i = 1}^{n} α_{i}})}^{\sum_{i = 1}^{n} α_{i}} {(\frac{α_{1}}{p_{1}})}^{α_{1}} \dots {(\frac{α_{n}}{p_{n}})}^{α_{n}}$

The following example shows when maximizing or minimizing a function subject to a constraint, we should also investigate the points where the gradient of the constraint is zero

\vec{\nabla} g (x) = 0

Example 5

Find the nearest point on the curve

y^{3} - x^{2} = 0

to the point

(0, - 1)

Solution

If we plot the level curve

g (x, y) = y^{3} - x^{2} = 0

(Fig. 3), it is clear that

(0, 0)

is the nearest point on this curve to the point

(0, - 1)

, and the distance is 1. If we want to use the method of Lagrange multipliers, we need to minimize the square of distance

f (x, y) = (x - 0)^{2} + (y - (- 1))^{2}

subject to the constraint

g (x, y) = y^{3} - x^{2} = 0

Figure 3.

Here
$\vec{\nabla} f (x, y) = (2 x, 2 y + 2), \vec{\nabla} g (x, y) = (- 2 x, 3 y^{2}) .$
Equations i take on the form
${\begin{cases} 2 x_{0} = - 2 λ x_{0} \\ 2 y_{0} + 2 = 3 λ y_{0}^{2} \\ y_{0}^{3} - x_{0}^{2} = 0 \end{cases}$
However the solution, $(x_{0}, y_{0}) = (0, 0)$ , does not satisfy the second equation. The reason is attributed to the fact that at this point the gradient of $g$ is $0$ , $\vec{\nabla} g (0, 0) = (0, 0)$ , so we cannot use Theorem 1. In addition, the level curve is not smooth at $(0, 0)$ .

If a function $f$ subject to a constraint $g (x) = c$ attains its extreme value at a point $x_{0}$ , then $x_{0}$ is one of the four types of the point:

$x_{0}$ is a point where $\vec{\nabla} f (x_{0}) = λ \vec{\nabla} g (x_{0})$ ,
$x_{0}$ is a point where $\vec{\nabla} g (x_{0}) = 0$ ,
$x_{0}$ is a rough point of $f$ or $g$ where $\vec{\nabla} f (x_{0})$ or $\vec{\nabla} g (x_{0})$ does not exist, or
$x_{0}$ is on the boundary of the level set $g (x) = c$ .

Multiple Constraints

The method of Lagrange multipliers extends to the case of multiple constraints: say to $f (x, y, z)$ subject to two constraints
$\begin{matrix} (ii) & g_{1} (x, y, z) = c_{1}, and g_{2} (x, y, z) = c_{2} . \end{matrix}$
Geometrically this means we seek the extreme values of $f (x, y, z)$ on a curve $C$ formed by the intersection of two level surfaces $g_{1} (x, y, z) = c_{1}$ and $g_{2} (x, y, z) = c_{2}$ (Fig. 4).

Figure 4. Because

\vec{\nabla} g_{1}

is perpendicular to the surface

g_{1} = c_{1}

and

\vec{\nabla} g_{2}

is perpendicular to the surface

g_{2} = c_{2}

, both

\vec{\nabla} g_{1}

and

\vec{\nabla} g_{2}

are perpendicular to C. At a point where

f

has a maximum or minimum, also

v e c \nabla f

is perpendicular to C.

Suppose $f$ subject to these constraints, $f | C$ , has an extremum at $P = (x_{0}, y_{0}, z_{0})$ , and the functions $f, g_{1}$ , and $g_{2}$ have continuous first partial derivatives near $P$ . Analogous to the case of one constraint, we can argue that $\vec{\nabla} f$ is perpendicular to $C$ at $P$ . Additionally because the gradient is perpendicular to the level surface, both $\vec{\nabla} g_{1}$ and $\vec{\nabla} g_{2}$ are perpendicular to $C$ . If $\vec{\nabla} g_{1} (x_{0}, y_{0}, z_{0})$ and $\vec{\nabla} g_{2} (x_{0}, y_{0}, z_{0})$ are neither zero vectors nor collinear vectors, then $\vec{\nabla} f (x_{0}, y_{0}, z_{0})$ lie in a plane spanned by the two vectors $\vec{\nabla} g_{1} (x_{0}, y_{0}, z_{0})$ and $\vec{\nabla} g_{2} (x_{0}, y_{0}, z_{0})$ and hence can be expressed as a linear combination of these two vectors, say:
$\begin{array}{r} (iii) & \vec{\nabla} f (x_{0}, y_{0}, z_{0}) = λ_{1} \vec{\nabla} g_{1} (x_{0}, y_{0}, z_{0}) + λ_{2} \vec{\nabla} g_{2} (x_{0}, y_{0}, z_{0}) \end{array}$
In this method, Equations ii and iii must be solved simultaneously for five unknowns $x_{0}, y_{0}, z_{0}, λ_{1}$ and $λ_{2}$ :
${\begin{cases} \frac{\partial f}{\partial x} (x_{0}, y_{0}, z_{0}) = λ_{1} \frac{\partial g_{1}}{\partial x} (x_{0}, y_{0}, z_{0}) + λ_{2} \frac{\partial g_{2}}{\partial x} (x_{0}, y_{0}, z_{0}) \\ \frac{\partial f}{\partial y} (x_{0}, y_{0}, z_{0}) = λ_{1} \frac{\partial g_{1}}{\partial y} (x_{0}, y_{0}, z_{0}) + λ_{2} \frac{\partial g_{2}}{\partial y} (x_{0}, y_{0}, z_{0}) \\ \frac{\partial f}{\partial z} (x_{0}, y_{0}, z_{0}) = λ_{1} \frac{\partial g_{1}}{\partial z} (x_{0}, y_{0}, z_{0}) + λ_{2} \frac{\partial g_{2}}{\partial z} (x_{0}, y_{0}, z_{0}) \\ g_{1} (x, y, z) = c_{1} \\ g_{2} (x, y, z) = c_{2} \end{cases}$

Example 6

Find the extreme points of the function

f (x, y, z) = x + 4 y - 2 z

on the curve of the intersection of the cylinder

x^{2} + y^{2} = 4

and the plane

2 y - z = 3

(Fig. 5).

Figure 5.

Solution

Here we are given two constraints:

g_{1} (x, y, z) = x^{2} + y^{2} = 4, g_{2} (x, y, z) = 2 y - z = 3.

Note that

\vec{\nabla} f = (1, 4, - 2), \vec{\nabla} g_{1} = (2 x, 2 y, 0), and \vec{\nabla} g_{2} = (0, 2, - 1) .

The vector

\vec{\nabla} g_{1} = (0, 0, 0)

only when

x = 0

and

y = 0

, which clearly does not satisfy the constraint

g_{1} = 4

. Thus, the two vectors

\vec{\nabla} g_{1}

and

\vec{\nabla} g_{2}

are clearly not parallel. Therefore, any constrained critical point

(x_{0}, y_{0}, z_{0})

must satisfy

\vec{\nabla} f (x_{0}, y_{0}, z_{0}) = λ_{1} \vec{\nabla} g_{1} (x_{0}, y_{0}, z_{0}) + λ_{2} \vec{\nabla} g_{2} (x_{0}, y_{0}, z_{0}) .

It follows that we must solve the following system of equations for five unknowns

x, y, z, λ_{1}

and

λ_{2}

{\begin{cases} 1 = 2 x λ_{1} + 0 \\ 4 = 2 y λ_{1} + 2 λ_{2} \\ - 2 = 0 - λ_{2} \\ x^{2} + y^{2} = 4 \\ 2 y - z = 3 \end{cases}

From the third equation, we know

λ_{2} = 2

. So the second equation becomes

2 y λ_{1} = 0

. Because from the first equation,

λ_{1} \neq 0

, it follows from

2 y λ_{1} = 0

that

y = 0

. If we substitute

y = 0

in the fourth and fifth equations, we get

x = \pm 2

and

z = 1

. Therefore,

f

may have extrema at

(\pm 2, 0, 1)

The condition $x^{2} + y^{2} = 4$ implies $- 2 \leq x, y \leq 2$ . Then it follows from $2 y - z = 3$ that $- 7 \leq z \leq 1$ . This means the constraint set, $S$ , is bounded. Because the constraint set $S$ is closed and bounded, and $f$ is a continuous function, $f | S$ assumes its maximum and minimum ( Theorem 1 in the previous section). Here we have only two potentials, therefore one of them $(2, 0, 1)$ is the maximum point and the other one $(- 2, 0, 1)$ is the minimum point.

By similar reasoning, we obtain equations for minimizing or maximizing
$f (x_{1}, \dots, x_{n})$ subject to several constraints
$\begin{array}{r} (iv) & {\begin{cases} g_{1} (x_{1}, \dots, x_{n}) = c_{1} \\ g_{2} (x_{1}, \dots, x_{n}) = c_{2} \\ ⋮ \\ g_{m} (x_{1}, \dots, x_{n}) = c_{m} \end{cases} \end{array}$
where $m < n$ . Assume $f (x_{1}, \dots, x_{n})$ has a relative extremum at $x_{0}$ when the variables are restricted by the constraint equations iv. If $f, g_{1}, \dots,$ and $g_{m}$ have continuous first partial derivatives at all points near $x_{0}$ and if each $\vec{\nabla} g_{i} (x_{0})$ is not a linear combination of the other $\vec{\nabla} g_{j} (x_{0})$ ( $j \neq i$ ), then there exists $m$ real numbers $λ_{1}, \dots, λ_{m}$ such that
$\vec{\nabla} f (x_{0}) = λ_{1} \vec{\nabla} g_{1} (x_{0}) + \dots + λ_{m} \vec{\nabla} g_{m} (x_{0}) .$

¹For instance, in this specific example,
${\begin{cases} x = X (t) = \frac{3}{\sqrt{2}} \cos t + 2 \sqrt{2} \sin t \\ y = Y (t) = - \frac{3}{\sqrt{2}} \cos t + 2 \sqrt{2} \sin t \end{cases} (0 \leq t \leq 2 π)$

²In other words, $f$ and $g$ have continuous partial derivatives.

Constrained Extrema

Lagrange Multipliers

Read the proof

Hide the proof

Multiple Constraints