## Introduction

The following section is about constrained optimization and lagrange multipliers. For more mathematical topics checkout other list.

## Lagrange multipliers Introduction

The Lagrange multiplier technique lets you find the maximum or minimum of a multivariable function $f(x, y, \dots)$ when there is some constraint $\color{red}{g(x,y,\dots)}$ on the input values $x$, $y$, $\dots$ you are allowed to use.

the gradient of $f$ evaluated at a point $(x_0, y_0)$ always gives a vector perpendicular to the contour line passing through that point. This means when the contour lines of two functions $\color{blue}f$ and $\color{red}g$ are tangent, their gradient vectors are parallel. This tangency means their gradient vectors align:

Tangency Condition $$\label{tangency} \nabla \color{blue}{f(x_0,y_0)} = \color{green}{\lambda_0} \nabla \color{red}{g(x_0,y_0)}$$

Here, $\color{green}{\lambda}$ represents some constant. Some authors use a negative constant, $-\color{green}{\lambda}$.

Let’s see what this looks like in our example where $\color{blue}{f(x,y)=2x+y}$ and $\color{red}{g(x,y)=x^2+y^2}$. The gradient of $f$ is

and the gradient of $g$ is

Therefore, the tangency condition from equation $\ref{tangency}$ ends up looking like this:

In the example from above, this leads to the following three equations and two three unknowns $x_0$, $y_0$ and $\lambda_0$.

Plugging $x_0$ and $y_0$ into the third equation results in a quadratic equation with two solutions for $\lambda_0$

This leads to two solutions for the input pair $(x_0,y_0)$ which can be plugged into the original function $f(x,y)$. One input leads to the minimum and the other to the maximum value of $f$.

In general the we can write these conditions by saying we are looking for constants $x_0$, $y_0$ and $\lambda_0$ that satisfy the constraint and the tangency condition \ref{tangency}.

Joseph Louis Lagrange wrote down a special new function which takes in all the same input variables as $f$ and $g$, along with the new kid in town $\lambda$, thought of now as a variable rather than a constant.

The Lagrangian $$\mathcal{L}(x,y,\dots,\color{green}{\lambda}) = \color{blue}{f(x,y,\dots)} - \color{green}{\lambda}(\color{red}{g(x,y,\dots) - c})$$ with the Lagrange multiplier $\color{green}{\lambda}$.

Tags:

Categories:

Updated: