I never expected one Twitter comment to generate so many e-mails. There are a lot of college students in the Flash community, many of which are taking calculus courses, so it probably makes sense. I could say check out Wikipedia, but there are a lot of topics for which a theoretical introduction is difficult to comprehend. When I work with calc. III students studying extrema of functions, I like to start out with this problem.
minimize x2 + y2 + z2
on the hyperbolic cylinder x2 – z2 = 1 [1]
This is a constrained optimization problem. Set the partial derivatives of the objective to zero. The partials are only zero at the point (0, 0, 0) which is not in the domain of the constraint. From the constraint, z2 = x2 – 1. Eliminate z from the objective which becomes
minimize w = 2x2 + y2 – 1 [2]
Now, δw/δx = 4x and δw/δy = 2y . These are only zero at x = 0 and y = 0. From [1], this implies z2 = -1 , yielding an imaginary solution. As an exercise, eliminate x from the objective, minimizing
w = 1 + y2 + 2z2 [3]
yielding y = 0, z = 0 and x = +/-1. The domain of the constraint is |x| ≥ 1. This constrains the domain of [2], while the domain of [3] is all reals for y and z. Since |x| must be ≥ 1, a minimum has been found.
Contrast this to solving the problem with Lagrange multipliers. This technique relaxes the constraint into the objective, turning a constrained optimization problem into an unconstrained problem. Create the function
L(x,y,z) = x2 + y2 + z2 – λ(x2 – z2 – 1)
Now, δL/δx = 2x(1-λ), δL/δy = 2y, δL/δz = 2z(1+λ), and δL/δλ = x2 – z2 – 1
Setting the first three partials to zero yields λ = 1, y = 0, and z = 0. Plugging z = 0 into the last partial and setting it equal to zero yields x2 = 1 or x = +/- 1.
Lagrange multipliers can be useful for finding stationary points of a constrained function, particularly when the classic approach breaks down or is cumbersome. Depending on the constraint(s), both maxima and minima may be found. Also, beware of saddle points.
If you want to read more about the method and why it works, here are some good introductions,
http://dbpubs.stanford.edu:8091/~klein/lagrange-multipliers.pdf
http://tutorial.math.lamar.edu/classes/calcIII/lagrangemultipliers.aspx
The Algorithmist 