| Lectures | Date | Subjects Covered | Lecture Supplementary Material |
| Lectures 32-34 | Mar. 23-27 | Barrier and Penalty Methods |
log-barrier method for (P), the quadratic penalty function method for (P) Chapter 6 in the text (and problems at the end of the chapter) |
| Lectures 24-31 | Mar. 9-20 | Strong Duality and the KKT optimality conditions | Chapter 5 in the text (and problems at the end of the chapter) |
| Lectures 20-23 | Mar. 7 | Convex Programming |
separating/supporting Hyperplanes cones, polar cones |
| Lectures 19 | Feb. 25 | Questions before the midterm | |
| Lectures 18 | Feb. 23 | Lagrange Multipliers | Proof of the Lagrange multiplier theorem for equality constraints using the implicit function theorem; linear independence constraint qualification (CQ); comparison (example) with simplex method for linear programming |
| Lectures 17 | Feb. 11 | Least Squares Problems | Least Squares Fit; Minimum Norm Solutions (underdetermined problems) |
| Lectures 16 | Feb. 9 | Trust Region Methods | trust region methods (Levenberg-Marquadt); solving the trust region subproblem |
| Lectures 15 | Feb. 6 | "Good" algorithm properties | beyond steepest descent; Wolfe conditions (existence theorem); start of trust region methods |
| Lectures 14 | Feb. 4 | optimtool (MATLAB) |
|
| Lectures 13 | Feb. 2 | Cauchy's Method of Steepest Descent | Example; and Theorem of orthogonal gradients at successive steps with proof. |
| Lectures 10-12 | Jan. 26-30 | Iterative Methods | Newton and Steepest Descent Methods |
| Lecture 9 | Jan. 23 | Convexity |
Examples of convex functions, compositions of convex functions;
start of iterative methods |
| Lecture 8 | Jan. 21 |
Convexity
Supplementary notes on optimality conditions and convexity; |
three characterizations of
convex functions (epigraph is a convex set; tangent plane lies below
graph; Hessian is positive semidefinite)
examples of convex/concave functions |
| Lecture 7 | Jan. 19 |
Convexity
Supplementary notes on optimality conditions and convexity; |
three characterizations of convex functions (epigraph is a convex set; tangent plane lies below graph; Hessian is positive semidefinite) |
| Lecture 6 | Jan. 16 |
Convex Sets (notes);
(
video lecture)
Supplementary notes on optimality conditions and convexity |
convexity; convex combinations; convex hull;
MATLAB example - Newton's method |
| Lecture 5 | Jan. 14 |
Unconstrained Minimization - Rn (Supplementary NOTES);
|
definitions of coercivity; convexity
Theorems and proofs on attainement for: (i) continuous functions on compact sets; (ii) coercive functions examples/applications of coercivity, e.g. least squares problems for overdetermined linear equations: min ||Ax-b||2 |
| Lecture 4 | Jan. 12 |
Unconstrained Minimization - Rn (Supplementary NOTES);
( Complete Supplementary course notes) |
definitions and characterizations of: positive (semi) definite matrices;
eigenvalues and orthogonal decomposition of symmetric matrices
(
supplementary notes on quadratic forms)
examples of recognizing local/global minima |
| Lecture 3 | Jan. 7 | Unconstrained Minimization - Rn (WIKI!!) |
Overview of WWW links to
NEOS,
(
WWW Form for unconstr NMTR);
LP and NLP FAQs
and
the NEOS WIKI minimization using matlab/plotting; file discrsimpleunc.m and resulting plot |
| Lecture 2 | Jan. 7 | Unconstrained Minimization |
Overview of WWW links to
NEOS,
LP and NLP FAQs topology: open and closed sets; continuity Definitions: (strict) global/local minimizers/maximizers, critical points second derivative tests for minimizer/maximizer/saddle points |
| Lecture 1 | Jan. 5 | Introduction to Continuous Optimization |
The General Nonlinear Optimization Problem Taylor Theorem; little o, big O notation in calculus; local optimizer and critical points (Fermat Theorem); constant, linear, quadratic functions on R; QUESTION: Can a quadratic function be bounded below but have its minimum value unattained? |