Regularization and Trust Regions in Optimization

ABSTRACT:

The trust region subproblem (the minimization of a quadratic objective subject to one quadratic constraint and denoted TRS) has many applications in diverse areas, e.g. function minimization, sequential quadratic programming, regularization, ridge regression, and discrete optimization. Recent advances in the theory and algorithmic development using Lanczos techniques have allowed TRS to be used to solve large scale optimization problems. We provide an overview of these new developments along with the relationships to Conjugate Gradient methods and regularization.