6. Unconstrained Nonlinear Optimization

Learning Objectives

1. Develop mathematical vocabular including convexity, vector spaces, eigendecomposition

2. Explain neccessary and sufficient optimality conditions

3. Implement basic algorithms including Newton’s method, trust regions, line searches in Python

4. Prove basic properties of algorithms such as local convergence rates

Sections

• 6.1. Linear Algebra Review and SciPy Basics
• 6.2. Mathematics Primer
• 6.3. Unconstrained Optimality Conditions
• 6.4. Newton-type Methods for Unconstrained Optimization
• 6.5. Quasi-Newton Methods for Unconstrained Optimization
• 6.6. Descent and Globalization