ÊýÖµ×îÓÅ»¯

Preface
1 Introduction
Mathematical Formulation
Example£º A Transportation Problem
Continuous versus Discrete Optimization
Constrained and Unconstrained Optimization
Global and Local Optimization
Stochastic and Deterministic Optimization
Optimization Algorithms
Convexity
Notes and References

2 Fundamentals of Unconstrained Optimization
2£®1 What Is a Solution£¿
Recognizing a Local Minimum
Nonsmooth Problems
2£®2 Overview of Algorithms
Two Strategies£º Line Search and Trust Region
Search Directions for Line Search Methods
Models for Trust¡ªRegion Methods
Scaling
Rates of Convergence
R¡ªRates of Convergence
Notes and References
Exercises

3 Line Search Methods
3£®1 Step Length
The Wolfe Conditions
The Goldstein Conditions
Sufficient Decrease and Backtracking
3£®2 Convergence of Line Search Methods
3£®3 Rate of Convergence
Convergence Rate of Steepest Descent
Quasi¡ªNewton Methods
Newton's Method
Coordinate Descent Methods
3£®4 Step¡ªLength Selection Algorithms
Interpolation
The Initial Step Length
A Line Search Algorithm for the Wolfe Conditions
Notes and References
Exerases

4 Trust¡ªRegion Methods
Outline of the Algorithm
4£®1 The Cauchy Point and Related Algorithms
The Cauchy Point
Improving on the Cauchy Point
The DoglegMethod
Two¡ªDimensional Subspace Minimization
Steihaug's Approach
4£®2 Using Nearly Exact Solutions to the Subproblem
Characterizing Exact Solutions
Calculating Nearly Exact Solutions
The Hard Case
Proof of Theorem 4£®3
4£®3 Global Convergence
Reduction Obtained by the Cauchy Point
Convergence to Stationary Points
Convergence of Algorithms Based on Nearly Exact Solutions
4£®4 Other Enhancements
Scaling
Non¡ªEuclidean Trust Regions
Notes and References
Exercises

5 Conjugate Gradient Methods
5£®1 The Linear Conjugate Gradient Method
Conjugate Direction Methods
Basic Properties of the Conjugate Gradient Method
A Practical Form of the Conjugate Gradient Method
Rate of Convergence
Preconditioning
Practical Preconditioners
5£®2 Nonlinear Conjugate Gradient Methods
The Fletcher¡ªReeves Method
The Polak¡ªRibiere Method
Quadratic Termination and Restarts
Numerical Performance
Behavior of the Fletcher¡ªReeves Method
Global Convergence
Notes and References
Exerases

6 Practical Newton Methods
6£®1 Inexact Newton Steps
6£®2 Line Search Newton Methods
Line Search Newton¡ªCG Method
Modified Newton's Method
6£®3 Hessian Modifications
Eigenvalue Modification
Adding a Multiple of the Identity
Modified Cholesky Factorization
Gershgorin Modification
Modified Symmetric Indefinite Factorization
6£®4 Trust¡ªRegion Newton Methods
Newton¡ªDogleg and Subspace¡ªMinimization Methods
Accurate Solution of the Trust¡ªRegion Problem
Trust¡ªRegion Newton¡ªCG Method
Preconditioning the Newton¡ªCG Method
Local Convergence of Trust¡ªRegion Newton Methods
Notes and References
Exerases

7 Calculating Derivatives
7£®1 Finite¡ªDifference Derivative Approximations
Approximating the Gradient
Approximating a Sparse Jacobian
Approximatingthe Hessian
Approximating a Sparse Hessian
7£®2 Automatic Differentiation
An Example
The Forward Mode
The Reverse Mode
Vector Functions and Partial Separability
Calculating Jacobians of Vector Functions
Calculating Hessians£º Forward Mode
Calculating Hessians£º Reverse Mode
Current Limitations
Notes and References
Exercises

8 Quasi¡ªNewton Methods
8£®1 The BFGS Method
Properties ofthe BFGS Method
Implementation
8£®2 The SR1 Method
Properties of SRl Updating
8£®3 The Broyden Class
Properties ofthe Broyden Class
8£®4 Convergence Analysis
Global Convergence ofthe BFGS Method
Superlinear Convergence of BFGS
Convergence Analysis of the SR1 Method
Notes and References
Exercises

9 Large¡ªScale Quasi¡ªNewton and Partially Separable Optimization
9£®1 Limited¡ªMemory BFGS
Relationship with Conjugate Gradient Methods
9£¬2 General Limited¡ªMemory Updating
Compact Representation of BFGS Updating
SR1 Matrices
Unrolling the Update
9£®3 Sparse Quasi¡ªNewton Updates
9£®4 Partially Separable Functions
A Simple Example
Internal Variables
9£®5 Invariant Subspaces and Partial Separability
Sparsity vs£®Partial Separability
Group Partial Separability
9£®6 Algorithms for Partially Separable Functions
Exploiting Partial Separabilityin Newton's Method
Quasi¡ªNewton Methods for Partially Separable Functions
Notes and References
Exercises
¡­¡­
10 Nonlinear Least¡ªSquares Problems
11 Nonlinear Equations
12 Theory of Constrained Optimization
13 Linear Programming£º The Simplex Method
14 Linear Programming£ºInterior¡ªPoint Methods
15 Fundamentals of Algorithms for Nonlinear Constrained Optimization
16 Quadratic Programnung
17 Penalty£¬ Barrier£¬ and Augmented Lagrangian Methods
18 Sequential Quadratic Programming
A Background Material
References
Index