This is an archived course. A more recent version may be available at ocw.mit.edu.

3.6 Introduction to Design Optimization

3.6.4 Finite Difference Method Applied to 1D Convection

Measurable Outcome 3.19

Generally the objective function and constraints are complex, nonlinear functions of the design variables which are not known analytically. We therefore cannot compute analytic derivatives for the required gradients and Hessian information. However, it is still possible to estimate the gradient and Hessian using other methods. The simplest approach is to use finite differences.

A one-sided finite difference requires \(n+1\) function evaluations. The gradient is estimated as

\[\frac{\partial J}{\partial x_ j} \approx \frac{J(x_1,x_2,\ldots ,x_ j+\Delta x_ j, x_{j+1},\ldots ,x_ n)-J(x_1,x_2,\ldots ,x_ j,x_{j+1},\ldots ,x_ n)}{\Delta x_ j}\] (3.75)

This approximation is first-order accurate: reducing \(\Delta x_ j\) by a factor of 2 reduces the error in the gradient by a factor of 2.

A more accurate estimate can be computed using a central difference, which requires \(2n\) function evaluations:

\[\frac{\partial J}{\partial x_ j} \approx \frac{J(x_1,x_2,\ldots ,x_ j+\Delta x_ j, x_{j+1},\ldots ,x_ n)-J(x_1,x_2,\ldots ,x_ j-\Delta x_ j,x_{j+1},\ldots ,x_ n)}{2\Delta x_ j}\] (3.76)

This approximation is second-order accurate: reducing \(\Delta x_ j\) by a factor of 2 reduces the error in the gradient by a factor of 4.

The Hessian matrix can also be estimated using finite differences. A second-order central approximation of the \((i,j)\) entry of the Hessian matrix is given by:

\[\frac{\partial ^2 J}{\partial x_ i \partial x_ j} \approx \frac{J(x_1,x_2,\ldots ,x_ i+\Delta x_ i,\ldots ,x_ j+\Delta x_ j, x_{j+1},\ldots ,x_ n)}{4\Delta x_ i\Delta x_ j} \\ + \frac{J(x_1,x_2,\ldots ,x_ i-\Delta x_ i,\ldots ,x_ j-\Delta x_ j, x_{j+1},\ldots ,x_ n)}{4\Delta x_ i\Delta x_ j} \\ - \frac{J(x_1,x_2,\ldots ,x_ i-\Delta x_ i,\ldots ,x_ j+\Delta x_ j, x_{j+1},\ldots ,x_ n)}{4\Delta x_ i\Delta x_ j} \\ - \frac{J(x_1,x_2,\ldots ,x_ i+\Delta x_ i,\ldots ,x_ j-\Delta x_ j, x_{j+1},\ldots ,x_ n)}{4\Delta x_ i\Delta x_ j}\] (3.77)