# Taylor series and gradient approximations

The approach of the finite difference method to represent a continuous reality in a grid with a resolution of a given distance $\Delta$ implies that some errors are introduced. This is illustrated in Fig. d.1, which shows three different ways to estimate the gradient of a continuous line using the finite difference method. In all three methods there is a gap between the reality represented by the continuous line, and the discretized world represented by the blue lines on the drawing.

The three different methods to implement the gradient are:
\begin{equation} \begin{aligned} & q'(x) = \frac{q(x+\Delta x)-q(x)}{\Delta x} \ \ \ \ \ \ \ \ \ \ \ \ \ \ \text{Forward} \\ & q'(x) = \frac{q(x)-q(x-\Delta x)}{\Delta x} \ \ \ \ \ \ \ \ \ \ \ \ \ \ \text{Backward} \\ & q'(x) = \frac{q(x+\Delta x)-q(x-\Delta x)}{2\Delta x} \ \ \ \text{Centered} \end{aligned} \end{equation}
Fig. d.1: Finite difference estimates of the gradient.

A way to put this into a more mathematical framework is to use Taylor series. The parameter $q$ in point $x+\Delta x$ expanded from point $x$ is written:
\begin{equation} q(x+\Delta x) = q(x) + q'(x) \Delta x + \frac{1}{2!}q''(x) (\Delta x)^2 + \frac{1}{3!}q'''(x) (\Delta x)^3 + \mathcal{O}(\Delta x)^4; \end{equation} while for the parameter $q$ in point $x-\Delta x$ expanded from point $x$:
\begin{equation} q(x-\Delta x) = q(x) - q'(x) \Delta x + \frac{1}{2!}q''(x) (\Delta x)^2 - \frac{1}{3!}q'''(x) (\Delta x)^3 + \mathcal{O}(\Delta x)^4. \end{equation} If the Taylor series is only written to the second order, $\mathcal{O}(\Delta x)^2$, then:
\begin{equation} q'(x) = \frac{q(x+\Delta x)-q(x)}{\Delta x} + \mathcal{O}(\Delta x); \end{equation} which shows that this is a first order approximation to $q'(x)$. A similar result may be found for the backward implementation, which also is a first order approximation.

For the centered approximation, the Taylor series $q(x+\Delta x)$ and $q(x-\Delta x)$ may be expanded to $O((\Delta x)^3)$ and then by insertion:
\begin{equation} q'(x) = \frac{q(x+\Delta x)-q(x-\Delta x)}{2\Delta x} + \mathcal{O}(\Delta x)^2; \end{equation} i.e. a second order approximation.

# Approximation of second order derivative

A straightforward way to approximate a second order derivative is first to estimate the gradients in points $x+\frac{1}{2}\Delta x$ and $x-\frac{1}{2}\Delta x$ respectively, using the forward and backward schemes, and then in a second step estimate the second derivative from:
\begin{equation} q''(x) = \frac{q'(x+\frac{1}{2}\Delta x)-q'(x-\frac{1}{2}\Delta x)}{\Delta x} = \frac{q(x+\Delta x)-2q(x)+q(x-\Delta x)}{(\Delta x)^2}. \end{equation} This procedure is illustrated in Fig. d.2.

Another approach is to expand the Taylor series $q(x+\Delta x)$ and $q(x-\Delta x)$ to $\mathcal{O}(\Delta x)^4$ and then add the two series to get:
\begin{equation} q(x+\Delta x) + q(x-\Delta x) = 2q(x) + q''(x)(\Delta x)^2 + \mathcal{O}(\Delta x)^4; \end{equation} which may be rewritten to:
\begin{equation} q''(x) = \frac{q(x+\Delta x) - 2q(x) + q(x-\Delta x)}{(\Delta x)^2} + \mathcal{O}(\Delta x)^2; \end{equation} which demonstrate that the straightforward implementation above is a second order approximation.

Fig. d.2: Simple method to estimate a second order derivative.

Read more about the Finite Differences - Higher Order Approximations