The finite difference method, used to approximate solutions to differential equations, is subject to two primary sources of error: truncation error and roundoff error.
Truncation Error
Truncation error arises because the finite difference method replaces derivatives with approximate difference quotients. These quotients are derived using Taylor series expansions, but these expansions are truncated after a certain number of terms to make the computations feasible. This truncation introduces an error that depends on the step size, h.
-
Explanation: We approximate the derivative (e.g., f'(x)) using a finite number of terms from its Taylor series expansion. The terms that are left out contribute to the truncation error.
-
Dependence on Step Size: The truncation error is typically O(hn), where n is an integer depending on the specific finite difference scheme used. This means that as the step size h decreases, the truncation error also decreases (at a rate proportional to hn).
-
Example: Using a first-order forward difference approximation for the first derivative, f'(x) ≈ (f(x + h) - f(x))/h, the truncation error is O(h). This comes from truncating the Taylor series: f(x+h) = f(x) + hf'(x) + (h2/2)f''(x) + ...
Roundoff Error
Roundoff error occurs due to the limitations of computer arithmetic. Computers use a finite number of digits to represent real numbers, which can lead to rounding errors in calculations. When using very small step sizes (h) in the finite difference method, roundoff errors can become significant.
-
Explanation: Computers store numbers with limited precision (e.g., single or double precision). Each arithmetic operation introduces a small roundoff error, which can accumulate, especially when subtracting nearly equal numbers.
-
Dependence on Step Size: As h gets smaller, the differences in function values (e.g., f(x + h) - f(x)) become smaller. The relative roundoff error can then become larger because we are subtracting two numbers that are very close, potentially losing significant digits. The roundoff error can be approximated as O(ε/h), where ε represents the machine epsilon (approximately 10-16 in double precision).
-
Example: Consider calculating (f(x+h) - f(x))/h where f(x+h) and f(x) are very close. If the computer can only store a limited number of digits, the subtraction may result in the loss of significant digits, leading to a larger error when divided by the small value of h.
The Trade-off
There's a trade-off between truncation and roundoff error. Decreasing the step size h reduces the truncation error, but it increases the roundoff error. Therefore, there is often an optimal value of h that minimizes the total error.
Summary
In summary, the errors in the finite difference method stem from:
- Truncation error: An inherent error due to approximating derivatives using finite difference formulas. Decreases with smaller step sizes.
- Roundoff error: An error introduced by the finite precision of computer arithmetic. Increases with smaller step sizes.
Choosing an appropriate step size h is crucial to balance these two competing error sources and obtain accurate results.
