Consider an unknown function . How can one approximate if (s)he know the information of the function , such as derivaties, at a certain , e.g. ?
One way is to use a polynomial with infinitely many terms using 's information around . The idea is that each term in the polynomial approximates a certain aspect of :
- The first term govern such that ;
- The second term govern such that ;
- The -th term should govern such that .
Consider the -th term in . From the construction, this term allows , hence
The equation above suggests that . In principle, we can use infinitely many terms, i.e. , yielding a series:
which is called Maclaurin series.
Considier . Let be the polynomial approximation upto degree :
The equation above tells us that we will make some error when we use only the first j+1 term approximation; thus, this technique gives accurate approximation only in a region near .
In general, one might have information at a particular point , i.e for . Using the same derivation as , the polynomial approximation at is
This is the Taylor series, a generalized version of the Maclaurin series into any arbitary . In essence, this means we do local expansion around : the farther we are away from the larger approximation error we get.
In this article, I have discussed the origin and the idea behind the Maclaurin and Taylor series. The Taylor series itself is a foundation in optimization and machine learning.
If you are would like to explore these two series on different functions, you might be interested in looking at my Colab notebook, which I used it to make the two figures for this article.