In A classification of the 'whys' and 'hows' of using history in mathematics education, U.T. Jankvist surveys a large body of research affirming the value of including the narrative of the historical development of mathematics in a math course. From this research, he concludes that there are two primary answers to "why" history should be included. One answer is that there is value in teaching the history itself, as it allows us to "show students that mathematics exists and evolves in time and space", dependent upon persons in a number of cultures. The other answer sees history as an important tool in aiding students' learning of mathematics. The arguments he collected in favor of this latter position include:
I find a number of these arguments compelling. In particular, I've noticed that providing historical connections provides context and gets students engaged in material.
For instance, the disputes between Newton and Leibniz regarding the credit for developing the Calculus are well-known; however, there is also a fantastic story of a feud between Descartes and Fermat that can be found in George Simmon's Calculus Gems: Brief Lives and Memorable Mathematics:
Before the development of the Calculus, Fermat has developed a method of finding tangents. "When the famous philosopher [Descartes] was informed of Fermat's method by Merseene, he attacked its generality, challenged Fermat to find the tangent to the curve x^3+y^3=3axy, and foolishly predicted that he would fail. Descartes was unable to cope with this problem himself, and was intensely irritated when Fermat solved it easily."
I incorporate this story into the lecture on implicit differentiation in a single-variable Calculus course, posing the challenge of Descartes: find the tangent line to the curve (called the 'folium of Descares') at an arbitrary point. It presents the need to develop a method of differentiation to solve such problems, which motivates us to develop implicit differentiation in order to save our honor and make Descartes "intensely irritated".
There are a number of other excellent historical accounts that I've found to have great value, such as the historical development of finding the volume of the sphere in a multivariable calculus course (noting Archimedes' insights and the very clever Cavalieri's principle). Many excellent books (such as Calculus Gems) have conveniently packaged these stories.