If you play the note A on the third string second fret of your guitar, this A would be, depending on tuning, approximately the frequency of 440 Hz. This frequency is the main pitch frequency in the sound and the easiest to recognize in the sound, and so it is called the fundamental frequency. Besides this frequency through, your guitar will produce many other frequencies, some of which will be higher (overtones) and some of which will be lower (undertones). This happens with your guitar because besides the vibration of your string, you get the vibration of the instrument, the vibration of pieces of the instrument, and vibrations of your string that you do not necessarily expect.
An overtone / undertone is a frequency in a sound that is different from the fundamental frequency.
Overtones are also known as "partial waves".
Overtones and undertones are important because having them changes the quality of the sound (the timbre of the sound) that is ultimately produced. Overtones and undertones are the reason why different instruments sound differently even when they are playing the same note. Overtones and undertones are sometimes more pronounced, such as with brass instruments and toms, and sometimes less. Certain instruments produce overtones / undertones that are always present, independently of the note that is played. Those are called formants and are essential for the timbre of the instrument.
Certain instruments produce harmonics, which are nicely sounding overtones. An overtone or undertone does not have to be harmonic though and it could be an inharmonic overtone. Harmonic overtones are those the frequency of which are an integer multiple of the fundamental frequency (for example, the harmonic overtones of 440 Hz are 880 Hz, 1320 Hz, 1760 Hz and so on). Inharmonic overtones, also known as "partial overtones" or "partial harmonics", are overtones that are not an integer multiple of the fundamental frequency.