What does an obnoxious person have in common with sandpaper? Both are abrasive. Anything that grates or irritates — physically or metaphorically — can be described using this adjective.
The history of the word abrasive illustrates how a word typically enters the English language and then changes with time. The English verb abrade "to wear down by scraping" entered the language from Latin abradere "to scrape off" in the late 1600s. Some 200 years later, the adjective form of the word — abrasive — came into use to describe a type of grinding tool. By the 1920s, abrasive had acquired the more figurative sense of annoying and infuriating. If you find someone abrasive, he or she grinds away at your nerves.