A New Pair of GloVes
This report documents, describes, and evaluates new 2024 English GloVe
(Global Vectors for Word Representation) models. While the original GloVe
models built in 2014 have been widely used and found useful, languages and the
world continue to evolve and we thought that current usage could benefit from
updated models. Moreover, the 2014 models were not carefully documented as to
the exact data versions and preprocessing that were used, and we rectify this
by documenting these new models. We trained two sets of word embeddings using
Wikipedia, Gigaword, and a subset of Dolma. Evaluation through vocabulary
comparison, direct testing, and NER tasks shows that the 2024 vectors
incorporate new culturally and linguistically relevant words, perform
comparably on structural tasks like analogy and similarity, and demonstrate
improved performance on recent, temporally dependent NER datasets such as
non-Western newswire data.
68832a97846d2e78b7548ad6
Riley Carlson
68832a97846d2e78b7548ad7
John Bauer
68832a97846d2e78b7548ad8
Christopher D. Manning
A New Pair of GloVes
This report documents, describes, and evaluates new 2024 English GloVe
(Global Vectors for Word Representation) models. While the original GloVe
models built in 2014 have been widely used and found useful, languages and the
world continue to evolve and we thought that current usage could benefit from
updated models. Moreover, the 2014 models were not carefully documented as to
the exact data versions and preprocessing that were used, and we rectify this
by documenting these new models. We trained two sets of word embeddings using
Wikipedia, Gigaword, and a subset of Dolma. Evaluation through vocabulary
comparison, direct testing, and NER tasks shows that the 2024 vectors
incorporate new culturally and linguistically relevant words, perform
comparably on structural tasks like analogy and similarity, and demonstrate
improved performance on recent, temporally dependent NER datasets such as
non-Western newswire data.
New 2024 GloVe models, trained on updated datasets, improve performance on temporally dependent NER tasks while maintaining structural task performance.
GloVe word embeddings Wikipedia Gigaword Dolma vocabulary comparison analogy similarity NER newswire data