Efficient Code Embeddings from Code Generation Models
jina-code-embeddings is a novel code embedding model suite designed to
retrieve code from natural language queries, perform technical
question-answering, and identify semantically similar code snippets across
programming languages. It makes innovative use of an autoregressive backbone
pre-trained on both text and code, generating embeddings via last-token
pooling. We outline the training recipe and demonstrate state-of-the-art
performance despite the relatively small size of the models, validating this
approach to code embedding model construction.
68b5221d851c6e7b001ecaa9
Daria Kryvosheieva
claimed_verified
2025-09-01T07:50:30.541Z
676e17d6fb9d2aa0ad730d0a
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/jC_o2O9JfK5-1Y99IgDmT.png
Daria Kryvosheieva
user
dariakryvosheieva
68b5221d851c6e7b001ecaaa
Saba Sturua
68b5221d851c6e7b001ecaab
Michael Günther
68b5221d851c6e7b001ecaac
Scott Martens
68b5221d851c6e7b001ecaad
Han Xiao
claimed_verified
2025-09-03T08:29:24.091Z
603763514de52ff951d89793
https://cdn-avatars.huggingface.co/v1/production/uploads/603763514de52ff951d89793/n-QouGYg7oE5QeDaAb3Ns.png
Han Xiao
user
hanxiao
Efficient Code Embeddings from Code Generation Models
jina-code-embeddings is a novel code embedding model suite designed to
retrieve code from natural language queries, perform technical
question-answering, and identify semantically similar code snippets across
programming languages. It makes innovative use of an autoregressive backbone
pre-trained on both text and code, generating embeddings via last-token
pooling. We outline the training recipe and demonstrate state-of-the-art
performance despite the relatively small size of the models, validating this
approach to code embedding model construction.
Jina-code-embeddings uses an autoregressive backbone pre-trained on text and code to generate embeddings for code retrieval, question-answering, and similarity identification.
autoregressive backbone last-token pooling code embedding model