File size: 4,335 Bytes
42b39ca dcb9e31 42b39ca dcb9e31 42b39ca dcb9e31 42b39ca aded020 42b39ca dcb9e31 42b39ca dcb9e31 42b39ca dcb9e31 42b39ca dcb9e31 42b39ca |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 |
---
base_model:
- Qwen/Qwen2.5-Coder-1.5B
license: cc-by-nc-4.0
---
<br><br>
<p align="center">
<img src="https://huggingface.co/datasets/jinaai/documentation-images/resolve/main/logo.webp" alt="Jina AI: Your Search Foundation, Supercharged!" width="150px">
</p>
<p align="center">
<b>The code embedding model trained by <a href="https://jina.ai/"><b>Jina AI</b></a>.</b>
</p>
# Jina Code Embeddings: A Small but Performant Code Embedding Model
## Intended Usage & Model Info
`jina-code-embeddings` is an embedding model for code retrieval.
The model supports various types of code retrieval (text-to-code, code-to-code, code-to-text, code-to-completion) and technical question answering across 15+ programming languages.
Built on [Qwen/Qwen2.5-Coder-1.5B](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B), `jina-code-embeddings-1.5b` features:
- **Multilingual support** (15+ programming languages) and compatibility with a wide range of domains, including web development, software development, machine learning, data science, and educational coding problems.
- **Task-specific instruction prefixes** for NL2Code, Code2Code, Code2NL, Code2Completion, and Technical QA, which can be selected at inference time.
- **Flexible embedding size**: dense embeddings are 1536-dimensional by default but can be truncated to as low as 128 with minimal performance loss.
Summary of features:
| Feature | Jina Code Embeddings 1.5B |
|------------|------------|
| Base Model | Qwen2.5-Coder-1.5B |
| Supported Tasks | `nl2code`, `code2code`, `code2nl`, `code2completion`, `qa` |
| Model DType | BFloat 16 |
| Max Sequence Length | 32768 |
| Embedding Vector Dimension | 1536 |
| Matryoshka dimensions | 128, 256, 512, 1024, 1536 |
| Pooling Strategy | Last-token pooling |
| Attention Mechanism | FlashAttention2 |
## Usage
<details>
<summary>Requirements</a></summary>
The following Python packages are required:
- `transformers>=4.53.0`
- `torch>=2.7.1`
### Optional / Recommended
- **flash-attention**: Installing [flash-attention](https://github.com/Dao-AILab/flash-attention) is recommended for improved inference speed and efficiency, but not mandatory.
- **sentence-transformers**: If you want to use the model via the `sentence-transformers` interface, install this package as well.
</details>
<details>
<summary>via <a href="https://huggingface.co/docs/transformers/en/index">transformers</a></summary>
```python
# !pip install transformers>=4.53.0 torch>=2.7.1
from transformers import AutoModel
import torch
# Initialize the model
model = AutoModel.from_pretrained("jinaai/jina-code-embeddings-1.5b", trust_remote_code=True)
model.to("cuda")
# Configure truncate_dim, max_length, batch_size in the encode function if needed
# Encode query
query_embeddings = model.encode(
["print hello world in python"],
task="nl2code",
prompt_name="query",
)
# Encode passage
passage_embeddings = model.encode(
["print('Hello World!')"],
task="nl2code",
prompt_name="passage",
)
```
</details>
<details>
<summary>via <a href="https://sbert.net/">sentence-transformers</a></summary>
```python
# !pip install sentence_transformers>=5.0.0 torch>=2.7.1
import torch
from sentence_transformers import SentenceTransformer
# Load the model
model = SentenceTransformer(
"jinaai/jina-code-embeddings-1.5b",
model_kwargs={
"torch_dtype": torch.bfloat16,
"attn_implementation": "flash_attention_2",
"device_map": "auto"
}
)
# The queries and documents to embed
queries = [
"print hello world in python",
"initialize array of 5 zeros in c++"
]
documents = [
"print('Hello World!')",
"int arr[5] = {0, 0, 0, 0, 0};"
]
query_embeddings = model.encode(queries, prompt_name="nl2code_query")
document_embeddings = model.encode(documents, prompt_name="nl2code_document")
# Compute the (cosine) similarity between the query and document embeddings
similarity = model.similarity(query_embeddings, document_embeddings)
print(similarity)
# tensor([[0.8157, 0.1222],
# [0.1201, 0.5500]])
```
</details>
## Training & Evaluation
Please refer to our technical report of jina-code-embeddings for training details and benchmarks.
## Contact
Join our [Discord community](https://discord.jina.ai) and chat with other community members about ideas. |