Update infinity section in readme
Browse files
README.md
CHANGED
|
@@ -2904,7 +2904,7 @@ from infinity_emb import AsyncEmbeddingEngine, EngineArgs
|
|
| 2904 |
|
| 2905 |
sentences = ["Embed this is sentence via Infinity.", "Paris is in France."]
|
| 2906 |
engine = AsyncEmbeddingEngine.from_args(
|
| 2907 |
-
EngineArgs(model_name_or_path = "BAAI/bge-
|
| 2908 |
))
|
| 2909 |
|
| 2910 |
async def main():
|
|
@@ -2913,6 +2913,13 @@ async def main():
|
|
| 2913 |
asyncio.run(main())
|
| 2914 |
```
|
| 2915 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2916 |
### Usage for Reranker
|
| 2917 |
|
| 2918 |
Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding.
|
|
|
|
| 2904 |
|
| 2905 |
sentences = ["Embed this is sentence via Infinity.", "Paris is in France."]
|
| 2906 |
engine = AsyncEmbeddingEngine.from_args(
|
| 2907 |
+
EngineArgs(model_name_or_path = "BAAI/bge-base-en-v1.5", device="cpu", engine="optimum" # or engine="torch"
|
| 2908 |
))
|
| 2909 |
|
| 2910 |
async def main():
|
|
|
|
| 2913 |
asyncio.run(main())
|
| 2914 |
```
|
| 2915 |
|
| 2916 |
+
for a docker based deployment with infinity using torch:
|
| 2917 |
+
```bash
|
| 2918 |
+
docker run --gpus all -v $PWD/data:/app/.cache -p "7997":"7997" \
|
| 2919 |
+
michaelf34/infinity:0.0.68 \
|
| 2920 |
+
v2 --model-id BAAI/bge-base-en-v1.5 --revision "main" --dtype float16 --batch-size 32 --engine torch --port 7997
|
| 2921 |
+
```
|
| 2922 |
+
|
| 2923 |
### Usage for Reranker
|
| 2924 |
|
| 2925 |
Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding.
|