Spaces:

ankanghosh
/

anveshak

Running

ankanghosh commited on 13 days ago

Commit

1d6a10a

verified ·

1 Parent(s): ec0a1d0

Update rag_engine.py

Files changed (1) hide show

rag_engine.py CHANGED Viewed

@@ -54,7 +54,7 @@ def cached_load_model():
         tokenizer = AutoTokenizer.from_pretrained(embedding_model)
         model = AutoModel.from_pretrained(
             embedding_model,
-            torch_dtype=torch.float16
         )
         # Move model to CPU and set to eval mode for inference

         tokenizer = AutoTokenizer.from_pretrained(embedding_model)
         model = AutoModel.from_pretrained(
             embedding_model,
+            torch_dtype=torch.bfloat16
         )
         # Move model to CPU and set to eval mode for inference