Any-to-Any
MLX
Safetensors
English
gemma4
quantized
apple-silicon
vision
audio
multimodal
8bit
8-bit precision
Instructions to use OsaurusAI/gemma-4-E4B-it-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use OsaurusAI/gemma-4-E4B-it-8bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir gemma-4-E4B-it-8bit OsaurusAI/gemma-4-E4B-it-8bit
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
Update usage to Osaurus branding
Browse files
README.md
CHANGED
|
@@ -61,9 +61,13 @@ Every tensor in every multimodal component was loaded and checked for `max(abs(t
|
|
| 61 |
|
| 62 |
## Usage
|
| 63 |
|
| 64 |
-
|
|
|
|
|
|
|
|
|
|
| 65 |
|
| 66 |
```python
|
|
|
|
| 67 |
from mlx_vlm import load, generate
|
| 68 |
|
| 69 |
model, processor = load("OsaurusAI/gemma-4-E4B-it-8bit")
|
|
@@ -75,12 +79,6 @@ output = generate(model, processor, "Explain quantum computing", max_tokens=500)
|
|
| 75 |
output = generate(model, processor, "Describe this image", ["path/to/image.jpg"], max_tokens=500)
|
| 76 |
```
|
| 77 |
|
| 78 |
-
### With vMLX / vllm-mlx
|
| 79 |
-
|
| 80 |
-
```bash
|
| 81 |
-
vllm-mlx serve OsaurusAI/gemma-4-E4B-it-8bit
|
| 82 |
-
```
|
| 83 |
-
|
| 84 |
## Conversion Details
|
| 85 |
|
| 86 |
| Detail | Value |
|
|
|
|
| 61 |
|
| 62 |
## Usage
|
| 63 |
|
| 64 |
+
```bash
|
| 65 |
+
# Requires Osaurus (https://osaurus.ai)
|
| 66 |
+
osaurus serve OsaurusAI/gemma-4-E4B-it-8bit
|
| 67 |
+
```
|
| 68 |
|
| 69 |
```python
|
| 70 |
+
# Python API
|
| 71 |
from mlx_vlm import load, generate
|
| 72 |
|
| 73 |
model, processor = load("OsaurusAI/gemma-4-E4B-it-8bit")
|
|
|
|
| 79 |
output = generate(model, processor, "Describe this image", ["path/to/image.jpg"], max_tokens=500)
|
| 80 |
```
|
| 81 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 82 |
## Conversion Details
|
| 83 |
|
| 84 |
| Detail | Value |
|