# Optimum for Intel® Gaudi® AI Accelerator

Optimum for Intel Gaudi AI accelerator is the interface between Hugging Face libraries (Transformers, Diffusers, Accelerate,...) and [Intel Gaudi AI Accelerators (HPUs)](https://docs.habana.ai/en/latest/index.html).
It provides a set of tools that enable easy model loading, training and inference on single- and multi-HPU settings for various downstream tasks as shown in the table below.

<div class="mt-10">
  <div class="w-full flex flex-col space-y-4 md:space-y-0 md:grid md:grid-cols-2 md:gap-y-4 md:gap-x-5">
    <a class="!no-underline border dark:border-gray-700 p-5 rounded-lg shadow hover:shadow-lg" href="./tutorials/overview"
      ><div class="w-full text-center bg-gradient-to-br from-blue-400 to-blue-500 rounded-lg py-1.5 font-semibold mb-5 text-white text-lg leading-relaxed">Tutorials</div>
      <p class="text-gray-700">Learn the basics and become familiar with training transformers on HPUs with 🤗 Optimum. Start here if you are using 🤗 Optimum for Intel Gaudi for the first time!</p>
    </a>
    <a class="!no-underline border dark:border-gray-700 p-5 rounded-lg shadow hover:shadow-lg" href="./usage_guides/overview"
      ><div class="w-full text-center bg-gradient-to-br from-indigo-400 to-indigo-500 rounded-lg py-1.5 font-semibold mb-5 text-white text-lg leading-relaxed">How-to guides</div>
      <p class="text-gray-700">Practical guides to help you achieve a specific goal. Take a look at these guides to learn how to use 🤗 Optimum for Intel Gaudi to solve real-world problems.</p>
    </a>
  </div>
</div>

The Intel Gaudi AI accelerator family currently includes three product generations:
[Intel Gaudi 1](https://habana.ai/products/gaudi/),
[Intel Gaudi 2](https://habana.ai/products/gaudi2/), and
[Intel Gaudi 3](https://habana.ai/products/gaudi3/).
Each server is equipped with 8 devices, known as Habana Processing Units (HPUs), providing 128GB of memory on Gaudi 3,
96GB on Gaudi 2, and 32GB on the first-gen Gaudi. For more details on the underlying hardware architecture, check out the
[Gaudi Architecture Overview](https://docs.habana.ai/en/latest/Gaudi_Overview/Gaudi_Architecture.html).
Optimum for Intel Gaudi library is fully compatible with all three generations of Gaudi accelerators.

For in-depth examples of running workloads on Gaudi, explore the following blog posts:
- [Benchmarking Intel Gaudi 2 with NVIDIA A100 GPUs](https://huggingface.co/blog/habana-gaudi-2-benchmark)
- [Accelerating Vision-Language Models: BridgeTower on Habana Gaudi2](https://huggingface.co/blog/bridgetower)

The following model architectures, tasks and device distributions have been validated for Optimum for Intel Gaudi:

<Tip>

In the tables below, ✅ means single-card, multi-card and DeepSpeed have all been validated.

</Tip>

- Transformers:

| Architecture | Training | Inference | Tasks |
|--------------|:--------:|:---------:|:------|
| BERT         | ✅       | ✅        | <li>[text classification](/examples/text-classification)</li><li>[question answering](/examples/question-answering)</li><li>[language modeling](/examples/language-modeling)</li><li>[text feature extraction](/examples/text-feature-extraction)</li> |
| RoBERTa      | ✅       | ✅        | <li>[question answering](/examples/question-answering)</li><li>[language modeling](/examples/language-modeling)</li> |
| ALBERT       | ✅       | ✅        | <li>[question answering](/examples/question-answering)</li><li>[language modeling](/examples/language-modeling)</li> |
| DistilBERT   | ✅       | ✅        | <li>[question answering](/examples/question-answering)</li><li>[language modeling](/examples/language-modeling)</li> |
| GPT2         | ✅       | ✅        | <li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li> |
| BLOOM(Z)     |          | <div style="text-align:left"><li>DeepSpeed</li></div> | <li>[text generation](/examples/text-generation)</li> |
| StarCoder / StarCoder2 | ✅ | <div style="text-align:left"><li>Single card</li></div> | <li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li> |
| GPT-J        | <div style="text-align:left"><li>DeepSpeed</li></div> | <div style="text-align:left"><li>Single card</li><li>DeepSpeed</li></div> | <li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li> |
| GPT-Neo      |          | <div style="text-align:left"><li>Single card</li></div> | <li>[text generation](/examples/text-generation)</li> |
| GPT-NeoX     | <div style="text-align:left"><li>DeepSpeed</li></div> | <div style="text-align:left"><li>DeepSpeed</li></div> | <li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li> |
| OPT          |          | <div style="text-align:left"><li>DeepSpeed</li></div> | <li>[text generation](/examples/text-generation)</li> |
| Llama 2 / CodeLlama / Llama 3 / Llama Guard / Granite | ✅ | ✅ | <li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li><li>[question answering](/examples/question-answering)</li><li>[text classification](/examples/text-classification) (Llama Guard)</li> |
| StableLM     |          | <div style="text-align:left"><li>Single card</li></div> | <li>[text generation](/examples/text-generation)</li> |
| Falcon       | <div style="text-align:left"><li>LoRA</li></div> | ✅ | <li>[text generation](/examples/text-generation)</li> |
| CodeGen      |          | <div style="text-align:left"><li>Single card</li></div> | <li>[text generation](/examples/text-generation)</li> |
| MPT          |          | <div style="text-align:left"><li>Single card</li></div> | <li>[text generation](/examples/text-generation)</li> |
| Mistral      |          | <div style="text-align:left"><li>Single card</li></div> | <li>[text generation](/examples/text-generation)</li> |
| Phi          | ✅       | <div style="text-align:left"><li>Single card</li></div> | <li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li> |
| Mixtral      |          | <div style="text-align:left"><li>Single card</li></div> | <li>[text generation](/examples/text-generation)</li> |
| Gemma        | ✅       | <div style="text-align:left"><li>Single card</li></div> | <li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li> |
| Gemma2       |           | ✅        | <li>[text generation](/examples/text-generation)</li> |
| Gemma3       |           | ✅        | <li>[text generation](/examples/text-generation)</li> |
| Qwen2 / Qwen3 | <div style="text-align:left"><li>Single card</li></div> | <div style="text-align:left"><li>Single card</li></div> | <li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li> |
| Qwen2-MoE    |          | <div style="text-align:left"><li>Single card</li></div> | <li>[text generation](/examples/text-generation)</li> |
| Persimmon    |          | <div style="text-align:left"><li>Single card</li></div> | <li>[text generation](/examples/text-generation)</li> |
| XGLM         |          | <div style="text-align:left"><li>Single card</li></div> | <li>[text generation](/examples/text-generation)</li> |
| Cohere       |          | <div style="text-align:left"><li>Single card</li></div> | <li>[text generation](/examples/text-generation)</li> |
| T5 / Flan T5 | ✅       | ✅        | <li>[summarization](/examples/summarization)</li><li>[translation](/examples/translation)</li><li>[question answering](/examples/question-answering#fine-tuning-t5-on-squad20)</li> |
| BART         |          | <div style="text-align:left"><li>Single card</li></div> | <li>[summarization](/examples/summarization)</li><li>[translation](/examples/translation)</li><li>[question answering](/examples/question-answering#fine-tuning-t5-on-squad20)</li> |
| ViT          | ✅       | ✅        | <li>[image classification](/examples/image-classification)</li> |
| Swin         | ✅       | ✅        | <li>[image classification](/examples/image-classification)</li> |
| Wav2Vec2     | ✅       | ✅        | <li>[audio classification](/examples/audio-classification)</li><li>[speech recognition](/examples/speech-recognition)</li> |
| Whisper      | ✅       | ✅        | <li>[speech recognition](/examples/speech-recognition)</li> |
| SpeechT5     |          | <div style="text-align:left"><li>Single card</li></div> | <li>[text to speech](/examples/text-to-speech)</li> |
| CLIP         | ✅       | ✅        | <li>[contrastive image-text training](/examples/contrastive-image-text)</li> |
| BridgeTower  | ✅       | ✅        | <li>[contrastive image-text training](/examples/contrastive-image-text)</li> |
| ESMFold      |          | <div style="text-align:left"><li>Single card</li></div> | <li>[protein folding](/examples/protein-folding)</li> |
| Blip         |          | <div style="text-align:left"><li>Single card</li></div> | <li>[visual question answering](/examples/visual-question-answering)</li><li>[image to text](/examples/image-to-text)</li> |
| OWLViT       |          | <div style="text-align:left"><li>Single card</li></div> | <li>[zero shot object detection](/examples/zero-shot-object-detection)</li> |
| ClipSeg      |          | <div style="text-align:left"><li>Single card</li></div> | <li>[object segmentation](/examples/object-segementation)</li> |
| Llava / Llava-next / Llava-onevision |    | <div style="text-align:left"><li>Single card</li></div> | <li>[image to text](/examples/image-to-text)</li> |
| Paligemma |    | <div style="text-align:left"><li>Single card</li></div> | <li>[image to text](/examples/image-to-text)</li> |
| idefics2     | <div style="text-align:left"><li>LoRA</li></div> | <div style="text-align:left"><li>Single card</li></div> | <li>[image to text](/examples/image-to-text)</li> |
| SAM          |          | <div style="text-align:left"><li>Single card</li></div> | <li>[object segmentation](/examples/object-segementation)</li> |
| VideoMAE |          | <div style="text-align:left"><li>Single card</li></div> | <li>[Video classification](/examples/video-classification)</li> |
| TableTransformer |       | <div style="text-align:left"><li>Single card</li></div> | <li>[table object detection](/examples/table-detection)</li> |
| DETR         |          | <div style="text-align:left"><li>Single card</li></div> | <li>[object detection](/examples/object-detection)</li> |
| Mllama     | <div style="text-align:left"><li>LoRA</li></div> |✅      | <li>[image to text](/examples/image-to-text)</li> |
| Video-LLaVA         |          | <div style="text-align:left"><li>Single card</li></div> | <li>[video comprehension](/examples/video-comprehension)</li> |
| MiniCPM3 |   | <div style="text-align:left"><li>Single card</li></div> | <li>[text generation](/examples/text-generation)</li> |
| Baichuan2 | <div style="text-align:left"><li>DeepSpeed</li></div> | <div style="text-align:left"><li>Single card</li></div> | <li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li> |
| DeepSeek-V2 | ✅ | ✅ | <li>[text generation](/examples/text-generation)</li> |
| DeepSeek-V3 / Moonlight |   | ✅ | <li>[text generation](/examples/text-generation)</li> |
| ChatGLM     | <div style="text-align:left"><li>DeepSpeed</li></div> |  <div style="text-align:left"><li>Single card</li></div> | <li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li> |
| Qwen2-VL |          |  <div style="text-align:left"><li>Single card</li></div> | <li>[image to text](/examples/image-to-text)</li> |
| Qwen2.5-VL |        |  <div style="text-align:left"><li>Single card</li></div> | <li>[image to text](/examples/image-to-text)</li> |
| GLM-4V |          |  <div style="text-align:left"><li>Single card</li></div> | <li>[image to text](/examples/image-to-text)</li> |
| Arctic |          |  <div style="text-align:left"><li>DeepSpeed</li></div> | <li>[text generation](/examples/text-generation)</li> |
| GPT-OSS |          |  <div style="text-align:left"><li>DeepSpeed</li></div> | <li>[text generation](/examples/text-generation)</li> |

- Diffusers

| Architecture        | Training | Inference | Tasks |
|---------------------|:--------:|:---------:|:------|
| Stable Diffusion    | ✅ | ✅ | <li>[text-to-image generation](/examples/stable-diffusion)</li> |
| Stable Diffusion XL | ✅ | ✅ | <li>[text-to-image generation](/examples/stable-diffusion)</li> |
| Stable Diffusion Depth2img | | <li>Single card</li> | <li>[depth-to-image generation](/examples/stable-diffusion)</li> |
| Stable Diffusion 3  | ✅ | <li>Single card</li> | <li>[text-to-image generation](/examples/stable-diffusion#stable-diffusion-3-and-35-sd3)</li> |
| LDM3D               |          | <div style="text-align:left"><li>Single card</li></div> | <li>[text-to-image generation](/examples/stable-diffusion)</li> |
| FLUX.1              | <li>LoRA</li> | <li>Single card</li> | <li>[text-to-image generation](/examples/stable-diffusion)</li> |
| Text to Video       |          | <li>Single card</li> | <li>[text-to-video generation](/examples/stable-diffusion#text-to-video-generation)</li> |
| Image to Video      |               | <li>Single card</li> | <li>[image-to-video generation](/examples/stable-diffusion#image-to-video-generation)</li> |
| i2vgen-xl       |          | <li>Single card</li> | <li>[image-to-video generation](/examples/stable-diffusion#I2vgen-xl)</li> |

- PyTorch Image Models/TIMM:

| Architecture        | Training | Inference | Tasks |
|---------------------|:--------:|:---------:|:------|
| FastViT             |          | <div style="text-align:left"><li>Single card</li></div> | <li>[image classification](/examples/image-classification)</li> |

- TRL:

| Architecture     | Training | Inference            | Tasks |
|------------------|:--------:|:--------------------:|:------|
| Llama 2          | ✅       |           | <li>[DPO Pipeline](/examples/trl)</li> |
| Llama 2          | ✅       |           | <li>[PPO Pipeline](/examples/trl)</li> |
| Stable Diffusion | ✅       |           | <li>[DDPO Pipeline](/examples/trl)</li> |


Other models and tasks supported by the 🤗 Transformers and 🤗 Diffusers library may also work.
You can refer to this [section](https://github.com/huggingface/optimum-habana#how-to-use-it) for using them with 🤗 Optimum for Intel Gaudi.
In addition, [this page](/examples) explains how to modify any [example](https://github.com/huggingface/transformers/tree/main/examples/pytorch) from the 🤗 Transformers library to make it work with 🤗 Optimum for Intel Gaudi.


<EditOnGithub source="https://github.com/huggingface/optimum-habana/blob/main/docs/source/index.mdx" />