# Optimum for Intel® Gaudi® AI Accelerator

Optimum for Intel Gaudi AI accelerator is the interface between Hugging Face libraries (Transformers, Diffusers, Accelerate,...) and [Intel Gaudi AI Accelerators (HPUs)](https://docs.habana.ai/en/latest/index.html).
It provides a set of tools that enable easy model loading, training and inference on single- and multi-HPU settings for various downstream tasks as shown in the table below.

<div class="mt-10">
  <div class="w-full flex flex-col space-y-4 md:space-y-0 md:grid md:grid-cols-2 md:gap-y-4 md:gap-x-5">
    <a class="!no-underline border dark:border-gray-700 p-5 rounded-lg shadow hover:shadow-lg" href="./tutorials/overview"
      ><div class="w-full text-center bg-gradient-to-br from-blue-400 to-blue-500 rounded-lg py-1.5 font-semibold mb-5 text-white text-lg leading-relaxed">Tutorials</div>
      <p class="text-gray-700">Learn the basics and become familiar with training transformers on HPUs with 🤗 Optimum. Start here if you are using 🤗 Optimum for Intel Gaudi for the first time!</p>
    </a>
    <a class="!no-underline border dark:border-gray-700 p-5 rounded-lg shadow hover:shadow-lg" href="./usage_guides/overview"
      ><div class="w-full text-center bg-gradient-to-br from-indigo-400 to-indigo-500 rounded-lg py-1.5 font-semibold mb-5 text-white text-lg leading-relaxed">How-to guides</div>
      <p class="text-gray-700">Practical guides to help you achieve a specific goal. Take a look at these guides to learn how to use 🤗 Optimum for Intel Gaudi to solve real-world problems.</p>
    </a>
  </div>
</div>

The Intel Gaudi AI accelerator family currently includes three product generations:
[Intel Gaudi 1](https://habana.ai/products/gaudi/),
[Intel Gaudi 2](https://habana.ai/products/gaudi2/), and
[Intel Gaudi 3](https://habana.ai/products/gaudi3/).
Each server is equipped with 8 devices, known as Habana Processing Units (HPUs), providing 128GB of memory on Gaudi 3,
96GB on Gaudi 2, and 32GB on the first-gen Gaudi. For more details on the underlying hardware architecture, check out the
[Gaudi Architecture Overview](https://docs.habana.ai/en/latest/Gaudi_Overview/Gaudi_Architecture.html).
Optimum for Intel Gaudi library is fully compatible with all three generations of Gaudi accelerators.

For in-depth examples of running workloads on Gaudi, explore the following blog posts:
- [Benchmarking Intel Gaudi 2 with NVIDIA A100 GPUs](https://huggingface.co/blog/habana-gaudi-2-benchmark)
- [Accelerating Vision-Language Models: BridgeTower on Habana Gaudi2](https://huggingface.co/blog/bridgetower)

The following model architectures, tasks and device distributions have been validated for Optimum for Intel Gaudi:

<Tip>

In the tables below, ✅ means single-card, multi-card and DeepSpeed have all been validated.

</Tip>

- Transformers:

| Architecture                                          | Training                      | Inference                                       | Tasks                                                                                                                                                                                                                                                           |
|:------------------------------------------------------|:-----------------------------:|:-----------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| BERT                                                  | ✅                            | ✅                                              | <ul><li>[text classification](/examples/text-classification)</li><li>[question answering](/examples/question-answering)</li><li>[language modeling](/examples/language-modeling)</li><li>[text feature extraction](/examples/text-feature-extraction)</li></ul> |
| RoBERTa                                               | ✅                            | ✅                                              | <ul><li>[question answering](/examples/question-answering)</li><li>[language modeling](/examples/language-modeling)</li></ul>                                                                                                                                   |
| ALBERT                                                | ✅                            | ✅                                              | <ul><li>[question answering](/examples/question-answering)</li><li>[language modeling](/examples/language-modeling)</li></ul>                                                                                                                                   |
| DistilBERT                                            | ✅                            | ✅                                              | <ul><li>[question answering](/examples/question-answering)</li><li>[language modeling](/examples/language-modeling)</li></ul>                                                                                                                                   |
| GPT2                                                  | ✅                            | ✅                                              | <ul><li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                         |
| BLOOM(Z)                                              |                               | <ul><li>DeepSpeed</li></ul>                     | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| StarCoder / StarCoder2                                | ✅                            | <ul><li>Single-card</li></ul>                   | <ul><li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                         |
| GPT-J                                                 | <ul><li>DeepSpeed</li></ul>   | <ul><li>Single card</li><li>DeepSpeed</li></ul> | <ul><li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                         |
| GPT-Neo                                               |                               | <ul><li>Single card</li></ul>                   | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| GPT-NeoX                                              | <ul><li>DeepSpeed</li></ul>   | <ul><li>DeepSpeed</li></ul>                     | <ul><li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                         |
| OPT                                                   |                               | <ul><li>DeepSpeed</li></ul>                     | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| Llama 2 / CodeLlama / Llama 3 / Llama Guard / Granite | ✅                            | ✅                                              | <ul><li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li><li>[question answering](/examples/question-answering)</li><li>[text classification](/examples/text-classification) (Llama Guard)</li></ul>   |
| StableLM                                              |                               | <ul><li>Single card</li></ul>                   | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| Falcon                                                | <ul><li>LoRA</li></ul>        | ✅                                              | <ul><li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                         |
| CodeGen                                               |                               | <ul><li>Single card</li></ul>                   | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| MPT                                                   |                               | <ul><li>Single card</li></ul>                   | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| Mistral                                               |                               | <ul><li>Single card</li></ul>                   | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| Phi                                                   | ✅                            | <ul><li>Single card</li></ul>                   | <ul><li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                         |
| Mixtral                                               |                               | <ul><li>Single card</li></ul>                   | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| Persimmon                                             |                               | <ul><li>Single card</li></ul>                   | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| Qwen2 / Qwen3                                         | <ul><li>Single card</li></ul> | <ul><li>Single card</li></ul>                   | <ul><li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                         |
| Qwen2-MoE                                             |                               | <ul><li>Single card</li></ul>                   | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| Gemma                                                 | ✅                            | <ul><li>Single card</li></ul>                   | <ul><li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                         |
| Gemma2                                                |                               | ✅                                              | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| Gemma3                                                |                               | ✅                                              | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| XGLM                                                  |                               | <ul><li>Single card</li></ul>                   | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| Cohere                                                |                               | <ul><li>Single card</li></ul>                   | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| T5 / Flan T5                                          | ✅                            | ✅                                              | <ul><li>[summarization](/examples/summarization)</li><li>[translation](/examples/translation)</li><li>[question answering](/examples/question-answering#fine-tuning-t5-on-squad20)</li></ul>                                                                    |
| BART                                                  |                               | <ul><li>Single card</li></ul>                   | <ul><li>[summarization](/examples/summarization)</li><li>[translation](/examples/translation)</li><li>[question answering](/examples/question-answering#fine-tuning-t5-on-squad20)</li></ul>                                                                    |
| ViT                                                   | ✅                            | ✅                                              | <ul><li>[image classification](/examples/image-classification)</li></ul>                                                                                                                                                                                        |
| Swin                                                  | ✅                            | ✅                                              | <ul><li>[image classification](/examples/image-classification)</li></ul>                                                                                                                                                                                        |
| Wav2Vec2                                              | ✅                            | ✅                                              | <ul><li>[audio classification](/examples/audio-classification)</li><li>[speech recognition](/examples/speech-recognition)</li></ul>                                                                                                                             |
| Whisper                                               | ✅                            | ✅                                              | <ul><li>[speech recognition](/examples/speech-recognition)</li></ul>                                                                                                                                                                                            |
| SpeechT5                                              |                               | <ul><li>Single card</li></ul>                   | <ul><li>[text to speech](/examples/text-to-speech)</li></ul>                                                                                                                                                                                                    |
| CLIP                                                  | ✅                            | ✅                                              | <ul><li>[contrastive image-text training](/examples/contrastive-image-text)</li></ul>                                                                                                                                                                           |
| BridgeTower                                           | ✅                            | ✅                                              | <ul><li>[contrastive image-text training](/examples/contrastive-image-text)</li></ul>                                                                                                                                                                           |
| ESMFold                                               |                               | <ul><li>Single card</li></ul>                   | <ul><li>[protein folding](/examples/protein-folding)</li></ul>                                                                                                                                                                                                  |
| Blip                                                  |                               | <ul><li>Single card</li></ul>                   | <ul><li>[visual question answering](/examples/visual-question-answering)</li><li>[image to text](/examples/image-to-text)</li></ul>                                                                                                                             |
| OWLViT                                                |                               | <ul><li>Single card</li></ul>                   | <ul><li>[zero shot object detection](/examples/zero-shot-object-detection)</li></ul>                                                                                                                                                                            |
| ClipSeg                                               |                               | <ul><li>Single card</li></ul>                   | <ul><li>[object segmentation](/examples/object-segementation)</li></ul>                                                                                                                                                                                         |
| Llava / Llava-next / Llava-onevision                  |                               | <ul><li>Single card</li></ul>                   | <ul><li>[image to text](/examples/image-to-text)</li></ul>                                                                                                                                                                                                      |
| idefics2                                              | <ul><li>LoRA</li></ul>        | <ul><li>Single card</li></ul>                   | <ul><li>[image to text](/examples/image-to-text)</li></ul>                                                                                                                                                                                                      |
| Paligemma                                             |                               | <ul><li>Single card</li></ul>                   | <ul><li>[image to text](/examples/image-to-text)</li></ul>                                                                                                                                                                                                      |
| Segment Anything Model                                |                               | <ul><li>Single card</li></ul>                   | <ul><li>[object segmentation](/examples/object-segementation)</li></ul>                                                                                                                                                                                         |
| VideoMAE                                              |                               | <ul><li>Single card</li></ul>                   | <ul><li>[Video classification](/examples/video-classification)</li></ul>                                                                                                                                                                                        |
| TableTransformer                                      |                               | <ul><li>Single card</li></ul>                   | <ul><li>[table object detection](/examples/table-detection) </li></ul>                                                                                                                                                                                          |
| DETR                                                  |                               | <ul><li>Single card</li></ul>                   | <ul><li>[object detection](/examples/object-detection)</li></ul>                                                                                                                                                                                                |
| Mllama                                                | <ul><li>LoRA</li></ul>        | ✅                                              | <ul><li>[image to text](/examples/image-to-text)</li></ul>                                                                                                                                                                                                      |
| MiniCPM3                                              |                               | <ul><li>Single card</li></ul>                   | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| Baichuan2                                             | <ul><li>DeepSpeed</li></ul>   | <ul><li>Single card</li></ul>                   | <ul><li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                         |
| DeepSeek-V2                                           | ✅                            | ✅                                              | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| DeepSeek-V3 / Moonlight                               |                               | ✅                                              | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| ChatGLM                                               | <ul><li>DeepSpeed</li></ul>   | <ul><li>Single card</li></ul>                   | <ul><li>[language modeling](/examples/language-modeling)</li><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                         |
| Qwen2-VL                                              |                               |  <ul><li>Single card</li></ul>                  | <ul><li>[image to text](/examples/image-to-text)</li></ul>                                                                                                                                                                                                      |
| Qwen2.5-VL                                            |                               |  <ul><li>Single card</li></ul>                  | <ul><li>[image to text](/examples/image-to-text)</li></ul>                                                                                                                                                                                                      |
| VideoLLaVA                                            |                               | <ul><li>Single card</li></ul>                   | <ul><li>[Video comprehension](/examples/video-comprehension)</li></ul>                                                                                                                                                                                          |
| GLM-4V                                                |                               |  <ul><li>Single card</li></ul>                  | <ul><li>[image to text](/examples/image-to-text)</li></ul>                                                                                                                                                                                                      |
| Arctic                                                |                               |  <ul><li>DeepSpeed</li></ul>                    | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |
| GPT-OSS                                               |                               |  <ul><li>DeepSpeed</li></ul>                    | <ul><li>[text generation](/examples/text-generation)</li></ul>                                                                                                                                                                                                  |

- Diffusers

| Architecture               | Training.              | Inference                     | Tasks                                                                                                                                                                                         |
|----------------------------|:----------------------:|:-----------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Stable Diffusion           | ✅                  | ✅                         | <ul><li>[text-to-image generation](/examples/stable-diffusion)</li></ul>                                                                                                                      |
| Stable Diffusion XL        | ✅                  | ✅                         | <ul><li>[text-to-image generation](/examples/stable-diffusion)</li></ul>                                                                                                                      |
| Stable Diffusion Depth2img |                        | <ul><li>Single card</li></ul> | <ul><li>[depth-to-image generation](/examples/stable-diffusion)</li></ul>                                                                                                                     |
| Stable Diffusion 3         | ✅                  | ✅                         | <ul><li>[text-to-image generation](/examples/stable-diffusion#stable-diffusion-3-and-35-sd3)</li></ul>                                                                                        |
| LDM3D                      |                        | <ul><li>Single card</li></ul> | <ul><li>[text-to-image generation](/examples/stable-diffusion)</li></ul>                                                                                                                      |
| FLUX.1                     | <ul><li>LoRA</li></ul> | <ul><li>Single card</li></ul> | <ul><li>[text-to-image generation](/examples/stable-diffusion)</li></ul>                                                                                                                      |
| Text to Video              |                        | <ul><li>Single card</li></ul> | <ul><li>[text-to-video generation](/examples/stable-diffusion#text-to-video-generation)</li></ul>                                                                                             |
| Image to Video             |                        | <ul><li>Single card</li></ul> | <ul><li>[image-to-video generation](/examples/stable-diffusion#image-to-video-generation)</li></ul>                                                                                           |
| i2vgen-xl                  |                        | <ul><li>Single card</li></ul> | <ul><li>[image-to-video generation](/examples/stable-diffusion#I2vgen-xl)</li></ul>                                                                                                           |
| Wan                        |                        | ✅                         | <ul><li>[text-to-video generation](/examples/stable-diffusion#text-to-video-with-wan-22)</li><li>[image-to-video generation](/examples/stable-diffusion#image-to-video-with-wan-22)</li></ul> |

- PyTorch Image Models/TIMM:

| Architecture        | Training | Inference                     | Tasks                                                                     |
|---------------------|:--------:|:-----------------------------:|:--------------------------------------------------------------------------|
| FastViT             |          | <ul><li>Single card</li></ul> |  <ul><li>[image classification](/examples/image-classification)</li></ul> |

- TRL:

| Architecture     | Training | Inference | Tasks                                            |
|------------------|:--------:|:---------:|:-------------------------------------------------|
| Llama 2          | ✅       |           | <ul><li>[DPO Pipeline](/examples/trl)</li></ul>  |
| Llama 2          | ✅       |           | <ul><li>[PPO Pipeline](/examples/trl)</li></ul>  |
| Stable Diffusion | ✅       |           | <ul><li>[DDPO Pipeline](/examples/trl)</li></ul> |


Other models and tasks supported by the 🤗 Transformers and 🤗 Diffusers library may also work.
You can refer to this [section](https://github.com/huggingface/optimum-habana#how-to-use-it) for using them with 🤗 Optimum for Intel Gaudi.
In addition, [this page](/examples) explains how to modify any [example](https://github.com/huggingface/transformers/tree/main/examples/pytorch) from the 🤗 Transformers library to make it work with 🤗 Optimum for Intel Gaudi.


<EditOnGithub source="https://github.com/huggingface/optimum-habana/blob/main/docs/source/index.mdx" />