# Quantization

## ORTQuantizer[[optimum.onnxruntime.ORTQuantizer]]

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>class optimum.onnxruntime.ORTQuantizer</name><anchor>optimum.onnxruntime.ORTQuantizer</anchor><source>https://github.com/huggingface/optimum-onnx/blob/v0.0.2/optimum/onnxruntime/quantization.py#L85</source><parameters>[{"name": "onnx_model_path", "val": ": Path"}, {"name": "config", "val": ": PretrainedConfig | None = None"}]</parameters></docstring>
Handles the ONNX Runtime quantization process for models shared on huggingface.co/models.


<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>compute_ranges</name><anchor>optimum.onnxruntime.ORTQuantizer.compute_ranges</anchor><source>https://github.com/huggingface/optimum-onnx/blob/v0.0.2/optimum/onnxruntime/quantization.py#L261</source><parameters>[]</parameters><retdesc>The dictionary mapping the nodes name to their quantization ranges.</retdesc></docstring>
Computes the quantization ranges.




</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>fit</name><anchor>optimum.onnxruntime.ORTQuantizer.fit</anchor><source>https://github.com/huggingface/optimum-onnx/blob/v0.0.2/optimum/onnxruntime/quantization.py#L159</source><parameters>[{"name": "dataset", "val": ": Dataset"}, {"name": "calibration_config", "val": ": CalibrationConfig"}, {"name": "onnx_augmented_model_name", "val": ": str | Path = 'augmented_model.onnx'"}, {"name": "operators_to_quantize", "val": ": list[str] | None = None"}, {"name": "batch_size", "val": ": int = 1"}, {"name": "use_external_data_format", "val": ": bool = False"}, {"name": "use_gpu", "val": ": bool = False"}, {"name": "force_symmetric_range", "val": ": bool = False"}]</parameters><paramsdesc>- **dataset** (`Dataset`) --
  The dataset to use when performing the calibration step.
- **calibration_config** (`~CalibrationConfig`) --
  The configuration containing the parameters related to the calibration step.
- **onnx_augmented_model_name** (`Union[str, Path]`, defaults to `"augmented_model.onnx"`) --
  The path used to save the augmented model used to collect the quantization ranges.
- **operators_to_quantize** (`Optional[List[str]]`, defaults to `None`) --
  List of the operators types to quantize.
- **batch_size** (`int`, defaults to 1) --
  The batch size to use when collecting the quantization ranges values.
- **use_external_data_format** (`bool`, defaults to `False`) --
  Whether to use external data format to store model which size is >= 2Gb.
- **use_gpu** (`bool`, defaults to `False`) --
  Whether to use the GPU when collecting the quantization ranges values.
- **force_symmetric_range** (`bool`, defaults to `False`) --
  Whether to make the quantization ranges symmetric.</paramsdesc><paramgroups>0</paramgroups><retdesc>The dictionary mapping the nodes name to their quantization ranges.</retdesc></docstring>
Performs the calibration step and computes the quantization ranges.






</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>from_pretrained</name><anchor>optimum.onnxruntime.ORTQuantizer.from_pretrained</anchor><source>https://github.com/huggingface/optimum-onnx/blob/v0.0.2/optimum/onnxruntime/quantization.py#L111</source><parameters>[{"name": "model_or_path", "val": ": ORTModel | str | Path"}, {"name": "file_name", "val": ": str | None = None"}]</parameters><paramsdesc>- **model_or_path** (`Union[ORTModel, str, Path]`) --
  Can be either:
  - A path to a saved exported ONNX Intermediate Representation (IR) model, e.g., `./my_model_directory/.
  - Or an `ORTModelForXX` class, e.g., `ORTModelForQuestionAnswering`.
- **file_name(`Optional[str]`,** defaults to `None`) --
  Overwrites the default model file name from `"model.onnx"` to `file_name`.
  This allows you to load different model files from the same repository or directory.</paramsdesc><paramgroups>0</paramgroups><retdesc>An instance of `ORTQuantizer`.</retdesc></docstring>
Instantiates a `ORTQuantizer` from an ONNX model file or an `ORTModel`.






</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>get_calibration_dataset</name><anchor>optimum.onnxruntime.ORTQuantizer.get_calibration_dataset</anchor><source>https://github.com/huggingface/optimum-onnx/blob/v0.0.2/optimum/onnxruntime/quantization.py#L416</source><parameters>[{"name": "dataset_name", "val": ": str"}, {"name": "num_samples", "val": ": int = 100"}, {"name": "dataset_config_name", "val": ": str | None = None"}, {"name": "dataset_split", "val": ": str | None = None"}, {"name": "preprocess_function", "val": ": Callable | None = None"}, {"name": "preprocess_batch", "val": ": bool = True"}, {"name": "seed", "val": ": int = 2016"}, {"name": "token", "val": ": bool | str | None = None"}]</parameters><paramsdesc>- **dataset_name** (`str`) --
  The dataset repository name on the Hugging Face Hub or path to a local directory containing data files
  to load to use for the calibration step.
- **num_samples** (`int`, defaults to 100) --
  The maximum number of samples composing the calibration dataset.
- **dataset_config_name** (`Optional[str]`, defaults to `None`) --
  The name of the dataset configuration.
- **dataset_split** (`Optional[str]`, defaults to `None`) --
  Which split of the dataset to use to perform the calibration step.
- **preprocess_function** (`Optional[Callable]`, defaults to `None`) --
  Processing function to apply to each example after loading dataset.
- **preprocess_batch** (`bool`, defaults to `True`) --
  Whether the `preprocess_function` should be batched.
- **seed** (`int`, defaults to 2016) --
  The random seed to use when shuffling the calibration dataset.
- **token** (`Optional[Union[bool,str]]`, defaults to `None`) --
  The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated
  when running `huggingface-cli login` (stored in `huggingface_hub.constants.HF_TOKEN_PATH`).</paramsdesc><paramgroups>0</paramgroups><retdesc>The calibration `datasets.Dataset` to use for the post-training static quantization calibration
step.</retdesc></docstring>
Creates the calibration `datasets.Dataset` to use for the post-training static quantization calibration step.






</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>partial_fit</name><anchor>optimum.onnxruntime.ORTQuantizer.partial_fit</anchor><source>https://github.com/huggingface/optimum-onnx/blob/v0.0.2/optimum/onnxruntime/quantization.py#L212</source><parameters>[{"name": "dataset", "val": ": Dataset"}, {"name": "calibration_config", "val": ": CalibrationConfig"}, {"name": "onnx_augmented_model_name", "val": ": str | Path = 'augmented_model.onnx'"}, {"name": "operators_to_quantize", "val": ": list[str] | None = None"}, {"name": "batch_size", "val": ": int = 1"}, {"name": "use_external_data_format", "val": ": bool = False"}, {"name": "use_gpu", "val": ": bool = False"}, {"name": "force_symmetric_range", "val": ": bool = False"}]</parameters><paramsdesc>- **dataset** (`Dataset`) --
  The dataset to use when performing the calibration step.
- **calibration_config** (`CalibrationConfig`) --
  The configuration containing the parameters related to the calibration step.
- **onnx_augmented_model_name** (`Union[str, Path]`, defaults to `"augmented_model.onnx"`) --
  The path used to save the augmented model used to collect the quantization ranges.
- **operators_to_quantize** (`Optional[List[str]]`, defaults to `None`) --
  List of the operators types to quantize.
- **batch_size** (`int`, defaults to 1) --
  The batch size to use when collecting the quantization ranges values.
- **use_external_data_format** (`bool`, defaults to `False`) --
  Whether uto se external data format to store model which size is >= 2Gb.
- **use_gpu** (`bool`, defaults to `False`) --
  Whether to use the GPU when collecting the quantization ranges values.
- **force_symmetric_range** (`bool`, defaults to `False`) --
  Whether to make the quantization ranges symmetric.</paramsdesc><paramgroups>0</paramgroups></docstring>
Performs the calibration step and collects the quantization ranges without computing them.




</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>quantize</name><anchor>optimum.onnxruntime.ORTQuantizer.quantize</anchor><source>https://github.com/huggingface/optimum-onnx/blob/v0.0.2/optimum/onnxruntime/quantization.py#L279</source><parameters>[{"name": "quantization_config", "val": ": QuantizationConfig"}, {"name": "save_dir", "val": ": str | Path"}, {"name": "file_suffix", "val": ": str | None = 'quantized'"}, {"name": "calibration_tensors_range", "val": ": dict[str, tuple[float, float]] | None = None"}, {"name": "use_external_data_format", "val": ": bool = False"}, {"name": "preprocessor", "val": ": QuantizationPreprocessor | None = None"}]</parameters><paramsdesc>- **quantization_config** (`QuantizationConfig`) --
  The configuration containing the parameters related to quantization.
- **save_dir** (`Union[str, Path]`) --
  The directory where the quantized model should be saved.
- **file_suffix** (`Optional[str]`, defaults to `"quantized"`) --
  The file_suffix used to save the quantized model.
- **calibration_tensors_range** (`Optional[Dict[str, Tuple[float, float]]]`, defaults to `None`) --
  The dictionary mapping the nodes name to their quantization ranges, used and required only when applying static quantization.
- **use_external_data_format** (`bool`, defaults to `False`) --
  Whether to use external data format to store model which size is >= 2Gb.
- **preprocessor** (`Optional[QuantizationPreprocessor]`, defaults to `None`) --
  The preprocessor to use to collect the nodes to include or exclude from quantization.</paramsdesc><paramgroups>0</paramgroups><retdesc>The path of the resulting quantized model.</retdesc></docstring>
Quantizes a model given the optimization specifications defined in `quantization_config`.






</div></div>