# DistributedRunner[[optimum.habana.distributed.DistributedRunner]]

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>class optimum.habana.distributed.DistributedRunner</name><anchor>optimum.habana.distributed.DistributedRunner</anchor><source>https://github.com/huggingface/optimum-habana/blob/v1.19.0/optimum/habana/distributed/distributed_runner.py#L32</source><parameters>[{"name": "command_list", "val": ": typing.List = []"}, {"name": "world_size", "val": ": int = 1"}, {"name": "hostfile", "val": ": typing.Union[str, pathlib.Path] = None"}, {"name": "use_mpi", "val": ": bool = False"}, {"name": "use_deepspeed", "val": ": bool = False"}, {"name": "master_port", "val": ": int = 29500"}, {"name": "use_env", "val": ": bool = False"}, {"name": "map_by", "val": ": bool = 'socket'"}, {"name": "multi_hls", "val": " = None"}]</parameters></docstring>

Set up training/inference hardware configurations and run distributed commands.



<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>create_multi_node_setup</name><anchor>optimum.habana.distributed.DistributedRunner.create_multi_node_setup</anchor><source>https://github.com/huggingface/optimum-habana/blob/v1.19.0/optimum/habana/distributed/distributed_runner.py#L188</source><parameters>[]</parameters></docstring>

Multi-node configuration setup for DeepSpeed.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>create_single_card_setup</name><anchor>optimum.habana.distributed.DistributedRunner.create_single_card_setup</anchor><source>https://github.com/huggingface/optimum-habana/blob/v1.19.0/optimum/habana/distributed/distributed_runner.py#L147</source><parameters>[{"name": "use_deepspeed", "val": " = False"}]</parameters></docstring>

Single-card setup.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>create_single_node_setup</name><anchor>optimum.habana.distributed.DistributedRunner.create_single_node_setup</anchor><source>https://github.com/huggingface/optimum-habana/blob/v1.19.0/optimum/habana/distributed/distributed_runner.py#L177</source><parameters>[]</parameters></docstring>

Single-node multi-card configuration setup.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>create_single_node_setup_deepspeed</name><anchor>optimum.habana.distributed.DistributedRunner.create_single_node_setup_deepspeed</anchor><source>https://github.com/huggingface/optimum-habana/blob/v1.19.0/optimum/habana/distributed/distributed_runner.py#L168</source><parameters>[]</parameters></docstring>

Single-node multi-card configuration setup for DeepSpeed.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>create_single_node_setup_mpirun</name><anchor>optimum.habana.distributed.DistributedRunner.create_single_node_setup_mpirun</anchor><source>https://github.com/huggingface/optimum-habana/blob/v1.19.0/optimum/habana/distributed/distributed_runner.py#L157</source><parameters>[]</parameters></docstring>

Single-node multi-card configuration setup for mpirun.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>process_hostfile</name><anchor>optimum.habana.distributed.DistributedRunner.process_hostfile</anchor><source>https://github.com/huggingface/optimum-habana/blob/v1.19.0/optimum/habana/distributed/distributed_runner.py#L230</source><parameters>[]</parameters><rettype>str</rettype><retdesc>address of the master node.</retdesc></docstring>

Returns the master address to use for multi-node runs with DeepSpeed.
Directly inspired from https://github.com/microsoft/DeepSpeed/blob/316c4a43e0802a979951ee17f735daf77ea9780f/deepspeed/autotuning/utils.py#L145.






</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>run</name><anchor>optimum.habana.distributed.DistributedRunner.run</anchor><source>https://github.com/huggingface/optimum-habana/blob/v1.19.0/optimum/habana/distributed/distributed_runner.py#L196</source><parameters>[]</parameters></docstring>

Runs the desired command with configuration specified by the user.


</div></div>

<EditOnGithub source="https://github.com/huggingface/optimum-habana/blob/main/docs/source/package_reference/distributed_runner.mdx" />