Horovod documentation

Horovod improves the speed, scale, and resource utilization of deep learning training.

Get started

Choose your deep learning framework to learn how to get started with Horovod.

To use Horovod with TensorFlow on your laptop:

  1. Install Open MPI 3.1.2 or 4.0.0, or another MPI implementation.
  2. If you've installed TensorFlow from PyPI, make sure that g++-5 or above is installed.
    If you've installed TensorFlow from Conda, make sure that the gxx_linux-64 Conda package is installed.
  3. Install the Horovod pip package: pip install horovod
  4. Read Horovod with TensorFlow for best practices and examples.
Or, use Horovod on GPUs, in Spark, Docker, Singularity, or Kubernetes (Kubeflow, MPI Operator, Helm Chart, and FfDL).

To use Horovod with Keras on your laptop:

  1. Install Open MPI 3.1.2 or 4.0.0, or another MPI implementation.
  2. If you've installed TensorFlow from PyPI, make sure that g++-5 or above is installed.
    If you've installed TensorFlow from Conda, make sure that the gxx_linux-64 Conda package is installed.
  3. Install the Horovod pip package: pip install horovod
  4. Read Horovod with Keras for best practices and examples.
Or, use Horovod on GPUs, in Spark, Docker, Singularity, or Kubernetes (Kubeflow, MPI Operator, Helm Chart, and FfDL).

To use Horovod with PyTorch on your laptop:

  1. Install Open MPI 3.1.2 or 4.0.0, or another MPI implementation.
  2. If you've installed PyTorch from PyPI, make sure that g++-5 or above is installed.
    If you've installed PyTorch from Conda, make sure that the gxx_linux-64 Conda package is installed.
  3. Install the Horovod pip package: pip install horovod
  4. Read Horovod with PyTorch for best practices and examples.
Or, use Horovod on GPUs, in Spark, Docker, Singularity, or Kubernetes (Kubeflow, MPI Operator, Helm Chart, and FfDL).

To use Horovod with Apache MXNet on your laptop:

  1. Install Open MPI 3.1.2 or 4.0.0, or another MPI implementation.
  2. Install the Horovod pip package: pip install horovod
  3. Read Horovod with MXNet for best practices and examples.
Or, use Horovod on GPUs, in Spark, Docker, Singularity, or Kubernetes (Kubeflow, MPI Operator, Helm Chart, and FfDL).

Guides

Indices and tables