Modify CNN training code to work with Horovod

Sat, 04 Nov 2023 00:00:00 +0000

Introduction

With Try Horovod in Docker, you can now use Horovod in your own environment (on-premises) and in a Docker environment. The next thing to do is to modify the training code running on a single server to apply it to distributed training using Horovod! For starters, I modified a relatively simple CNN code to allow distributed learning using Horovod, which is summarized in the following article.

Try Horovod in Docker

Sat, 14 Oct 2023 00:00:00 +0000

Motivation

I have been interested in Distributed Training for about a year. I have been experimenting with a distributed learning framework called Horovod on multiple TITAN-V-capable machines. I finally got a distributed training sample working, so I am posting it here.

Horovod on akenji's lab

Modify CNN training code to work with Horovod

Introduction

Try Horovod in Docker

Motivation