Performance evaluation of Horovod Cluster in ELKH Cloud

The sheer volume of the input datasets used in the process can have a significant negative effect on the training time of the model. As a consequence of physical limitations, the resources of a single node cannot be scaled to a sufficient level for efficient training.

As such, different parallel and distributed solutions exist in order to solve this problem. In this paper, we measured the scalability of Horovod, a distributed deep learning framework (available as a reference architecture) with different parameters on the research infrastructure called ELKH Cloud. The experiment was conducted in order to verify the scalability of the framework on the general purpose infrastructure, which was not designed pri- marily for distributed deep learning applications.