Optical Circuit Switching (OCS) refers to a network switching technique used to establish dedicated optical paths between devices in a datacenter. This technique allows for high-speed, low-latency data transmission by creating temporary, circuit-like connections using optical fibers.
We propose to apply optical circuit switching to enable dynamic AllReduce reconfiguration for accelerating distributed machine learning. With simulated annealing-based optimization, theproposed AllReduce reconfiguration approach achieves 31% less average training time than existing solutions.
We propose an efficient routing strategy for AllReduce transfers, which compromise of the dominant traffic in machine learning-centric datacenters, to achieve fast parameter synchronization in distributed machine learning, improving the average training time by 9%.