Dynamic Reconfiguration is the process of adaptively adjusting communication structures in distributed machine learning during runtime. Instead of relying on a fixed AllReduce topology, the system uses optical circuit switching to reconfigure network connections as training progresses. Guided by simulated annealing optimization, these changes respond to workload and traffic patterns to minimize communication bottlenecks. This flexible approach enables more efficient synchronization across nodes, leading to faster overall training performance.
We propose to apply optical circuit switching to enable dynamic AllReduce reconfiguration for accelerating distributed machine learning. With simulated annealing-based optimization, theproposed AllReduce reconfiguration approach achieves 31% less average training time than existing solutions.
Princeton Office
4 Independence Way, Suite 200
Princeton, NJ 08540
+1 609-520-1555
San Jose Office
2033 Gateway Place, Suite 200
San Jose, CA 95110
+1 408-863-6007
NEC Laboratories America, Inc. (NEC Labs) is the US-based center for NEC Corporation’s global network of corporate research laboratories. Our diverse research groups collaborate with industry, academia and governments to provide disruptive solutions to complex problems. A leader in the integration of IT and network technologies with more than 100 years of expertise, NEC provides a combination of products and solutions that cross-utilize the company’s experience and global resources to meet the complex and ever-changing needs of its customers.