Analyze the Convergence Bounds of Federated Learning with Closer-to-Practice Constraints
Implementing Organization
Indian Institute Of Technology (Bhu)
Principal Investigator
Dr. Hari Prabhat Gupta
Indian Institute of Technology (IIT)
Project Overview
Federated Learning (FL) is a popular technique for training machine learning models on decentralized data. However, analyzing convergence bounds while considering practical constraints like heterogeneity of clients, non-i.i.d. data distribution, noisy datasets, and without-replacement sampling method is challenging. This work analyzes the convergence bounds of FL for $L$-smooth functions satisfying the $\mu$-Polyak-Łojasiewicz condition with practical constraints. High probability bounds are obtained, and incorporating practice constraints into FL increases communication rounds but leads to faster convergence when the number of epochs is sufficiently large. The upper bound obtained is more precise when subjected to constraints like smoothness, strong convexity, and fixed noise ratio. The study also proposes a modification to the analyzed convergence bounds by integrating the partially-with-replacement sampling method.