Reducing Communication Costs of Federated Contrastive Learning by Particle Swarm Optimization

Federated learning and contrastive learning are two important machine learning paradigms, and federated contrastive learning is their combination, allowing pretraining models using unlabeled data in the context of data silos while protecting the data privacy and security. However, existing federated contrastive learning algorithms require transmission of model weights and data representations among the sever and clients, which is costly and greatly increases data security risks. Therefore, how to reduce the amount of data communication, especially the representations, while maintaining the performance of the model is an important problem that has not yet received any attention. In this study, we propose Federated Contrastive Learning Algorithm using Particle Swarm Optimization to reduce data communication by transmitting the client model scores instead. In the experiments, our proposed method can reduce 40% weight uploads and 78.97% representation uploads compared with the baseline algorithm, while with only 1.93% decrease in accuracy.