Aggregation Design for Personalized Federated Multi-Modal Learning over Wireless Networks
Abstract
Federated Multi-Modal Learning (FMML) is an emerging field that integrates information from different modalities in federated learning to improve the learning performance. In this letter, we develop a parameter scheduling scheme to improve personalized performance and communication efficiency in personalized FMML, considering the non-independent and nonidentically distributed (non-IID) data along with the modality heterogeneity. Specifically, a learning-based approach is utilized to obtain the aggregation coefficients for parameters of different modalities on distinct devices. Based on the aggregation coefficients and channel state, a subset of parameters is scheduled to be uploaded to a server for each modality. Experimental results show that the proposed algorithm can effectively improve the personalized performance of FMML.
- Publication:
-
arXiv e-prints
- Pub Date:
- June 2024
- DOI:
- 10.48550/arXiv.2406.07915
- arXiv:
- arXiv:2406.07915
- Bibcode:
- 2024arXiv240607915Y
- Keywords:
-
- Computer Science - Information Theory;
- Electrical Engineering and Systems Science - Signal Processing
- E-Print:
- accepted by IEEE Communications Letters