Personalized Federated Learning via Gradient Modulation for Heterogeneous Text Summarization

04/23/2023
by   Rongfeng Pan, et al.
0

Text summarization is essential for information aggregation and demands large amounts of training data. However, concerns about data privacy and security limit data collection and model training. To eliminate this concern, we propose a federated learning text summarization scheme, which allows users to share the global model in a cooperative learning manner without sharing raw data. Personalized federated learning (PFL) balances personalization and generalization in the process of optimizing the global model, to guide the training of local models. However, multiple local data have different distributions of semantics and context, which may cause the local model to learn deviated semantic and context information. In this paper, we propose FedSUMM, a dynamic gradient adapter to provide more appropriate local parameters for local model. Simultaneously, FedSUMM uses differential privacy to prevent parameter leakage during distributed training. Experimental evidence verifies FedSUMM can achieve faster model convergence on PFL algorithm for task-specific text summarization, and the method achieves superior performance for different optimization metrics for text summarization.

READ FULL TEXT
research
08/21/2023

Adaptive Local Steps Federated Learning with Differential Privacy Driven by Convergence Analysis

Federated Learning (FL) is a distributed machine learning technique that...
research
06/08/2022

Hypernetwork-based Personalized Federated Learning for Multi-Institutional CT Imaging

Computed tomography (CT) is of great importance in clinical practice due...
research
11/19/2022

Personalized Federated Learning with Hidden Information on Personalized Prior

Federated learning (FL for simplification) is a distributed machine lear...
research
06/08/2020

Responsive Web User Interface to Recover Training Data from User Gradients in Federated Learning

Local differential privacy (LDP) is an emerging privacy standard to prot...
research
06/17/2022

Avoid Overfitting User Specific Information in Federated Keyword Spotting

Keyword spotting (KWS) aims to discriminate a specific wake-up word from...
research
06/22/2023

Reinforcement Federated Learning Method Based on Adaptive OPTICS Clustering

Federated learning is a distributed machine learning technology, which r...
research
05/09/2023

Privacy-Preserving Collaborative Chinese Text Recognition with Federated Learning

In Chinese text recognition, to compensate for the insufficient local da...

Please sign up or login with your details

Forgot password? Click here to reset