Privacy Leakage of Real-World Vertical Federated Learning

by   Haiqin Weng, et al.

Federated learning enables mutually distrusting participants to collaboratively learn a distributed machine learning model without revealing anything but the model's output. Generic federated learning has been studied extensively, and several learning protocols, as well as open-source frameworks, have been developed. Yet, their over pursuit of computing efficiency and fast implementation might diminish the security and privacy guarantees of participant's training data, about which little is known thus far. In this paper, we consider an honest-but-curious adversary who participants in training a distributed ML model, does not deviate from the defined learning protocol, but attempts to infer private training data from the legitimately received information. In this setting, we design and implement two practical attacks, reverse sum attack and reverse multiplication attack, neither of which will affect the accuracy of the learned model. By empirically studying the privacy leakage of two learning protocols, we show that our attacks are (1) effective - the adversary successfully steal the private training data, even when the intermediate outputs are encrypted to protect data privacy; (2) evasive - the adversary's malicious behavior does not deviate from the protocol specification and deteriorate any accuracy of the target model; and (3) easy - the adversary needs little prior knowledge about the data distribution of the target participant. We also experimentally show that the leaked information is as effective as the raw training data through training an alternative classifier on the leaked information. We further discuss potential countermeasures and their challenges, which we hope may lead to several promising research directions.


page 1

page 8


Quantification of the Leakage in Federated Learning

With the growing emphasis on users' privacy, federated learning has beco...

BEAS: Blockchain Enabled Asynchronous Secure Federated Machine Learning

Federated Learning (FL) enables multiple parties to distributively train...

Enhanced Security with Encrypted Vision Transformer in Federated Learning

Federated learning is a learning method for training models over multipl...

BadVFL: Backdoor Attacks in Vertical Federated Learning

Federated learning (FL) enables multiple parties to collaboratively trai...

All You Need Is Hashing: Defending Against Data Reconstruction Attack in Vertical Federated Learning

Vertical federated learning is a trending solution for multi-party colla...

What's a little leakage between friends?

This paper introduces a new attack on recent messaging systems that prot...

Model-Reuse Attacks on Deep Learning Systems

Many of today's machine learning (ML) systems are built by reusing an ar...

Please sign up or login with your details

Forgot password? Click here to reset