Privacy-Preserving Training of Tree Ensembles over Continuous Data

06/05/2021
by   Samuel Adams, et al.
0

Most existing Secure Multi-Party Computation (MPC) protocols for privacy-preserving training of decision trees over distributed data assume that the features are categorical. In real-life applications, features are often numerical. The standard “in the clear” algorithm to grow decision trees on data with continuous values requires sorting of training examples for each feature in the quest for an optimal cut-point in the range of feature values in each node. Sorting is an expensive operation in MPC, hence finding secure protocols that avoid such an expensive step is a relevant problem in privacy-preserving machine learning. In this paper we propose three more efficient alternatives for secure training of decision tree based models on data with continuous features, namely: (1) secure discretization of the data, followed by secure training of a decision tree over the discretized data; (2) secure discretization of the data, followed by secure training of a random forest over the discretized data; and (3) secure training of extremely randomized trees (“extra-trees”) on the original data. Approaches (2) and (3) both involve randomizing feature choices. In addition, in approach (3) cut-points are chosen randomly as well, thereby alleviating the need to sort or to discretize the data up front. We implemented all proposed solutions in the semi-honest setting with additive secret sharing based MPC. In addition to mathematically proving that all proposed approaches are correct and secure, we experimentally evaluated and compared them in terms of classification accuracy and runtime. We privately train tree ensembles over data sets with 1000s of instances or features in a few minutes, with accuracies that are at par with those obtained in the clear. This makes our solution orders of magnitude more efficient than the existing approaches, which are based on oblivious sorting.

READ FULL TEXT

page 1

page 15

page 16

research
02/06/2021

Privacy-Preserving Feature Selection with Secure Multiparty Computation

Existing work on privacy-preserving machine learning with Secure Multipa...
research
10/06/2022

Federated Boosted Decision Trees with Differential Privacy

There is great demand for scalable, secure, and efficient privacy-preser...
research
05/01/2023

GTree: GPU-Friendly Privacy-preserving Decision Tree Training and Inference

Decision tree (DT) is a widely used machine learning model due to its ve...
research
07/24/2020

MPC-enabled Privacy-Preserving Neural Network Training against Malicious Attack

In the past decades, the application of secure multiparty computation (M...
research
12/24/2021

Efficient decision tree training with new data structure for secure multi-party computation

We propose a secure multi-party computation (MPC) protocol that construc...
research
03/05/2021

Efficient Encrypted Inference on Ensembles of Decision Trees

Data privacy concerns often prevent the use of cloud-based machine learn...
research
05/03/2022

Scalable Private Decision Tree Evaluation with Sublinear Communication

Private decision tree evaluation (PDTE) allows a decision tree holder to...

Please sign up or login with your details

Forgot password? Click here to reset