Privacy-Preserving Technology to Help Millions of People: Federated Prediction Model for Stroke Prevention

06/15/2020 ∙ by Ce Ju, et al. ∙ Lehigh University Tsinghua University Nanyang Technological University Tencent The Hong Kong University of Science and Technology 0

prevention of stroke with its associated risk factors has been one of the public health priorities worldwide. Emerging artificial intelligence technology is being increasingly adopted to predict stroke. Because of privacy concerns, patient data are stored in distributed electronic health record (EHR) databases, voluminous clinical datasets, which prevent patient data from being aggregated and restrains AI technology to boost the accuracy of stroke prediction with centralized training data. In this work, our scientists and engineers propose a privacy-preserving scheme to predict the risk of stroke and deploy our federated prediction model on cloud servers. Our system of federated prediction model asynchronously supports any number of client connections and arbitrary local gradient iterations in each communication round. It adopts federated averaging during the model training process, without patient data being taken out of the hospitals during the whole process of model training and forecasting. With the privacy-preserving mechanism, our federated prediction model trains over all the healthcare data from hospitals in a certain city without actual data sharing among them. Therefore, it is not only secure but also more accurate than any single prediction model that trains over the data only from one single hospital. Especially for small hospitals with few confirmed stroke cases, our federated model boosts model performance by 10 in several machine learning metrics. To help stroke experts comprehend the advantage of our prediction system more intuitively, we developed a mobile app that collects the key information of patients' statistics and demonstrates performance comparisons between the federated prediction model and the single prediction model during the federated training process.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Figure 1: Workflow of the Stroke Prediction Model

Recent data privacy concerns begin with an investigation of Facebook’s data privacy violations in collecting user information for uninformed use [20]. Hence, acts like the General Data Protection Regulation (GDPR) are proposed to prohibit organizations from exchanging data without explicit user approval [18]. Data, collected by health care providers, include a record of clinical metrics along with information pertinent to health and wellness. Although the use of big data in healthcare advances and gains popularity, it may lead to severe data privacy violations [7, 17]. Therefore, it is crucial to design and implement a privacy-preserving mechanism for AI technology in the field of healthcare. We adopt an emerging and powerful technique, federated learning framework [16, 11], to address the data privacy because it can jointly train a machine learning model using data from different clients without the need for actually sharing data between clients. The federated model performs not only more accurate but also more secure than the traditional statistical model in medical research [1].

In this work, Tencent and WeBank collaboratively develop a privacy-preserving technology for the prediction of stroke. Tencent’s tech-experts have successfully developed a prediction model for stroke with advanced AI technology supported by Tencent’s cloud server in the past year. To solve the scalability problem due to privacy concerns, We- Bank’s tech-experts help establish and implement a federated version for Tencent’s prediction model based on their self- developed privacy-preserving framework with FATE 222FedAI (https://www.fedai.org/) is a community that helps businesses and organizations build AI models effectively and collaboratively, by using data in accordance with user privacy protection, data security, data confidentiality, and government regulations.

, an industrial level open-source secure computing framework. Our privacy-preserving framework makes the impossible practical problem possible. Combing advanced AI technology and great sensitivity to privacy protection, our privacy-preserving framework works both accurately and securely. This stroke prediction application in healthcare is the second project hit- ting the ground for the new privacy-preserving technology yet proposed by WeBank from China, which may help millions of people to prevent stroke.

2 Related Work

Federated learning is a machine learning setting in which clients solve a machine learning problem through collaboration [9, 22]. During the model training, the data never leave the database of clients. Because health care data are strictly regulated by data compliance rules in China, clinical data in hospitals are very appropriate for this setting. Existing literature demonstrates the great potential of this approach in health care. Brisimi[2] developed an iterative cluster Primal-Dual Splitting (cPDS) algorithm for patients EHR in a decentralized fashion. Sheller [19] introduced the first use of federated learning in the medical image for multi-institutional collaboration. Li [13] investigated the feasibility of applying differential-privacy techniques to protect the patient data in a federated learning setup for brain tumor segmentation. Ju and Gao [6, 8] are first to investigate the feasibility of the federated learning framework to enable a distributed training of brain-computer interface models from multiple datasets with heterogeneous configurations.

3 Methodology

3.1 Workflow of the Stroke Prediction Model

Figure 1 illustrates workflow of the stroke prediction model.

3.1.1 Data Preprocessing

Our tech-experts clean and normalize raw data of patients collected from hospitals in a city of Central China with medical knowledge and AI techniques, such as natural language processing and relation networks, into structured clinical data. Key information of these structured clinical data includes out-patient/in-patient EHRs, out-patient/in-patient prescriptions, and inspections, which are distributed stored in each hospital’s private cloud. Each hospital’s private cloud is not permitted to communicate with each other. Furthermore, the structured clinical data is extracted to model features and grouped in a disease feature library in a private cloud.

3.1.2 Missing Data Imputation

An omission in clinical data often yields an inaccuracy in a prediction model. Data imputation is typically used to remedy the omission 

[5]. We fill-in missing entries using the following methods:

  • Column mean: replace the missing value with the mean of non-missing values.

  • Column median: replace the missing value with the me- dian of non-missing values.

We rounded the imputed value as the nearest discrete value after the above post-processing step.

3.1.3 Feature Selection

Because only a small number of attributes in clinical data is highly relevant to stroke, our medical experts manually select 119 model features based on risk factors analyzed by medical studies, including systolic blood pressure, use of antihypertensive therapy, prior coronary disease, etc. [3, 15, 21]

3.1.4 Prediction Classifier

Our prediction classifier for the risk of stroke, outputting the risk score, is established on the model features in the feature library. We pick the following approaches as the prediction classifier:

The computing platform for AI is supported by our Tencent Cloud server 333Tencent Cloud (https://intl.cloud.tencent.com/) is a secure, reliable and high-performance cloud compute service provided by Tencent.. The risk score can be found in a mini-program on the instant messaging WeChat 444WeChat (https://www.wechat.com/en/) is a Chinese multi-purpose messaging, social media and mobile payment app developed by Tencent.. The results of the medical test for patients will form a feedback loop in our workflow to upgrade our feature library and prediction model.

Figure 2: Illustration of Architecture of Federated Prediction Model
Experimental Setting AUC Mean AUC Std.
Hospital A Local Training 0.812 0.027
Hospital B Local Training 0.799 0.062
Hospital C Local Training 0.797 0.067
Hospital D Local Training 0.720 0.035
Hospital E Local Training 0.701 0.026
Federated Training 0.813 0.018
Centralized Training 0.814 0.014
Table 1: Cross-validation accuracy of stroke prediction model on different experimental settings.

3.2 Federated Prediction Model

Our federated prediction system adopts the baseline method, federated averaging [16], during model training. Federated averaging is conducted on each client’s (i.e. hospital) private cloud for feature mapping aggregation and classifier aggregation. Specifically, as illustrated in the blue part of Figure 2, the techniques of federated learning follow a server-client setting. The Tencent Cloud sever acts as a model aggregator. In each round , the server collects updated feature mapping models with model weights and classifiers from each client for model aggregation as follows,

After model aggregation, the Tencent Cloud server sends the updated global model to each client. When a client receives the model sent by the server, it updates the model with its lo- cal data in the private cloud. The training process continues until the model converges. Our system of federated prediction model is asynchronous supporting any number of client connections and any local gradient iteration in each communication round.

3.3 Monitoring and Visualization System

We develop a monitoring and visualization system as shown in the red right part of Figure 2 to better explain and control the federated training process by detecting the data flow through clients and the sever. The user interface for the clients in our system, a mini-program called FedAI Stroke Prediction, is illustrated in Figure 3. On the first page of the user interface, each user, doctors in hospitals, can view the basic statistics of patients, including numbers of male and female, numbers of positive and negative samples, communication rounds, feature dimensions in model and others. There are also histograms for several patient statistics (i.e. patient age). The performance, such as the area under the curve (AUC) score, between the federated prediction model and the single prediction model, is plotted on the second page of our user interface.

Figure 3: Illustration of a Mini-program FedAI Stroke Prediction.

4 Experimental Analysis

In this part, we evaluate our federated prediction model in the hospitals in a city of Central China. Hospitals A, B, and C are Top 3 3A hospitals with clinically confirmed stroke patients. The confirmed ratio in Hospital D or E is less than

. The stroke prediction model is a 3-layer neural network classifier. We conduct 5 experiments to calculate the standard error of the AUC score. The test set for each experiment is fixed and sampled from patients in the whole city. From Table

1

, we find that the model’s AUC mean under the federated training experiment is almost the same as the one under the centralized training experiment with a little bigger variance. Because the number of patients in Hospital A makes up around

of the total patient number, the model performance of Hospital A dominates in the joint training, close to the one of centralized training experiments. With the privacy-preserving mechanism, our federated prediction model trains over all the healthcare information from hospitals in a city and works better than any single prediction model that trains over the information only in one single hospital, especially the small hospital with few confirmed cases, increased by

in model evaluation metric AUC score.

5 Discussion

Overall, in this work, we design a privacy-preserving mechanism to boost the performance of the prediction model for stroke and establish the first hitting-ground privacy-preserving pipeline in healthcare worldwide, which has pro- found social value.

References

  • [1] R. Bender, T. Augustin, and M. Blettner (2005) Generating survival times to simulate cox proportional hazards models. Statistics in medicine 24 (11), pp. 1713–1723. Cited by: §1.
  • [2] T. S. Brisimi, R. Chen, T. Mela, A. Olshevsky, I. C. Paschalidis, and W. Shi (2018) Federated learning of predictive models from federated electronic health records. International journal of medical informatics 112, pp. 59–67. Cited by: §2.
  • [3] L. E. Chambless, G. Heiss, E. Shahar, M. J. Earp, and J. Toole (2004) Prediction of ischemic stroke risk in the atherosclerosis risk in communities study. American journal of epidemiology 160 (3), pp. 259–269. Cited by: §3.1.3.
  • [4] T. Chen and C. Guestrin (2016) Xgboost: a scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785–794. Cited by: 3rd item.
  • [5] B. Efron (1994) Missing data, imputation, and the bootstrap. Journal of the American Statistical Association 89 (426), pp. 463–475. Cited by: §3.1.2.
  • [6] D. Gao, C. Ju, X. Wei, Y. Liu, T. Chen, and Q. Yang (2019) HHHFL: hierarchical heterogeneous horizontal federated learning for electroencephalography. NeurIPS Workshop on Federated Learning for Data Privacy and Confidentiality. Cited by: §2.
  • [7] F. Jiang, Y. Jiang, H. Zhi, Y. Dong, H. Li, S. Ma, Y. Wang, Q. Dong, H. Shen, and Y. Wang (2017) Artificial intelligence in healthcare: past, present and future. Stroke and vascular neurology 2 (4), pp. 230–243. Cited by: §1.
  • [8] C. Ju, D. Gao, R. Mane, B. Tan, Y. Liu, and C. Guan (2020)

    Federated transfer learning for eeg signal classification

    .
    IEEE Engineering in Medicine and Biology Society.. Cited by: §2.
  • [9] P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings, et al. (2019) Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977. Cited by: §2.
  • [10] D. G. Kleinbaum, K. Dietz, M. Gail, M. Klein, and M. Klein (2002) Logistic regression. Springer. Cited by: 1st item.
  • [11] J. Konecný, H. B. McMahan, D. Ramage, and P. Richtárik (2016) Federated optimization: distributed machine learning for on-device intelligence. CoRR abs/1610.02527. External Links: Link, 1610.02527 Cited by: §1.
  • [12] Y. LeCun, Y. Bengio, and G. Hinton (2015) Deep learning. nature 521 (7553), pp. 436–444. Cited by: 4th item.
  • [13] W. Li, F. Milletarì, D. Xu, N. Rieke, J. Hancox, W. Zhu, M. Baust, Y. Cheng, S. Ourselin, M. J. Cardoso, et al. (2019) Privacy-preserving federated brain tumour segmentation. In International Workshop on Machine Learning in Medical Imaging, pp. 133–141. Cited by: §2.
  • [14] A. Liaw, M. Wiener, et al. (2002) Classification and regression by randomforest. R news 2 (3), pp. 18–22. Cited by: 2nd item.
  • [15] T. Lumley, R. A. Kronmal, M. Cushman, T. A. Manolio, and S. Goldstein (2002) A stroke prediction score in the elderly: validation and web-based application. Journal of clinical epidemiology 55 (2), pp. 129–136. Cited by: §3.1.3.
  • [16] H. B. McMahan, E. Moore, D. Ramage, and B. A. y Arcas (2016) Federated learning of deep networks using model averaging. CoRR abs/1602.05629. External Links: 1602.05629 Cited by: §1, §3.2.
  • [17] R. Miotto, F. Wang, S. Wang, X. Jiang, and J. T. Dudley (2018) Deep learning for healthcare: review, opportunities and challenges. Briefings in bioinformatics 19 (6), pp. 1236–1246. Cited by: §1.
  • [18] G. D. P. Regulation (2016) Regulation (eu) 2016/679 of the european parliament and of the council of 27 april 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46. Official Journal of the European Union (OJ) 59 (1-88), pp. 294. Cited by: §1.
  • [19] M. J. Sheller, G. A. Reina, B. Edwards, J. Martin, and S. Bakas (2018) Multi-institutional deep learning modeling without sharing patient data: a feasibility study on brain tumor segmentation. In International MICCAI Brainlesion Workshop, pp. 92–104. Cited by: §2.
  • [20] H. Tuttle (2018) Facebook scandal raises data privacy concerns. Risk Management 65 (5), pp. 6–9. Cited by: §1.
  • [21] Z. Vokó, M. Hollander, P. J. Koudstaal, A. Hofman, and M. M. Breteler (2004) How do american stroke risk functions perform in a western european population?. Neuroepidemiology 23 (5), pp. 247–253. Cited by: §3.1.3.
  • [22] Q. Yang, Y. Liu, Y. Cheng, Y. Kang, T. Chen, and H. Yu (2019) Federated learning. Synthesis Lectures on Artificial Intelligence and Machine Learning 13 (3), pp. 1–207. Cited by: §2.