The dramatic demographic change in most western countries will increase the need for development of new Ambient Intelligence (AmI) technologies making use of Artificial Intelligence (AI) and machine learning (ML). The new EU Data Protection regulations applying from 2018 onwards  will make privacy aware machine learning necessary . Consequently, issues of privacy, security, safety and data protection move more and more into the focus of AI and ML, thereby fostering an integrated ML approach 
, which emphasizes the importance of the human-in-the-loop. Currently, major threats to privacy come from personal data aggregation and the increasing power of data mining and pattern recognition techniques, as well as from healthcare data sharing and analysis. As the number of information sources increases the potential to combine these sources, profile the users and learn sensitive information about them also increases, which makes it a great threat to individual privacy. This an important issue especially in the field of AAL where the users are not always aware of the privacy risks they might face.
To address the threats mentioned previously we propose a model for the encoding and sharing of combined healthcare and AAL data. The model aims to achieve the privacy of input data before they are distributed to various stakeholders. To protect the privacy a Long Short-Term Memory (LSTM) encoder-decoder system is designed that allows the creation of different data views to correspond to the access level of the receiver.
The remainder of the paper is organized as follows. Section 2 presents an overview of existing privacy techniques and related work for deep learning in privacy. Section 3 introduces the proposed privacy model. Section 4 describes the case study and explains the performed experiments while Section 5 presents the results and the performance of the algorithm. Lastly, Section 6 includes the conclusion and suggestions for future work.
Ii Related Work
This section is divided into three parts. The first part gives an overview of privacy definitions and identifiers. The second describes previous work done in regards to privacy preserving techniques while the third part gives an introduction to deep learning and overview of existing work in privacy protection with the use of deep learning techniques.
Ii-a Private Data
The new EU General Data Protection Regulation (GDPR)  defines personal data as “any information relating to an identified or identifiable natural person” and specifically acknowledges that this includes both ‘direct’ and ‘indirect’ identification. The identification can be by means of “an identification number or to one or more factors specific to his physical, physiological, mental, economic, cultural or social identity”. While so far there is not one privacy definition yet that is able to encompass all the different aspects of privacy, there are guidelines that list the possible identifiers that could be used to identify a person from a group  :
Names, Geographical subdivisions smaller than a state, Dates (other than year)
Phone & Fax Numbers
Electronic mail addresses
Social Security, Medical Record & Health plan beneficiary numbers
Account & Certificate/license numbers
Vehicle identifiers and serial numbers (including license plate numbers)
Device identifiers and serial numbers
Web Uniform Resource Locators (URLs) & Internet Protocol (IP) address numbers
Biometric identifiers, including finger, retinal and voice prints
Full face photographic images and any comparable images
Any other unique identifying number, characteristic or code
However, the above list of identifiers is not exhaustive, as technology advances more potential identifiers could emerge.
Ii-B Privacy Preservation with Anonymization Methods
Privacy enhancing technologies protect the users’ privacy based on technology, and can offer additional levels of protection than just relying on laws and policies. In order to address the privacy concerns of the users, several approaches have been proposed by the research community. These approaches include information manipulation, privacy and context awareness, access control and data anonymization. In the sections below the anonymization are further analyzed since they are the methods most commonly used for privacy preservation.
A very well-known method to anonymize data before releasing them is -anonymity . In a k-anonymized dataset, each record is indistinguishable from at least other records in regards to specific identifying attributes . -anonymity is achieved by suppressing (deleting an attribute value from the data and replacing it with a random value that matches any possible attribute value) or generalizing the attributes in the data, which means that an attribute is replaced with a less specific but semantically consistent value . The utility and privacy of the data are connected. There is no way so far that can increase the data privacy without also decreasing the data utility . The objective in these problems is to maximize utility by minimizing the amount of generalization and suppression. Achieving -anonymity by generalization with this objective as a constraint is a Non-deterministic Polynomial-time hard (NP hard) problem which cannot be solved fully automatically  . In most cases -anonymity is able to prevent identity disclosure so that a record in a k-anonymized data set cannot be connected again to the corresponding record in the original data set. But in some cases, it may fail to protect against attribute disclosure.
This method was developed to address the weaknesses of -anonymity, which as shown, does not guarantee privacy against adversaries that use background knowledge or in cases where data are lacking diversity. For -diversity, the anonymization conditions are satisfied if, for each group of records sharing a combination of key attributes, there are at least -“well-represented” values for each confidential attribute . The disadvantage of this method is that it depends on the range of the sensitive attributes. If
-diversity is to be applied to a sensitive attribute that does not have many different values, artificial data will have to be inserted. The use of artificial data will improve the privacy but may result in problems with the analysis thus ruining the utility of the data. Also, this method is vulnerable to skewness and similarity attack so it cannot always prevent attribute disclosure.
As shown -diversity might not always be sufficient in preventing attribute disclosure. Since it does not account for the semantic closeness of the sensitive values. A new method named - closeness was proposed in  to address these problems. This method requires the distribution of the sensitive attributes in an equivalent class to be close to the distribution of the attribute in the overall table, which in turn means that the distance between the two distributions should be no more than a specified threshold . While the authors in  describe ways to check t-closeness (using several distances between distributions), no computational procedure to enforce this property is given . The authors proposing the -closing method  mention that -closeness limits the amount of useful information that is released. The only way to increase the utility of the data is to increase the threshold t, which in turn decreases the privacy protection.
As seen from the overview of the strengths and weaknesses of each technique -anonymity and the other anonymization methods are not always successful in guarantying that no information is leaked while ensuring usable data levels. While the methods of k-anonymity and -diversity do not always accomplish complete privacy, the method of -closeness provides it. But sometimes it is at the expense of the correlations between confidential attributes and key attributes. Also, the computational method for a specific dataset to be anonymized is an additional problem of this method. The papers defining -anonymity and -diversity propose approaches based on generalization and suppression which a lot of times can cause numerical attributes to become categorical. In the case of -closeness, a computational procedure to reach it is not described. Thus, some issues in this field are still open, both at a conceptual and computational level, which can be improved by defining better properties and by creating more effective methods.
Ii-C Privacy Preservation with Deep Learning
Deep learning is a promising area of machine learning research with significant success in recent years. So far the applications of deep learning are being used in various systems such as image and speech recognition, data analysis, social media, bioinformatics, medicine, and healthcare. Usually, deep learning architectures are constructed as multi-layer neural networks. There are several different neural network architectures, such as the Recurrent Neural Network (RNN)4]
and the Deep Belief Network (DBN). Deep learning has the ability to transform original data into a higher level with more abstract expressions. That means that high-dimensional original data can be converted to low-dimensional data by training multiple neural networks on how to reconstruct the high-dimensional input data.
However, the existing literature on privacy protection mostly focuses on traditional privacy preserving methods, as described in the previous section, and not on deep learning. Differential privacy proposed by Dwork  is one of the few approaches of privacy protection that makes use of machine learning methods. Applications of Differential Privacy include boosting 7]
, linear and logistic regression[5, 32]support vector machines , risk minimization [6, 30] and continuous data processing . However, the most relevant work to this paper is that of Dai et al.  in which they used an Encoder-Decoder system to protect private information in videos by extracting the privacy region and scrambling it while encoding. The system allows the users to fully restore the original video only if they have a legitimate key, otherwise, they can only see the non private regions in the video.
Iii LSTM Encoder-Decoder Model
Long Short Term Memory networks are a special kind of RNN, capable of learning long-term dependencies . A basic sequence-to-sequence model, as introduced in , consists of two recurrent neural networks (RNNs). The first is an encoder that processes the input and the second a decoder that generates the output. The recurrently connected blocks in LSTM layers are known as memory blocks. Each block contains one or more memory cells which are composed of three units: an input gate, a forget gate and an output gate. These gates modulate the interactions between the memory cell and the environment. Figure 1 shows the single cell of LSTM memory block. LSTM can assist in error minimization because the error can be back-propagated through time and layers. By maintaining a more constant error, the recurrent network can continue to learn over many time steps, and thus be able to link causes and effects.
The model that was developed in this work uses the multi-layered Long Short-Term Memory (LSTM) encoder to map the input sequence to a vector of a fixed dimensionality, and then another LSTM is used to decode the target sequence from the vector. The encoder network is the part of the network that takes the input sequence and maps it to an encoded representation of the sequence. The encoded representation is then used by the decoder network to generate an output sequence. This makes the framework have a lock and key analogy, where only someone with the correct key (decoder) will be able to access the resources behind the lock (encoder). The multiple hidden layers of neural networks have characteristics that enable this kind of learning , along with the mapping characteristic of the encoder-decoder models which are able to create corresponding pairs could make them appropriate for privacy preserving frameworks.
Iv Experiment Design
The experiments were conducted using as basis the LSTM Encoder-Decoder model introduced in the previous section. In the following sections we describe the evaluation use case scenario and details of the simulated dataset used for the study followed by the modeling and training methodology.
Iv-a Use case
John is 80 years old and lives in an ambient assisted living environment. He is widowed and was recently diagnosed with Alzheimer’s disease. Currently, he lives alone but he likes to stay in touch with his family and friends. The AAL environment he lives in gives him independence and allows him to control his home automation system, for example, he can remotely open and close windows/doors, control the lighting, heating, and the alarm system. Also, it allows monitoring of his vital signs and offers him reminders about medication and appointments. The sensors deployed in the home send the collected information to the cloud offering access to, family members, care givers, doctors, and researchers.
In this use case scenario, four different views of the data are being created (Figure 2) depending on the access level of the receiver and the preferences of the user (Table I). The user in this scenario has a very close relationship with his family and trusts ,  his caregiver so he has selected almost all his information to be accessible to them, especially because he feels safer knowing they will be able to help him in case of emergency. With regards to doctors, he has allowed only some basic personal and medical information to be visible to them so a different view is created for them. And finally for research purposes, the view that is created does not show any explicit personal data and most of the other sensitive attributes are generalized.
For the purposes of this work data related to AAL were simulated. The type of data that were selected are divided into three categories: personal, medical and smart home sensor attributes. Each entry of the simulated data includes these three kinds of attributes. Personal attributes are those that can explicitly identify a person such as name, address and phone number. The second includes attributes that could potentially identify a person such as gender and birth date. Lastly, the third category includes sensitive medical information and sensor data, like blood pressure, medical history and presence sensors (Table I).
|External Door Sensor||F||D||F||D|
Abbreviations F: fully disclosed, G: generalized, D: deleted
The simulation of the data was based on real world data collected from AAL environments and it included personal information, health care data as well as smart home sensor data. The data were simulated for 10000 users, which each user having 100 entries.
Iv-C Model Configuration
As described previously, the proposed model makes use of the LSTM neural network architecture that learns to encode a variable-length input sequence into a fixed-length vector representation and to decode a given fixed-length vector representation back into a variable-length sequence. On this neural network three types of operations are applied to the encoder input by the decoder. These three operations are: 1) Disclosure which means keeping the data as it is 2) Deletion by removing the data or 3) Generalization which means replacing the value with a less specific but semantically consistent value. So the data from each entry can be fully disclosed to the receiver, generalized or deleted. Each value for a given attribute has different range aligned with real life values. Four separate views are created for different receivers, family member, doctor, caregiver, and researcher. Each receiver has a different decoder output due to their privacy clearances on patient information (Table I).
In Figure 3 the overall functionality of the encoder and decoder LSTM layers of the model are depicted. While in Figure 4 we show in more detail how the privacy operations work when the model reads an input sentence such as “John has Alzheimer” which will produce “* has Dementia” as the output sentence. In this instance the operation of deletion is applied on the Name attribute so ‘John’ is transformed to ‘*’ and the operation of generalization is applied on the Disease attribute which changes ‘Alzheimer’ to ‘Dementia’. The model stops making predictions after outputting the end-of-string token ‘eos’.
Iv-D Model Training and Testing
For the experiments the models were trained with 800.000 data entries and tested with 200.000 unseen entries. Each user entry consisted of different attribute as shown in Table I and their corresponding values, each entry had 160 characters at most. Different attributes are separated with ‘
’ in order to easily distinguish between different attributes in an entry. A 40 character set was used as dictionary. Each sequence is maximum 160 characters long and ends with special token ‘eos’. In order to handle different sequence lengths, we zero padded each entry to the maximum number of characters which in our case is 160. The encoder and decoder comprise an LSTM network which has 256 hidden units and is trained with Adam with a learning rate of 0.0004.
We qualitatively analyzed the trained model’s results by comparing the decoded outputs with those of the expected model output after the privacy operations. The qualitative analysis shows that the LSTM Encoder – Decoder is very good at learning the privacy operations of disclosure, generalization and deletion as well as at capturing the specified preferences in the access to information table. An example of the results of the model can be seen in the following figures.
In Figure 5, it can be seen that with the use of the right encoder-decoder mechanism the user’s information can be transferred almost perfectly to the researcher with the appropriate privacy rules applied for the researcher’s access level. And since only the encoded vector is shared it is not possible to get user information without the right decoder.
Figure 6, showcases one of the most beneficial parts of our model. In this case, we explore the possibility that a Caregiver tries to access the data meant for the Researcher. Because the encoded vector of user information was shared, it would not be possible to decode it unless the right decoder was used. If the encoder and decoder do not match it is not possible to decode and access the user’s private information. If the end receiver does not have the correct decoder it is almost impossible to find the right decoder weights since it is very high dimensional floating point vector.
Vi Conclusion & Future Work
Our LSTM Encoder-Decoder model is able to learn privacy operations such as disclosure, generalization, and deletion of data, therefore it can generate different data views for data receivers in an encrypted way. It allows users to train independently on their own datasets and selectively share small subsets of their key attributes to specific receivers by creating different views for each one. This offers an attractive point in regards to the utility and privacy trade-off. Two goals are achieved in an end-to-end manner by using the LSTM based encoder and decoder. One is to get an encoded version of user information while the second one is to decode this encoded information according to privacy rules defined by the user. The encoded private information of the user can not be decoded unless the right decoder is used. If an adversary tried using a different decoder on encoded information the system would not disclose any information. Without the right decoder, it would not be possible to train a decoder on encoded information due to the very high dimensionality of the LSTM hidden state vector and possible values of each LSTM network parameter. This way the users preserve the privacy of their data while receiving the benefits of the AAL environment. Moreover, the model can handle raw data even in text format, which is very beneficial in the case of medical records which usually include a lot of doctors’ and nurses’ notes. This work is a preliminary experimental study in preserving privacy with the use of LSTM Encoders-Decoders. One of the model’s limitations, that will be addressed in future work, is the tokenization of the attributes to improve the performance. Another limitation is the use of simulated data which means they do not contain missing or abnormal values to evaluate the robustness of the method. Future work will also include the expansion of the model to real world data, as well as more complex data formats such as meta-data and multimedia formats.
This work has been funded by the European Union Horizon 2020 MSCA ITN ACROSSING project (GA no. 616757). The authors would like to thank the members of the project’s consortium for their valuable inputs.
-  Council of Europe, Committee of Ministers, Recommendation No. R (97) 5 on the Protection of Medical Data. http://hrlibrary.umn.edu/instree/coerecr97-5.html.
-  EU General Data Protection Regulation. http://www.eugdpr.org.
-  Health Insurance Portability and Accountability Act Of 1996. https://www.gpo.gov/fdsys/pkg/PLAW-104publ191/html/PLAW-104publ191.htm.
-  George Bebis and Michael Georgiopoulos. Feed-forward neural networks. IEEE Potentials, 13(4):27–31, 1994.
-  Kamalika Chaudhuri and Claire Monteleoni. Privacy-preserving logistic regression. In Advances in Neural Information Processing Systems, pages 289–296, 2009.
-  Kamalika Chaudhuri, Claire Monteleoni, and Anand D Sarwate. Differentially private empirical risk minimization. Journal of Machine Learning Research, 12(Mar):1069–1109, 2011.
-  Kamalika Chaudhuri, Anand D Sarwate, and Kaushik Sinha. A near-optimal algorithm for differentially-private principal components. Journal of Machine Learning Research, 14(1):2905–2943, 2013.
-  Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014.
-  Feng Dai, Dongming Zhang, and Jintao Li. Encoder/decoder for privacy protection video with privacy region detection and scrambling. In MMM (2), pages 525–527, 2013.
-  Josep Domingo-Ferrer and Vicenç Torra. A critique of k-anonymity and some of its enhancements. In ARES 2008, pages 990–993. IEEE, 2008.
-  Cynthia Dwork. Differential privacy. In Encyclopedia of Cryptography and Security, pages 338–340. Springer, 2011.
-  Cynthia Dwork, Guy N Rothblum, and Salil Vadhan. Boosting and differential privacy. In Foundations of Computer Science (FOCS), 2010 51st Annual IEEE Symposium on, pages 51–60. IEEE, 2010.
-  Christoph Goller and Andreas Kuchler. In Neural Networks, 1996., IEEE International Conference on, volume 1, pages 347–352. IEEE, 1996.
-  Geoffrey E Hinton. Deep belief networks. Scholarpedia, 4(5):5947, 2009.
-  Geoffrey E Hinton and Ruslan R Salakhutdinov. Reducing the dimensionality of data with neural networks. science, 313(5786):504–507, 2006.
-  Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
-  Andreas Holzinger, Randy Goebel, Vasile Palade, and Massimo Ferri. Towards integrative machine learning and knowledge extraction. In Lecture Notes in Artificial Intelligence LNAI 10344, pages 1–12. Springer, Cham, 2017.
-  Andreas Holzinger, Markus Plass, Katharina Holzinger, Gloria Cerasela Crisan, Camelia-M. Pintea, and Vasile Palade. A glass-box interactive machine learning approach for solving np-hard problems with the human-in-the-loop. arXiv:1708.01104, 2017.
-  Andreas Holzinger, Klaus Schaupp, and Walter Eder-Halbedl. An investigation on acceptance of ubiquitous devices for the elderly in an geriatric hospital environment: using the example of person tracking. In Lecture Notes in Computer Science, LNCS 5105, pages 22–29. Springer, Heidelberg, 2008.
-  D. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
-  Ninghui Li, Tiancheng Li, and Suresh Venkatasubramanian. t-closeness: Privacy beyond k-anonymity and l-diversity. In Data Engineering, 2007. ICDE 2007. IEEE 23rd International Conference on, pages 106–115. IEEE, 2007.
-  Ashwin Machanavajjhala, Daniel Kifer, Johannes Gehrke, and Muthuramakrishnan Venkitasubramaniam. l-diversity: Privacy beyond k-anonymity. ACM Transactions on Knowledge Discovery from Data (TKDD), 1(1):3, 2007.
-  Bernd Malle, Peter Kieseberg, Sebastian Schrittwieser, and Andreas Holzinger. Privacy aware machine learning and the “right to be forgotten”. ERCIM News (special theme: machine learning), 107(3):22–23, 2016.
-  Paul Ohm. Broken promises of privacy: Responding to the surprising failure of anonymization. 2009.
-  Hyoungmin Park and Kyuseok Shim. Approximate algorithms for k-anonymity. In Proceedings of the 2007 ACM SIGMOD international conference on Management of data, pages 67–78. ACM, 2007.
-  Benjamin IP Rubinstein, Peter L Bartlett, Ling Huang, and Nina Taft. Learning in a large function space: Privacy-preserving mechanisms for svm learning. arXiv preprint arXiv:0911.5708, 2009.
-  Anand D Sarwate and Kamalika Chaudhuri. Signal processing and machine learning with differential privacy: Algorithms and challenges for continuous data. IEEE signal processing magazine, 30(5):86–94, 2013.
-  Deepika Singh, Johannes Kropf, Sten Hanke, and Andreas Holzinger. Ambient assisted living technologies from the perspectives of older people and professionals. In Lecture Notes in Computer Science LNCS 10410, pages 255–266. Springer, Cham, 2017.
-  Latanya Sweeney. k-anonymity: A model for protecting privacy. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 10(05):557–570, 2002.
-  Martin J Wainwright, Michael I Jordan, and John C Duchi. Privacy aware learning. In Advances in Neural Information Processing Systems, pages 1430–1438, 2012.
-  Xiaokui Xiao and Yufei Tao. Personalized privacy preservation. In Proceedings of the 2006 ACM SIGMOD international conference on Management of data, pages 229–240. ACM, 2006.
Jun Zhang, Zhenjie Zhang, Xiaokui Xiao, Yin Yang, and Marianne Winslett.
Functional mechanism: regression analysis under differential privacy.Proceedings of the VLDB Endowment, 5(11):1364–1375, 2012.
-  Shu Zhang, Dequan Zheng, Xinchen Hu, and Ming Yang. Bidirectional long short-term memory networks for relation classification. In PACLIC, 2015.
-  Martina Ziefle, Carsten Röcker, and Andreas Holzinger. Medical technology in smart homes: Exploring the user’s perspective on privacy, intimacy and trust. In 35th COMPSAC, pages 410–415. IEEE, Munich, 2011.