Privacy-Protecting Techniques for Behavioral Data: A Survey

09/09/2021
by   Simon Hanisch, et al.
KIT
TU Dresden
0

Our behavior (the way we talk, walk, or think) is unique and can be used as a biometric trait. It also correlates with sensitive attributes like emotions. Hence, techniques to protect individuals privacy against unwanted inferences are required. To consolidate knowledge in this area, we systematically reviewed applicable anonymization techniques. We taxonomize and compare existing solutions regarding privacy goals, conceptual operation, advantages, and limitations. Our analysis shows that some behavioral traits (e.g., voice) have received much attention, while others (e.g., eye-gaze, brainwaves) are mostly neglected. We also find that the evaluation methodology of behavioral anonymization techniques can be further improved.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

01/28/2018

A Survey on Behavioral Biometric Authentication on Smartphones

Recent research has shown the possibility of using smartphones' sensors ...
01/04/2021

Covert Embodied Choice: Decision-Making and the Limits of Privacy Under Biometric Surveillance

Algorithms engineered to leverage rich behavioral and biometric data to ...
12/19/2018

Privacy-Aware Eye Tracking Using Differential Privacy

With eye tracking being increasingly integrated into virtual and augment...
10/29/2020

Acoustic Correlates of the Voice Qualifiers: A Survey

Our voices are as distinctive as our faces and fingerprints. There is a ...
03/28/2021

An Affective Approach for Behavioral Performance Estimation and Induction

Emotions have a major interactive role in defining how humans interact w...
06/18/2020

"And then they died": Using Action Sequences for Data Driven,Context Aware Gameplay Analysis

Many successful games rely heavily on data analytics to understand playe...
12/10/2019

edBB: Biometrics and Behavior for Assessing Remote Education

We present a platform for student monitoring in remote education consist...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

The ongoing digital transformation is leading to an increasingly comprehensive data collection on citizens. Ever improving peripherals, like augmented reality (AR)/ virtual reality (VR) goggles, motion capturing suits and gloves, force-feedback input devices, sensor-rich cell phones, smart watches, and other wearables drastically increase the coverage and resolution at which biometrics and behavioral data of individuals become available for processing at the same time.

A large amount of such data is shared knowingly, when users post their latest achievements, photos, or opinions on products and current affairs. A much larger amount is collected unnoticed, when individuals browse Web pages, use location services and similar apps, or simply enter smart spaces that are enriched with anything from voice assistants to CCTV cameras.

The corresponding behavioral data is highly descriptive of the captured individual and it reveals a multitude of attributes. They contain strong indicators for routines, habits, and also medical conditions and ‘ticks’. Known correlations between physiological features and medical conditions include the detection of depression or consumption of anti-depressants in facial pictures, detection of organ insufficiencies due to the coloration of eyes (hepatitis), or skin (alcohol abuse (de Carvalho Bruno et al., 2013), general fitness (Perrett et al., 2020), and others). A large number of studies have also reported correlations between behavioral data and psychological traits as well as characteristics. Behavioral data can also be used to uniquely identify individuals. Prominent examples across the spectrum include identifying personal traits and characteristics from social media feeds (Kosinski et al., 2013), identifying users by their mobility patterns (de Montjoye et al., 2013) and web-browsing behavior (Deuser et al., 2020). Gait very prominently has been used to identify individuals (Yovel and O’Toole, 2016; Wan et al., 2019), and it obviously reveals individual attributes like age, gender, and physiological conditions (Pollick et al., 2005; Troje, 2002). This also increases capabilities for credit and social scoring, based on aggregated digital dossiers.

Preserving the privacy, and terminally the dignity of individuals who come in the range of sensors and are captured in their behavior requires more sophisticated approaches than removing direct identifiers (IP address, social security number (SSN), blurring a face) or intuitive quasi identifiers (gender, age, ethnicity) in databases. Note, that the behavioral data captured from humans has both temporal dependencies, as it is captured as a time-series, and physiological dependencies, as human bodies must adhere to both their physiological and general physical limitations. Due to the strong dependency between observations and to the underlying models, the efficacy of randomized, perturbative anonymization also must be critically reviewed. Context information and habits being represented as strong signals in the data further complicate effective anonymization.

A growing corpus of studies is addressing this challenge of anonymizing behavioral data. They focus on a variety of different human traits, ranging from the voice, over gait, to less prominent examples like gestures, heartbeat, and others. A systematic review of all these approaches, which bridges the attempts in extracting the shared conceptual and methodological similarities and highlights both differences as well as roads less traveled is missing, to the best of our knowledge.

For this paper, we hence set out to systematize the corresponding literature. We are more interested in privacy than confidentiality: we do not consider approaches in which an entity encrypts its own data to hide it from access by unintended audiences. We are rather interested in approaches that protect from unintended revelation of information contained in data that is collected and shared for a different, explicit purpose (Cristofaro, 2021). In other words, we are interested in privacy-enhancing technologies (PETs) for scenarios in which behavioral data are collected by or shared with third parties to perform a specific operation. We deem ‘confidential computing’, processing based on homomorphic cryptography, or similar approaches in which the data owner is the only entity that learns anything from the data, out of scope of our analysis.

For our study, we followed Kitchenham’s guidelines (Kitchenham, 2004) to discover and survey the current state of the art, comprising of 78 distinct studies, extracted from a corpus of 237 initially discovered publications. We identify common applications of behavioral data, to extract sensible measures of utility, as well as common privacy threats with corresponding adversary models. We define a taxonomy of anonymization approaches, informed by the related work from the fields of database publication and anonymous communications. We then provide a detailed overview of the different anonymization approaches, sorted by the trait they aim to protect. We provide insight into the corresponding applications that define utility metrics, and into the privacy threats, privacy goals, applied anonymization concepts, and the evaluation the corresponding scientists performed, together with the data they chose for their studies. Our main findings are that some traits (e.g., voice) received a lot of research interest while others are mostly neglected (e.g., brain activity, eye-gaze). One reason we found for this is the lack of available datasets for these neglect traits. Further, we find that the general evaluation methodology for behavioral biometric anonymization can be improved by taking stronger adversaries into account.

The rest of the article is organized as follows: section 2 describes the background on privacy terminology, as well as the related work and our survey approach. section 3 introduces behavioral data, applications, and related privacy concerns. We define our taxonomy of concepts in section 4, and survey the field, sorting anonymization techniques by the trait the authors addressed and the conceptual approach taken, in section 5. We discuss our insights and general lessons learned in section 6 and conclude the article with a summary in section 7.

2. Background

In this section we first review the relevant terminology utilized throughout this work and the existing surveys on anonymization techniques. We then present the methodology we used to perform the systematic literature review.

2.1. Terminology

Our use of the term privacy enhancement or protection shall refer to the obfuscation of information from internal and external observers, including the information or service provider, regardless of whether this obfuscation consists in data access control, encryption, minimization of the data revealed, or data modification, perturbation, partial or full, in any manner. In the most abstract sense, the behavioral information to be protected may be composed of various elements, including links or relationships among several pieces of information.

Another important type of information to be obfuscated is directly a user’s identity, by itself or accompanied with behavioral or profile information. The close relation between personal devices (such as smartphones or wearables) and their users makes distinctive features in said devices potentially unique identifiers. In this respect, we adhere to the terminological convention of regarding anonymity as a particular case of privacy, when the data to be protected, without being direct identifiers111Direct identifiers allow to unequivocally identify individuals. For example, it would be the case of SSNs or full names. In an data-anonymization process, direct identifiers are always removed in the very first phase., may be linked with external information to reidentify the individual to whom the data refer.

In the field of statistical disclosure control (SDC) (Willenborg and de Waal, 2001), the aim is to protect a microdata set, that is, a database whose records contain information at the level of individual respondents, while ensuring that those data are still useful for researchers. In this field, the concepts of identity and attribute disclosure refer to the goal of an attacker to ascertain either the identity of an individual in the microdata set or the confidential attribute/s thereof. In this work, our interpretation of the terms anonymity and privacy will be in the sense meant by the aforementioned concepts of identity and attribute disclosure, respectively.

We shall employ the term utility to refer to a quantification of the degree of functionality maintained with respect to that intended by a personalized or information service, despite the implementation of privacy mechanisms that may hide or perturb part of the data, along with the degree of quality of service maintained, despite processing, storage, communication and scalability overheads incurred by such mechanisms. We stress that utility in this context does not refer to user-interface design.

As pointed out above in the introduction, any PETs poses a trade-off between privacy and functionality. The optimization of the privacy-functionality (or privacy-utility) trade-off will refer to the design and tuning of PETs in order to maximize privacy for a desired functionality, or vice versa.

2.2. Related Surveys

Most of the surveys on behavioral data focus on analyzing the uniqueness and suitability of behavioral traits to identify people, comparing the accuracy of different approaches and their applicability. In this line of research, we find surveys covering a range of existing behavioral biometrics for user authentication (Alzubaidi and Kalita, 2016; Meng et al., 2015; Mahfouz et al., 2017; Liang et al., 2020), and others focusing on the review of specific traits, such as gait recognition (Wan et al., 2019), keystrokes (Banerjee and Woodard, 2012; Teh et al., 2013), eye gaze (Katsini et al., 2020), or brainwave biometrics (Gui et al., 2019). However, the treatment of privacy issues is limited to mentioning that there is potential for sensitive inferences or identity leaks but there is no in-depth discussion about privacy countermeasures.

There is an important stream of research on potential privacy attacks to behavioral data focusing on attribute inferences (Bales et al., 2016; Inoue et al., 2020; Frank et al., 2017; Buriro et al., 2016), or dealing with user de-identification (i.e., trying to identify a person by their behavioral data) (Ye et al., 2021; Emam et al., 2011; Dwork et al., 2017; Henriksen-Bulmer and Jeary, 2016). Dantcheva et al. (Dantcheva et al., 2016) provide an extensive overview of which sensitive attributes, so called soft biometrics (gender, age, ethnicity, weight, etc.), can be inferred from primary biometrics extracted from image and video data. This survey highlights that protecting privacy of inferred attributes is an open research challenge.

While the current literature on behavioral data underscores the need for privacy defences, work on this area is still emerging and scattered but no comprehensive view of the problem, existing solutions, and challenges has been carried out yet. Ribaric et al. (Ribaric et al., 2016) review techniques to protect user’s visual and multimedia data from attribute inferences and re-identification (Ribaric et al., 2016). Though they include a section on behavioral data protection, it only covers a limited number of traits (voice, gait, and gesture) and anonymization techniques that apply when these data has been captured as video, audio, or images, but no other sensors are considered. Also closely related, Nhat Tran et al. (Tran et al., 2021) survey biometric template protection techniques, but they do it generally without entering in details of the anonymization needs of behavioral biometrics. In our article, we go beyond the state of the art by systematically reviewing research works on behavioral data anonymization techniques, examining a comprehensive set of traditional and modern types of behavioral traits for which solutions have been proposed, and considering different types of recording sensors and use-cases. We categorize and compare existing techniques, analyze their associated evaluation approaches and results, and present a summary of challenges pointing at research directions that need attention in future work.

2.3. Methodology

We performed a systematic literature review following Kitchenham’s guidelines (Kitchenham, 2004) to identify relevant studies on behavioral data privacy techniques, as it is depicted in Figure 1.

Figure 1. Summary of the procedure for identifying and selecting relevant studies on behavioral data privacy techniques. We first analyzed the literature on biometrics to determine behavioral traits for person identification. We then used these traits as key terms to search for privacy-related publications, following Kitchenham’s guidelines for systematic literature reviews (Kitchenham, 2004). The complete list of behavioral traits we searched includes: brain activity, eye gaze, facial expression, gait, gesture, handwriting, haptic, heartbeat, kesytrokes, lip, motion, mouse, thermal, touch, and voice.

Our guiding research question is “What techniques are applicable to protect behavioral data privacy?” From this starting point, the goal is to understand how these techniques work, what is the level of protection provided, and what are the limitations and existing open challenges. To answer these questions, we first explored the literature on biometrics (Alzubaidi and Kalita, 2016; Dantcheva et al., 2016; Meng et al., 2015; Mahfouz et al., 2017; Yampolskiy and Govindaraju, 2010; Pfeuffer et al., 2019; Ackad et al., 2012; Griffiths et al., 2018) to determine what kind of behavioral traits can be used to identify a person. Next, we used this list of traits combined with the keyword “privacy” and the semantically similar terms “anonymization” and “de-identification”, as search strings in the main academic databases for computer science. Based on these search terms, we compiled works with no constraints on publication date, obtaining a set of 237 papers spanning from 2007 to 2020, after filtering duplicates. During pre-screening, we built a taxonomy of privacy solutions and decided to narrow-down the scope of the survey to anonymization techniques focused on protecting the publication of behavioral data from identity and attribute disclosure attacks. We consider approaches that assume collection, sanitization, and subsequent publishing of data, which must be anonymized but also keep a level of utility to provide behavioral data driven services. Accordingly, the down-selection of primary studies to be analyzed in this survey considered the following criteria. Documents were excluded if:

  1. The publication format was other than peer-reviewed academic journal or conference paper.

  2. The paper could not be retrieved using IEEE Explore, ACM Digital Library, DBLP, or Google Scholar.

  3. The publication language was not English.

  4. Another paper by the same authors superseded the work, in which case the most complete work was considered.

  5. The privacy protection technique was other than identity or attribute anonymization with data utility.

  6. The anonymization approach was described at a high level and not enough details were provided to properly address the guiding research question.

The search and selection protocol yielded a final corpus of 78 peer-reviewed works on behavioral data anonymization, which we clustered according to the behavioral trait being protected: gait, brain activity, heartbeat, eye gaze, voice, and hand motions (handwriting, keystrokes, mouse movements, and hand gestures)222We found no papers on facial expression, lip, touch, and haptic traits that fulfil our criteria.. We first describe the different applications of these traits, motivating the need for privacy (Section 3

). Then, we define a taxonomy for classifying anonymization techniques (Section

4). We use this taxonomy to review the papers for each behavioral trait (Section 5), analyzing the proposed anonymization technique, its performance, as well as the main advantages and disadvantages. We then examine and discuss the literature in a consolidated way, identifying overall gaps and future challenges to advance research on behavioral data privacy (Section 6).

3. Behavioral Data Applications and Privacy Concerns

Behavioral data can be leveraged to provide valuable services for both users and companies. In this section, we summarize the application model, the main usages of behavioral data and the related emergent privacy issues, which motivate the need for our survey.

3.1. Behavioral Biometric Data

Behavioral biometric data are a subclass of biometric data which encompasses all human behavior. While in SDC the columns of a microdata set that should be protected are explicit, it is not so easy for behavioral biometrics as it is not apparent which part of the data is privacy sensitive. As behavioral biometric data are captured from a human, it contains a lot of implicit dependencies between individual data points and across traits. For example the motion of a foot is highly depended on the motion of the corresponding leg. Another dependency to consider is the temporal dependency between data points as behavioral biometrics are usually captured as a time-series. These dependencies make the anonymization of behavioral biometric data challenging as an attacker can exploit them to reverse the anonymization.

3.2. Scenario

Figure 2. The data-publishing scenario of the survey.

In this survey, we assume a data-publishing scenario (see Figure 2) in which the data are first transformed in a privacy protective manner and then published or shared with a service or application. This also includes involuntary publication, which for example can occur when the biometric templates of an authentication system are leaked. We assume that the utility of the protected, modified data is preserved to the extent that the received service (e.g., a personalized recommendation) is still meaningful and effective.

3.3. Applications

One of the most important and well researched application area of behavioral data is biometric authentication (Hogben, 2010; Alzubaidi and Kalita, 2016; Meng et al., 2015; Mahfouz et al., 2017). A person’s behavior, such as the way of walking or typing on a keyboard, contain unique inherent patterns that allow for verifying the identity of that person. Given that these patterns can be sensed implicitly while the person interacts with, wears, or carries a device, behavioral biometrics are generally considered more usable than other traditional biometrics like fingerprints (Bonneau et al., 2012, 2015), and therefore a good alternative or complement to password-based authentication. Academic research has shown the feasibility of numerous behavioral traits for user identification , to name a few: keystroke patterns (Teh et al., 2013), gait (Wan et al., 2019), touch (Teh et al., 2016), mouse movement (Zheng et al., 2016), brain activity (Gui et al., 2019), or even breathing patterns (Chauhan et al., 2018, 2017). And some of them are already developed in commercial solutions, specially in the financial sector to prevent fraud through detecting behavior anomalies (BehavioSec, ; TypingDNA, ; Nymi, ; Voice Vault, ).

In general the entire of field of human computer interaction captures and processes behavioral biometric data, as each input over time also comprises a behavior. Keystroke patterns and mouse movement are our main input modality for computer systems today, however new input modality such as touch, voice, and gestures are on the rise and will likely become more relevant in the coming years.

Another area where behavioral data are useful is healthcare

. Advances in sensors and machine learning techniques enabled the development of applications for activity recognition, fall detection, and remote health monitoring that facilitate caring of elderly, sick, or disabled people and eases diagnosis 

(Pansiot et al., 2007; Pogorelc et al., 2011; de Lima et al., 2017). Typical collected data are gait and motion information coming from accelerometers and gyroscopes embedded in user devices, and biosignals like heartbeat or brain activity. This data can be also processed to give health-related feedback to users, for example to guide them through relaxation or to detect and signal cognitive states, such as being stressed, so the user can act on it.

Besides biometric authentication and healthcare, a great deal of behavioral data driven applications are focused on personalization. In this category we find adaptive interfaces and services that change their content or appearance according to the predicted user preferences based on their behavior. Furthermore, personalization can be applied in many areas. To give some examples, behavioral data are used to personalize online games adapting to the player profile for a more satisfactory experience (e.g., adjusting the level of difficulty) (Zohaib, 2018), in recommender systems to suggest online content or advertisements (Reddy et al., 2019), or in education to taylor the learning experience to the student mental state (level of attention, stress, etc.) (Joe Louis Paul et al., 2019).

3.4. Utility

Depending on each application the behavioral biometric data may be obviously utilized for one purpose or another. For example, in an application for biometric authentication, an evident measure of utility for the provider is its ability to verify the identity of an individual. Likewise, in an application based on human computer-interaction, the provider may require the behavior to still work as reliable input modality for computer systems. In a healthcare application, on the other hand, the service provider may be interested in detecting abnormal behavior patterns, monitoring specific aspects of the behavior such as counting steps or inferring the preferences of a user for personalization, and the utility of the provided service may be assessed as the performance in carrying out those tasks.

3.5. Privacy Concerns

There are also troubling privacy implications derived from the significant amount of personal information implicitly collected in behavioral data driven applications. As we have seen, behavioral data can be used as biometrics because they are rich in individuating information. The counterpart is that any entity that collects behavioral data could use it to identify people even if that is not the main purpose of the service they provide. What aggravates this problem is that people might not be aware that they are being measured, either because of the lack of transparency and adequate consent frameworks, or because the surveillance is meant to be covert. But besides identity, behavioral data carry a wealth of potentially sensitive information that can also be abused. For example, behavioral traits like our voice, eye gaze, gait, or brain responses, are correlated with different diseases (Yang et al., 2012; de Lima et al., 2017), mental states and emotions (Sur and Sinha, 2009; Yacoub et al., 2003), and specific involuntary reactions (such as pupil dilation) can signal our interests (Kröger et al., 2020).

Technically, the general process for inferring identity or other information about an individual from their behavioral data follows four steps, depicted in Figure 3. First, there is a data acquisition step in which the behavioral data are recorded and digitised. Then a feature representation that is suitable for the latter inference is extracted from the raw data. This feature representation is then usually reduced to lower the number of dimensions. In the last step the reduced feature representation is used to perform the inference of either identity or specific attributes. Thus, machine learning techniques are applied to classify the user data as belonging to an existing user profile or not, or as belonging to a specific attribute class (man, woman). Regression models can also be applied to assign the target individual with a measure (e.g., degree of depression on a continuous 1–5 scale). Based on this general workflow, a service that uses a voice-controlled personal assistant could apply the process to classify the user commanding to open an email application as the owner of the account (authentication). But it could also exploit the voice features to classify the mood of the user and offer them highly targeted advertisements, a practice that often comes with discrimination and threatens user’s autonomy. Amazon, for instance, has a patent on technology to extract emotions from user’s voice (Huafeng and Shuo, 2017).

Figure 3. The general behavioral-based inference process.

While big companies already collect a huge amount of behavioral data, the advent of affordable consumer wearables with numerous sensors (e.g., VR/AR devices with eyetracking, head pose and electroencephalograpy (EEG) sensors) exacerbates the issue. Once the data are collected, even if for a legitimate, user-consented functionality like fraud detection based on behavior anomaly, these data can be exploited to learn private information. Hence, the need for techniques to protect behavioral data is poignant. To establish a map of current research on the topic, we categorize and analyze the existing of protection approaches to prevent from identity and attribute disclosure.

3.6. Attacker Model

Our adversary is a malicious service or application provider that wishes to infer private information about the user. As the service provider the adversary has full access to the behavioral biometric data and can freely select an inference technique. Further, she also might have access to additional prior knowledge about the user such as biometric templates or soft biometrics.

4. A Taxonomy of Solutions for Behavioral Data Privacy

Based on our literature analysis, we identify two main privacy threats that apply to behavioral data collected/processed by a third party and can be explained in terms of the related attacker model:

  • Identity Disclosure. The attacker’s goal is to use the behavioral data to identify the user. In this threat model, we assume that the attacker is able to link the target’s behavioral data to the target’s identity and now wants to identify them in another scenario. For example, linking the user account and data in a work-related application to their account in an entertainment application. This linkage would allow the attacker to learn more about the user activity. An example of this type of attacker, as presented in (Steil et al., 2019), could be a VR company with devices that record eye-tracking offering several services (e.g., games, adult content, professional training apps). This company would be able to determine if a user is the same person across these applications using their eye-tracking data, even if the user takes care to create accounts with different names or fake personal data. Moreover, it is not uncommon that behavioral data are sold to third parties or released unintentionally through a breach or hack.

  • Attribute Disclosure. In this threat model, the attacker goal is not to re-identify the user across accounts, but to derive sensitive attributes included within the available behavioral data that the user did not intend to disclose, such as gender, age, or mental state. The attacker might have had previous access or could have collected a dataset where to train the machine learning model for the targeted inference. For example, based on publicly available electroencephalogram datasets of alcoholic and non-alcoholic persons (Neurodynamics Laboratory, 1999; Karamzadeh et al., 2015), it could be possible to build a classifier that determines if newly gathered data from a entertainment application using a brain-computer interface (BCI) belong to a user with an alcohol problem.

Figure 4. Taxonomy of anonymization techniques for behavioral data protection according to the privacy goal.

From the privacy threats, we can derive the two anonymization goals with which techniques can be categorized, i.e., focused on protecting user identity and focused on protecting specific attributes, as depicted in Figure 4.

  • Identity Protection. The process of transforming the behavioral biometric data of person in such a way that that the person can no longer be linked to the data.

  • Attribute Protection. The process of transforming the behavioral biometric data a person in such a way that specific private attributes of the person can no longer be inferred from the data. Attribute encompasses both long-living attributes such as age or gender and short-living attributes such as mental state or temporary health conditions. An extreme version of attribute protection is template protection. For template protection the identification of the person, in the context of an authentication system, should be still possible while all attributes are protected. Further, multiple templates of the same person must not be linkable to each other.

Figure 5. Taxonomy of anonymization techniques for behavioral data protection according to the type of data transformation applied.

We taxonomize anonymization solutions for behavioral biometric data according to the type of transformation applied, as depicted in Figure 5. We include only fundemental concepts, some of the anonymization techniques combine multiple of them. The basic and shared characteristic of all anonymization methods is that they are irreversible transformations. The first distinction of our taxonomy is if they are deterministic or randomized techniques. Non-Deterministic methods rely on randomness in their transformation, which can yield different results for the same input and deterministic methods always give the same result. There are several methods under these two approaches, as we detail in the following.

  • Non-Deterministic methods.

    • Random perturbations. A random transformation into a different domain.

    • Noise injection. Methods that add random noise to the data points.

  • Deterministic methods. Are further split into removal and conversion. The removal method eliminates data points from the data such that the data points do not have an influence on the anonymized result. Conversion methods transform the data points into a new representation, which typically depends on the original domain.

    • Removal. Can happen in two forms: coarsening and feature removal. Coarsening refers to removing parts of each data point or making the data more sparse. Feature removal refers to removing data points belonging to a specific feature altogether.

    • Conversion. Can be discrete or continuous, depending on if the result of the conversion is a discrete or continuous value.

5. Anonymization Techniques

We organize the surveyed techniques according to the behavioral biometric trait they seek to protect. The first trait is voice, then we move on to gait, hand motions, eye-gaze, heartbeat, and brain activity. For each of the traits, we analyze their utility, threat space, anonymization techniques, and evaluation methodology.

5.1. Voice

Voice processing and analysis (Bäckström et al., ) have long been performed and hence a large set of specific terminology exists to describe it. A sound is a change in air pressure, which is often described as airwaves. The sound of the human voice is created by the Larynx and then travels via the vocal tract, which transforms and filters the sound before it leaves the mouth. Due to its approximate tube shape, the vocal tract produces resonances of the sound which are dependent on the length of the vocal tract. Human speech

is a sequence of sounds that convey meaning. A Phoneme is the smallest unit of sound that distinguishes one word from another and an utterance is a unit of speech between two clear pauses. The frequency spectrum is the range in which the frequency of a sound signal can vary, it is gained from the original signal by using a fast Fourier transform (FFT). An important variant of the spectrum is the log-spectrum which allows a better human interpretation of a signal because the human perception of the magnitude of the signal is roughly approximated by the log transformation. Connecting the peaks in the log-spectrum gives the formant frequencies which correspond to the resonances in the vocal tract and uniquely identify vowels. By using a domain transformation (FFT or cosine) on the log-spectrum we get the cepstrum (see Figure 

6

). The cepstrum is useful because it allows easy estimation of the fundamental frequency (f0) of the signal. The perceived fundamental frequency by humans is known as pitch. A widely used scale to transform the fundamental frequency to the pitch is the Mel scale. Using the Mel scale the cepstrum can be sampled at frequencies with the same perceived distance using weighted sums. Applying an FFT on those sums gives the Mel-frequency cepstral coefficients (MFCC). The MFCCs are an approximate quantification of the signal spectrum that focuses on the macrostructure of the signal.

Figure 6. A windowed speech segment (left) and its corresponding Cepstrum (right), Source: https://wiki.aalto.fi/display/ITSP/Cepstrum+and+MFCC.

The following gives a short overview of the field of speaker recognition which aims to establish the identity of a speaker. Gaussian mixture models 

(Reynolds, 1995)

(GMM) represent speakers as the distribution of their feature vectors. The feature vectors are extracted from the speech (most often represented as MFCC) of the speaker and then modeled as Gaussian mixture density. A GMM assumes that the data points are generated by a finite number of Gaussian distributions with unknown parameters. Each feature vector is represented as a linear combination of Gaussian densities. A universal background model (UBM) is a GMM that models a wide variety of non-target speakers, representing possible imposters. The means of the UBM are then adjusted to the target speaker by using a maximum aposteriori adaption 

(Reynolds et al., 2000)

resulting in a GMM for the target speaker. The benefit of this approach is that the Gaussians used to model the target speaker are the same as in the UBM. For the classification of a speaker, the log-likelihood of the target speaker GMM is compared to that of the UBM to determine if the speaker should be accepted. An alternative to the log-likelihood approach is to get a GMM for each speaker recording through a maximum a posteriori probability (MAP) adaptation of the UBM and then map these GMM to a new feature vector, called Supervector 

(Campbell et al., 2006)

. Supervectors can be classified using traditional methods like support vector machines. A common extension of Supervectors is the total variability (TV) 

(Dehak et al., 2011) approach. Which maps the Supervectors to a low-dimensional space that models both the speaker and the channel variability. The resulting vector is called i-vector and is the de facto state-of-art in speaker identification. An alternative to i-vectors are x-vectors (Snyder et al., 2018)

which are extracted for each utterance via a deep neural network (DNN).

5.1.1. Utility

The main usage of voice recordings is the transmission of information between humans, however, in recent years voice also became an important input modality for computer systems. In both cases, it is important that the content of the speech is intelligible for the intended listeners. But also the mere detection of speech in audio samples can be useful, for example for crowd detection. Further, voices uniquely identify their speaker, making them suitable both for authentication and recognition purposes.

5.1.2. Threat Space

The privacy threats for human voices range from the identification of individuals, over the inference of private attributes, to identity theft via fake recordings. The identification of individuals via their voice has long been apparent to humans. But voices convey more information than just identity, they also allow us to infer attributes such as gender (Ertam, 2019), or emotional state (Yacoub et al., 2003). Further, modern speech synthesis methods allow the creation of fake voice recordings for a target speaker, enabling identity theft or the circumvention of speaker authentication systems. Other than the other behavioral biometric traits voice and its resulting speech can also carry a semantic meaning, which can be privacy sensitive.

5.1.3. Privacy Goals

Voice has speech blurring as an additional privacy goal, which aims at destroying the intelligibility of the speech to protect its semantic content from unintended listener.

5.1.4. Anonymization Techniques

We now present the surveyed anoymization techniques that deal with protecting human voices.

Random Perturbation

In (Parthasarathi et al., 2013) Parthasarathi et al. extend their feature removal methods (Parthasarathi et al., 2011) by additionally shuffling the voice blocks. Mtibaa et al. (Mtibaa et al., 2018) propose a template protection scheme that relies on shuffling the feature vector of a GMM-UBM speaker identification system.

Noise Injection

Tamesue et al. (Tamesue and Saeki, 2014) propose a very simple method to make speech unintelligible by simply playing pink noise between 180 and 5630 Hz with various dBs. Hashimoto et al. (Hashimoto et al., 2016)

proposes a system to preserve speech privacy in physical spaces. The core idea is to add white noise to prevent recordings of speakers to be used for identity theft. They found that increasing the Signal-to-noise ratio (SNR) is bad for the intelligibility of the speech and experiment with filtering the white noise in frequency ranges from 0 to 8 kHz to boost the performance of the scheme. They conclude that preventing speaker identity is possible while at the same time keeping the intelligibility of the speech at a high level. Hamm et al. 

(Hamm, 2017) proposes a differential private min-max filter. The mix-max filter minimizes the privacy risk while maximizing utility risk with a given utility and private task. The differential privacy is achieved by adding noises either in front of the filter or after the filter. Ohshio et al. (Ohshio et al., 2018) train multiple so-called babble maskers from pre-recorded speakers by segmenting the speech and then averaging the segments. When a speaker should be de-identified the babble masker is selected based on the fundamental frequency and the pitch of the person. Vaidya et al. (Vaidya and Sherr, 2019) proposes to add random noise to four features: pitch, tempo, pause, and MFCC.

Feature Removal

In (Parthasarathi et al., 2009) Parthasarathi et al. propose three feature removal methods for privacy-aware speaker change detection. Adaptive filtering assumes that the excitation source is independent of the vocal tract response. They perform short-term linear prediction analysis to estimate an all-pole model (Lim and Oppenheim, 1978)(representing the vocal tract), a residual (representing the excitation source), and the gain. Then the residual is used to estimate its real cepstrum. Their second method is to remove all subbands except the one from 1.5 kHz to 2.5 kHz and from 3.5 kHz to 4.5 kHz. They represent the two subbands as MFCC coefficients and log-energy from a single filter. Their last method only uses the spectral slope of the speaker represented as cepstral coefficients. In another work (Parthasarathi et al., 2011)

also propose similar feature removal methods for speaker diarisation using the real cepstrum and MFCC as features. Their analysis finds that MFCC works better than real cepstrum. Additionally, they add subband frequency information between 2.5 kHz and 3.5 kHz and the spectral slope. The privacy is evaluated by trying to recognize phonemes in the anonymized speech. They use an hidden Markov model (HMM) GMM speaker diarisation method as an evaluation system.

Wyatt et al. (Wyatt et al., 2007) propose a feature removal method for speaker segmentation and conversation detection. They segment the audio into segments and save for each the non-initial maximum autocorrelation peak, the total number of autocorrelation peaks, the relative spectral entropy, and the energy of the frame. Zhang et al. (Zhang and Yaginuma, 2012) uses the same features as proposes by Wyatt et al. except for the energy of the frame and then use an HMM to perform the conversation detection.

In (Nelus and Martin, 2018) and (Nelus and Martin, 2019) Nelus et al. propose to use a DNN to extract features from a speaker that allow gender recognition but not speaker identification. Cohen-Hadria et al. (Cohen-Hadria et al., 2019)

use a convolutional neural network called U-net to extract the voices from recordings that consist of both background and voice noise in which the voices should be anonymized. They remove attributes with two methods. The first method simply low-pass filters the voice at 250 Hz. The second method extracts the MFCC from the voice and then uses the first 5 components to create a new voice. In the end, the blurred speech is recombined with the background noise.

Discrete Conversion

In (Pathak and Raj, 2012) Pathak et al. present a hashing algorithm to protect voice data for authentication purposes. The supervector of a speaker is gained by performing the MAP adaptation of a universal background model for each utterance of the speaker and concatenating the means of the adapted model. This supervector is the feature vector for the classification. The locality sensitive hashing is then performed with the supervector which transforms it into a low dimensional space, which is referred to as a bucket. This operation is an approximation of the nearest neighbors algorithm. Now the results can be compared to find to authenticate the individual. In order to make the representation more privacy preserving the salted hash of the result is computed. In (Portelo et al., 2013) and (Portelo et al., 2014) Portelo et al. propose a template protection scheme based on secure binary embeddings. The authors use a speaker identification system that uses supervectors and i-vectors to represent the features of a speaker’s voice. The feature vectors are then encoded with secure binary embeddings which have the property that if the euclidian distance of the two vectors is below a certain threshold then the hamming distance of the resulting hashes is proportional to the euclidian distance. This allows the comparison of the encoded vectors by using a support-vector machine (SVM) with a hamming distance-based kernel. Billeb et al. (Billeb et al., 2015) propose a template protection scheme that is based on fuzzy commitment. They first extract the frequency spectrum via an FFT and then extract features from the magnitude spectrum. Then the MAP adaptation of a GMM-UBM speaker identification system is applied and additional statistics are extracted. The template is then stored as a combination of error-correcting code and hash algorithm.

Continuous Conversion

Speaker transformation is the process of manipulating the voice characteristics of a speaker (not the linguistic features) to make the voice sound like a target speaker. A target speaker can be either a specific natural speaker or a synthetic speaker. For the synthetic speaker either an existing speaker is used or a new one is generated, for example by averaging multiple speakers into one. The general approach of speaker transformation is that the voice characteristics of the source speaker are extracted and then transformed to match the target speaker. In the last step, the new speaker is synthesized.

Jin et al. (Jin et al., 2009) evaluate four methods for speaker transformation for identity protection. Their base method uses a GMM-mapping based speaker transformation system to transfer speakers to a target synthetic voice called kal-diphone. Further, they test duration transformation in which the length of utterances of the source speaker is scaled to match the ones of the target speaker. Double voice transformation simply repeats the process of mapping the source to the target twice. Lastly, they try an extrapolated transformation in which they use the linear mapping of the source to the target to extrapolate beyond the target. Pobar et al. (Pobar and Ipsic, 2014) also use a speaker transformation system based on GMM mapping but combine it with a harmonic stochastic model. The system is trained on a set of speakers to learn the transformation functions. Instead of retraining the system for a new speaker one of the existing transformation functions is applied. This removes the need for a parallel corpus for the speakers that should be protected. The target speaker is a synthetic speaker. In (Sojka et al., 2014) and (Justin et al., 2015) Justin et al. investigate the intelligibility of transformed speakers. They test with a diphone speech synthesis system and an HMM-based speech synthesis system to transform speakers into a synthetic speaker. They performed a survey with human listeners to evaluate the intelligibility of the protected speakers, measuring the word error rate. Abou-Zleikha et al. (Abou-Zleikha et al., 2015) do not propose a speaker transformation method themself but explore how to select a target speaker to achieve the lowest identification rate and have good results when the speaker is transformed back to the source speaker. They formulate this as an optimization problem and measure the distance between two speakers with a confusion factor, for which they evaluate entropy and Gini index as metrics. Pribil et al. (Přibil et al., 2018) propose a speaker de-identification method that relies on modifying several features of the source speaker. In the first step, the prosodic and spectral features are extracted from the source speaker. They then modify the features to make the speaker sound older, younger, more female, and more male by using manually defined transformation functions and feature differences for each class. After the features are modified the de-identified speaker is synthesized.

Bahamanienezhad et al. (Bahmaninezhad et al., 2018)

have developed a speaker transformation method that uses a convolutional encoder/decoder network. They, first extract spectral features and excitation features (f0) from the source speaker. The spectral features are then mapped via the encoder/decoder framework to a target speaker. The resulting speech is fused together either via taking the average or via a gender-based average to create an average speaker. From the excitation features, only the fundamental frequency is transformed via linear transformation, the remaining features stay the same. Both spectral and excitation features are used to synthesize the de-identified speaker. Fang et al. 

(Fang et al., 2019) use a similar averaging approach but rely on x-vectors. They extract the x-vector of a speaker and then use a set of random x-vectors of unrelated speakers to calculate a mean x-vector. They also propose to construct an altogether new x-vector that has a similarity scoring of s to the original x-vector. Further, they keep the fundamental frequency of the speaker the same. Kesking et al. (Keskin et al., 2019)

do not study de-identification directly but instead try to create an imposter transformation for a target speaker. They use a cycle generative adversarial networks (GAN) voice converter to transform speakers and then evaluate against four speaker identification systems to see if the target speaker is recognized.

Frequency warping is a technique that is similar to speaker transformation, the main difference is that frequency warping focuses on transforming the frequency spectrum of a speaker and usually does not try to transform the source into a specific target speaker. It is mostly used for identity and gender protection. A common goal of frequency warping is vocal tract length normalization in which the resonances that are specific to an individual’s vocal tract length should be removed or altered.

Faundez-Zanuy et al. (Faundez-Zanuy et al., 2015) explore two approaches for gender protection: Phase vocoder and vocal tract length normalization. The vocoder approach detects peaks in the voice signal. For each peak, a bin is defined and compared to its two neighbors to define a region of influence. Then the peak and its region of influence are shifted by a peak specific frequency. In the last step artifacts from the shift are removed. The vocal tract length normalization approach defines frames on the signal spectrum and stretches or compresses them using a frequency warping function.

In  (Abad et al., 2016) Valdivielso et al. present a speaker protection approach that transforms the pitch and the frequency axis. Further, the parameters of the transformation are embedded into the signal for later re-identification. Lopez-Otero et al. (Lopez-Otero et al., 2017) rely on frequency warping and amplitude scaling for speaker protection in the context of depression detection. They implement both operations as an affine transformation in the cepstral domain and manually define piece-wise linear transformation functions.

Magarinos et al. (Magariños et al., 2017)

also relies on frequency and amplitude warping for speaker protection. First, they extract the cepstral voice vectors from the speaker and then convert them into a discrete spectrum. Then dynamic frequency warping (DFW) is applied to map the source spectrum bins to the target spectrum. As multiple source bins can have the same target bin, all source bins that map to the same target bin are averaged. Additionally to the frequency and amplitude warping the fundamental frequency is adjusted regarding its mean and variance. Aloufi et al. 

(Aloufi et al., 2019) try to hide the emotional state of speakers before their speech is sent to a voice-based cloud service. They first extract the fundamental frequency, spectral envelope, and aperiodicity. The features are then transformed via a cycle GAN from emotional speech to neutral speech.

Srivastava et al. (Srivastava et al., 2020) evaluate multiple speaker protection methods against an informed attacker. They work with three attacker models: An ignorant attack that is not aware that the voice data is de-identified, a semi-informed attacker that knows that the data is de-identified, and an informed attacker that knows the de-identification method and its parameters. The first method is a vocal tract length normalization approach. The speaker is represented as a set of centroid spectra. The algorithm then calculates the closest path between the source set and the target set to get the parameters for the warping. The second method uses a neural net encoder/decoder approach to transform the speaker.

Continuous Conversion + Random Perturbation

Canuto et al. (Canuto et al., 2014) proposes a new method for template protection in which the feature vector is shuffled via a randomized sum. For each feature vector, the elements are shuffled based on a secret key. Two random vectors of the same length are derived from the key. These vectors give the position of the attributes that should be summed. The reorganized feature vector is summed up with the vectors resulting when the position vectors are applied to the original feature vector.

Continuous Conversion + Noise Injection

In (Kondo et al., 2013) and (Kondo and Sakurai, 2014) Kondo et al. create so-called babble maskers by segmenting speech into ten second segments and then averaging them into babble maskers. Besides speaker-dependent maskers, they also create gender-based babble maskers based on multiple speakers of the same gender. The babble masker is then applied to the recording of the speaker. Qian et al. (Qian et al., 2018) present a method to sanitize speech before it is sent to the server of a virtual assistant. Their main method is to perform vocal tract length normalization via a compound frequency warping function consisting of a bilinear and a quadratic function to avoid re-identification attacks. The parameters of the warping function are selected randomly. Additionally, they add Laplace noise after the warping function to make the anonymization more robust. For the result, they claim to achieve differential privacy. In  (Qian et al., 2021) the same authors further investigate the security of their scheme. Srivastava et al. (Srivastava et al., 2020) also investigate the security of the scheme with stronger attackers.

5.1.5. Evaluations

Qian et al. (Qian et al., 2018) present a framework to reason about the privacy and utility of voice anonymization techniques. For this, they present the measure of p-leak limit which should give a maximum privacy leakage per speaker for a published dataset. Zhang et al. (Zhang et al., 2020) propose a theoretical framework to quantify the privacy leakage risk and utility loss for speech data publishing. They identify three main data properties to anonymize: dataset description, speech content, and speaker voice. For speaker de-identification they do not describe their own speaker de-identification techniques but give a framework for quantifying the utility privacy loss.

Most of the reviewed works evaluate the quality of the de-identification by comparing the recognition rates of attributes or identities on unmodified and de-identified data. The recognition is done via machine learning models or human listeners. As metrics to measure the recognition rate the papers mostly rely on the equal error rate (EER), false positive rate (FPR), false negative rate (FNR), recall, precision, and F1 score. Abou-Zleikha et al. (Abou-Zleikha et al., 2015) also use entropy and the Gini index to evaluate the de-identification performance.

Additionally to the de-identification, some works evaluate the loss of utility. One important goal in regards to human listeners is to achieve a natural-sounding de-identified voice. The naturalness is evaluated by human listeners using the mean opinion score. Another important aspect is the intelligibility of the de-identified speech. Intelligibility can be evaluated via human listeners or machine learning models using the word error rate, phoneme error rate, or short-time objective intelligibility.

A common limitation we observed is that in most evaluations use the clear data to train the recognition model and then test it against the anonymized data. This approach implicitly assumes that the attacker is not aware of the anonymization and hence does not try to circumvent it. A work that explicitly assumes an attack on the anonymization is (Srivastava et al., 2020). Here the authors propose attackers with varying degrees of information about the performed anonymization.

5.2. Gait

The human gait is the pattern in which humans move their limbs during locomotion, multiple manners of gait exist such as trotting, walking, or running. Gait can be broken down into individual gait cycle (Stöckel et al., 2015) (see Figure 7) which is the shortest repetitive task during the gait. The gait cycle spans from a specific gait event of one foot until the same foot reaches the same gait event. It consists of a stance phase, in which the foot is on the ground, and a swing phase, in which the foot is in the air. The two phases alternate for each foot. Due to its usefulness as a behavioral biometric trait for identifying individuals, gait has long been a research interest of both computer science and psychology. For example, Yovel et al. (Yovel and O’Toole, 2016) find that it plays an important part for humans to identify people at a distance, and Pollick et al. (Pollick et al., 2005) show that it is possible for humans to infer the gender of a walker, even when the walker is only shown as a set of points, as so-called point-light-display. The following section deals with the anonymization of gait patterns.

Figure 7. The phases of the gait cycle, source: (Stöckel et al., 2015).

Gait recognition methods have been an active research topic in the past, hence a large set of different methods for various capture methods exists. Wan et al. (Wan et al., 2019) performed a recent survey on the subject and list recognition methods for cameras, accelerometers, floor sensors, and radars. The main portion of the works focuses on camera based gait recognition which is classified by Wan et al. as either model-based or model-free. Model-based methods use a specific model of the walker, for example, a pendulum model of the legs, to then match the walker to it. Model-free methods, however, do not have an explicit model but rather use the entire capture of the gait to perform the recognition, for example by averaging the silhouette of the walker over time as a gait energy image. Accelerometer-based systems also average the gait into a feature representation either by segmenting the gait into its gait cycles or by using frames with a fixed size.

5.2.1. Utility

The human gait is omnipresent in everyday life and as such often captured as a byproduct of recordings being made. As such it is often not necessary to preserve the utility of the gait, but rather the utility of the recording. One example of this would be video recordings of people walking, the gait pattern itself is not so important but rather that the video looks natural and convincing to its viewers. But there also exist use-cases in which the gait pattern itself should be captured, for example for medical examinations by a physician to find gait abnormalities. Another more casual example would be the recording of the gait pattern to count the steps a person has performed during a day.

5.2.2. Threat space

Due to its omnipresence in everyday life, human gait is easy to capture, especially because most capturing methods are unintrusive and do not require the participation of the victim. Additionally, it has been shown that gait recognition is very robust to video quality and obfuscation making it very much suited for surveillance systems (Wan et al., 2019). Besides identifying humans it has also been shown that gait can be used to infer private attributes like gender (Pollick et al., 2005). Considering all this the threat to gait biometrics is already large. What’s more, with recent developments in richer capturing methods such as LiDAR (Galai and Benedek, 2015) or cheap motion capture suits, it is to be expected that the threat space for gait will even increase in the coming years.

5.2.3. Anonymization Techniques

In the following, we present the gait anonymization methods found in the literature, sorted by our taxonomy.

Random Perturbation

Hoang et al. (Hoang et al., 2015)

propose a fuzzy commitment scheme based on Bose–Chaudhuri–Hocquenghem (BCH) codes for storing accelerometer gait templates. After the feature extraction and binarization of the accelerometer data the reliable bits are extracted. These bits are then XORed with the BCH encoded secret key to gain the secure

. Additionally to the , the hash of the secret key and some helper data are stored. During the authentication phase, the extracted reliable bits are XORed with the secure and then decoded with BCH. The result can then be hashed and compared to the hash of the secret key.

Noise Injection

The influence of noise injection on the performance of accelerometer/gyroscope authentication systems was studied by Matovu et al. (Matovu et al., 2018)

. For their approach, they generate a time series of noise values drawn from a uniform distribution and then merge the original time series with the generated one. The two traits evaluated are gait and handwriting.

A noise injection approach for gait in videos was developed by Tieu et al. (Tieu et al., 2017). They use a convolutional neural network (CNN) to mix the gait of a second person (noise gait) into the original gait. In the first step, the silhouette for both the original and noise gait is extracted from a black and white representation of the input videos. The noise gait is selected hereby to have the same size and view angle as the original gait to achieve a more natural result. The silhouettes are then fed into the CNN which uses shared weights networks to abstract them and then merges the abstracted representations via a third network. In a post-processing step, the original gait is replaced with the newly merged gait. The authors further improve their method in (Tieu et al., 2019). Here the noise gait is generated via a generative adversarial network (GAN) that takes Gaussian noise as input and outputs noise silhouette. Instead of using a CNN they then use a self-growing and pruning GAN (SP-GAN) to fuse the noise and original gait. Further, they propose an approach to colorize the resulting black and white silhouette. In  (Tieu et al., 2019) it is proposed to use a deep convolutional GAN to fuse original and noise gait.

Feature Removal

A feature removal approach for privacy-preserving activity recognition via accelerometers is proposed by Jourdan et al. (Jourdan et al., 2018). They extract various temporal and frequency features from the accelerometer data such as mean, correlation, energy, or entropy. Via experiments, they then determine the influence of each feature for activity and identity recognition. They find that the temporal features contribute more to identity recognition and frequency features more to activity recognition, therefore they remove the temporal features.

Continuous Conversion

A continuous conversion approach is blurring, in which persons in videos, including their gait, should be de-identified. As a first step, the silhouettes of the persons in the videos are tracked and segmented to then apply the blur. Agrawal et al. (Agrawal and Narayanan, 2011) proposed two blurring approaches exponential blur and line integral convolution (LIC). Exponential blur regards the video as a 3D space with the time as the z-axis and then calculates a weighted average of the neighbors of each voxel to blur via an exponential function. LIC works with the bounding box of the walker silhouette and maps it onto a vector field which is then used to calculate the output pixels. To counter reversal attacks against the blur randomization of the blurring functions at each pixel is proposed. Another blurring approach is proposed by Ivasic-Kos et al. (Ivasic-Kos et al., 2014). They apply a gaussian filter to blur the silhouettes of walkers. The filter calculates a weighted average of the color of the neighboring pixels, with the weights decreasing monotonically from the central pixel.

Continuous Conversion + Discrete Conversion

An approach that combines both continuous and discrete conversions for walkers in videos is proposed by Hirose et al. (Hirose et al., 2019). First, they extract the silhouette and the gait cycle of the walker. The silhouette is then transformed via a deconvolutional neural network encoder into a silhouette code. The code is converted by using a k-same approach in which the k-nearest neighbors of the input code are selected and then a weighted average is computed. The gait cycle is transformed via a continuous, differentiable, and monotonically increasing function. In the last step, the new video is generated by feeding the perturbed silhouette code and gait cycle into the convolutional neural network decoder.

5.2.4. Evaluation

Gait de-identification is evaluated in the literature via gait recognition systems or human observers with the recognition accuracy as the main metric, but there are also usages of the F1 score, equal error rate (EER), or false acceptance rate (FAR). To access the utility loss there is a larger variety of metrics, usually to either quantify the naturalness of the de-identified gait or to perform another kind of recognition, such as activity. One specific evaluation method we observed was  (Matovu et al., 2018) in which the authors use the biometric menagerie to observe the de-identification influence on different types of users in biometric authentication systems.

5.3. Hand Motions

Hand motions are the wide variety of movements humans can perform using their hands. As they are such a universal part of human interaction with their environment there exist multiple approaches for using hand motions a behavioral biometric factors: handwriting, keystrokes, mouse movements, and hand gestures. They differ by how the hand motions are recorded and which task the person performs. Handwriting is a hand motion in which the person performing it uses a pen to write text. Due to the uniqueness of people’s handwriting, it has long been established that humans can be identified by it. Signatures are the written name of a person which are intended for identification purposes, for example on legal documents. Handwriting can be captured offline in which the produced text is captured via a picture or online in which the movement of the pen is captured during the writing process. For this survey, we only consider the uniqueness of one writing style and not the linguistic style (Stylometry) of the written text. In modern life, handwriting has been largely replaced by typing on a keyboard. Besides writing texts, keyboards are also used as a general input modality for computer systems. The keystrokes and the timings a human produces while using a keyboard are also a biometric factor. Another input modality that captures hand motions are mouse movements. Hand gestures are the wide range of hand motions humans perform to communicate nonverbally. While a normal part of human communication, hand gestures only recently became important as an input modality for computer systems with the rise of swipe gestures on smartphones. This trend is continuing with freehand gestures for wearables such as smartwatches or augmented reality headsets.

Hand motion recognition encompasses multiple recognition techniques for different capture modalities, here we give an overview of handwriting, mouse movements, keystrokes, and gestures. For handwriting bases hand motion recognition the input handwriting sequence is often adjusted for its baseline, scaled to a normal writing style, and segmented to meet the demands of the classifier (Plamondon and Srihari, 2000). Handwriting is further dependent if it was captured while the person was writing (online handwriting), for example with a digital pen, or only handwriting itself is capture after the person has finished (offline handwriting). The recognition for mouse movements relies on the trajectory, speed, single, and double clicks performed with a mouse as features. Keystroke-based hand motion recognition is based primarily on the timing differences between button up, down, and hold events. Besides individual events, the differences between two successive events or even three successive events are also used as features (Zhong and Deng, 2015). Hand motion recognition via gestures can be split into 2D gestures which are performed on a flat surface (e.g. on a smartphone) and 3D gestures which are performed in mid-air.  (Sherman et al., 2014)

uses the trajectories of each finger and first resample them using a cubic spline interpolation to get a lower sampling rate, removing unwanted jitter. They then use a mutual information metric to classify the gesture. 

(Sae-Bae et al., 2014) In the first step they label each finger. Then the distance between every two fingers and each finger position and its following position is calculated. To find the distance between two gestures dynamic time warping is employed with various distance metrics.  (Tian et al., 2013) The 3D gesture recognition works similar to the 2D one as first the fingertips of each finger are found and then after scaling and smoothing multiple features based on the fingertips are selected. The classification is again performed with dynamic time warping.

5.3.1. Utility

The utility range for hand motions is a large and diverse field. For handwriting the resulting text must be readable either by humans or computers, the particular handwriting style is usually not important. This is different for signatures, as their main purpose is to facilitate the identification and verification of the signers identity, hence their particular style is important, while the readability of the name is less important. Since the other hand motions mostly serve as input modalities for computer systems their utility as input modality must be kept precise and timely to keep their utility as an input modality. For hand gestures, there is additionally its utility for non-verbal communication.

5.3.2. Threat Space

Handwriting used to be essential to human communication but with the rise of computers, it has become less important and was mostly replaced by digital communication. Due to its decline as a communication medium, it has become difficult to get handwriting samples of a particular person. Since the sensitive nature of signatures as a biometric factor is commonly known humans are usually cautious at leaving their signature, however, due to them widely being used in everyday life there still at risk of being collected by an adversary. Most hand motion capturing today happens implicitly when humans use their hands to control computer systems. Each time we use a mouse or keyboard our hand motion is recorded and as such at risk. Most applications or websites could be used to capture both mouse movements and keystrokes. But even without direct access to the keyboard attackers could collect these biometrics via side-channels such as network latency. Hand gestures are a rather new input modality for computer systems and only became widely popular with the rise of smartphones. Due to their exposed nature and the fact that we often perform gestures in public hand gestures are relatively easy to capture by an adversary, for example by using a camera. It is to be expected that with the rise of mixed reality and its applications hand gestures gain more importance as an input modality and therefore will be at a higher risk.

5.3.3. Anonymization Techniques

In the following, we present the suitable methods for hand motion anonymization, with the exception of mouse movements as we did not find any suitable papers for it.

Random Perturbation

Maiorana et al. (Maiorana et al., 2011) propose a template protection method for online handwriting which splits a handwriting sequence into segments and then randomly mixes the segments before convoluting them. The same shuffling approach is taken by Maiti et al. (Maiti et al., 2016) to prevent keystroke inference attacks via wrist-worn accelerometers, however, they do not convolute the segments. Another study investigating the permutation of keystrokes is performed by Vassallo et al. (Vassallo et al., 2017). Goubaru et al. (Goubaru et al., 2014) propose a template protection scheme for online handwriting templates. They extract the pattern ID for a user by using a common template. The pattern ID is then XORed with a secret that was encoded by an error-correcting code. The result is stored as the template. For the verification, the pattern ID is again extracted and then XORed with the template.

Noise Injection

To prevent the identification in browsers via keystroke timings Monaco et al. (Monaco and Tappert, 2017) investigate two noise injection strategies: delay mixing and interval mixing. Delay mixing adds random noise to the timing of a keystroke and interval mixing which draws a new arrival time for each keystroke, depending on a randomly drawn interval. A similar approach to delay mixing is also investigated by Migdal et al. (Migdal and Rosenberger, 2019) which also add delays to keystroke timings.

Coarsening

Vassallo et al. (Vassallo et al., 2017) explore suppression of keystrokes to preserve the content of the typed text in a continuous authentication scenario. Maiti et al. (Maiti et al., 2016) propose two coarsening methods to prevent keystroke inference attacks via wrist-worn accelerometers. In their first approach, they simply detect if a user is typing via several features and then block the access to the accelerometer data to prevent attacks. Their second method reduces the sampling rate of the accelerometer.

Discrete Conversion

An online handwriting template protection scheme is proposed by Sae-Bae et al. (Sae-Bae and Memon, 2013) which decomposes signatures into histograms on which the authentication is performed. They use one-dimensional histograms to capture the distribution of single features and two-dimensional histograms to capture the dependence between two features. Migdal et al. (Migdal and Rosenberger, 2019) propose a template protection scheme for multiple modalities, including keystrokes. Their scheme combines multiple pieces of information, such as ip addresses, with the keystroke information and then computes a biohash on it.

Leinonen et al. (Leinonen et al., 2017) investigate the anonymization of keystroke timing data using two rounding approaches which effectively sort the timings into buckets. Vassallo et al. (Vassallo et al., 2017) explore substitution of keys with a random nearby key to preserve the content of the typed text in a continuous authentication scenario. Figueiredo et al. (Figueiredo et al., 2016) have developed a modeling language that can be used to design new gestures for applications. Each gesture is validated to check if the gesture can harm the person performing it and if an existing gesture is overwritten by it. The gestures can then be recognized on the recording hardware, eliminating the need to give the application access to the clear data.

Continuous Conversion

Maiorana et al. (Maiorana et al., 2011) propose two continuous conversions for online handwriting templates: A baseline conversion which first splits a handwriting sequence into multiple segments based on a secret key and then convolutes the segments. And a shifting transformation that applies a shift to the initial sequence. The template matching is performed on the protected template.

5.3.4. Evaluation

Hand motion anonymization is mostly evaluated in the context of authentication and as such the false positive rate (FPR), false negative rate (FNR), and equal error rate (EER) are important metrics for evaluating the performance. But there is also the usage of recognition approaches for the evaluation for example by (Monaco and Tappert, 2017) which uses the accuracy of identity, age, gender, and handedness inference. A unique evaluation approach we found was used by Goubaru et al. (Goubaru et al., 2014) who used the randomness of the template bits via occurrences and autocorrelation to evaluate their approach.

5.4. Eye-Gaze

Eye gaze involves two type of movements: fixations and saccades. Our eyes alternate between them during visual tasks, such as reading (see Figure 8). Fixations refer to maintained visual focus on a single stimulus, while saccades are rapid eye movements between fixations to reorient our gaze. Besides, even during fixations, our eyes are not completely still, but constantly producing involuntary micro movements (hundreds per second) known as microsaccades (Abrams et al., 1989).

Figure 8. Fixation and saccades while reading, from a study of speed reading made by Humanistlaboratoriet, Lund University, in 2005. Source:http://en.wikipedia.org/wiki/File:Rea.

Eye-tracking technologies are becoming increasingly available in the consumer and research market. The most common type of tracking technology works by illuminating the eye with an array of non-visible light sources that generate a corneal reflection. These reflections are sensed and analyzed to extract eye rotation from changes in reflections. There is a wide range of hardware configurations for eye-tracking, including embedded cameras in computers, smartphones and virtual reality headsets, dedicated external hardware, or mobile eye-wear. These sensors allow to extract measurements not only regarding movement data related to fixations and saccades (speed, gaze angle, attention spots, scan path), but also additional features, such as pupil size variations and blink behavior. Combinations of these features provide valuable information to implement eye-gaze driven applications.

5.4.1. Utility

Eye movements have been studied, analyzed and used for more than a century in different research domains. In the medical field, gaze provides useful information about our cognitive and visual processing (Harezlak and Kasprowski, 2018; Bahill et al., 1975), which can be used for diagnosing different diseases. In computer science, eye gaze is used as a form of human computer interaction to improve accessibility, user experience, and to adapt system behavior (Majaranta and Bulling, 2014; Poole and Ball, 2006; Conati et al., 2007). More recently, security and privacy researchers have focused on analyzing stable unique features of eye movement to build biometric authentication systems (Katsini et al., 2020). Behavioral eye biometrics have been subject of intense investigation in the last decade, showing EERs as low as 1.8% (Eberz et al., 2018). Across all these different domains, the utility to be preserved would depend on the underlying application, e.g., accuracy in predicting the next eye movement, in diagnosing a mental disease, in detecting the focus of user attention, or in recognizing a user.

5.4.2. Threat Space

Eye movement data is rich in information that can be exploited by malicious entities or curious service providers to uncover user sensitive attributes beyond those disclosed intentionally and required for the purpose of the service or to directly identify a person. Besides the biometric information carried by eye movement data, research has also documented their correlation with multiple disorders and mental conditions, such as Alzheimer’s (Hutton et al., 1984), schizophrenia (Levy et al., 2010; Holzman et al., 1973), Parkinson (Kuechenmeister et al., 1977) bipolar disorder (García-Blanco et al., 2014), mild cognitive impairment (Yang et al., 2012) multi sclerosis (Derwenskus et al., 2005), Autism (Boraston and Blakemore, 2007; Wang et al., 2015), or psychosis (Ettinger et al., 2004), to name a few. Furthermore, pupil size is known to be an indicator of a person’s interest in a scene (Hess and Polt, 1960) and a proxy for detecting cognitive load (Matthews et al., 1991; Krejtz et al., 2018). Other recent works demonstrated that eye data can be used to infer gender and age, or even personality traits (Kröger et al., 2020; Berkovsky et al., 2019). Given the richness of eye data and the increased availability of consumer tracking devices and the advent of eye-gaze driven applications, there is a significant and imminent privacy threat potential (Adams et al., 2018).

The two main threats that endanger eye privacy are re-identification and attributes’ inference.

5.4.3. Anonymization Techniques

We found three proposals to protect the privacy of eye movement data (Liu et al., 2019; Bozkir et al., 2020; Steil et al., 2019), all of them are guided by on differential privacy (DP). The general idea of differentially private algorithms is to add a certain amount of randomly generated noise to the original signal, so that it is difficult to say whether or not an individual contributed their data.

Noise Injection

Steil et al. (Steil et al., 2019) propose a DP-based technique to protect eye movement data collected while users read different types of document (comic, newspaper, textbook) in a VR setting. The utility goal is to accurately predict the type of document to provide enhanced features in the reader application. Additionally, the privacy goals are to avoid gender inferences from eye movement data and to protect against re-identification when the attacker has prior knowledge of a dataset including the target user eye data and identity. To achieve these goals, the exponential mechanism (Dwork and Roth, 2013) is applied to a database of users’ eye features by a trusted curator prior to its release. This sanitised database can be then used for training classifiers to provide the enhanced reader functionality. The experiments testing at various noise level show that utility with regard to document classification can be partly preserved (55-70%) while reducing gender accuracy inference to the level of random guesses (50%).

Based on Steil et al.’s dataset, Bozkir et al. (Bozkir et al., 2020) evaluate two types of DP-based perturbations, the standard Laplacian perturbation algorithm (LPA) (Dwork et al., 2017) and the Fourier perturbation algorithm (FPA) (Rastogi and Nath, 2010). They also propose a modification of the FPA algorithm that splits eye data in chunks before adding noise, in order to reduce temporal correlations, which is a source of reduced utility as more noise is required to protect privacy. With this modification, they obtain document type classification (comic, newspaper, textbook) results results similar tho those in (Steil et al., 2019) for the case of 50% gender classification, while adding more noise to the data (better privacy guarantee).

Liu et al. (Liu et al., 2019) present a DP-based solution to anonymize eye tracking data aggregated as a heatmap. A heatmap, or attentional landscape, is a popular method for visualizing eye movement data that represents aggregate fixations (Duchowski, 2017). This means that the intensity of every pixel is adjusted relative to the number of fixations over that region. The privacy goal in this case is to protect individual gaze maps while preserving the utility of the aggregated heatmap. Their experiments with random selection and additive noise (Gaussian, Laplacian) show that Gaussian noise is the best option to obtain good privacy guarantees for the individuals’ gaze maps without visually distorting the hotspots in the aggregated heatmap, i.e., keeping a certain utility.

5.4.4. Evaluation

The proposals by Steil et al. (Steil et al., 2019) and Bozkir et al. (Bozkir et al., 2020), measure the quality of their anonymization techniques for attribute inference protection using the classification accuracy metric for the main task and the attribute inference task. For the re-identification protection case, it is assumed that the attacker has previous knowledge of a database of users’ eye data and their identities. To simulate this knowledge, they train the classifiers on the clean data and test them on the anonymized data , using also the accuracy metric to report privacy protection. Besides, these works also report the so called privacy loss parameter (or ) from DP theory, which quantifies the maximum difference between the data points of two individuals in the dataset. Furthermore, Bozkir et al. use the inverse of the normalized mean square error (NMSE) between the actual eye feature values and the perturbed ones as a utility metric. However, the interpretation and implications of these privacy loss and utility metrics are not developed.

Liu et al. (Liu et al., 2019) analyzed the privacy-utility tradeoff of anonymized heatmaps using the correlation coefficient (CC) and mean square error (MSE) of noisy heatmaps under different privacy levels (different values of ). The CC and MSE give an idea of the similarity between the original and the anonymized heatmaps and the provides information about the privacy guarantee (the smaller, the better privacy). These metrics are accompanied by the visual representation of the noisy heatmap, in order to aid the relevant stakeholders in deciding what level of noise is acceptable for a given application.

Regarding datasets, Steil et al. (Steil et al., 2019) collect data from 20 participants (10 male, 10 female, aged 21-45) while reading documents using a VR headset. Each recording is divided into three sessions (reading a comic, newspaper, or textbook), lasting 30 minutes in total. They extract 52 eye movement features related to fixations, saccades, blinks, and pupil diameter. The dataset has been publicly released 333https://www.mpi-inf.mpg.de/departments/computer-vision-and-machine-learning/research/visual-privacy/privacy-aware-eye-tracking-using-differential-privacy by the authors and Bozkir et al. (Bozkir et al., 2020) use it as the basis to evaluate their proposal. In the heatmaps anonymization study, Liu et al. use a synthetic simulated dataset to illustrate their privacy analysis. Besides the technical privacy analysis, Steil et al. (Steil et al., 2019) is one of the few works considering user privacy concerns regarding behavioral data collection. They conduct a large scale user survey (with N=164 participants) to explore with whom, for which services, and to what extent users are willing to share their gaze data. Their report shows that people are uncomfortable with inferences (gender, race, sexual orientation) and would object to share their data if these attributes can be leaked. The results also show that people generally agree to share their eye tracking data if a governmental health agency or for research purposes, but would object to do so if the data owners are companies. These insights are a first step towards understanding user privacy awareness and privacy needs, but more work is required in this field to guide the design of user-centered privacy protective techniques for behavioral data.

5.5. Heartbeat

An electrocardiogram (ECG) is a graph of voltage over time that captures the electrical activities of cardiac muscle depolarization followed by repolarization during each heartbeat. Shown in Figure 9, the ECG graph of a normal beat is composed of a sequence of waves: a P-wave reflecting the atrial depolarization process, a QRS complex representing the ventricular depolarization process, and a T-wave denoting the ventricular repolarization. Other portions of the ECG signal encompass the PR, ST, and QT intervals (Zheng et al., 2020).

Figure 9. Waveform of an ECG signal with normal cardiac cycle. Source: https://www.nottingham.ac.uk/nursing/practice/resources/cardiology/function/normal_duration.php.

As per the current screening and diagnostic practices, cardiologists review ECG data, find the right diagnosis and implement subsequent treatment plans such as a medication regime or the removal of a radiofrequency catheter. Nonetheless, the demand for highly accurate, automated heart-condition diagnoses has increased significantly, in part due to the new public health regulations of implementing more extensive screening processes as well as the adoption of ECG-enabled wearable devices.

It is well known that certain types of cardiovascular conditions, such as atrial fibrillation, have a wide and severe impact on public health, quality of life, and medical expenditures. The long-term ECG monitoring is a vital, non-invasive tool for detecting such conditions. For evident computational and intellectual property reasons, however, the analysis of such data is never conducted at the wearable device but at automated, machine-learning based systems typically hosted in hospitals or external service providers. This necessarily implies the transmission of ECG data from patients to non-fully trusted entities, which inevitably poses evident privacy risks.

However, the disclosure of sensitive data not only represents a threat to patients’ privacy: it may also prompt a serious security risk to any biometric-authentication system that relies on those data. The advantage of ECG-data-based systems over other biometrics systems (like fingerprint, face or iris), though, is the intrinsic nature of ECGs and also their inherent indication of life, which make them very difficult to forge or steal (Wu et al., 2019)

. Compared to fingerprint and facial recognition systems, where extra sensors —other than those required for medical monitoring purpose— are needed, ECGs are a more suitable choice in practical applications and have been shown to be extremely accurate in identification tasks 

(Singh and Gupta, 2008).

Like other biometric systems applied to identification tasks, ECGs are typically converted into abstract, compressed representations, typically referred to as biometric templates444As already mentioned in previous subsections, the functioning principle of biometric templates is that an original signal can be recovered from its template., before the task is conducted555Bear in mind that ECG signals are generally collected over long periods of time and at high resolutions. This leads to large volumes of data being collected. For example, for a sampling rate of 500 Hz and a data resolution of 8 bits per sample, a 24-h record amounts to about 43.2 Mbytes per channel.. Biometric-template methods can be classified depending on the exploited features of the ECG data. The most popular ones are fiducial-based, non-fiducial-based and hybrid methods (Odinaka et al., 2012). On the one hand, fiducial-based techniques utilize characteristic points on the ECG signal to extract temporal, amplitude, envelope, slope and area features. Characteristic points are the locations that correspond to the peaks and boundaries of the P, QRS and T-waves of the ECG signal. On the other hand, the non-fiducial-based methods do not rely on the ECG characteristic points, and examples include autocorrelation coefficients, Fourier and wavelet transforms. Hybrid methods combine both fiducial-based and non-fiducial-based features.

Biometric templates are therefore an attempt to reduce data storage in identification services. In other type of services, ECG signals are expectedly compressed to allow efficient transmission and storage as well. As we shall elaborate later, techniques aimed to protect the transmission of ECG data will be classified depending on whether they are applied before or after compression.

5.5.1. Utility

ECG data find application in healthcare and biometrics systems, the latter being intended for identification and authentication, as discussed in the preliminaries of this section.

In healthcare, ECGs are utilized for remote diagnosis and in-home health monitoring. Typically, there is a stand-alone service or a complete e-health system where the service provider, in addition to offering a repository of personal medical data, may allow to remotely process such data. In any case, the aim is to provide real-time feedback to patients and hospitals, either as a warning of impending medical emergency or as a monitoring aid during physical exercises.

Although it is well known that ECG data may help diagnose a patient’s physiological or pathological condition, other probably lesser-known inferences include cocaine use (Hossain et al., 2014) and stress (Plarre et al., 2011), which may be sensitive to the patient and obviously should be kept private. The fact that the very same time series data allows drawing both desirable inferences (i.e., for healthcare) and sensitive inferences (that need to be protected) poses a dilemma of great practical relevance.

5.5.2. Threat Space

Regardless of the application (i.e., identification, authentication or healthcare), ECGs are health data and, as such, are considered sensitive by data-protection regulations and need to be protected. Consider the case, for example, of a user who might see their insurance premium increased or suffer discrimination during a job application due to a medical condition inferred from their ECGs.

The general scenario where ECG data are collected, processed and stored is shown in Figure 10. The scenario is composed of three entities: a patient (in the case of healthcare applications) or user (in the case of identification and authentication applications); a wearable or internal666In the sense of within a patient or user’s premises. device collecting patient’s ECG data; and an external entity that receives the data collected by the internal device, and processes and stores such information as a biometric template or in raw or compressed format, so as to provide a service.

Figure 10. Key entities in a scenario where ECG data are collected, processed and stored.

Although the internal device is typically assumed to be trusted, both this and the external entity may be either trusted, partially trusted or fully untrusted. In the latter two cases, at the external entity the access of ECG data (including biometric templates) by unauthorized personnel poses an evident privacy threat and therefore should be prevented. As we shall elaborate in the coming subsections, privacy-protecting techniques will need to be put in place to allow only authorized personnel (e.g., medical personnel or cardiologists) to have access to ECG data or be able to reconstruct them from a biometric template.

Another aspect to consider within the spectrum of potential threats is the algorithm itself used by the service provider (e.g., a company or a hospital) to process ECG data. We have already mentioned that patients or users of the service could see their privacy compromised if their personal signal information or their biometric template was disclosed to a non-fully trusted-third party (not necessarily a hospital or doctor). However, protection is not only required by patients or users, but also by the service provider itself, which may not be willing to provide the end-user with its proprietary protocols because of fear of disclosing valuable intellectual property to third parties or compromising the basis for its service (Lazzeretti et al., 2012).

To conclude, a service provider might want to make the inferences model available to any health professional, e.g., through controlled queries, and/or like to publish anonymized ECG data as a means to crowdsourcing algorithmic development777The Netflix Prize (Bennet and Lanning, 2007) is probably the best-known example of collaborative-problem solving in the computer-science community.. In both cases, the threats space would include the learned model and the released or published data.

5.5.3. Anonymization Techniques

Next we survey the most relevant privacy-protection techniques for ECG data.

Random Perturbation

As mentioned in the preliminaries, large volumes of data are collected in ECG-monitoring applications, and compression is very often needed for their transmission and storage. In this sense, Liu et al. (Liu et al., 2018)

propose combining compression and encryption to provide privacy and confidentiality. Their proposal, however, differs from the typical compression-then-encryption approach, which may be problematic when untrusted network providers may conduct the compression task but do not have access to the private keys. The encryption-then-compression technique proposed by Liu et al. is composed of two steps. First, the ECG data, which are stored in a matrix, are multiplied by an orthogonal, randomly-generated key matrix. Then, singular-value decomposition (SVD) —a popular dimensionality-reduction technique— is applied to the encrypted data to provide compression.

Another approach based on compressive sensing (CS) (Candes et al., 2006) is proposed by Djelouat et al in (Djelouat et al., 2018). CS is a signal processing technique that combines both sampling and compression through random projections. Building on this technique, the authors propose compressing the ECG signal by sampling it at the time of sensing. This reduces the need to even store the sensitive ECG data at the wearable device, thereby providing protection against that entity. The theoretical properties of this compression technique ensure that, under certain assumptions on the random projection, a good reconstruction of the original ECG signal can be obtained at the provider side.

Feature Removal

Kalai et al. (Zaghouani et al., 2017) present a solution to secure the transmission of the ECG template between a wearable device and a service provider. In a first phase, the authors propose computing the discrete cosine transform (DCT) of the ECG signal’s autocorrelation coefficients, and then removing those DCT coefficients with the lowest energy. The remaining DCT coefficients constitute the biometric template. In a second phase, two keys are obtained from the template. One is transmitted to the target application the user wishes to authenticate. The other functions as a private key, which is derived from the complete DCT already stored in the server.

A similar approach is presented by the same authors in (Zaghouani et al., 2017) that uses a quantization step once the DCT-template is obtained. This latter approach is evaluated on the PTB dataset but no experimental comparison is conducted between the two proposed solutions.

Another similar proposal is (Mahmoud, 2016), which decomposes the ECG signal into its wavelet transform, eliminates the low-frequency coefficients and reconstructs the ECG signal for release. At the provider side, only authorized personnel with access to a secret key (derived from the wavelet-transform template) is able to reconstruct the original ECG from the released, protected signal. To which extent these released data may safeguard patients’ privacy is evaluated through the percentage root mean square difference (PRD), a simple and widely used distortion measure in ECG signal processing applications (Manikandan and Dandapat, 2008) that quantifies the difference between the original ECG and its protected version.

Utilizing the same transform, (Sufi et al., 2008) proposes that, after the decomposition, the essential parts of the coefficients (which consists in the P, QRS and T signatures of the ECG) are treated differently, as follows. The non-essential parts of the signal are uploaded to a public repository in the clear, whereas the essential parts are encrypted and distributed among the healthcare experts in charge of analyzing patients’ ECG data. In this process, the encrypted essential coefficients act as a key to reconstruct the original ECG, which can only accessed by authorized personnel.

Random Perturbation + Noise Injection

Although encryption based on the idea of CS can achieve a computational notion of secrecy through the random projection step, it has been shown this technique is vulnerable from an information-theoretic perspective (Rachlin and Baron, 2008). To address this problem, Chou et al. (Chou et al., 2018)

propose using principal component analysis and SVD on a CS scheme, where the ECG data is encrypted at the wearable sensor by adding signal-dependent noise. They measure privacy as the mutual information between the original ECG signal and its encrypted version, and show that high classification accuracy can be achieved while providing privacy beyond computational secrecy.

Discrete Conversion + Noise Injection

Unlike the works surveyed previously, the goal of (Zare-Mirakabad et al., 2013) is to publish suitable representations of ECG data with certain privacy guarantees. To do this, Zare-Mirakabad et al. propose converting ECG time series into symbolic representations over time. They use the popular Symbolic Aggregate approXimation (SAX) to replace continuous numerical values with strings of symbols (see Figure 11

). With this new symbol representation, the proposed anonymization technique first builds an n-gram model from the complete time-series string, and then ensures that each n-gram has a minimum frequency of occurrence, similar to the

-anonymity criterion. To ensure this version of -anonymity is satisfied over the string of symbols, the authors contemplate adding fake n-grams to the original string. Experimental results on the Eamonn Discord Dataset show that (a measure of) information loss is hardly affected for values of up to 20.

Continuous Conversion + Random Perturbation

In (Chen et al., 2017) and subsequent work (Wu et al., 2019), the authors address the problem of making ECG-based biometric templates revocable, exactly as keys or passwords, a property they consider indispensable in order for ECGs to be used in practice. To enable template revocability, the common practice is to associate distinct templates with the same biometrics by perturbing them in a different manner. To protect user privacy, however, this process needs to ensure the recovery of the original biometric from its template is either infeasible or computationally hard.

Essentially, cancelable templates are obtained as random projections of a user’s ECG data block. Unlike common approaches, however, (Wu et al., 2019) puts no restrictions on the generator matrix. Accordingly, the idea is that each realization of this matrix allows cancelling their corresponding templates. Reidentification is then conducted with the multiple-signal classification algorithm (Bienvenu and Kopp, 1980), reporting rates of over 95% in the Physikalisch Technische Bundesanstalt Database.

A distinct approach is (Hong et al., 2019)

, which proposes a template-free identification system to prevent any privacy issue from compromised or stolen templates. The system converts ECG-data into images through various spatial and temporal correlations methods and uses deep-learning techniques to train a classifier. The authors conduct experiments on the Pysikalisch-Technische Bundesanstalt database and report identification rates of over 90% with sampling rates of 1 000 Hz.

Continuous Conversion + Noise Injection

Sufi et al. (Sufi et al., 2008) propose building templates of the waves P, QRS and T through cross-correlations of the ECG signal. Each of those templates are then obfuscated in a concatenated fashion with additive noise generated synthetically, so that the obfuscation of a wave serves as input to obfuscate the next wave. The upshot are noisy forms of the three waves and noisy templates thereof. All this information constitutes the key available to authorized personnel, who will be able to reconstruct the original ECG from the noisy version (which is shared or made publicly available by the patient or user themselves). Unauthorized personnel, per contra, will only have access to the noisy ECG signal, which, according to the authors, may prevent identity and attribute disclosure.

Chen et al. (Chen et al., 2019) tackle the problem of federated learning, where the goal is to train of a machine-learning classifier with ECG data distributed over a set of entities (e.g., health institutions). The authors assume the central server coordinating the learning process and updating the global model is untrusted and resorts to block-chain technology to address this issue. Differential privacy is the privacy model used to guarantee the privacy of entities’ patients. Specifically, the authors rely on the common approach of adding noise to the local gradients, and address the asynchronous problem that arises when local gradients are missing or delayed in each iteration, by adopting the solution proposed in (Zheng et al., 2017). Experimental results on the MIT-BIH ECG Arrhythmia Database (Moody and Mark, 1990) show the classification performance over ten types of cardiac arrhythmia is around 20% in test error for and 600 iterations.

Huang et al. (Huang et al., 2019) propose an authentication system that protects the privacy of ECG templates in a database with differential privacy. The authors assume the interactive setting of this privacy notion, where an analyst queries the database to obtain ECG data. Specifically, the analyst is supposed to ask for the coefficients of a Legendre polynomial, that the anonymization system utilizes to fit and compress the ECG signal. Laplace noise is calibrated to the sensitivity of those coefficients and added to them, and the noisy response is returned to the analyst. The parameter of DP therefore regulates the trade-off between user privacy and authentication accuracy, the latter aspect depending on two sources of error: the polynomial fitting approximation and the injected noise. The authors evaluate the system in the MIT-BIH ECG and MIT-BIH Noise Strees databases, reporting decent authentication accuracy. However, they appear to misunderstand how the sensitivity of the coefficients is computed and therefore their results seem to have been obtained incorrectly.

Saleheen et al. (Saleheen et al., 2016)

investigates if sensitive inferences from segments of time series data can be drawn by a dynamic Bayesian network adversary. The adversary is assumed to estimate a range of behavioral states about the user, including, for example, whether or not they are in a conversation, running, smoking and stress, at the time the data is gathered. When the adversary is likely to infer sensitive aspects of a user, the corresponding segments of data are substituted for most-plausible, non-sensitive data. To estimate the privacy provided by these substitutions of data, the authors propose a variation of the differential-privacy notion that bounds the information leaked resulting from the substitutions. In other words, the proposed metric ensures that the information leaked about a sensitive inference from a substituted segment is always bounded. Utility loss is, on the other hand, computed as the absolute difference between the probability of inference about each non-sensitive behavioral state from actual data, and the same probability from released data. Although experimental results show relatively small values of utility loss for

, the proposed solution has two main limitations: first, protection is provided only for dynamic Bayesian network adversaries; and secondly, it assumes all time-series data are available beforehand, which precludes its application in real-time scenarios.

In (Delaney et al., 2019)

, Delaney et al. investigate the ability of generative adversarial networks (GANs) to produce realistic medical time series data. Typically, the access to medical data is highly restricted due to its sensitive nature, which prevents communities from using this data for research or clinical training. The aim of this work is to generate synthetic ECG signals representative of normal ECG waveforms without concerns over privacy. On the one hand, the authors measure utility as maximum mean discrepancy (MMD) and dynamic time warping (DTW), two common approaches to estimate the dissimilarity between two probability distributions and two time series, respectively. On the other, user privacy is evaluated as the accuracy of a membership inference attack who strives to ascertain whether or not a user’s data was used for training. Experimental results on MIT-BIH Arrythmia Database 

(Moody and Mark, 1990) show that MMD favours GANs that generate a diverse range of outputs, while DTW is more robust against training instability. Although the authors report low accuracy results for such inference attacks, it is unclear if their solution would protect against more recent, sophisticated (Chen et al., 2020) versions of those attacks.

Figure 11. A time series is converted into the string “acdbbdca”. Source: https://cs.gmu.edu/~jessica/sax.htm.

5.5.4. Evaluation

The reviewed techniques measure how service functionality is degraded due to anonymization with common machine learning metrics like precision, recall and accuracy, and less frequently with the DTW and PRD quantities, which assess the similarity between original and protected time series. As for privacy, the level of protection is assessed through a variety notions and measures, including the accuracy of a membership inference attack, the parameter of differential privacy, the mutual information between the original ECG signal and its encrypted version, the probability of correct inferences on sensitive attributes with and without protection, and through a notion similar to -anonymity.

5.6. Brain Activity

Brainwaves are patterns of measurable electrical impulses emitted as a result of the interaction of billions of neurons inside the human brain. Since the first human electroencephalogram was recorded in 1924 

(Haas, 2003), both the hardware devices to measure brain activity and the analysis techniques to process these signals have significantly improved. Current technologies to measure brainwaves can be classified as invasive and noninvasive methods. Invasive methods record signals within the cortex by directly implanting electrodes near the surface of the brain (Kanaga et al., 2017). These methods are far too risky for usage under noncritical circumstances and only used in clinical applications. Instead, non-invasive methods are most frequently used and applicable to many areas other than the medical realm, such as brain-controlled interfaces. The most portable and commonly used of these techniques is electroencephalograpy (EEG), which records electrical activity through sensors placed on the scalp surface.

An EEG signal is a combination of different brainwaves occurring at different frequencies. Every type of wave carries different kinds of information, which can be used to gain insights about the current state of the brain (Almehmadi and El-Khatib, 2013). Researchers have tried to identify certain mental states associated to each brainwave. Table 1 presents a summary of the most important wave types, their respecttive frequencies, their originating location in the brain, and their associated mental state.

Brain-computer interface (BCI) technologies mostly work on continuous EEG data recordings, i.e., time series data. But there are also many applications based on the extraction of time-locked brain variations that appear in reaction to external stimuli. These variations, called event related potentials (ERPs), are widely used to detect neurological diseases. In both cases, either using ERPs or a longer EEG series, features are computed for the brainwave data-driven application built on top. These features can belong to the time and/or frequency domain, and to one or multiple channels. Examples of commonly used features include Autoregressive coefficients, Fourier and Wavelet transforms.

Brain Wave Type Freq. (Hz) Originating Location in the Brain Mental State
Gamma 30-100 Somatosensory cortex Active information processing, strong response to visual stimuli (Abo-Zahhad et al., 2015)
Beta 13-30 Both hemispheres, frontal lobe Increased alertness, anxious thinking, focused attention
Alpha 8-13 Posterior regions, both hemispheres; High amplitude waves Resting, eyes closed, no attention (Khalifa et al., 2012); Most dominant rhythm
Theta 4-8 No special location Idling, dreaming, imagining, quiet focus, memory retrieval
Delta 0.5-4 Frontal regions; High amplitude waves Dreamless and deep sleep, unconsciousness

Table 1. Overview of EEG brainwaves - based on (Almehmadi and El-Khatib, 2013) and (Abo-Zahhad et al., 2015).

5.6.1. Utility.

The utility that should be preserved when processing brainwave data is highly dependent on the application. For clinical applications, for example, the raw information could be needed for a proper diagnosis or a safe brain controlled prosthesis. In these cases, regulations like the HIPAA Privacy Rule (Assistance, 2003) are usually in place to protect personal identifiable information. When moving to other less regulated fields of application, the need for full raw EEG data is not necessarily justified. The most prominent EEG applications include user authentication, personalization of gaming experiences, and brain controlled-interfaces. In these cases, the utility to be preserved should be enough to provide a useful application, i.e., recognize the user, offer personalized options and responsive interfaces all with a tolerable error that does not hamper the security and usability of the service.

5.6.2. Threat space.

Brain activity is rich in information. It can be used to uniquely identify individuals given their unique characteristics and, in fact, several biometric systems based on brainwaves have been proposed (Gui et al., 2019). Besides, the acquisition of EEG signals raises privacy issues because brainwaves correlate, among others, with our mental states, cognitive abilities, and medical conditions (Sur and Sinha, 2009). A third party in possession of neural data could try to make inferences of private attributes that were not intentionally disclosed by the user subscribing to its service, and thus non consented. Furthermore, if this entity has the ability to control the stimuli presented to the user when collecting their brainwave activity, such as the images shown in the computer screen, it could manipulate them to obtain private data. Martinovic et al. (Martinovic et al., 2012) where among the first to demonstrate the feasibility of these type of attacks. Focusing on users of low cost EEG readers, they successfully proved that, by manipulating the images presented to the users, their EEG signals could reveal private information about, e.g., bank cards, PIN numbers, area of living, or if the user knew a particular person. In another work, Frank et al. (Frank et al., 2017) show how to obtain private data from EEG recordings but, in this case, through subliminal stimuli (short duration images embedded in visual content) that cannot even be consciously detected by users. On the positive side, contrary to other behavioral traits like keystrokes or gait, brainwaves cannot be observed from the outside, which limits the possibility to misuse observed data to identify users without consent (Korany et al., 2019). Overall, the two main threats that apply to brainwave data collected/processed by a third party service provider are re-identification and inference of private attributes. In the first case, the attack would consist of linking the brain data of the user to brain data collected by other service or available in public databases, gaining additional information about the user that can potentially be identifiable or reveal sensitive information. In the second case, the attack is oriented to uncover attributes correlated with the brainwave data, such as emotions, for which the user did not consent.

5.6.3. Anonymization Techniques

We found two works on brainwave data anonymization (Matovu et al., 2018; Yao et al., 2019), both of them targeting the privacy goal of avoiding sensitive attribute inferences, more specifically, being an alcoholic , through feature removal.

Feature removal

Matovu et al. (Matovu and Serwadda, 2016) explore how to reduce the leakage of private information from EEG user authentication templates. They assume an insider type of attacker, such as an unscrupulous database administrator, who misuses their privilege to maliciously exploit the templates. The attacker wants to infer, specifically, if the user associated with a template is an alcoholic. Their envisioned anonymization technique aims at concealing the alcoholism information while still providing good authentication accuracy. It is, therefore, an attribute protection mechanism. Conceptually, it lies on the hypothesis that different template designs (features, channels, frequencies) will have an impact on the amount of non-authentication information (emotions, health conditions) that can be inferred. The authors demonstrate this hypothesis by choosing two different templates and calculating the predictive capability to authenticate users and to determine their alcohol consumption behavior. One of the template designs shows a good trade-off between accuracy and alcoholism obfuscation, while the other template provides better accuracy at the expense of leaking alcohol consumption behavior. While these results support the hypothesis, the article does not propose a concrete and systematic methodology to design the templates.

In the same direction of feature selection, Yao et al. 

(Yao et al., 2019) propose the usage of Generative Adversarial Networks (GANs) (Goodfellow et al., 2020) to filter sensitive information out of EEG data. Their goal is to reduce the possibility of inferring alcoholism while keeping the brain activity recordings useful to detect mental tasks, specifically to predict which visual stimulus the user is looking at. The GAN-based proposed filter involves deep neural networks that perform domain transformation, that is, translating EEGs from a source domain distribution X with both desired and privacy-related features to a target domain distribution Y with desired features only. Their results after applying the filtering technique show a significant reduction in the percentage of EEG sequences from alcoholic users that can be classified as such (from 90.6% to 0.6%). At the same time, the mental task classification accuracy does not drop significantly (4.2% less). However, the original mental task classifier accuracy was not strong before filtering the privacy-sensitive features and it remains to be studied if this technique would work in other classification scenarios.

5.6.4. Evaluation

The reviewed works, similar to the proposals for anonymizing gait, evaluate the quality of inference protection by comparing the prediction accuracy for the protected attribute before and after modifying the EEG data. The metrics used for this analysis are typical machine learning metrics, including accuracy, false positive rates, and false negative rates. Similarly, the loss of utility is evaluated by measuring the reduction in classification accuracy when using the original and anonymized EEG data.

Both works used the same publicly available dataset for evaluating their anonymization proposals, the SUNY medical dataset with EEG data of 25 alcoholic subjects and 25 control subjects while looking at visual stimuli (Neurodynamics Laboratory, 1999; Karamzadeh et al., 2015).

6. Discussion

All reviewed behavioral biometric traits have in common that they are captured as a time-series tracking the change of the trait over time. Most traits, such as gait, hand motions, voice, and eye gaze are overt traits that can be observed from a distance and do not require the participation of the subject. These traits are often captured as a byproduct for other recordings, for example, video recordings. EEG and ECG on the other hand are secret traits that can mostly only be recorded by directly attaching sensors to the subject to measure them. Due to the missing requirement of user participation for the observable traits they are more prone to be abused for surveillance, or identity theft. Given that we, and others, upload/store a lot of info about ourselves, there is plenty of basis for making inferences. Therefore it is necessary to protect these traits more severely from being stolen or abused.

The utility of these traits is very diverse and is mostly unique to each trait and the application using it. It ranges from utilities such as the naturalness of a motion to the intelligibility of utterances.

Regarding their threat space, the traits are similar to each other because the instances they are recorded are increasing with the pervasiveness of digital capturing devices such as smartphones and wearables in our everyday life. Wearables are of especial interest as they are attached to the subject and can therefore allow continuous capture of behavioral data. As our literature review has shown all traits can be used for both identity and attribute inference, which then can be used in a wide variety of privacy threats such as surveillance, identity theft, or private attribute inference. The privacy goals, identity protection, and attribute protection are also the same for all the traits. However, voice has an additional privacy goal in which the content of the speech should be made unintelligible.

For the techniques (see Table 2) that we reviewed, we found that most of them fall into the category of continuous conversion, followed by feature removal and noise injection. Next are random perturbation and discrete conversion, with most discrete conversion methods aiming at template protection. Coarsening is the category with the least amount of methods. We observe several differences for the categories of our taxonomy, for the removal methods we find that the removal is not directly reversible, however, due to the high redundancy in behavioral biometric data it still might be possible to reconstruct the removed data. For the conversion methods, we often observe that the parameter space for the anonymizations is often rather small, making it possible that an attacker can link clear and anonymized data by brute forcing the parameters when the anonymization technique is known. In general, we find that the reversibility of conversion techniques should be evaluated. For noise injection techniques we find that the strong dependency both temporal and physiological a problem since they allow can be used to filter out the noise.

With regard to the techniques providing differential privacy, we have observed that none of them can be used continuously over time without completely compromising patient or user privacy. The reason lies in that the privacy budget is necessarily finite, which means, by the sequential composition property of differential privacy (McSherry, 2009), that it will be consumed completely at some time instant. Surprisingly, this appears to be in contradiction to the intended use of most of the applications where differential privacy is guaranteed, namely, continuous monitoring in healthcare scenarios, and identification and authentication services (which clearly are not single-use services). In that respect, the use of related privacy notions intended for continuous observations (e.g., -event differential privacy (Kellaris et al., 2014)) may come in handy.

We found the most anonymization methods for voice and the least for EEG. For the traits touch, thermal, lip-facial, and motion we could not find any methods. We made the observation that most methods do not manipulate the temporal aspect of their data. Notable exceptions are Hirose et al. (Hirose et al., 2019) and Maiti et al. (Maiti et al., 2016). Since all traits result in time series data manipulating the temporal order or time differences between events could lead to some general anonymization techniques which work for multiple traits. For attribute protection we find anonymizing intrinsic attributes (e.g., age, sex) to be difficult as it is not clear which part of the behavioral data is relevant for these attributes. Further, we noticed a lack of even basic understanding of users’ privacy awareness and concerns about behavioral privacy. These are necessary to design protection techniques that consider user needs and requirements.

We found that the evaluation methodology between the traits and methods is rather similar. In general, an inference/recognition system is being used on the clear and on the anonymized data and then the difference in accuracy is reported, often without retraining the inference system on the anonymized data. We find this methodology too simple as the underlying assumption is that the attacker is not aware of the anonymization. Besides training the recognition model on the anonymized data the evaluations should also consider an attacker that actively tries to reverse the anonymization and knows the anonymization technique and its parameters. To allow the comparison between multiple methods the attacker models should be made explicit and common, similar to attacker models in cryptography. Only a handful of papers compare their own methods to that of others and due to the differences in attacker models and data sources, it is difficult to compare their results to one another. We also found that there are not many approaches (Zhang et al., 2020; Qian et al., 2018) to formalize the privacy of behavioral biometric anonymization methods and most of the evaluations rely on empirical privacy estimations. Another problem is that the evaluation methodology is too close to the recognition system evaluation methodology which seeks to infer persons in a large dataset with poor data quality, while an anonymization method should also work on a small group size with high data quality. We believe that the lack of available datasets (see Table 3) is one of the main problems which keeps the less researched behavioral biometric traits back.

[innerwidth=3.0cm,height=1.3cm]MethodTrait Voice Gait Hand motion Eye-Gaze Brain
activity
Heartbeat
continuous conversion (Jin et al., 2009) (Pobar and Ipsic, 2014) (Sojka et al., 2014) (Justin et al., 2015) (Abou-Zleikha et al., 2015) (Přibil et al., 2018) (Bahmaninezhad et al., 2018) (Fang et al., 2019) (Keskin et al., 2019) (Faundez-Zanuy et al., 2015) (Abad et al., 2016) (Lopez-Otero et al., 2017) (Magariños et al., 2017) (Aloufi et al., 2019) (Srivastava et al., 2020) (Canuto et al., 2014) (Kondo et al., 2013) (Kondo and Sakurai, 2014) (Qian et al., 2018) (Qian et al., 2021) (Srivastava et al., 2020) (Agrawal and Narayanan, 2011) (Ivasic-Kos et al., 2014) (Hirose et al., 2019) (Maiorana et al., 2011) (Chen et al., 2017) (Wu et al., 2019) (Hong et al., 2019) (Sufi et al., 2008) (Chen et al., 2019) (Huang et al., 2019) (Saleheen et al., 2016) (Delaney et al., 2019)
discrete conversion (Pathak and Raj, 2012) (Portelo et al., 2013) (Portelo et al., 2014) (Billeb et al., 2015) (Sae-Bae and Memon, 2013) (Leinonen et al., 2017) (Migdal and Rosenberger, 2019) (Vassallo et al., 2017) (Figueiredo et al., 2016) (Zare-Mirakabad et al., 2013)
feature removal (Parthasarathi et al., 2009) (Parthasarathi et al., 2011) (Wyatt et al., 2007) (Zhang and Yaginuma, 2012) (Nelus and Martin, 2018) (Nelus and Martin, 2019) (Cohen-Hadria et al., 2019) (Jourdan et al., 2018) (Matovu and Serwadda, 2016) (Yao et al., 2019) (Zaghouani et al., 2017) (Zaghouani et al., 2017) (Mahmoud, 2016) (Sufi et al., 2008)
coarsening (Maiti et al., 2016) (Vassallo et al., 2017)
noise injection (Tamesue and Saeki, 2014) (Hashimoto et al., 2016) (Hamm, 2017) (Ohshio et al., 2018) (Vaidya and Sherr, 2019) (Tieu et al., 2017) (Tieu et al., 2019) (Tieu et al., 2019) (Matovu et al., 2018) (Migdal and Rosenberger, 2019) (Monaco and Tappert, 2017) (Steil et al., 2019) (Bozkir et al., 2020) (Liu et al., 2019)
random perturbation (Parthasarathi et al., 2013) (Mtibaa et al., 2018) (Hoang et al., 2015) (Maiorana et al., 2011) (Goubaru et al., 2014) (Maiti et al., 2016) (Vassallo et al., 2017) (Liu et al., 2018) (Djelouat et al., 2018) (Chou et al., 2018)
Table 2. An overview of all found methods classified by trait and method. Papers that propose multiple methods can appear in multiple rows. Papers that combine multiple methods are marked the following: plus noise injection, plus random perturbation, plus discrete conversion.
Name Participants Published Source Trait
TIMIT 630 1993 (Garofolo et al., 1992) Voice
Albayzin 164 1993 (Moreno et al., 1993) Voice
YOHO 137 1994 (Campbell, Joseph and Higgins, Alan, 1994) Voice
BioSecureID 400 2009 (Fierrez et al., 2009) Voice
Billeb et al. 701 2014 (Billeb et al., 2015) Voice
Librispeech 1166 2015 (Panayotov et al., 2015) Voice
RSR2015 300 2015 (Larcher et al., 2012) Voice
VoxCeleb 1251 2018 (Nagrani et al., 2017) Voice
CASIA-B 124 2005 (Zheng et al., 2011) Gait
i3DPost 8 2009 (Gkalelis et al., 2009) Gait
BEHAVE 125 2010 (Blunsden and Fisher, 2010) Gait
OU-ISIR 200 2012 (Makihara et al., 2012) Gait
MCYT baseline corpus 330 2003 (Ortega-Garcia et al., 2003) Hand motion
SVC2004 100 2004 (Yeung et al., 2004) Hand motion
GREYC 133 2009 (Giot et al., 2009) Hand motion
MNIST 500 2012 (Deng, 2012) Hand motion
Web-based keystroke 83 2012 (Giot et al., 2012) Hand motion
SUNY EEG database 50 1999 (Neurodynamics Laboratory, 1999) Brain activity
MIT-BIH ECG Arrhythmia 47 1979 (Moody and Mark, 1990) Heartbeat
MIT-BIH Noise Stress Test 2 1984 (Moody and Muldrow, 1990) Heartbeat
Physikalisch Technische Bundesanstalt 290 1995 (R, 1995) Heartbeat


Table 3. An overview of used behavioral biometric datasets.

7. Concluding Remarks

Anonymizing behavioral biometric data is an important task for protecting people’s privacy. In our literature review, we found many different behavioral traits that need to be considered and developed a taxonomy to classify the anonymization techniques that can be applied to them by the type of data transformation they perform. While voice anonymization is already a research field with many available solutions, most behavioral biometric traits only got little attention in the literature and therefore anonymizing them remains an open research question. We further found that most anonymization techniques are only evaluated rudimentarily with the assumption of a weak attacker. Improving the evaluation methodology is therefore another open research question. Lastly, we find that the temporal aspect of the data was mostly neglected, both for offering privacy for data streams and for anonymizing the data.

References

  • A. Abad, A. Ortega, A. Teixeira, C. G. Mateo, C. D. M. Hinarejos, F. Perdigão, F. Batista, and N. Mamede (Eds.) (2016) Advances in speech and language technologies for iberian languages. Lecture Notes in Computer Science, Vol. 10077, Springer International Publishing. External Links: ISBN 978-3-319-49168-4 978-3-319-49169-1, Document, Link Cited by: §5.1.4, Table 2.
  • M. Abo-Zahhad, S. M. Ahmed, and S. N. Abbas (2015) State-of-the-art methods and future perspectives for personal recognition based on electroencephalogram signals. IET Biometrics 4 (3), pp. 179–190. External Links: Document Cited by: Table 1, Table 1.
  • M. Abou-Zleikha, Z. Tan, M. G. Christensen, and S. H. Jensen (2015) A discriminative approach for speaker selection in speaker de-identification systems. In 2015 23rd European Signal Processing Conference (EUSIPCO), pp. 2102–2106. External Links: Document, ISBN 978-0-9928626-3-3, Link Cited by: §5.1.4, §5.1.5, Table 2.
  • R. A. Abrams, D. E. Meyer, and S. Kornblum (1989) Speed and accuracy of saccadic eye movements: characteristics of impulse variability in the oculomotor system.. Journal of Experimental Psychology: Human Perception and Performance 15 (3), pp. 529. External Links: Document Cited by: §5.4.
  • C. Ackad, A. Clayphan, R. M. Maldonado, and J. Kay (2012) Seamless and continuous user identification for interactive tabletops using personal device handshaking and body tracking. In CHI '12 Extended Abstracts on Human Factors in Computing Systems, pp. 1775–1780. External Links: Document Cited by: §2.3.
  • D. Adams, A. Bah, C. Barwulor, N. Musaby, K. Pitkin, and E. M. Redmiles (2018) Ethics emerging: the story of privacy and security perceptions in virtual reality. In Fourteenth Symposium on Usable Privacy and Security (SOUPS 2018), Baltimore, MD, pp. 427–442. External Links: ISBN 978-1-939133-10-6, Link Cited by: §5.4.2.
  • P. Agrawal and P. J. Narayanan (2011) Person de-identification in videos. IEEE Transactions on Circuits and Systems for Video Technology 21 (3), pp. 299–310. External Links: ISSN 1051-8215, 1558-2205, Document, Link Cited by: §5.2.3, Table 2.
  • A. Almehmadi and K. El-Khatib (2013) The state of the art in electroencephalogram and access control. In 2013 Third International Conference on Communications and Information Technology (ICCIT), Beirut, Lebanon, pp. 49–54. External Links: Document Cited by: §5.6, Table 1.
  • R. Aloufi, H. Haddadi, and D. Boyle (2019) Emotionless: privacy-preserving speech analysis for voice assistants. External Links: 1908.03632 Cited by: §5.1.4, Table 2.
  • A. Alzubaidi and J. Kalita (2016) Authentication of smartphone users using behavioral biometrics. IEEE Communications Surveys & Tutorials 18 (3), pp. 1998–2026. External Links: Document Cited by: §2.2, §2.3, §3.3.
  • H. C. Assistance (2003) Summary of the hipaa privacy rule. Office for Civil Rights. Cited by: §5.6.1.
  • [12] T. Bäckström, O. Räsänen, A. Zewoudie, and P. P. Zarazaga Introduction to Speech Processing. Note: WebPageAccessed: 02.02.2021 External Links: Link Cited by: §5.1.
  • A. Bahill, M. R. Clark, and L. Stark (1975) The main sequence, a tool for studying human eye movements. Mathematical Biosciences 24 (3-4), pp. 191–204. External Links: Document Cited by: §5.4.1.
  • F. Bahmaninezhad, C. Zhang, and J. Hansen (2018) Convolutional neural network based speaker de-identification. In Odyssey 2018 The Speaker and Language Recognition Workshop, pp. 255–260. External Links: Document, Link Cited by: §5.1.4, Table 2.
  • D. Bales, P. A. Tarazaga, M. Kasarda, D. Batra, A. G. Woolard, J. D. Poston, and V. V. N. S. Malladi (2016) Gender classification of walkers via underfloor accelerometer measurements. IEEE Internet of Things Journal 3 (6), pp. 1259–1266. External Links: Document Cited by: §2.2.
  • S. P. Banerjee and D. Woodard (2012) Biometric authentication and identification using keystroke dynamics: a survey.

    Journal of Pattern Recognition Research

    7 (1), pp. 116–139.
    External Links: Document Cited by: §2.2.
  • [17] BehavioSec Continuous Authentication Through Behavioral Biometrics. Note: WebpageAccessed: 17.05.2019 External Links: Link Cited by: §3.3.
  • J. Bennet and S. Lanning (2007) The netflix prize. In Proceedings of the KDD Cup Workshop 2007, pp. 3–6. External Links: Link Cited by: footnote 7.
  • S. Berkovsky, R. Taib, I. Koprinska, E. Wang, Y. Zeng, J. Li, and S. Kleitman (2019) Detecting personality traits using eye-tracking data. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–12. External Links: Document Cited by: §5.4.2.
  • G. Bienvenu and L. Kopp (1980) Adaptivity to background noise spatial coherence for high resolution passive methods. In ICASSP '80. IEEE International Conference on Acoustics, Speech, and Signal Processing, Vol. 5, pp. 307–310. External Links: Document Cited by: §5.5.3.
  • S. Billeb, C. Rathgeb, H. Reininger, K. Kasper, and C. Busch (2015) Biometric template protection for speaker recognition based on universal background models. IET Biometrics 4 (2), pp. 116–126. External Links: ISSN 2047-4938, 2047-4946, Document, Link Cited by: §5.1.4, Table 2, Table 3.
  • S. Blunsden and R. Fisher (2010) The behave video dataset: ground truthed video for multi-person behavior classification. Annals of the BMVA 4 (1-12), pp. 4. Cited by: Table 3.
  • J. Bonneau, C. Herley, P. C. van Oorschot, and F. Stajano (2012) The quest to replace passwords: a framework for comparative evaluation of web authentication schemes. In 2012 IEEE Symposium on Security and Privacy, pp. 553–567. External Links: Document Cited by: §3.3.
  • J. Bonneau, C. Herley, P. C. van Oorschot, and F. Stajano (2015) Passwords and the evolution of imperfect authentication. Communications of the ACM 58 (7), pp. 78–87. External Links: Document Cited by: §3.3.
  • Z. Boraston and S. Blakemore (2007) The application of eye-tracking technology in the study of autism. The Journal of Physiology 581 (3), pp. 893–898. External Links: Document Cited by: §5.4.2.
  • E. Bozkir, O. Günlü, W. Fuhl, R. F. Schaefer, and E. Kasneci (2020) Differential privacy for eye tracking with temporal correlations. External Links: 2002.08972 Cited by: §5.4.3, §5.4.3, §5.4.4, §5.4.4, Table 2.
  • A. Buriro, Z. Akhtar, B. Crispo, and F. D. Frari (2016) Age, gender and operating-hand estimation on smart mobile devices. In 2016 International Conference of the Biometrics Special Interest Group (BIOSIG), pp. 1–5. External Links: Document Cited by: §2.2.
  • W.M. Campbell, D. Sturim, and D.A. Reynolds (2006) Support vector machines using GMM supervectors for speaker verification. IEEE Signal Processing Letters 13 (5), pp. 308–311. External Links: ISSN 1070-9908, Document, Link Cited by: §5.1.
  • Campbell, Joseph and Higgins, Alan (1994) YOHO speaker verification corpus. Linguistic Data Consortium. External Links: Document Cited by: Table 3.
  • E. J. Candes, J. Romberg, and T. Tao (2006) Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information Theory 52 (2), pp. 489–509. External Links: Document Cited by: §5.5.3.
  • A. M. P. Canuto, F. Pintro, and M. C. Fairhurst (2014) An effective template protection method for face and voice cancellable identification. International Journal of Hybrid Intelligent Systems 11 (3), pp. 157–166. External Links: ISSN 18758819, 14485869, Document, Link Cited by: §5.1.4, Table 2.
  • J. Chauhan, Y. Hu, S. Seneviratne, A. Misra, A. Seneviratne, and Y. Lee (2017) BreathPrint: breathing acoustics-based user authentication. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services, pp. 278–291. External Links: Document Cited by: §3.3.
  • J. Chauhan, S. Seneviratne, Y. Hu, A. Misra, A. Seneviratne, and Y. Lee (2018)

    Breathing-based authentication on resource-constrained IoT devices using recurrent neural networks

    .
    Computer 51 (5), pp. 60–67. External Links: Document Cited by: §3.3.
  • D. Chen, N. Yu, Y. Zhang, and M. Fritz (2020) GAN-leaks: a taxonomy of membership inference attacks against generative models. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, External Links: Document Cited by: §5.5.3.
  • P. Chen, S. Wu, and J. Hsieh (2017) A cancelable biometric scheme based on multi-lead ECGs. In 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 3497–3500. External Links: Document Cited by: §5.5.3, Table 2.
  • X. Chen, X. Wang, and K. Yang (2019) Asynchronous blockchain-based privacy-preserving training framework for disease diagnosis. In 2019 IEEE International Conference on Big Data (Big Data), pp. 5469–5473. External Links: Document Cited by: §5.5.3, Table 2.
  • C. Chou, E. Chang, H. Li, and A. Wu (2018) Low-complexity privacy-preserving compressive analysis using subspace-based dictionary for ECG telemonitoring system. IEEE Transactions on Biomedical Circuits and Systems 12 (4), pp. 801–811. External Links: Document Cited by: §5.5.3, Table 2.
  • A. Cohen-Hadria, M. Cartwright, B. McFee, and J. P. Bello (2019) Voice anonymization in urban sound recordings. In 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1–6. External Links: Document, ISBN 978-1-72810-824-7, Link Cited by: §5.1.4, Table 2.
  • C. Conati, C. Merten, S. Amershi, and K. Muldner (2007) Using eye-tracking data for high-level user modeling in adaptive interfaces. In AAAI, pp. 1614–1617. Cited by: §5.4.1.
  • E. D. Cristofaro (2021) A critical overview of privacy in machine learning. IEEE Security & Privacy 19 (4), pp. 19–27. External Links: Document Cited by: §1.
  • A. Dantcheva, P. Elia, and A. Ross (2016) What else does your biometric data reveal? a survey on soft biometrics. IEEE Transactions on Information Forensics and Security 11 (3), pp. 441–467. External Links: Document Cited by: §2.2, §2.3.
  • M. C. T. de Carvalho Bruno, M. A. C. Vilela, and C. A. B. M. de Oliveira (2013) Study on dermatoses and their prevalence in groups of confirmed alcoholic individuals in comparison to a non-alcoholic group of individuals. Anais Brasileiros de Dermatologia 88 (3), pp. 368–375. External Links: Document Cited by: §1.
  • A. L. S. de Lima, L. J. W. Evers, T. Hahn, L. Bataille, J. L. Hamilton, M. A. Little, Y. Okuma, B. R. Bloem, and M. J. Faber (2017) Freezing of gait and fall detection in parkinson’s disease using wearable sensors: a systematic review. Journal of Neurology 264 (8), pp. 1642–1654. External Links: Document Cited by: §3.3, §3.5.
  • Y. de Montjoye, C. A. Hidalgo, M. Verleysen, and V. D. Blondel (2013) Unique in the crowd: the privacy bounds of human mobility. Scientific Reports 3 (1), pp. 1376. External Links: Document Cited by: §1.
  • N. Dehak, P. J. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet (2011) Front-end factor analysis for speaker verification. IEEE Transactions on Audio, Speech, and Language Processing 19 (4), pp. 788–798. External Links: ISSN 1558-7916, 1558-7924, Document, Link Cited by: §5.1.
  • A. M. Delaney, E. Brophy, and T. E. Ward (2019) Synthesis of realistic ecg using generative adversarial networks. External Links: 1909.09150 Cited by: §5.5.3, Table 2.
  • L. Deng (2012) The MNIST database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Processing Magazine 29 (6), pp. 141–142. External Links: ISSN 1053-5888, Document, Link Cited by: Table 3.
  • J. Derwenskus, J. C. Rucker, A. Serra, J. S. Stahl, D. L. Downey, N. L. Adams, and R. J. Leigh (2005) Abnormal eye movements predict disability in MS: two-year follow-up. Annals of the New York Academy of Sciences 1039 (1), pp. 521–523. External Links: Document Cited by: §5.4.2.
  • C. Deuser, S. Passmann, and T. Strufe (2020) Browsing unicity: on the limits of anonymizing web tracking data. In 2020 IEEE Symposium on Security and Privacy (SP), pp. 279–292. External Links: Document Cited by: §1.
  • H. Djelouat, X. Zhai, M. A. Disi, A. Amira, and F. Bensaali (2018) System-on-chip solution for patients biometric: a compressive sensing-based approach. IEEE Sensors Journal 18 (23), pp. 9629–9639. External Links: Document Cited by: §5.5.3, Table 2.
  • A. T. Duchowski (2017) Eye tracking methodology. Springer International Publishing. External Links: Document Cited by: §5.4.3.
  • C. Dwork, F. McSherry, K. Nissim, and A. Smith (2017) Calibrating noise to sensitivity in private data analysis. Journal of Privacy and Confidentiality 7 (3), pp. 17–51. External Links: Document Cited by: §5.4.3.
  • C. Dwork and A. Roth (2013) The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science 9 (3-4), pp. 211–407. External Links: Document Cited by: §5.4.3.
  • C. Dwork, A. Smith, T. Steinke, and J. Ullman (2017) Exposed! a survey of attacks on private data. Annual Review of Statistics and Its Application 4 (1), pp. 61–84. External Links: Document Cited by: §2.2.
  • S. Eberz, G. Lovisotto, A. Patane, M. Kwiatkowska, V. Lenders, and I. Martinovic (2018) When your fitness tracker betrays you: quantifying the predictability of biometric features across contexts. In 2018 IEEE Symposium on Security and Privacy (SP), pp. 889–905. External Links: Document Cited by: §5.4.1.
  • K. E. Emam, E. Jonker, L. Arbuckle, and B. Malin (2011) A systematic review of re-identification attacks on health data. PLoS ONE 6 (12), pp. e28071. External Links: Document Cited by: §2.2.
  • F. Ertam (2019) An effective gender recognition approach using voice data via deeper LSTM networks. Applied Acoustics 156, pp. 351–358. External Links: ISSN 0003-682X, Document, Link Cited by: §5.1.2.
  • U. Ettinger, V. Kumari, X. A. Chitnis, P. J. Corr, T. J. Crawford, D. G. Fannon, S. O’Ceallaigh, A. L. Sumich, V. C. Doku, and T. Sharma (2004) Volumetric neural correlates of antisaccade eye movements in first-episode psychosis. American Journal of Psychiatry 161 (10), pp. 1918–1921. External Links: Document Cited by: §5.4.2.
  • F. Fang, X. Wang, J. Yamagishi, I. Echizen, M. Todisco, N. Evans, and J. Bonastre (2019) Speaker anonymization using x-vector and neural waveform models. 10th ISCA Speech Synthesis Workshop. External Links: Document, Link Cited by: §5.1.4, Table 2.
  • M. Faundez-Zanuy, E. Sesa-Nogueras, and S. Marinozzi (2015) Speaker identification experiments under gender de-identification. In 2015 International Carnahan Conference on Security Technology (ICCST), pp. 1–6. External Links: Document, ISBN 978-1-4799-8690-3 978-1-4799-8691-0, Link Cited by: §5.1.4, Table 2.
  • J. Fierrez, J. Galbally, J. Ortega-Garcia, M. R. Freire, F. Alonso-Fernandez, D. Ramos, D. T. Toledano, J. Gonzalez-Rodriguez, J. A. Siguenza, J. Garrido-Salas, E. Anguiano, G. Gonzalez-de-Rivera, R. Ribalda, M. Faundez-Zanuy, J. A. Ortega, V. Cardeñoso-Payo, A. Viloria, C. E. Vivaracho, Q. I. Moro, J. J. Igarza, J. Sanchez, I. Hernaez, C. Orrite-Uruñuela, F. Martinez-Contreras, and J. J. Gracia-Roche (2009) BiosecurID: a multimodal biometric database. Pattern Analysis and Applications 13 (2), pp. 235–246. External Links: Document Cited by: Table 3.
  • L. S. Figueiredo, B. Livshits, D. Molnar, and M. Veanes (2016) Prepose: privacy, security, and reliability for gesture-based programming. In 2016 IEEE Symposium on Security and Privacy (SP), pp. 122–137. External Links: Document, ISBN 978-1-5090-0824-7, Link Cited by: §5.3.3, Table 2.
  • M. Frank, T. Hwu, S. Jain, R. Knight, I. Martinovic, P. Mittal, D. Perito, and D. Song (2017) Subliminal probing for private information via eeg-based bci devices. External Links: 1312.6052 Cited by: §2.2, §5.6.2.
  • B. Galai and C. Benedek (2015) Feature selection for lidar-based gait recognition. In 2015 International Workshop on Computational Intelligence for Multimedia Understanding (IWCIM), pp. 1–5. External Links: Document Cited by: §5.2.2.
  • A. García-Blanco, L. Salmerón, M. Perea, and L. Livianos (2014) Attentional biases toward emotional images in the different episodes of bipolar disorder: an eye-tracking study. Psychiatry Research 215 (3), pp. 628–633. External Links: Document Cited by: §5.4.2.
  • J. Garofolo, L. Lamel, W. Fisher, J. Fiscus, D. Pallett, N. Dahlgren, and V. Zue (1992) TIMIT acoustic-phonetic continuous speech corpus. Linguistic Data Consortium. Cited by: Table 3.
  • R. Giot, M. El-Abed, and C. Rosenberger (2009) GREYC keystroke: a benchmark for keystroke dynamics biometric systems. In 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems, pp. 1–6. External Links: Document, ISBN 978-1-4244-5019-0, Link Cited by: Table 3.
  • R. Giot, M. El-Abed, and C. Rosenberger (2012) Web-based benchmark for keystroke dynamics biometric systems: a statistical analysis. In 2012 Eighth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, pp. 11–15. External Links: Document, ISBN 978-1-4673-1741-2 978-0-7695-4712-1, Link Cited by: Table 3.
  • N. Gkalelis, H. Kim, A. Hilton, N. Nikolaidis, and I. Pitas (2009) The i3dpost multi-view and 3d human action/interaction database. In 2009 Conference for Visual Media Production, pp. 159–168. External Links: Document Cited by: Table 3.
  • I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio (2020) Generative adversarial networks. Communications of the ACM 63 (11), pp. 139–144. External Links: Document Cited by: §5.6.3.
  • Y. Goubaru, Y. Yamazaki, T. Miyazaki, and T. Ohki (2014) A consideration on a common template-based biometric cryptosystem using on-line signatures. In 2014 IEEE 3rd Global Conference on Consumer Electronics (GCCE), pp. 131–135. External Links: Document, ISBN 978-1-4799-5145-1, Link Cited by: §5.3.3, §5.3.4, Table 2.
  • E. Griffiths, S. Assana, and K. Whitehouse (2018) Privacy-preserving image processing with binocular thermal cameras. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1 (4), pp. 1–25. External Links: Document Cited by: §2.3.
  • Q. Gui, M. V. Ruiz-Blondet, S. Laszlo, and Z. Jin (2019) A survey on brain biometrics. ACM Computing Surveys 51 (6), pp. 1–38. External Links: Document Cited by: §2.2, §3.3, §5.6.2.
  • L. F. Haas (2003) Hans berger (1873-1941), richard caton (1842-1926), and electroencephalography. Journal of Neurology, Neurosurgery & Psychiatry 74 (1), pp. 9–9. External Links: Document Cited by: §5.6.
  • J. Hamm (2017) Enhancing utility and privacy with noisy minimax filters. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6389–6393. External Links: Document, ISBN 978-1-5090-4117-6, Link Cited by: §5.1.4, Table 2.
  • K. Harezlak and P. Kasprowski (2018) Application of eye tracking in medicine: a survey, research issues and challenges. Computerized Medical Imaging and Graphics 65, pp. 176–190. External Links: Document Cited by: §5.4.1.
  • K. Hashimoto, J. Yamagishi, and I. Echizen (2016) Privacy-preserving sound to degrade automatic speaker verification performance. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5500–5504. External Links: Document, ISBN 978-1-4799-9988-0, Link Cited by: §5.1.4, Table 2.
  • J. Henriksen-Bulmer and S. Jeary (2016) Re-identification attacks—a systematic literature review. International Journal of Information Management 36 (6), pp. 1184–1192. External Links: Document Cited by: §2.2.
  • E. H. Hess and J. M. Polt (1960) Pupil size as related to interest value of visual stimuli. Science 132 (3423), pp. 349–350. External Links: Document Cited by: §5.4.2.
  • Y. Hirose, K. Nakamura, N. Nitta, and N. Babaguchi (2019) Anonymization of gait silhouette video by perturbing its phase and shape components. In 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp. 1679–1685. External Links: Document, ISBN 978-1-72813-248-8, Link Cited by: §5.2.3, Table 2, §6.
  • T. Hoang, D. Choi, and T. Nguyen (2015) Gait authentication on mobile phone using biometric cryptosystem and fuzzy commitment scheme. International Journal of Information Security 14 (6), pp. 549–560. External Links: ISSN 1615-5262, 1615-5270, Document, Link Cited by: §5.2.3, Table 2.
  • G. Hogben (2010) ENISA briefing: behavioural biometrics. Computational Intelligence. Cited by: §3.3.
  • P. S. Holzman, L. R. Proctor, and D. W. Hughes (1973) Eye-tracking patterns in schizophrenia. Science 181 (4095), pp. 179–181. External Links: Document Cited by: §5.4.2.
  • P. Hong, J. Hsiao, C. Chung, Y. Feng, and S. Wu (2019) ECG biometric recognition: template-free approaches based on deep learning. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 2633–2636. External Links: Document Cited by: §5.5.3, Table 2.
  • S. M. Hossain, A. A. Ali, Md. M. Rahman, E. E. D. Epstein, A. Kennedy, K. Preston, A. Umbricht, Y. Chen, and S. Kumar (2014) Identifying drug (cocaine) intake events from acute physiological response in the presence of free-living physical activity. In IPSN-14 Proceedings of the 13th International Symposium on Information Processing in Sensor Networks, IPSN ’14, pp. 71–82. External Links: Document Cited by: §5.5.1.
  • J. Huafeng and W. Shuo (2017) Voice-based determination of physical and emotional characteristics of users. Note: U.S. Patent 10 096 319B1 Cited by: §3.5.
  • P. Huang, L. Guo, M. Li, and Y. Fang (2019) Practical privacy-preserving ECG-based authentication for IoT-based healthcare. IEEE Internet of Things Journal 6 (5), pp. 9200–9210. External Links: Document Cited by: §5.5.3, Table 2.
  • J. T. Hutton, J. Nagel, and R. B. Loewenson (1984) Eye tracking dysfunction in alzheimer-type dementia. Neurology 34 (1), pp. 99–99. External Links: Document Cited by: §5.4.2.
  • M. Inoue, M. Nishiyama, and Y. Iwai (2020) Gender classification using the gaze distributions of observers on privacy-protected training images. In

    Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications

    ,
    pp. 149–156. External Links: Document Cited by: §2.2.
  • M. Ivasic-Kos, A. Iosifidis, A. Tefas, and I. Pitas (2014) Person de-identification in activity videos. In 2014 37th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pp. 1294–1299. External Links: Document, ISBN 978-953-233-077-9 978-953-233-081-6, Link Cited by: §5.2.3, Table 2.
  • Q. Jin, A. R. Toth, T. Schultz, and A. W. Black (2009) Voice convergin: speaker de-identification by voice transformation. In 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3909–3912. External Links: Document, ISBN 978-1-4244-2353-8, Link Cited by: §5.1.4, Table 2.
  • I. Joe Louis Paul, S. Sasirekha, S. Uma Maheswari, K. A. M. Ajith, S. M. Arjun, and S. Athesh Kumar (2019) Eye gaze tracking-based adaptive e-learning for enhancing teaching and learning in virtual classrooms. In Information and Communication Technology for Competitive Strategies, pp. 165–176. External Links: ISBN 978-981-13-0586-3 Cited by: §3.3.
  • T. Jourdan, A. Boutet, and C. Frindel (2018) Toward privacy in IoT mobile devices for activity recognition. In Proceedings of the 15th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services, pp. 155–165. External Links: Document, ISBN 978-1-4503-6093-7, Link Cited by: §5.2.3, Table 2.
  • T. Justin, V. Struc, S. Dobrisek, B. Vesnicer, I. Ipsic, and F. Mihelic (2015) Speaker de-identification using diphone recognition and speech synthesis. In 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–7. External Links: Document, ISBN 978-1-4799-6026-2, Link Cited by: §5.1.4, Table 2.
  • E. G. M. Kanaga, R. M. Kumaran, M. Hema, R. G. Manohari, and T. A. Thomas (2017) An experimental investigations on classifiers for brain computer interface (BCI) based authentication. In 2017 International Conference on Trends in Electronics and Informatics (ICEI), pp. 1–6. External Links: Document Cited by: §5.6.
  • N. Karamzadeh, Y. Ardeshirpour, M. Kellman, F. Chowdhry, A. Anderson, D. Chorlian, E. Wegman, and A. Gandjbakhche (2015) Relative brain signature: a population-based feature extraction procedure to identify functional biomarkers in the brain of alcoholics. Brain and Behavior 5 (7), pp. e00335. External Links: Document Cited by: 2nd item, §5.6.4.
  • C. Katsini, Y. Abdrabou, G. E. Raptis, M. Khamis, and F. Alt (2020) The role of eye gaze in security and privacy applications: survey and future HCI research directions. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–21. External Links: Document Cited by: §2.2, §5.4.1.
  • G. Kellaris, S. Papadopoulos, X. Xiao, and D. Papadias (2014) Differentially private event sequences over infinite streams. Proceedings of the VLDB Endowment 7 (12), pp. 1155–1166. External Links: Document Cited by: §6.
  • G. Keskin, T. Lee, C. Stephenson, and O. H. Elibol (2019)

    Measuring the effectiveness of voice conversion on speaker identification and automatic speech recognition systems

    .
    External Links: 1905.12531 Cited by: §5.1.4, Table 2.
  • W. Khalifa, A. Salem, and M. Roushdy (2012) A Survey of EEG Based User Authentication Schemes. In The 8th International Conference on INFOrmatics and Systems (INFOS2012), pp. 55–60. Cited by: Table 1.
  • B. Kitchenham (2004) Procedures for performing systematic reviews. Technical report Technical Report TR/SE-0401, Keele University, Keele, UK. Cited by: §1, Figure 1, §2.3.
  • K. Kondo, T. Komiyama, and S. Kashiwada (2013) Towards gender-dependent babble maskers for speech privacy protection. In 2013 Ninth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, pp. 275–278. External Links: Document, ISBN 978-0-7695-5120-3, Link Cited by: §5.1.4, Table 2.
  • K. Kondo and H. Sakurai (2014) Gender-dependent babble maskers created from multi-speaker speech for speech privacy protection. In 2014 Tenth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, pp. 251–254. External Links: Document, ISBN 978-1-4799-5390-5 978-1-4799-5389-9, Link Cited by: §5.1.4, Table 2.
  • B. Korany, C. R. Karanam, H. Cai, and Y. Mostofi (2019) XModal-id: using wifi for through-wall person identification from candidate video footage. In The 25th Annual International Conference on Mobile Computing and Networking, pp. 1–15. External Links: Document Cited by: §5.6.2.
  • M. Kosinski, D. Stillwell, and T. Graepel (2013) Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences 110 (15), pp. 5802–5805. External Links: Document Cited by: §1.
  • K. Krejtz, A. T. Duchowski, A. Niedzielska, C. Biele, and I. Krejtz (2018) Eye tracking cognitive load using pupil diameter and microsaccades with fixed gaze. PLOS ONE 13 (9), pp. e0203629. External Links: Document Cited by: §5.4.2.
  • J. L. Kröger, O. H. Lutz, and F. Müller (2020) What does your gaze reveal about you? on the privacy implications of eye tracking. In Privacy and Identity Management. Data for Better Living: AI and Privacy, Springer, pp. 226–241. External Links: Document Cited by: §3.5, §5.4.2.
  • C. A. Kuechenmeister, P. H. Linton, T. V. Mueller, and H. B. White (1977) Eye tracking in relation to age, sex, and illness. Archives of General Psychiatry 34 (5), pp. 578–579. External Links: Document Cited by: §5.4.2.
  • A. Larcher, K. A. Lee, B. Ma, and H. Li (2012) The rsr2015: database for text-dependent speaker verification using multiple pass-phrases. 13th Annual Conference of the International Speech Communication Association 2012, INTERSPEECH 2012 2, pp. 1578–1581. Cited by: Table 3.
  • R. Lazzeretti, J. Guajardo, and M. Barni (2012) Privacy preserving ECG quality evaluation. In Proceedings of the on Multimedia and security - MM&Sec '12, MM&Sec ’12, pp. 165–174. External Links: Document Cited by: §5.5.2.
  • J. Leinonen, P. Ihantola, and A. Hellas (2017) Preventing keystroke based identification in open data sets. In Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale, pp. 101–109. External Links: Document, ISBN 978-1-4503-4450-0, Link Cited by: §5.3.3, Table 2.
  • D. L. Levy, A. B. Sereno, D. C. Gooding, and G. A. O’Driscoll (2010) Eye tracking dysfunction in schizophrenia: characterization and pathophysiology. In Behavioral Neurobiology of Schizophrenia and Its Treatment, pp. 311–347. External Links: Document Cited by: §5.4.2.
  • Y. Liang, S. Samtani, B. Guo, and Z. Yu (2020)

    Behavioral biometrics for continuous authentication in the internet-of-things era: an artificial intelligence perspective

    .
    IEEE Internet of Things Journal 7 (9), pp. 9128–9143. External Links: Document Cited by: §2.2.
  • J. Lim and A. Oppenheim (1978) All-pole modeling of degraded speech. IEEE Transactions on Acoustics, Speech, and Signal Processing 26 (3), pp. 197–210. External Links: Document Cited by: §5.1.4.
  • A. Liu, L. Xia, A. Duchowski, R. Bailey, K. Holmqvist, and E. Jain (2019) Differential privacy for eye-tracking data. In Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications, pp. 1–10. External Links: Document Cited by: §5.4.3, §5.4.3, §5.4.4, Table 2.
  • T. Y. Liu, K. J. Lin, and H. C. Wu (2018)

    ECG data encryption then compression using singular value decomposition

    .
    IEEE Journal of Biomedical and Health Informatics 22 (3), pp. 707–713. External Links: Document Cited by: §5.5.3, Table 2.
  • P. Lopez-Otero, C. Magariños, L. Docio-Fernandez, E. Rodriguez-Banga, D. Erro, and C. Garcia-Mateo (2017) Influence of speaker de-identification in depression detection. IET Signal Processing 11 (9), pp. 1023–1030. External Links: ISSN 1751-9675, 1751-9683, Document, Link Cited by: §5.1.4, Table 2.
  • C. Magariños, P. Lopez-Otero, L. Docio-Fernandez, E. Rodriguez-Banga, D. Erro, and C. Garcia-Mateo (2017) Reversible speaker de-identification using pre-trained transformation functions. Computer Speech & Language 46, pp. 36–52. External Links: ISSN 08852308, Document, Link Cited by: §5.1.4, Table 2.
  • A. Mahfouz, T. M. Mahmoud, and A. S. Eldin (2017) A survey on behavioral biometric authentication on smartphones. Journal of Information Security and Applications 37, pp. 28–37. External Links: Document Cited by: §2.2, §2.3, §3.3.
  • S. S. Mahmoud (2016) A generalised wavelet packet-based anonymisation approach for ECG security application. Security and Communication Networks 9 (18), pp. 6137–6147. External Links: Document Cited by: §5.5.3, Table 2.
  • E. Maiorana, P. Campisi, and A. Neri (2011) Bioconvolving: cancelable templates for a multi-biometrics signature recognition system. In 2011 IEEE International Systems Conference, pp. 495–500. External Links: Document, ISBN 978-1-4244-9494-1, Link Cited by: §5.3.3, §5.3.3, Table 2.
  • A. Maiti, O. Armbruster, M. Jadliwala, and J. He (2016) Smartwatch-based keystroke inference attacks and context-aware protection mechanisms. In Proceedings of the 11th ACM on Asia Conference on Computer and Communications Security, pp. 795–806. External Links: Document, ISBN 978-1-4503-4233-9, Link Cited by: §5.3.3, §5.3.3, Table 2, §6.
  • P. Majaranta and A. Bulling (2014) Eye tracking and eye-based human–computer interaction. In Human–Computer Interaction Series, pp. 39–65. External Links: Document Cited by: §5.4.1.
  • Y. Makihara, H. Mannami, A. Tsuji, Md. A. Hossain, K. Sugiura, A. Mori, and Y. Yagi (2012) The OU-ISIR gait database comprising the treadmill dataset. IPSJ Transactions on Computer Vision and Applications 4, pp. 53–62. External Links: Document Cited by: Table 3.
  • M. S. Manikandan and S. Dandapat (2008) ECG distortion measures and their effectiveness. In 2008 First International Conference on Emerging Trends in Engineering and Technology, pp. 705–710. External Links: Document Cited by: §5.5.3.
  • I. Martinovic, D. Davies, M. Frank, D. Perito, T. Ros, and D. Song (2012) On the feasibility of side-channel attacks with brain-computer interfaces. In 21st USENIX Security Symposium (USENIX Security 12), Bellevue, WA, pp. 143–158. External Links: ISBN 978-931971-95-9, Link Cited by: §5.6.2.
  • R. Matovu, A. Serwadda, D. Irakiza, and I. Griswold-Steiner (2018) Jekyll and hyde: on the double-faced nature of smart-phone sensor noise injection. In 2018 International Conference of the Biometrics Special Interest Group (BIOSIG), pp. 1–6. External Links: Document, ISBN 978-3-88579-676-3, Link Cited by: §5.2.3, §5.2.4, §5.6.3, Table 2.
  • R. Matovu and A. Serwadda (2016) Your substance abuse disorder is an open secret! gleaning sensitive personal information from templates in an EEG-based authentication system. In 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS), pp. 1–7. External Links: Document Cited by: §5.6.3, Table 2.
  • G. Matthews, W. Middleton, B. Gilmartin, and M. A. Bullimore (1991) Pupillary diameter and cognitive load.. Journal of Psychophysiology. Cited by: §5.4.2.
  • F. D. McSherry (2009) Privacy integrated queries: an extensible platform for privacy-preserving data analysis. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of data, pp. 19–30. External Links: Document Cited by: §6.
  • W. Meng, D. S. Wong, S. Furnell, and J. Zhou (2015) Surveying the development of biometric user authentication on mobile phones. IEEE Communications Surveys & Tutorials 17 (3), pp. 1268–1293. External Links: Document Cited by: §2.2, §2.3, §3.3.
  • D. Migdal and C. Rosenberger (2019) My behavior is my privacy & secure password !. In 2019 International Conference on Cyberworlds (CW), pp. 299–307. External Links: Document, ISBN 978-1-72812-297-7, Link Cited by: §5.3.3, Table 2.
  • D. Migdal and C. Rosenberger (2019) Keystroke dynamics anonymization system. In Proceedings of the 16th International Joint Conference on e-Business and Telecommunications, pp. 448–455. External Links: Document, ISBN 978-989-758-378-0, Link Cited by: §5.3.3, Table 2.
  • J. V. Monaco and C. C. Tappert (2017) Obfuscating keystroke time intervals to avoid identification and impersonation. External Links: 1609.07612 Cited by: §5.3.3, §5.3.4, Table 2.
  • G. B. Moody and Muldrow (1990) A noise stress test for arrhythmia detectors. Computers in Cardiology 11, pp. 381–384. Cited by: Table 3.
  • G.B. Moody and R.G. Mark (1990) The MIT-BIH arrhythmia database on CD-ROM and software for use with it. In [1990] Proceedings Computers in Cardiology, pp. 185–188. External Links: Document Cited by: §5.5.3, §5.5.3, Table 3.
  • A. Moreno, D. Poch, A. Bonafonte, E. Lleida, J. Llisterri, J. Mariño, and C. Nadeu (1993) Albayzin speech database: design of the phonetic corpus. Vol. 1, pp. . Cited by: Table 3.
  • A. Mtibaa, D. Petrovska-Delacretaz, and A. B. Hamida (2018) Cancelable speaker verification system based on binary gaussian mixtures. In 2018 4th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), pp. 1–6. External Links: Document, ISBN 978-1-5386-5239-8, Link Cited by: §5.1.4, Table 2.
  • A. Nagrani, J. S. Chung, and A. Zisserman (2017) VoxCeleb: a large-scale speaker identification dataset. In Interspeech 2017, Vol. abs/1706.08612. External Links: Document, Link Cited by: Table 3.
  • A. Nelus and R. Martin (2018) Gender discrimination versus speaker identification through privacy-aware adversarial feature extraction. In Speech Communication; 13th ITG-Symposium, Cited by: §5.1.4, Table 2.
  • A. Nelus and R. Martin (2019) Privacy-aware feature extraction for gender discrimination versus speaker identification. In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 671–674. External Links: Document, ISBN 978-1-4799-8131-1, Link Cited by: §5.1.4, Table 2.
  • S. D. M. C. Neurodynamics Laboratory (1999) EEG database. External Links: Link Cited by: 2nd item, §5.6.4, Table 3.
  • [143] Nymi Always On Authantication. Note: WebpageAccessed: 01.06.2019 External Links: Link Cited by: §3.3.
  • I. Odinaka, P. Lai, A. D. Kaplan, J. A. O'Sullivan, E. J. Sirevaag, and J. W. Rohrbaugh (2012) ECG biometric recognition: a comparative analysis. IEEE Transactions on Information Forensics and Security 7 (6), pp. 1812–1824. External Links: Document Cited by: §5.5.
  • Y. Ohshio, H. Adachi, K. Iwai, T. Nishiura, and Y. Yamashita (2018) Active speech obscuration with speaker-dependent human speech-like noise for speech privacy. In 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp. 1252–1255. External Links: Document, ISBN 978-988-14768-5-2, Link Cited by: §5.1.4, Table 2.
  • J. Ortega-Garcia, J. Fierrez-Aguilar, D. Simon, J. Gonzalez, M. Faundez-Zanuy, V. Espinosa, A. Satue, I. Hernaez, J.-J. Igarza, C. Vivaracho, D. Escudero, and Q.-I. Moro (2003) MCYT baseline corpus: a bimodal biometric database. IEE Proceedings - Vision, Image, and Signal Processing 150 (6), pp. 395. External Links: ISSN 1350245X, Document, Link Cited by: Table 3.
  • V. Panayotov, G. Chen, D. Povey, and S. Khudanpur (2015) Librispeech: an ASR corpus based on public domain audio books. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5206–5210. External Links: Document Cited by: Table 3.
  • J. Pansiot, D. Stoyanov, D. McIlwraith, B. P.L. Lo, and G. Z. Yang (2007) Ambient and wearable sensor fusion for activity recognition in healthcare monitoring systems. In 4th International Workshop on Wearable and Implantable Body Sensor Networks (BSN 2007), pp. 208–212. External Links: Document Cited by: §3.3.
  • S. H. K. Parthasarathi, H. Bourlard, and D. Gatica-Perez (2013) Wordless sounds: robust speaker diarization using privacy-preserving audio representations. IEEE Transactions on Audio, Speech, and Language Processing 21 (1), pp. 85–98. External Links: ISSN 1558-7916, 1558-7924, Document, Link Cited by: §5.1.4, Table 2.
  • S. H. K. Parthasarathi, H. Bourlard, and D. Gatica-Perez (2011) LP residual features for robust, privacy-sensitive speaker diarization. In Interspeech, Cited by: §5.1.4, §5.1.4, Table 2.
  • S. H. K. Parthasarathi, M. Magimai.-Doss, D. Gatica-Perez, and H. Bourlard (2009) Speaker change detection with privacy-preserving audio cues. In Proceedings of the 2009 international conference on Multimodal interfaces - ICMI-MLMI '09, pp. 343. External Links: Document, ISBN 978-1-60558-772-1, Link Cited by: §5.1.4, Table 2.
  • M. A. Pathak and B. Raj (2012) Privacy-preserving speaker verification as password matching. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1849–1852. External Links: Document, ISBN 978-1-4673-0046-9 978-1-4673-0045-2 978-1-4673-0044-5, Link Cited by: §5.1.4, Table 2.
  • D. I. Perrett, S. N. Talamas, P. Cairns, and A. J. Henderson (2020) Skin color cues to human health: carotenoids, aerobic fitness, and body fat. Frontiers in Psychology 11. External Links: Document, Link Cited by: §1.
  • K. Pfeuffer, M. J. Geiger, S. Prange, L. Mecke, D. Buschek, and F. Alt (2019) Behavioural biometrics in VR. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–12. External Links: Document Cited by: §2.3.
  • R. Plamondon and S.N. Srihari (2000) Online and off-line handwriting recognition: a comprehensive survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 22 (1), pp. 63–84. External Links: Document Cited by: §5.3.
  • K. Plarre, A. Raij, S. M. Hossain, A. A. Ali, M. Nakajima, M. Al’absi, E. Ertin, T. Kamarck, S. Kumar, M. Scott, D. Siewiorek, A. Smailagic, and L. E. Wittmers (2011) Continuous inference of psychological stress from sensory measurements collected in the natural environment. In Proceedings of the 10th ACM/IEEE International Conference on Information Processing in Sensor Networks, pp. 97–108. Cited by: §5.5.1.
  • M. Pobar and I. Ipsic (2014) Online speaker de-identification using voice transformation. In 2014 37th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pp. 1264–1267. External Links: Document, ISBN 978-953-233-077-9 978-953-233-081-6, Link Cited by: §5.1.4, Table 2.
  • B. Pogorelc, Z. Bosnić, and M. Gams (2011) Automatic recognition of gait-related health problems in the elderly using machine learning. Multimedia Tools and Applications 58 (2), pp. 333–354. External Links: Document Cited by: §3.3.
  • F. E. Pollick, J. W. Kay, K. Heim, and R. Stringer (2005) Gender recognition from point-light walkers.. Journal of Experimental Psychology: Human Perception and Performance 31 (6), pp. 1247–1265. External Links: ISSN 1939-1277, 0096-1523, Document, Link Cited by: §1, §5.2.2, §5.2.
  • A. Poole and L. J. Ball (2006) Eye tracking in HCI and usability research. In Encyclopedia of Human Computer Interaction, pp. 211–219. External Links: Document Cited by: §5.4.1.
  • J. Portelo, A. Abad, B. Raj, and I. Trancoso (2013) Secure binary embeddings of front-end factor analysis for privacy preserving speaker verification. pp. 2494–2498. Cited by: §5.1.4, Table 2.
  • J. Portelo, B. Raj, A. Abad, and I. Trancoso (2014) Privacy-preserving speaker verification using secure binary embeddings. In 2014 37th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pp. 1268–1272. External Links: Document, ISBN 978-953-233-077-9 978-953-233-081-6, Link Cited by: §5.1.4, Table 2.
  • J. Přibil, A. Přibilová, and J. Matoušek (2018) Evaluation of speaker de-identification based on voice gender and age conversion. Journal of Electrical Engineering 69 (2), pp. 138–147. External Links: ISSN 1339-309X, Document, Link Cited by: §5.1.4, Table 2.
  • J. Qian, H. Du, J. Hou, L. Chen, T. Jung, and X. Li (2018) Hidebehind: enjoy voice input with voiceprint unclonability and anonymity. In Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems, pp. 82–94. External Links: Document, ISBN 978-1-4503-5952-8, Link Cited by: §5.1.4, Table 2.
  • J. Qian, H. Du, J. Hou, L. Chen, T. Jung, and X. Li (2021) Speech sanitizer: speech content desensitization and voice anonymization. IEEE Transactions on Dependable and Secure Computing, pp. 1–1. External Links: ISSN 1545-5971, 1941-0018, 2160-9209, Document, Link Cited by: §5.1.4, Table 2.
  • J. Qian, F. Han, J. Hou, C. Zhang, Y. Wang, and X. Li (2018) Towards privacy-preserving speech data publishing. In IEEE INFOCOM 2018 - IEEE Conference on Computer Communications, pp. 1079–1087. External Links: Document, ISBN 978-1-5386-4128-6, Link Cited by: §5.1.5, §6.
  • B. R (1995) Nutzung der ekg-signaldatenbank cardiodat der ptb über das internet. Biomedizinische Technik 40 (1). Cited by: Table 3.
  • Y. Rachlin and D. Baron (2008) The secrecy of compressed sensing measurements. In 2008 46th Annual Allerton Conference on Communication, Control, and Computing, pp. 813–817. External Links: Document Cited by: §5.5.3.
  • V. Rastogi and S. Nath (2010) Differentially private aggregation of distributed time-series with transformation and encryption. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of data, pp. 735–746. External Links: Document Cited by: §5.4.3.
  • S. Reddy, S. Nalluri, S. Kunisetti, S. Ashok, and B. Venkatesh (2019) Content-based movie recommendation system using genre correlation. In Smart Intelligent Computing and Applications, pp. 391–397. External Links: ISBN 978-981-13-1927-3 Cited by: §3.3.
  • D. A. Reynolds, T. F. Quatieri, and R. B. Dunn (2000) Speaker verification using adapted gaussian mixture models. Digital Signal Processing 10 (1), pp. 19–41. External Links: ISSN 10512004, Document, Link Cited by: §5.1.
  • D. A. Reynolds (1995) Speaker identification and verification using gaussian mixture speaker models. Speech Communication 17 (1), pp. 91–108. External Links: ISSN 01676393, Document, Link Cited by: §5.1.
  • S. Ribaric, A. Ariyaeeinia, and N. Pavesic (2016) De-identification for privacy protection in multimedia content: a survey. Signal Processing: Image Communication 47, pp. 131–151. External Links: ISSN 09235965, Document, Link Cited by: §2.2.
  • N. Sae-Bae, N. Memon, K. Isbister, and K. Ahmed (2014) Multitouch gesture-based authentication. IEEE Transactions on Information Forensics and Security 9 (4), pp. 568–582. External Links: Document Cited by: §5.3.
  • N. Sae-Bae and N. Memon (2013) A simple and effective method for online signature verification. pp. 1–12. Cited by: §5.3.3, Table 2.
  • N. Saleheen, S. Chakraborty, N. Ali, M. M. Rahman, S. M. Hossain, R. Bari, E. Buder, M. Srivastava, and S. Kumar (2016) MSieve: differential behavioral privacy in time series of mobile sensor data. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ’16, New York, NY, USA, pp. 706–717. External Links: Document Cited by: §5.5.3, Table 2.
  • M. Sherman, G. Clark, Y. Yang, S. Sugrim, A. Modig, J. Lindqvist, A. Oulasvirta, and T. Roos (2014) User-generated free-form gestures for authentication. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services, MobiSys ’14, New York, NY, USA, pp. 176–189. External Links: Document, ISBN 9781450327930, Link Cited by: §5.3.
  • Y. N. Singh and P. Gupta (2008) ECG to individual identification. In 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems, pp. 1–8. External Links: Document Cited by: §5.5.
  • D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur (2018) X-vectors: robust DNN embeddings for speaker recognition. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5329–5333. External Links: Document, ISBN 978-1-5386-4658-8, Link Cited by: §5.1.
  • P. Sojka, A. Horák, I. Kopeček, and K. Pala (Eds.) (2014) Text, speech and dialogue: 17th international conference, TSD 2014, brno, czech republic, september 8-12, 2014. proceedings. Lecture Notes in Computer Science, Vol. 8655, Springer International Publishing. External Links: ISBN 978-3-319-10815-5 978-3-319-10816-2, Document, Link Cited by: §5.1.4, Table 2.
  • B. M. L. Srivastava, N. Vauquier, M. Sahidullah, A. Bellet, M. Tommasi, and E. Vincent (2020) Evaluating voice conversion-based privacy protection against informed attackers. External Links: Document, Link Cited by: §5.1.4, §5.1.4, §5.1.5, Table 2.
  • J. Steil, I. Hagestedt, M. X. Huang, and A. Bulling (2019) Privacy-aware eye tracking using differential privacy. In Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications, ETRA ’19, New York, NY, USA. External Links: Document, ISBN 9781450367097, Link Cited by: 1st item, §5.4.3, §5.4.3, §5.4.3, §5.4.4, §5.4.4, Table 2.
  • T. Stöckel, R. Jacksteit, M. Behrens, R. Skripitz, R. Bader, and A. Mau-Moeller (2015) The mental representation of the human gait in young and older adults. Frontiers in Psychology 6, pp. 943. External Links: Link, Document, ISSN 1664-1078 Cited by: Figure 7, §5.2.
  • F. Sufi, S. Mahmoud, and I. Khalil (2008) A new ECG obfuscation method: a joint feature extraction & corruption approach. In 2008 International Conference on Information Technology and Applications in Biomedicine, pp. 334–337. External Links: Document Cited by: §5.5.3, Table 2.
  • F. Sufi, S. Mahmoud, and I. Khalil (2008) A wavelet based secured ECG distribution technique for patient centric cpproach. In 2008 5th International Summer School and Symposium on Medical Devices and Biosensors, pp. 301–304. External Links: Document Cited by: §5.5.3, Table 2.
  • S. Sur and V. Sinha (2009) Event-related potential: an overview. Industrial Psychiatry Journal 18 (1), pp. 70. External Links: Document Cited by: §3.5, §5.6.2.
  • T. Tamesue and T. Saeki (2014) Sound masking for achieving speech privacy with parametric acoustic array speaker. In 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems (SCIS) and 15th International Symposium on Advanced Intelligent Systems (ISIS), pp. 1134–1137. External Links: Document, ISBN 978-1-4799-5955-6, Link Cited by: §5.1.4, Table 2.
  • P. S. Teh, A. B. J. Teoh, and S. Yue (2013) A survey of keystroke dynamics biometrics. The Scientific World Journal 2013, pp. 1–24. External Links: Document Cited by: §2.2, §3.3.
  • P. S. Teh, N. Zhang, A. B. J. Teoh, and K. Chen (2016) A survey on touch dynamics authentication in mobile devices. Computers & Security 59, pp. 210–235. External Links: Document Cited by: §3.3.
  • J. Tian, C. Qu, W. Xu, and S. Wang (2013) KinWrite: handwriting-based authentication using kinect. 93, pp. 94. Cited by: §5.3.
  • N. T. Tieu, H. H. Nguyen, F. Fang, J. Yamagishi, and I. Echizen (2019) An RGB gait anonymization model for low-quality silhouettes. In 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp. 1686–1693. External Links: Document, ISBN 978-1-72813-248-8, Link Cited by: §5.2.3, Table 2.
  • N. T. Tieu, H. H. Nguyen, H. Nguyen-Son, J. Yamagishi, and I. Echizen (2017) An approach for gait anonymization using deep learning. In 2017 IEEE Workshop on Information Forensics and Security (WIFS), pp. 1–6. External Links: Document, ISBN 978-1-5090-6769-5, Link Cited by: §5.2.3, Table 2.
  • N. T. Tieu, H. H. Nguyen, H. Nguyen-Son, J. Yamagishi, and I. Echizen (2019) Spatio-temporal generative adversarial network for gait anonymization. Journal of Information Security and Applications 46, pp. 307–319. External Links: ISSN 22142126, Document, Link Cited by: §5.2.3, Table 2.
  • Q. N. Tran, B. P. Turnbull, and J. Hu (2021) Biometrics and privacy-preservation: how do they evolve?. IEEE Open Journal of the Computer Society 2, pp. 179–191. External Links: Document Cited by: §2.2.
  • N. F. Troje (2002) Decomposing biological motion: a framework for analysis and synthesis of human gait patterns. Journal of Vision 2 (5), pp. 2. External Links: ISSN 1534-7362, Document, Link Cited by: §1.
  • [196] TypingDNA Note: WebpageAccessed: 01.06.2019 External Links: Link Cited by: §3.3.
  • T. Vaidya and M. Sherr (2019) You talk too much: limiting privacy exposure via voice input. In 2019 IEEE Security and Privacy Workshops (SPW), pp. 84–91. External Links: Document, ISBN 978-1-72813-508-3, Link Cited by: §5.1.4, Table 2.
  • G. Vassallo, T. V. hamme, D. Preuveneers, and W. Joosen (2017) Privacy-preserving behavioral authentication on smartphones. In Proceedings of the First International Workshop on Human-centered Sensing, Networking, and Systems, pp. 1–6. External Links: Document, ISBN 978-1-4503-5480-6, Link Cited by: §5.3.3, §5.3.3, §5.3.3, Table 2.
  • [199] Voice Vault VoiceVault Voice Biometric Authentication. Note: WebpageAccessed: 01.06.2019 External Links: Link Cited by: §3.3.
  • C. Wan, L. Wang, and V. V. Phoha (2019) A survey on gait recognition. ACM Computing Surveys 51 (5), pp. 1–35. External Links: ISSN 0360-0300, 1557-7341, Document, Link Cited by: §1, §2.2, §3.3, §5.2.2, §5.2.
  • S. Wang, M. Jiang, X. M. Duchesne, E. A. Laugeson, D. P. Kennedy, R. Adolphs, and Q. Zhao (2015) Atypical visual saliency in autism spectrum disorder quantified through model-based eye tracking. Neuron 88 (3), pp. 604–616. External Links: Document Cited by: §5.4.2.
  • L. Willenborg and T. de Waal (2001) Elements of statistical disclosure control. Springer New York, New York. External Links: Document Cited by: §2.1.
  • S. Wu, P. Chen, A. L. Swindlehurst, and P. Hung (2019) Cancelable biometric recognition with ECGs: subspace-based approaches. IEEE Transactions on Information Forensics and Security 14 (5), pp. 1323–1336. External Links: Document Cited by: §5.5.3, §5.5.3, §5.5, Table 2.
  • D. Wyatt, T. Choudhury, and J. Bilmes (2007) Conversation detection and speaker segmentation in privacy-sensitive situated speech data. In Eighth Annual Conference of the International Speech Communication Association, Cited by: §5.1.4, Table 2.
  • S. Yacoub, S. Simske, X. Lin, and J. Burns (2003) Recognition of emotions in interactive voice response systems. In Eighth European conference on speech communication and technology, Cited by: §3.5, §5.1.2.
  • R. V. Yampolskiy and V. Govindaraju (2010) Taxonomy of behavioural biometrics. In Behavioral Biometrics for Human Identification, pp. 1–43. External Links: Document Cited by: §2.3.
  • Q. Yang, T. Wang, N. Su, S. Xiao, and Z. Kapoula (2012) Specific saccade deficits in patients with alzheimer’s disease at mild to moderate stage and in patients with amnestic mild cognitive impairment. AGE 35 (4), pp. 1287–1298. External Links: Document Cited by: §3.5, §5.4.2.
  • Y. Yao, J. Plested, T. Gedeon, Y. Liu, and Z. Wang (2019) Improved techniques for building EEG feature filters. In 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–6. External Links: Document Cited by: §5.6.3, §5.6.3, Table 2.
  • M. Ye, J. Shen, G. Lin, T. Xiang, L. Shao, and S. C.H. Hoi (2021) Deep learning for person re-identification: a survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–1. External Links: Document Cited by: §2.2.
  • D. Yeung, H. Chang, Y. Xiong, S. George, R. Kashi, T. Matsumoto, and G. Rigoll (2004) SVC2004: first international signature verification competition. In Biometric Authentication, D. Zhang and A. K. Jain (Eds.), Vol. 3072, pp. 16–22. Note: Series Title: Lecture Notes in Computer Science External Links: ISBN 978-3-540-22146-3 978-3-540-25948-0, Document, Link Cited by: Table 3.
  • G. Yovel and A. J. O’Toole (2016) Recognizing people in motion. Trends in Cognitive Sciences 20 (5), pp. 383–395. External Links: ISSN 13646613, Document, Link Cited by: §1, §5.2.
  • E. K. Zaghouani, A. Benzina, and R. Attia (2017) ECG based authentication for e-healthcare systems: towards a secured ECG features transmission. In 2017 13th International Wireless Communications and Mobile Computing Conference (IWCMC), pp. 1777–1783. External Links: Document Cited by: §5.5.3, Table 2.
  • E. K. Zaghouani, A. Benzina, and R. Attia (2017) ECG biometrie template protection based on secure sketch scheme. In 2017 25th International Conference on Software, Telecommunications and Computer Networks (SoftCOM), pp. 1–5. External Links: Document Cited by: §5.5.3, Table 2.
  • M. Zare-Mirakabad, F. Kaveh-Yazdy, and M. Tahmasebi (2013) Privacy preservation by k-anonymizing ngrams of time series. In 2013 10th International ISC Conference on Information Security and Cryptology (ISCISC), Vol. , pp. 1–6. External Links: Document Cited by: §5.5.3, Table 2.
  • G. Zhang, S. Ni, and P. Zhao (2020) Enhancing privacy preservation in speech data publishing. IEEE Internet of Things Journal 7 (8), pp. 7357–7367. External Links: ISSN 2327-4662, 2372-2541, Document, Link Cited by: §5.1.5, §6.
  • N. Zhang and Y. Yaginuma (2012) A privacy-preserving and language-independent speaking detecting and speaker diarization approach for spontaneous conversation using microphones. In 2012 IEEE 11th International Conference on Signal Processing, pp. 499–502. External Links: Document, ISBN 978-1-4673-2197-6 978-1-4673-2196-9 978-1-4673-2195-2, Link Cited by: §5.1.4, Table 2.
  • J. Zheng, J. Zhang, S. Danioko, H. Yao, H. Guo, and C. Rakovski (2020) A 12-lead electrocardiogram database for arrhythmia research covering more than 10,000 patients. Scientific Data 7 (1). External Links: Document Cited by: §5.5.
  • N. Zheng, A. Paloski, and H. Wang (2016) An efficient user verification system using angle-based mouse movement biometrics. ACM Transactions on Information and System Security 18 (3), pp. 1–27. External Links: Document Cited by: §3.3.
  • S. Zheng, J. Zhang, K. Huang, R. He, and T. Tan (2011) Robust view transformation model for gait recognition. In 2011 18th IEEE International Conference on Image Processing, pp. 2073–2076. External Links: Document Cited by: Table 3.
  • S. Zheng, Q. Meng, T. Wang, W. Chen, N. Yu, Z. Ma, and T. Liu (2017)

    Asynchronous stochastic gradient descent with delay compensation

    .
    In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, pp. 4120–4129. External Links: Document Cited by: §5.5.3.
  • Y. Zhong and Y. Deng (2015) A survey on keystroke dynamics biometrics: approaches, advances, and evaluations. In Gate to Computer Science and Research, pp. 1–22. External Links: Document Cited by: §5.3.
  • M. Zohaib (2018) Dynamic difficulty adjustment (DDA) in computer games: a review. Advances in Human-Computer Interaction 2018, pp. 1–12. External Links: Document Cited by: §3.3.