Log In Sign Up

VindiCo: Privacy Safeguard Against Adaptation Based Spyware in Human-in-the-Loop IoT

by   Salma Elmalaki, et al.

Personalized IoT adapts their behavior based on contextual information, such as user behavior and location. Unfortunately, the fact that personalized IoT adapts to user context opens a side-channel that leaks private information about the user. To that end, we start by studying the extent to which a malicious eavesdropper can monitor the actions taken by an IoT system and extract users' private information. In particular, we show two concrete instantiations (in the context of mobile phones and smart homes) of a new category of spyware which we refer to as Context-Aware Adaptation Based Spyware (SpyCon). Experimental evaluations show that the developed SpyCon can predict users' daily behavior with an accuracy of 90.3 devoted to introducing VindiCo, a software mechanism designed to detect and mitigate possible SpyCon. Being new spyware with no known prior signature or behavior, traditional spyware detection that is based on code signature or app behavior is not adequate to detect SpyCon. Therefore, VindiCo proposes a novel information-based detection engine along with several mitigation techniques to restrain the ability of the detected SpyCon to extract private information. By having general detection and mitigation engines, VindiCo is agnostic to the inference algorithm used by SpyCon. Our results show that VindiCo reduces the ability of SpyCon to infer user context from 90.3 (accuracy based on random guesses) with negligible execution overhead.


page 5

page 8

page 15

page 17


Preserving Privacy in Personalized Models for Distributed Mobile Services

The ubiquity of mobile devices has led to the proliferation of mobile se...

A Context-aware Framework for Detecting Sensor-based Threats on Smart Devices

Sensors (e.g., light, gyroscope, accelerometer) and sensing-enabled appl...

PETGEN: Personalized Text Generation Attack on Deep Sequence Embedding-based Classification Models

What should a malicious user write next to fool a detection model? Ident...

Data-driven Design of Context-aware Monitors for Hazard Prediction in Artificial Pancreas Systems

Medical Cyber-physical Systems (MCPS) are vulnerable to accidental or ma...

Phishing Mitigation Techniques: A Literature Survey

Email is a channel of communication which is considered to be a confiden...

1. Introduction

Context-aware systems provide personalized services that are adaptive according to user context and surrounding environments. These pervasive systems have enabled a multitude of applications in several IoT sectors including smart cities (Ma et al., 2017), health care (Larson et al., 2011), smart classrooms (Elmalaki et al., 2021; Taherisadr et al., 2021), and automotive systems (Li et al., 2018; Elmalaki et al., 2018b; Ahadi-Sarkani and Elmalaki, 2021). However, these enhanced capabilities come at the expense of privacy weaknesses (Han et al., 2017; Sicari et al., 2015; Sadeghi et al., 2015; Elmalaki et al., 2019) and sometimes lack of fairness in context-adaptation (Elmalaki, 2021).

Figure 1.

A cartoon that illustrates the flow of information in a personalized IoT. The IoT uses devices (typically on the edge) in order to sense and infer the human state. The IoT then adapts to the human state by applying some actions. (top) the proposed personalized IoT Spyware monitors the actions taken by the personalized IoT and reverse engineers it in order to estimate the human state (bottom) the proposed VindiCo framework which prevents modifies the IoT actions in a fashion that prevents any spyware from estimating the human state.

Unfortunately, previous work ignores the fact that decisions taken by IoT are often triggered by human beings. For example, a smart NEST thermostat333NEST - can automatically turn on and off the HVAC equipment based on users’ presence. Such a coupling between human behaviors and decisions taken by IoT system can open a side channel, leaking sensitive information about users. As pictorially illustrated in Figure 1, a human-in-the-loop (HITL) IoT utilizes edge devices (e.g., mobile phones and wearables) to sense and infer human states. Such states are then used by the IoT to produce actions and adapt its behavior to match the human state. Resting on this observation, this paper raises the following questions: (1) Can an eavesdropper who monitors the actions taken by the IoT be able to reverse engineer these actions in order to estimate human states and leak sensitive information? (2) Can we develop new software mechanisms that can detect and mitigate such privacy leaks?

To explore the answers, we introduce a new type of spyware that exploits privacy leaks in context-aware adaptations which we call SpyCon. Besides, we purpose VindiCo, a novel detection and mitigation engine that is designed to protect against privacy leaks due to the actions taken by HILT IoT but observed by SpyCon. Contrary to traditional malware detection approaches, VindiCo does not rely on prior information about the code signatures or run-time behaviors of known malware. Instead, VindiCo is designed to be generic and agnostic to the implementation of SpyCon.

1.1. Related Work

Context Monitoring Malware on Mobile Platforms: Mobile systems are becoming an integral component in multiple IoT systems due to their sensing capabilities (Nahrstedt et al., 2016). While mobile users benefit from these sensing technologies, there are increasing privacy and security concerns. The permission systems on both Android and iOS become the first line of defense to protect users from leaking sensitive information. However, the traditional grant-all-or-none policy allows third-party apps to have all permissions (Hornyack et al., 2011). Even worse, most users have trouble with realizing the potential privacy hazards after granting such permissions. For example, though seems innocuous, ACCESS_WIFI_STATE becomes a heavily privacy intrusive permission since local MAC address can serve as a unique device identifier (Achara et al., 2014). Felt et al.(Felt et al., 2012) shows that as little as 17% of users pay attention to the permissions during app installation phase.

Different side-channel attacks have been proposed, for example, using inertial sensors and touch screen to infer user input such as passwords (Han et al., 2012; Miluzzo et al., 2012; Owusu et al., 2012; Marquardt et al., 2011). Besides, we witnessed how to exploit cellular signal strengths, air pressure, or power consumption for locations (Michalevsky et al., 2015; Ho et al., 2015; Michalevsky et al., 2015), gyroscope for eavesdropping conversations (Michalevsky et al., 2014), system-level aggregate statistics for user’s real world identity (Zhou et al., 2013), and the state of shared memory for foreground apps, and even, activity transition sequences (Nan et al., 2015). There is a trend that malicious apps are adapting to wearable devices (Raij et al., 2011). For example, MoLe (Wang et al., 2015) exploits the wrist motion derived from smartwatches to infer keystroke inputs. So far we’ve provided many examples showing “Your apps are watching you” (Kane and Thurm, 2010) in a broad spectrum which a majority of users will never realize, and for sure “These aren’t the droid you’re looking for” (Hornyack et al., 2011).

Contrary to the aforementioned side-channel attacks, we consider a spyware which does not have access to sensor information like inertial or gyroscope sensors, a spyware which can monitor only the actions that are triggered—by HITL IoT—based on changes in these sensory data.

Malware Detection Techniques: Several techniques have been proposed for malware detection and can be broadly categorized into two groups. (1)Code signature-based approach (Egele et al., 2011; Grace et al., 2012; Enck et al., 2011; Lu et al., 2012; Arzt et al., 2014) detects stealthy behavior based on the code flow. (2)Behavior-based approach (Yan and Yin, 2012; Zhu et al., 2009; Kang et al., 2011; Enck et al., 2014) performs information leakage detection in execution time, but tackling the issue from different layers of an operating system. DroidRanger (Zhou et al., 2012) points out that an app can download binary payload in the runtime, which code-signature based approach can never diagnose its intention but raise a warning of potential hazard. Nevertheless, VindiCo is distinct from the above techniques by considering the fact that a malicious app can learn sensitive information based on the adaptations made by HITL IoT.

Malware Mitigation Techniques: Different mitigation techniques have been proposed, including sensory value perturbation (Beresford et al., 2011; Hornyack et al., 2011; Chakraborty et al., 2014), finer-grained permission control (Jeon et al., 2012; Nauman et al., 2010), and permission recommendation systems (Agarwal and Hall, 2013). Box (Lee et al., 2013) and SemaDroid (Xu and Zhu, 2015) introduce a notion of privacy budget and seek a balance between utilization and privacy sacrifice. VindiCo shares the similar goal to quantify the degree of information being leaked and choose an appropriate data perturbation method and according mitigation magnitude based on the desired degree of data distortion.

1.2. Paper Contribution

Our contributions can be divided into two categories: 
SpyCon: a new category of spyware targeting HITL IoT systems:

  • [noitemsep, nolistsep]

  • We exploit a new side-channel attack vector arising from monitoring actions and decisions taken by IoT systems. We call this new set of attacks a

    context-aware adaptation based spyware, or in short, SpyCon.

  • We show two concrete instantiation of SpyCon. The first instantiation targets mobile phones in which the SpyCon can maliciously infer user’s behavior by monitoring the decisions taken by context-based apps. We assess the performance of the developed SpyCon through a one-month user study involving human subjects. The second instantiation targets smart homes, in which the SpyCon monitors HVAC activity reported to the cloud and use it to infer human activity. We assess the performance of the developed SpyCon through an industrial-level simulation engines simulating HVAC systems.

VindiCo: A safeguard against SpyCon:

  • [noitemsep, nolistsep]

  • We design, implement, and demonstrate VindiCo, a safeguard against SpyCon. We introduce a novel detection mechanism (named information-based detection) along with two genres of mitigation policies.

  • We re-examine the proposed mitigation policies against the developed SpyCon to assess it and show how its performance decreases after applying mitigation policies.

2. SpyCon: A Context-aware Adaptation based Spyware

(a) SpyCon at the Edge
(b) SpyCon at the Cloud
Figure 2. SpyCon mounted at different places (Edge, Cloud) in HITL IoT

We define a spyware that spies on context-aware adaptations a SpyCon. Since adaptions are dependent on the user context, observing adaptions made by HITL IoT can reveal sensitive user information. In this section, we show two examples of SpyCon, one targets IoT mobile/edge devices while the other one targets the IoT cloud (Elmalaki et al., 2019). We refer to these two SpyCon as “SpyCon Edge” and “SpyCon Cloud”, respectively.

2.1. SpyCon Edge: Popular Phone Manager Apps

A typical pipeline of an IoT application includes edge devices which generate sensor data, an IoT cloud which stores and forwards these data, and a mobile phone which consumes the data, computes the actions, and presents them to IoT users. Mobile phones are arguably a primary target for spyware due to their sensing capabilities and the integrations with IoT systems. In this scenario, we consider SpyCon is mounted on a mobile phone to leak sensitive information, depicted in Figure 1(a). The SpyCon, for example, can reveal a user location such as next to the smart fridge (the sensitive information) if a notification reminder of a grocery list is observed (the adaption). Location-based phone settings management is one of the most popular context-aware applications444By the time this paper was written, context-aware phone settings management applications ranked 3rd in the Productivity category in the Android Developer Challenge (joaomgcd, 2021).. Due to their capability to adapt to user context, apps like Llama (KebabApps, 2022), Tasker (joaomgcd, 2021), and Locale (FOUR, 2022) have gained more than 1 million downloads from Google Play Store. Moreover, these location-based apps are increasingly integrated as part of larger IoT systems. Motivated by the popularity of these location-based context-aware apps, we choose user location as the sensitive data for which SpyCon is trying to leak.

PS Description PS Description
R Ringer mode P Wallpaper
H Touch sound D Dialpad sound
W Enable WiFi A Alarm volume
I Ringer volume M Media volume
T Display timeout B Screen brightness
V Vibration on touch L Screen locking sound
Table 1. Phone settings (PS)

2.1.1. Spyware Description

We design a SpyCon that monitors changes in phone settings to demonstrate it is possible to leak user location, an arguably the most sensitive type of user information (Timing, ). The phone setting changes are triggered by the decisions taken by a location-based context-aware app as part of an IoT system. We start by making the following two important remarks:

  • No user permissions: Unlike location information, many phone settings can be monitored without seeking user permissions. For example, SpyCon can easily get current screen brightness or alarm volume without user consent.

  • Ambiguity on setting changes: Manual adjustment can make changes in phone settings through physical buttons. Although SpyCon can not discriminate a priori

    between the changes in the phone settings done by a location change or by manual adjustment from users, machine learning algorithms can be handy in discovering repetitive patterns in the data.

The operation of the designed SpyCon is divided into two phases:

  • Logging:SpyCon monitors all the changes in phone settings and records a timestamped value upon a change is detected. A list of phone settings that we consider in our SpyCon is given in Table 1.

  • Data Mining: Once enough data is collected, SpyCon analyzes these data to discover repeated patterns and hence infers user’s daily behavior. More details about the data mining algorithm are given in Section 2.1.3 after we discuss the user study setup.

2.1.2. SpyCon User Study

Shadow Logging Application

To understand how much information is leaked by context-aware apps like Tasker and Locale, we developed a shadow app that resembles the functionality of Tasker and Locale in order to collect the ground truth data. First, we ask users to enter the same profiles which they would provide in the context-aware apps (Tasker or Locale). To be more specific, users have to enter a fixed-radius circular geo-fence as a context trigger, as well as a set of actions (e.g., adjusting screen brightness or changing ringer mode to vibration) that would be activated when the user enters these geofences. The full phone settings we considered are listed in Table 1. Secondly, in order to keep track of the golden output (ground truth) for later evaluation, the shadow app keeps and timestamps a record whenever the active profile is changed, implying that the user moves to a different location.

SpyCon Application

We developed a SpyCon whose only task is to log phone settings in the background without any interaction with all the other apps, including the context-aware app555In the real world, this SpyCon can provide some functionality but collect data stealthily, which is a typical way a spyware hides its true intention.. All the settings collected by the SpyCon app can be accessed without permissions in Android OS, including those listed in Table 1.

However, it should be noted that any SpyCon app may benefit from knowing whether a context-aware adaption application is installed. This information can be retrieved through different ways such as the getInstalledApplications() API 666 Even though Android may protect this API by adding a permission in the future, studies have shown that it is hard for most users to relate the side-channel privacy implications to the granted permissions in different apps (Felt et al., 2012)..

(a) Golden output
(b) Clustered by all features
(c) Clustered by dominant features
Figure 3. One example of profile timeline from user #2.
User Study

We implemented both apps mentioned above on Android 5.0.1 running on Nexus 4 and Nexus 5. Seven participants are recruited in our user study, including four males and three females. Each participant carries our phone for four weeks. Users can choose the settings/profiles based on their personal preferences, and they are allowed to change the phone settings manually. Based on the data we collected during the user study, we explore what sensitive information can be mined maliciously as shown in the following experiment.

2.1.3. Experiment 1a: Data Mining by Clustering

Revealing the semantics of the user location trace, or equivalently, the active profile sequence from phone settings is challenging since both the profile and the phone settings do not always have a one-to-one mapping. This is because (1) Users configure only a subset of the settings listed in Table 1 in their profiles and hence it is not known a priori which subset of settings are used by the user. Furthermore, different profiles may include different phone settings to be changed. (2) Users can manually override the phone settings by, for instance, pressing the volume buttons or adjusting the brightness through the Settings App

. Thus, we use a clustering technique to approach the user data mining problem, and in particular, we use k-means algorithm.

Deciding the number of clusters in the k-means algorithm is known to be hard in general and is usually application dependent. Since our SpyCon does not know how many profiles are defined by users, we brute-forcedly set to be any value between 2 through 7 (selected based on the maximum number of profiles defined by our participants). Our algorithm returns the clustering result with the highest silhouette score.

Critical Phone Settings

Inspired by how most unsupervised machine learning algorithms work, we implement a greedy algorithm to find dominant phone settings. The algorithm procedures are provided below:

  1. [noitemsep, nolistsep]

  2. Initialize the selected feature set .

  3. We examine every other setting not in by performing k-means with feature set . The silhouette score is computed accordingly.

  4. Denote as the maximum from the previous step. If is larger than previous silhouette score, then and go back to step 2. Otherwise, the algorithm terminates.

UID # clusters using all features Dominant
base 2 3 2-7 features
1 75.2 +18.9 +22.9 +19.1 +21.8 W,R,V
2 56 +17.2 +24 +18.3 +24.1 R,B,W
3 80.5 +12.9 +14.4 +13.6 +16.7 R
4 45.6 +37.3 +34.2 +35.6 +35.9 W,R,L
5 42 +24 +35.2 +24 +41.8 T,R,A
6 57.9 +4.4 +36 +4.4 +40.7 A,R,B,W
7 78 +15 +15.5 +15 +15.6 R,O
Avg. 62.2 +18.5 +25.8 +18.2 +28.1
Table 2. Clustering accuracy (in percentage) of all users compared to the baseline accuracy (the accuracy that the SpyCon can have based on blind guesses) by applying k-means using the settings in Table 1
Privacy Implications

The clustering result of one participant in our study is demonstrated in Figure 3. Figure 2(a) shows the actual user profile changes across the day (the golden output as explained in Section 2.1.2). Figure 2(b) shows the k-means clustering result (using an adaptive number of clusters) and demonstrates similar patterns with the golden output in Figure 2(a)

. Our algorithm is able to capture subtle events, for example, learning that the user regularly went to a particular place (which turns out to be the child care) after leaving or before returning home, despite the portion of time this user spent in child care is short. Clustering result derived by dominant features from our feature selection algorithm is shown in Figure

2(c). Figures 2(b) and 2(c) clearly indicate the ability of the developed SpyCon to reconstruct user context (switching profiles in this case) by just monitoring its side effect (changes in phone settings)777If the user specifies two profiles with the same settings, SpyCon will recognize them as the same profile. However, the incentive of the user to define the same settings for multiple profiles defies the idea of the context-aware app..

The overall accuracy of our clustering algorithm is reported in Table 2. We define baseline accuracy by using blind guesses, that is, the SpyCon always reports a user is at home without observing any phone settings. The results in the rest of the columns are the additional information (the increase in accuracy) the SpyCon gains over the baseline accuracy if an inference is used based on a different number of clusters. The accuracy derived from dominant features is slightly higher because the feature selection algorithm excludes noisy features leading to a better result. We report dominant features for each user in the last column of Table 2. We observed that the ringer mode is a dominant feature.

In summary, this study shows that the designed SpyCon can estimate and learn with an average accuracy of the user behavior, in particular:

  1. [noitemsep, itemindent=!, nolistsep]

  2. Average commuting time between home and work.

  3. Average time spent at work and at home.

  4. Weekend behavior, such as if a specific place is frequently visited on Sundays and average time spent at home.

2.1.4. Experiment 2: Detection Using Current Antivirus Apps

After we had implemented this spyware app, we examined it using 5 well-known anti-virus applications 888These 5 anti-virus applications are AVG AntiVirus, Symantec Norton Security & AntiVirus, AVAST Mobile Security & Antivirus, McAfee Security & Power Booster, and Kaspersky Internet Security for Android.. None of them reported this app as malware. This follows from the fact that the proposed SpyCon does not have any suspicious code signature. This motivates the need to find a new detection technique that suits this kind of spyware.

2.1.5. Experiment 3: Beyond Location SpyCon

While the previous experiment aims at studying how the proposed SpyCon can leak the semantics of the user location, we further explore how acquiring side-channels can reveal other sensitive user information. To this end, we study several context-aware apps in the Android market and report the monitored context and the corresponding actions taken by these apps in Table 3. Since other apps can observe these actions (even without asking for user permissions), these actions open a side-channel that leaks information about the user behavior. For example, monitoring changes in the played music media999While Android framework does not provide an API to directly retrieve which music is playing, our experiments show that a SpyCon can retrieve such information by reading the metadata associated with the currently active media. can leak information about the user biometrics (heart rate and running pace) and user mood whenever such context-aware apps are used. In general, the proposed SpyCon can take advantage of any pair of get and set methods that are present in the Android framework APIs.

Context-aware App Context Side-channel
Tasker (joaomgcd, 2021) location phone settings
Locale (FOUR, 2022) location phone settings
RockMyRun (Rock My World, ) biometrics music played
HABU music (Mobile, 2014) mood music played
Table 3. Context-aware apps in the market and their side-channel
(a) Golden output for human #1 activity
(b) Clustered into 6 profiles
(c) Clustered into 2 profiles
Figure 4. Activity timeline from simulated human #1.

2.2. SpyCon Cloud: Smart HVAC System

Cloud servers continue to be one of the weakest points when it comes to data breaches (Almorsy et al., 2016). We argue in this example, that even non-sensitive information collected on HITL IoT clouds can be used to leak sensitive user information. To that end, we choose the personalized smart HVAC system as an example of a HITL IoT application (Jung and Jazizadeh, 2017; Hang-yat and Wang, 2013). The personalized HVAC incorporate the human’s activity to change the HVAC set point to maximize his comfort level (Elmalaki et al., 2018a). In this scenario, as pictorially shown in Figure 1(b), human’s activity such as cooking, sleeping, sitting, etc. is used with the current temperature in the house to tune the HVAC set point. The activity of the human is a wealth of information that should be kept safe as it leaks the behavioral pattern of the human along with his house occupancy patterns. Although the HVAC set point is calculated on the phone and pushed to the cloud service that controls the smart thermostat, a SpyCon operating on the cloud could monitor the changes in temperature set points along with the current home temperature to infer the human’s daily behavior.

2.2.1. Spyware Description

We conduct a simulation-based experiment using EnergyPlus (Crawley et al., 2000), an industrial-level physics-based simulation engine, to model HVAC systems. We use the weather reports in Colorado-Denver during January 2018 (energy plus, 2018). The user activity is used to control the set point of the HVAC (Elmalaki et al., 2018a) across the day to maximize the human thermal comfort measured in Prediction Mean Vote (PMV) (Fanger, 1970). A SpyCon mounted in the cloud can monitor the following information to infer the user’s daily behavior: (1) Changes in the HVAC set point triggered by the IoT, (2) current house temperature, (3) time of the day, and (4) day of the week.

2.2.2. Experiment 1b: Data Mining by Clustering

Using the same procedure in Experiment 1a (Section 2.1.3), we simulated several humans independently in EnergyPlus. Due to space, we only show the ground-truth activity and the occupancy of one human (human #1) across time for a month in Figure 3(a). The results of the clustering algorithm used by the SpyCon to infer the human’s daily behavior and the home occupancy are shown in Figure 3(b) and 3(c), respectively.

Privacy Implications

The results shown in Figure 3(b) suggest that SpyCon operating in the cloud can infer sensitive information like when the IoT user wakes up and goes to sleep (prof3), and when the user leaves the house and comes back (prof1). SpyCon Cloud is also able to detect the occurrence of a periodic user activity just after returning home from work which is not performed during the weekends (prof6). The accuracy of the clustering is listed in Table 4. By using two clusters to detect the occupancy (home/away), SpyCon Cloud achieves an accuracy of for human #1 and for human #2. To detect the daily behavioral patterns, we increase the amount of clusters and achieved an accuracy of for human #1 and for human #2.

UID Number of Clusters (k-means)
2 (Occupancy) 4 5 6 7 8
1 75.65 73.47 72.59 76.71 85.31 73.06
2 91.72 70.08 70.14 73 63.23 50.52
Table 4. Clustering accuracy (in percentage) to detect the human’s daily behavioral pattern for two simulated humans in a house using EnergyPlus simulator to simulate HVAC system. The clustering algorithm (k-means) uses features mentioned in Section 2.2.1

3. VindiCo System Architecture

In VindiCo, we focus on the general question of how to design software mechanisms that can detect and mitigate SpyCon. Our entry point is that mobile phones play an important role in connecting the user to numerous HITL IoT. For example, in both the scenarios discussed in Section 2, namely when a SpyCon operated on the edge/phone and the cloud, sensitive user information are collected from the user’s phone. Being the source of several user’s sensitive information, a natural choice of implementing a privacy safeguard would be on the user’s phone. Indeed, the same concepts developed in this section can be applied to other sources of sensitive user information before it is used by the rest of the HITL IoT.

3.1. Threat Model and Design Objectives

Motivated by the observations from the previous section, we define our threat model as follows.
[T1] No prior signature or behavior: Since SpyCon is a new class of spyware, we assume neither prior knowledge of code signatures nor prior recorded suspicious behaviors exist.
[T2] Unknown inference algorithm: We assume no knowledge of the algorithms used by SpyCon to infer the sensitive information.
[T3] Access to information: We assume that SpyCon has access to the APIs101010Our study in the section 2.1 shows that many of such Android APIs do not require permissions. on the phone and have access to the information stored in the cloud.
[T4] Knowledge of Existence of VindiCo: Finally, we assume that SpyCon is aware of the existence of VindiCo. That is, SpyCon has full knowledge of VindiCo detection and mitigation algorithms and SpyCon may adapt its behavior accordingly.

T1 and T2 imply that traditional malware detection schemes (e.g., (Xie et al., 2010)) are not adequate in our problem setup. Hence, a new SpyCon detection scheme has to be designed without possessing any prior information (signature or behavior) of such spyware. In particular, we propose the following design objectives in VindiCo:
[O1] Generic Detection: VindiCo should be able to detect any possible privacy leak without prior knowledge of spyware signatures or behavior.
[O2] Mitigation: VindiCo should be able to mitigate the impact of a possible SpyCon while minimally affecting the performance of the HITL IoT.
[O3] Performance: VindiCo should be lightweight and add minimal execution overhead.

Motivated by these three design objectives, we designed VindiCo as discussed in this section in three main blocks: Context-adaptation registration, Detection Engine, and Mitigation Engine. The details of each building block are given in the subsequent sections.

3.2. Context-adaptation Registration

In order to protect the actions and decisions made by the HITL IoT, VindiCo needs to know the context-action relations within the IoT system. One action in an adaption can be made based on several context, and one context can be shared by different actions. For example, a user may use a camera app and another app for listening to music in the background while running. Upon detecting that the user is running, the camera app adjusts the hardware camera focus while the music app changes the music playlist. Both actions from these two apps, i.e., changing camera properties and altering music playlist, are made based on the same context, i.e., user’s physical activity. Providing the context-action relations is essential for VindiCo to monitor how likely a SpyCon is inferring associated context.

In order to automatically retrieve the context-action relations, we use FlowDroid (Arzt et al., 2014) (which augments the Java code analysis tool Soot (Soot, 2017)) in order to analyze the call flow graph of the context-aware app (e.g., Tasker) and extract all relations between the setter APIs and getter APIs along with the information about which data will be sent and recieved over the network interface to the cloud. The final output is an XML file that maps context (getter APIs and data received from the cloud) to actions (setter APIs and data sent to the cloud). We call this XML file a registry file. The developer can then view the automatically generated XML file to verify or modify it before being used by VindiCo.

In the registry file example shown in Figure 5, there are two adaptation policies. Each adaptation represents one context-action relation. The first adaptation makes decisions based on GPS location and updates ringer mode and alarm volume accordingly. In the second adaptation, the application adjusts the camera settings to accommodate battery capacity, GPS location, and transportation modality.

During application installation phase, VindiCo checks the existence of the registry file. If the file exists, VindiCo considers the associated application as a context-aware app and will offer detection and mitigation mechanisms to protect this app from potential context exposures against any other suspicious SpyCon.

        <context> getGPSlocation() </context>
        <action>  setRingerMode()  </action>
        <action>  setAlarmVolume() </action>
        <context> getTransportationModality() </context>
        <context> getBatteryCapacity()        </context>
        <context> getGPSlocation()            </context>
        <action>  setFocusMode()              </action>
        <action>  setJpegQuality()            </action>
        <action>  setSceneMode()              </action>
Figure 5. Snippet of a VindiCo registry file.

3.3. Information-Based Detection Engine

It follows from assumptions T1 and T2, in our threat model, that existing signature-based detection mechanisms are not able to detect the developed SpyCon. Similarly, behavior-based detection techniques are not suitable to detect such spyware since no prior behavior is known. Exacerbating the situation, the behavior of SpyCon is coupled to the behavior of the authentic context-aware apps that are part of the HITL IoT. That is, if the IoT (and hence the authentic context-aware app) triggers actions more frequently, SpyCon will monitor the actions by calling the getter values APIs for these actions more frequently. This behavior coupling hinders the usability of the behavior-based detection techniques.

The idea behind the information-based detection is to keep track of the ability of SpyCon to infer the context through monitoring actions triggered by this context. Recall that we do not have any prior knowledge or assumption on how SpyCon performs its inference (T2). To this end, we draw on the literature of information theory and leverage mutual information

to quantify the amount of correlation (or dependence) between two random variables. In our scenario, we use the mutual information between

context and action as a metric to measure how certain a SpyCon can infer context from observed actions. Mutual information provides a theoretical bound on the inference capability of any learning algorithm. Generally speaking, the lower the mutual information between context and actions is, the lower the accuracy any inference algorithm can get. Push into one extreme, if the mutual information is zero, then no algorithm can infer context from monitored actions.

Based on this intuition, our detection engine constantly tracks what actions have been monitored by each app, computes an estimate of mutual information between the actual context and those actions monitored by this app, and assigns a suspicion score accordingly. This score is then passed to our Mitigation Engine, as an indicator of the magnitude of countermeasures, which aims at reducing the amount of information possessed by suspicious apps.

3.4. Mitigation Engine

Once the suspicion score, assigned to each possibly running SpyCon, increases beyond a certain threshold (named alarm threshold), the Mitigation Engine informs the user with the possibility of having a SpyCon and asks the user permission to apply countermeasures against the suspicious app. Upon the user consent, the Mitigation Engine seeks a general way to hinder the SpyCon’s capability of revealing user context. While completely blocking all side-channels may not be practical, our goal is to drastically reduce its bandwidth. This process needs to be done without any prior assumption on the type of inference algorithms used by SpyCon (T2). There are two solution regimes to achieve this: 1) imposing delays so that a SpyCon cannot get the latest action updates in real time, and 2) tearing down the correlation between the actions monitored by SpyCon and the associated context. We call the mitigation method in the first regime delay and introduce three different mitigation methods in the second regime: suppression, row-masking, and feature-masking.

Before delving into the details of various mitigation algorithms, we would like to stress the following facts:

  • Complementary and adaptive mitigation: The two regimes of mitigation methods achieve different goals and can complement each other. In practical scenarios, we combine both delay mitigation with one in the second regime which reduces the mutual information. VindiCo does not assume any mitigation can always outperform the others. In fact, a mitigation technique may be effective only in a certain situation. Hence, VindiCo starts by selecting a mitigation method and applies it accordingly. Next, VindiCo tracks the effectiveness of the imposed mitigation technique by constant monitoring of changes in the suspicion score. If a SpyCon can still survive under current mitigation treatment, VindiCo increases the mitigation magnitude first and eventually switches to a different method.

  • The Scope of mitigation: The mitigation is applied on a per-app basis instead of a system-wide configuration. Hence, well-behaved apps (including the protected context-aware app) can receive correct action values. Therefore, mitigations cause no negative impact on context-aware apps.

The rest of this section we explain the details of the mitigation techniques we have in VindiCo.

3.4.1. Mitigate by Delay

Our first strategy is to let VindiCo delay the information about when adaptation actions are taken so that a SpyCon can only get an outdated context instead of the latest one. That is, whenever an app with high suspicious score or whenever the cloud is getting information about actions taken based on changes in user contexts, VindiCo will send old information that are delayed minutes which is selected randomly (to prevent the SpyCon from revealing the delay).

Figure 6. Architecture and data structures used by VindiCo. At installation time, VindiCo checks the existence of a registry file and starts processing it accordingly. Registry file processing constructs all necessary data structures that are needed by the detection and mitigation engines

3.4.2. Mitigate by Suppression

With the previous mitigation method, even though SpyCon may not be able to get the real-time update from delay mitigation, a SpyCon can retrofit users’ daily routine by aligning with a priori knowledge, for example, when a person usually goes to school or work. Once the personal schedule is derived, this mitigation will have no effect in the future. The fundamental reason is that simply imposing a delay will not reduce the mutual information. Suppression mitigation, on the other hand, is designed to decrease the information a SpyCon can harvest. The main idea of this mitigation technique is to give SpyCon false adaptation actions. For SpyCon that are detected on the phone (i.e., associated with high-suspicious score), VindiCo can return action values associated with another context without affecting the actual actions taken by the trusted app and hence eliminating any negative affect on the system. On other hand, if the cloud is associated with a high-suspicious score, VindiCo will corrupt the actions before they are sent to the cloud leading to some negative effect on the whole system performance.

3.4.3. Mitigate by Masking

Yet another mitigation to increase obfuscation is to mask some action values, i.e., returning zeros. We explore two variants of the masking approach: row-masking and feature-masking

. In row-masking, our system returns correct action values, but with a certain probability

, our system returns 0 to SpyCon for all the action values after context changes. The masking effect takes place until the next adaptation is made. In consequence, SpyCon cannot infer anything during the masked periods, but can still make inferences based on the unmasked observations. The feature-masking approach, in contrast, considers each action value individually. Upon an adaptation is made, each action has a certain probability to be masked. As a result, at any given context change, the SpyCon can only observe partial action values after an adaptation occurs. The decision of masking all actions in row-masking or which action values to be masked in feature-masking depends on flipping a biased coin with a selected probability , which serves as the parameter of mitigation effectiveness. The evaluation of the mitigation techniques is shown in Section 5.3.

4. Implementation

VindiCo is implemented by modifying the Android system image for platform 5.0.1 API 22 (Android, 2022). We extend the Android system layer to add the three main parts of VindiCo as shown in explained in Section 3. VindiCo increases the sizes of system image by 4.5% (original image size is 998 MByte). The overhead on the API call is shown in Section 4.3

Upon installation time (as shown in Figure 6) VindiCo searches for the registry file and creates the necessary data structures to track all changes in user contexts and actions associated with these changes. It then creates a list called Protection Lists and Mutual Information tables that keeps track of the available information at each app and the cloud. To update these tables, VindiCo intercepts all the calls between apps and the Android OS. Whenever an app (or the cloud) gets information about context changes, VindiCo starts to suspect this app (or the cloud) by assigning a suspicion score to this app (or the cloud service). The value of this suspicion score is equal to the mutual information associated with the getter API (invoked by the app) which is stored in the mutual information tables. Since the same app may have called several getter APIs belonging to different Protection Lists, VindiCo associates a set of suspicion scores (instead of just one suspicion score), one per Protection List in a table that is called applications suspicion scores table.

4.1. VindiCo Detection Engine

An important aspect that is related to the use of mutual information is the convergence of estimated mutual information to the actual mutual information from the data samples accessed by VindiCo. That is, it is well known that calculating an estimate of mutual information with enough precision requires the access to multiple samples from both context and triggered actions111111An estimate of the mutual information converges to actual mutual information asymptotically.. Hence, one can argue that a malicious spyware—who is aware of the existence of VindiCo as per assumption T4 in our threat model—may be tempted to reduce the frequency for which it monitors the actions with the goal of decreasing the number of samples used to estimate his mutual information and hence deceives VindiCo by reducing the associated suspicion score121212On average the actual mutual information is the upper bound for the estimate mutual information.. To avoid such situation, we associate mutual information to actions instead of apps. That is, whenever an action takes place, we update the amount of mutual information between this particular action and all the context associated with it. At any time point, once an app monitors this action, we copy the mutual information associated with this action to the app. Therefore, regardless of how frequent an app monitors actions, we associate to it the mutual information based on all samples in history—recall that the number of these samples are not controlled by the malicious app—not only on those to which it has access.

4.2. VindiCo Mitigation Engine

As mentioned in Section 3.4, VindiCo does not presume any mitigation treatment outperforms the others, and opportunistically selects a method and see the effect and gradually increases the mitigation. If the suspicion score cannot be effectively decreased, VindiCo will switch to another mitigation method. To optimize the run-time of mitigation engine, we cache the decisions taken by the mitigation engine on this API call for each calling application while a separate thread update this cache based on mutual information values.

4.3. Timing Analysis of VindiCo

The execution time is obtained using Android traceview (SDK, 2018). Table 5 shows the CPU execution time when the complexity of the context-aware app increases, which is majorly reflected by the number of adaptation tags in the registry file. Parsing and processing the registry file take approximately . Fortunately, this overhead takes place only during the installation of a new package on the phone. On the other hand, at runtime, VindiCo adds negligible overhead (less than ) which is approximately increase from the average execution time of the original API call.

Description Overhead in CPU Time (ms)
2 adapt 3 adapt 4 adapt
Parsing registry file
Registry file processing
API call in context-aware apps
API call in other apps
Table 5. Timing analysis of VindiCo against increasing complexity of context-aware apps measured by number of adaptation tags in the registry file.
Figure 7. Accuracy of leaking sensitive information about phone user from the data collected by 45 of the most downloaded free apps; (blue) accuracy of identifying the semantics of the user location when a location-based context-aware app (Tasker/Locale) is used to change the phone settings and (red) accuracy of identifying user calendar profile when a calendar-based context-aware app (Silence 2.0) is used to change the phone settings.

5. Evaluation

In this section, we discuss the performance of the proposed VindiCo in mitigating possible SpyCon Edge and SpyCon Cloud.

5.1. Experiment 4: How many SpyCon Edge in the market?

We start by examining how many apps currently in the market and have the capability of being a successful SpyCon Edge app. While identifying whether a real app is actually exploiting the proposed side-channel is hard to be checked without the knowledge of the app behavior, we focus in this experiment, instead, on identifying real apps in the market that possess enough information to leak sensitive information about phone user.

To that end, we performed static analysis on 45 of the most downloaded free apps from the Google Play store to check which getter APIs are being used by these apps. Next, we intercepted these getter APIs to retrieve the information possessed by these applications. Finally, we use this data to leak information about the phone user when a location-based context adaptation apps like Tasker/Locale and Silence (a calendar events-based context adaptation app) are taking actions by changing the phone setups based on location and calendar events, respectively. As mentioned in Section 2, context-based apps are increasingly used by IoT platforms like IBM Watson IoT to allow the designed IoT system to adapt its functionality to the human user.

Figure 7 shows the accuracy of retrieving the user context from the data available to the 45 apps. Our results show that of the top downloaded free apps have enough information to leak user context with some of the apps scoring more than accuracy in leaking user context when a location-based context aware app is used. Similarly, for calendar-based context, of these apps can leak sensitive information with an accuracy more than showing the significance of this side-channel information.

Figure 8. Performance of the VindiCo information-based detector (in terms of false positive and false negative rates) versus different alarm thresholds.

5.2. Experiment 5: Performance of Information-Based Detection

5.2.1. Detection Accuracy

We investigate the effect of choosing the alarm threshold (the threshold on the suspicion score above which an app is considered malicious) on the performance of the proposed information-based detection algorithm. In particular, we evaluate the performance of the detection engine regarding both the false positive rate and false negative rate. In this context, an application is considered malicious whenever its clustering accuracy exceeds the baseline accuracy by blind guesses defined in Section 2.1.3. A false positive flags the case when the information-based detector claims that an application possesses enough information to accurately identify the user behavior with accuracy more than the baseline accuracy while the real accuracy of the app is indeed lower than the baseline accuracy. A false negative is defined similarly. Similar to the previous experiment, we focus again on the case when SpyCon is monitoring phone settings changed by a location-based context-aware app (Tasker/Locale) and when these settings are changed by a calendar events-based context-aware app (Silence).

(a) Suppression mitigation (3 rows)
(b) Row-masking mitigation (p=0.4)
(c) Feature-masking mitigation (p=0.4)
Figure 9. Profile timeline of user #2 after VindiCo applies the mitigation techniques
(a) MI by suppression
(b) MI by row-masking
(c) MI by feature-masking
(d) Accuracy by suppression
(e) Accuracy by row-masking
(f) Accuracy by feature-masking
Figure 10. Mutual information (MI) and clustering accuracy difference (Acc diff)—with respect to the baseline accuracy—after applying different mitigation methods. When mitigation magnitude increases, both mutual information and accuracy decrease

Figure 8 reports the false positive and false negative rates obtained from 8000 data points collected from our user studies versus different alarm thresholds. As the alarm threshold increases, the information-based detection becomes less aggressive leading to a significant decrease in the false positive rates while sacrificing the ability to detect malicious apps (reflected by the increase in the false negative rates). The results reported in Figure 8 suggest that an alarm threshold of 0.65 leads to a compromise between false positive and false negative rates (false positive rate = 0.1 and false negative rate = 0.15).

5.2.2. Detection of SpyCon in the Market

Next, we ran the proposed information-based detection algorithm (with alarm threshold set to 0.65) against the 45 real applications from the market used in Experiment 4. Recall that Experiment 4 asserts most apps can identify the user context with accuracy more than 80% which in turn indicates that each app possesses high amount of information. Running the proposed information-based detection algorithm against these applications results into suspicion scores that are greater than the alarm threshold for all the 45 apps. Therefore, VindiCo marks all these apps as malicious reflecting the fact that these apps possess enough information to leak sensitive details about the user behavior. For space limits, we report here the suspicion score of the extreme cases (the top two and bottom two apps in terms of accuracy in Experiment 4) for the SpyCon that monitors changes based on location as follows: 360 Security Antivirus (0.974), Clean Master (0.971), pinterest (0.722), and Madden NFL Mobile (0.784). Similar suspicion scores are obtained when SpyCon is monitoring changes based on calendar events.

5.3. Experiment 6: Performance of Mitigation Algorithms

To evaluate the performance of the proposed mitigation technique, we applied the proposed mitigation methods on SpyCon Edge and SpyCon Cloud.

5.3.1. Mitigation of SpyCon Edge

In Figure 9, we plot the profile timeline of user 2 (the profile of the same user was analyzed and discussed previously in Section 2.1) after applying different mitigation techniques. Comparing the results in Figure 2(c) and Figure 9, we notice the distortion in the patterns of user profile compared to the case when no mitigation is applied. To better judge the effect of the proposed mitigation algorithms, we plot in Figure 10 the accuracy of the developed clustering algorithm when different mitigation techniques are applied along with the corresponding mutual information. These results show how the SpyCon accuracy (measured as the accuracy increase above the baseline accuracy) drops in general to the baseline accuracy. Moreover, and as expected from the theoretical underpinnings of mutual information, whenever mutual information decreases—as a consequence of the mitigation parameter (number of records in suppression case and the probability of masking in the row-masking and feature masking case)— the accuracy of the developed SpyCon must decrease (on average). We also observe that the suppression method leads to a sharp decrease in the mutual information with a fast saturation whenever as shown in Figure 9(a). In contrast, the other two proposed mask mitigation methods (row-masking and feature-masking) lead to a gradual decrease in the mutual information as the probability of masking increases (Figure 9(b),  9(c)). This gradual degradation provides more flexibility for VindiCo to adjust to a desired mutual information.

Finally, though we expect mutual information should monotonically decrease when the mitigation is stronger (higher masking probability), 9(c) shows that mutual information goes higher when masking probability . One possible explanation is that noisy settings are cleared with this configuration, causing higher correlation between mitigated data and user profile. However, the mutual information decreases sufficiently when larger masking probability is applied (i.e., ).

(a) Profiles detection after masking mitigation (p=0.4)
(b) Profiles detection after masking mitigation (p=0.8)
(c) Occupancy detection after masking mitigation (p=0.8)
Figure 11. Activity and occupancy timeline of human #1 after VindiCo applies the mitigation techniques

5.3.2. Mitigation of SpyCon Cloud

Using the same mitigation procedure as SpyCon Edge (Section 5.3.1), we plot the mitigation results of human #1 in Figure 11. We applied row masking and feature masking simultaneously with different probabilities. We observe that the performance of the SpyCon Cloud in detecting the user profile is adversely affected by increasing the probability of masking. Similarly, as shown in Figure 10(c), the SpyCon ability to detect home occupancy degraded severely as a consequence of applying the mitigation. The decrease in accuracy for both human 1 and human 2 across different probabilities of mitigation is shown in Figure 12.

(a) Accuracy of activity detection after mitigation
(b) Accuracy of occupancy detection after mitigation
Figure 12. Clustering accuracy after applying mitigation methods on SpyCon Cloud. When mitigation magnitude increases, the accuracy decreases

5.3.3. Effect of Mitigation on Human Thermal Comfort

Applying row and feature masking in the context of SpyCon Cloud entails changing the HVAC set-point in order to hinder the ability of SpyCon to infer the human behavior can affect the human thermal comfort. Accordingly, we compared the Prediction Mean Vote (PMV) values—as a measure for the human thermal comfort (Fanger, 1970)—before and after mitigation in Figure 13. The PMV score ranges from to which is the range of thermal sensation from very cold () to very hot (). According to ISO standard ASHRAE 55 (ASHRAE/ANSI Standard 55-2010 American Society of Heating, Refrigerating, and Air-Conditioning Engineers, 2010), a PMV in the range of and for an interior space is recommended to achieve thermal comfort. We used the Fanger model in EnergyPlus to estimate the PMV value (Fanger, 1970). In particular, as seen in Figure 13, a choice of masking probability equal lead to user PMV in the range of to while achieving a degradation in SpyCon accuracy by (as shown in Figure 12).

(a) Before mitigation
(b) After mitigation by masking()
(c) After mitigation by masking()
Figure 13. PMV score before and after mitigation with different masking probabilities across 15 days

6. Conclusion

In this paper, we introduced SpyCon, a new class of privacy threatening spyware that monitors the adaptation actions and the decisions made by HITL IoT in an attempt to leak sensitive information about the IoT user. We showed two concrete examples of SpyCon namely SpyCon Edge and SpyCon Cloud. In the first example (SpyCon Edge), we showed through our user study how a SpyCon Edge could infer user behavior and the semantics of the user location in realtime. To exacerbate the situation, our experiments showed that this new spyware is undetectable using off-the-shelf antivirus packages and moreover many of the top 45 downloadable free Android apps have enough information to reveal sensitive user information. In the second example (SpyCon Cloud), we showed a simulated example of a smart HVAC, in which the cloud service has a mounted SpyCon. In our experiments, we showed how the mounted SpyCon can infer the user’s occupancy at home and his daily schedule (awake, sleep).

To circumvent such scenarios, we designed VindiCo, a safeguard which protects authentic HITL IoT applications against leaking private information via this side-channel. VindiCo employs a detection technique based on mutual information which is agnostic to the SpyCon implementation details. VindiCo uses three mitigation techniques. Our mitigation techniques have shown a degradation of SpyCon Edge inference accuracy from to the baseline accuracy and a degradation in SpyCon Cloud by and by only adding negligible overhead () on the API call performance. Future work includes studying the effect of collaborative SpyCon. In such scenario, multiple IoT devices collect different (or partial) information from the proposed side-channel and fuse them to leak sensitive user information while maintaining small individual mutual information and hence bypass the information-based detection algorithm.


  • J. P. Achara, M. Cunche, V. Roca, and A. Francillon (2014) Short paper: wifileaks: underestimated privacy implications of the access_wifi_state android permission. In Proceedings of the 2014 ACM conference on Security and privacy in wireless & mobile networks, pp. 231–236. Cited by: §1.1.
  • Y. Agarwal and M. Hall (2013) ProtectMyPrivacy: detecting and mitigating privacy leaks on ios devices using crowdsourcing. In Proceeding of the 11th annual international conference on Mobile systems, applications, and services, pp. 97–110. Cited by: §1.1.
  • A. Ahadi-Sarkani and S. Elmalaki (2021)

    ADAS-rl: adaptive vector scaling reinforcement learning for human-in-the-loop lane departure warning

    In Proceedings of the First International Workshop on Cyber-Physical-Human System Design and Implementation, pp. 13–18. Cited by: §1.
  • M. Almorsy, J. Grundy, and I. Müller (2016) An analysis of the cloud computing security problem. arXiv preprint arXiv:1609.01107. Cited by: §2.2.
  • Android (2022)

    Android Open Source Project

    Note:; accessed Feb 1, 2022 Cited by: §4.
  • S. Arzt, S. Rasthofer, C. Fritz, E. Bodden, A. Bartel, J. Klein, Y. Le Traon, D. Octeau, and P. McDaniel (2014) Flowdroid: precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for android apps. In ACM SIGPLAN Notices, Vol. 49, pp. 259–269. Cited by: §1.1, §3.2.
  • ASHRAE/ANSI Standard 55-2010 American Society of Heating, Refrigerating, and Air-Conditioning Engineers (2010) Thermal environmental conditions for human occupancy. Inc.Atlanta, GA, USA. Cited by: §5.3.3.
  • A. R. Beresford, A. Rice, N. Skehin, and R. Sohan (2011) MockDroid: trading privacy for application functionality on smartphones. In Proceedings of the 12th Workshop on Mobile Computing Systems and Applications, HotMobile ’11, New York, NY, USA, pp. 49–54. External Links: Document, ISBN 978-1-4503-0649-2, Link Cited by: §1.1.
  • S. Chakraborty, C. Shen, K. R. Raghavan, Y. Shoukry, M. Millar, and M. Srivastava (2014) IpShield: a framework for enforcing context-aware privacy. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14), pp. 143–156. Cited by: §1.1.
  • D. B. Crawley, L. K. Lawrie, C. O. Pedersen, and F. C. Winkelmann (2000) Energy plus: energy simulation program. ASHRAE journal 42 (4), pp. 49–56. Cited by: §2.2.1.
  • M. Egele, C. Kruegel, E. Kirda, and G. Vigna (2011) PiOS: detecting privacy leaks in ios applications.. In NDSS, Cited by: §1.1.
  • S. Elmalaki, B. U. Demirel, M. Taherisadr, S. Stern-Nezer, J. J. Lin, and M. A. Al Faruque (2021) Towards internet-of-things for wearable neurotechnology. In 2021 22nd International Symposium on Quality Electronic Design (ISQED), pp. 559–565. Cited by: §1.
  • S. Elmalaki, B. Ho, M. Alzantot, Y. Shoukry, and M. Srivastava (2019) Spycon: adaptation based spyware in human-in-the-loop iot. In 2019 IEEE Security and Privacy Workshops (SPW), pp. 163–168. Cited by: §1, §2, footnote 2.
  • S. Elmalaki, Y. Shoukry, and M. Srivastava (2018a) Internet of personalized and autonomous things (iopat): smart homes case study. In Proceedings of the 1st ACM International Workshop on Smart Cities and Fog Computing, pp. 35–40. Cited by: §2.2.1, §2.2, footnote 2.
  • S. Elmalaki, H. Tsai, and M. Srivastava (2018b) Sentio: driver-in-the-loop forward collision warning using multisample reinforcement learning. In Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems, pp. 28–40. Cited by: §1.
  • S. Elmalaki (2021) FaiR-iot: fairness-aware human-in-the-loop reinforcement learning for harnessing human variability in personalized iot. In Proceedings of the International Conference on Internet-of-Things Design and Implementation, pp. 119–132. Cited by: §1.
  • W. Enck, P. Gilbert, S. Han, V. Tendulkar, B. Chun, L. P. Cox, J. Jung, P. McDaniel, and A. N. Sheth (2014) TaintDroid: an information-flow tracking system for realtime privacy monitoring on smartphones. ACM Transactions on Computer Systems (TOCS) 32 (2), pp. 5. Cited by: §1.1.
  • W. Enck, D. Octeau, P. McDaniel, and S. Chaudhuri (2011) A study of android application security.. In USENIX security symposium, Vol. 2, pp. 2. Cited by: §1.1.
  • energy plus (2018) Weather data by location. Note:; accessed Feb 1, 2022 Cited by: §2.2.1.
  • P. O. Fanger (1970) Thermal comfort. analysis and applications in environmental engineering.. Thermal comfort. Analysis and applications in environmental engineering.. Cited by: §2.2.1, §5.3.3.
  • A. P. Felt, E. Ha, S. Egelman, A. Haney, E. Chin, and D. Wagner (2012) Android permissions: user attention, comprehension, and behavior. In Proceedings of the Eighth Symposium on Usable Privacy and Security, pp. 3. Cited by: §1.1, footnote 6.
  • T. F. FOUR (2022) Locale application. Note:; accessed Feb 1, 2022 Cited by: §2.1, Table 3.
  • M. Grace, Y. Zhou, Q. Zhang, S. Zou, and X. Jiang (2012) Riskranker: scalable and accurate zero-day android malware detection. In Proceedings of the 10th international conference on Mobile systems, applications, and services, pp. 281–294. Cited by: §1.1.
  • J. Han, A. J. Chung, and P. Tague (2017) Pitchln: eavesdropping via intelligible speech reconstruction using non-acoustic sensor fusion. In Proceedings of the 16th ACM/IEEE International Conference on Information Processing in Sensor Networks, pp. 181–192. Cited by: §1.
  • J. Han, E. Owusu, L. T. Nguyen, A. Perrig, and J. Zhang (2012) Accomplice: location inference using accelerometers on smartphones. In Communication Systems and Networks (COMSNETS), 2012 Fourth International Conference on, pp. 1–9. Cited by: §1.1.
  • L. A. Hang-yat and D. Wang (2013) Carrying my environment with me: a participatory-sensing approach to enhance thermal comfort. In Proceedings of the 5th ACM Workshop on Embedded Systems For Energy-Efficient Buildings, pp. 1–8. Cited by: §2.2.
  • B. Ho, P. Martin, P. Swaminathan, and M. Srivastava (2015) From pressure to path: barometer-based vehicle tracking. In Proceedings of the 2nd ACM International Conference on Embedded Systems for Energy-Efficient Built Environments, pp. 65–74. Cited by: §1.1.
  • P. Hornyack, S. Han, J. Jung, S. Schechter, and D. Wetherall (2011) These aren’t the droids you’re looking for: retrofitting android to protect data from imperious applications. In Proceedings of the 18th ACM conference on Computer and communications security, pp. 639–652. Cited by: §1.1, §1.1, §1.1.
  • J. Jeon, K. K. Micinski, J. A. Vaughan, A. Fogel, N. Reddy, J. S. Foster, and T. Millstein (2012) Dr. android and mr. hide: fine-grained permissions in android applications. In Proceedings of the Second ACM Workshop on Security and Privacy in Smartphones and Mobile Devices, SPSM ’12, New York, NY, USA, pp. 3–14. External Links: Document, ISBN 978-1-4503-1666-8, Link Cited by: §1.1.
  • joaomgcd (2021) Tasker app. Note:; accessed Feb 1, 2022] Cited by: §2.1, Table 3, footnote 4.
  • W. Jung and F. Jazizadeh (2017) Towards integration of doppler radar sensors into personalized thermoregulation-based control of hvac. In Proceedings of the 4th ACM International Conference on Systems for Energy-Efficient Built Environments, pp. 21. Cited by: §2.2.
  • S. T. Y. I. Kane and S. Thurm (2010) Your apps are watching you. WALL ST. J.(Dec. 17, 2010), http://on. wsj. com/wq7Wiw. Cited by: §1.1.
  • M. G. Kang, S. McCamant, P. Poosankam, and D. Song (2011) DTA++: dynamic taint analysis with targeted control-flow propagation.. In NDSS, Cited by: §1.1.
  • KebabApps (2022) Llama - Location Profiles Application. Note:; accessed Feb 1, 2022 Cited by: §2.1.
  • E. C. Larson, T. Lee, S. Liu, M. Rosenfeld, and S. N. Patel (2011) Accurate and privacy preserving cough sensing using a low-cost microphone. In Proceedings of the 13th international conference on Ubiquitous computing, pp. 375–384. Cited by: §1.
  • S. Lee, E. L. Wong, D. Goel, M. Dahlin, and V. Shmatikov (2013) ?Box: a platform for privacy-preserving apps. In Presented as part of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI 13), Lombard, IL, pp. 501–514. External Links: ISBN 978-1-931971-00-3, Link Cited by: §1.1.
  • J. Li, X. Sun, Q. Liu, W. Zheng, H. Liu, and J. A. Stankovic (2018) Planning electric vehicle charging stations based on user charging behavior. In Proceedings of the Third International Conference on Internet-of-Things Design and Implementation, pp. 225–236. Cited by: §1.
  • L. Lu, Z. Li, Z. Wu, W. Lee, and G. Jiang (2012) Chex: statically vetting android apps for component hijacking vulnerabilities. In Proceedings of the 2012 ACM conference on Computer and communications security, pp. 229–240. Cited by: §1.1.
  • M. Ma, S. M. Preum, and J. A. Stankovic (2017) Cityguard: a watchdog for safety-aware conflict detection in smart cities. In Proceedings of the Second International Conference on Internet-of-Things Design and Implementation, pp. 259–270. Cited by: §1.
  • P. Marquardt, A. Verma, H. Carter, and P. Traynor (2011) (Sp) iphone: decoding vibrations from nearby keyboards using mobile phone accelerometers. In Proceedings of the 18th ACM conference on Computer and communications security, pp. 551–562. Cited by: §1.1.
  • Y. Michalevsky, D. Boneh, and G. Nakibly (2014) Gyrophone: recognizing speech from gyroscope signals. In 23rd USENIX Security Symposium (USENIX Security 14), pp. 1053–1067. Cited by: §1.1.
  • Y. Michalevsky, A. Schulman, G. A. Veerapandian, D. Boneh, and G. Nakibly (2015) PowerSpy: location tracking using mobile device power analysis. In 24th USENIX Security Symposium (USENIX Security 15), Washington, D.C., pp. 785–800. External Links: ISBN 978-1-931971-232, Link Cited by: §1.1.
  • Y. Michalevsky, A. Schulman, G. A. Veerapandian, D. Boneh, and G. Nakibly (2015) PowerSpy: location tracking using mobile device power analysis.. In USENIX Security, pp. 785–800. Cited by: §1.1.
  • E. Miluzzo, A. Varshavsky, S. Balakrishnan, and R. R. Choudhury (2012) Tapprints: your finger taps have fingerprints. In Proceedings of the 10th international conference on Mobile systems, applications, and services, pp. 323–336. Cited by: §1.1.
  • G. Mobile (2014) HABU music. Note:; accessed Feb 1, 2022 Cited by: Table 3.
  • K. Nahrstedt, H. Li, P. Nguyen, S. Chang, and L. Vu (2016) Internet of mobile things: mobility-driven challenges, designs and implementations. In Internet-of-Things Design and Implementation (IoTDI), 2016 IEEE First International Conference on, pp. 25–36. Cited by: §1.1.
  • Y. Nan, M. Yang, Z. Yang, S. Zhou, G. Gu, and X. Wang (2015) UIPicker: user-input privacy identification in mobile applications. In 24th USENIX Security Symposium (USENIX Security 15), Washington, D.C., pp. 993–1008. External Links: ISBN 978-1-931971-232, Link Cited by: §1.1.
  • M. Nauman, S. Khan, and X. Zhang (2010) Apex: extending android permission model and enforcement with user-defined runtime constraints. In Proceedings of the 5th ACM Symposium on Information, Computer and Communications Security, pp. 328–332. Cited by: §1.1.
  • E. Owusu, J. Han, S. Das, A. Perrig, and J. Zhang (2012) ACCessory: password inference using accelerometers on smartphones. In Proceedings of the Twelfth Workshop on Mobile Computing Systems & Applications, pp. 9. Cited by: §1.1.
  • A. Raij, A. Ghosh, S. Kumar, and M. Srivastava (2011) Privacy risks emerging from the adoption of innocuous wearable sensors in the mobile environment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 11–20. Cited by: §1.1.
  • [51] Inc. Rock My World Rock my run. Note: www.rockmyrun.comOnline; accessed Feb 1, 2022 Cited by: Table 3.
  • A. Sadeghi, C. Wachsmann, and M. Waidner (2015) Security and privacy challenges in industrial internet of things. In The 52nd ACM/EDAC/IEEE Design Automation Conference (DAC), 2015, pp. 1–6. Cited by: §1.
  • A. SDK (2018) Profiling with traceview. Note: Cited by: §4.3.
  • S. Sicari, A. Rizzardi, L. A. Grieco, and A. Coen-Porisini (2015) Security, privacy and trust in internet of things: the road ahead. Computer networks 76, pp. 146–164. Cited by: §1.
  • Soot (2017) Soot - a framework for analyzing and transforming java and android applications. Note:; accessed Feb 1, 2022 Cited by: §3.2.
  • M. Taherisadr, B. U. Demirel, M. A. A. Faruque, and S. Elmalaki (2021) Future of smart classroom in the era of wearable neurotechnology. arXiv preprint arXiv:2110.11475. Cited by: §1.
  • [57] Timing Geolocation privacy legislation. Note:; accessed Feb 2022 Cited by: §2.1.1.
  • H. Wang, T. T. Lai, and R. Roy Choudhury (2015) Mole: motion leaks through smartwatch sensors. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, pp. 155–166. Cited by: §1.1.
  • L. Xie, X. Zhang, J. Seifert, and S. Zhu (2010) PBMDS: a behavior-based malware detection system for cellphone devices. In Proceedings of the third ACM conference on Wireless network security, pp. 37–48. Cited by: §3.1.
  • Z. Xu and S. Zhu (2015) SemaDroid: a privacy-aware sensor management framework for smartphones. In Proceedings of the 5th ACM Conference on Data and Application Security and Privacy, pp. 61–72. Cited by: §1.1.
  • L. K. Yan and H. Yin (2012) Droidscope: seamlessly reconstructing the os and dalvik semantic views for dynamic android malware analysis. In Presented as part of the 21st USENIX Security Symposium (USENIX Security 12), pp. 569–584. Cited by: §1.1.
  • X. Zhou, S. Demetriou, D. He, M. Naveed, X. Pan, X. Wang, C. A. Gunter, and K. Nahrstedt (2013) Identity, location, disease and more: inferring your secrets from android public resources. In Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security, pp. 1017–1028. Cited by: §1.1.
  • Y. Zhou, Z. Wang, W. Zhou, and X. Jiang (2012) Hey, you, get off of my market: detecting malicious apps in official and alternative android markets.. In NDSS, Cited by: §1.1.
  • Y. Zhu, J. Jung, D. Song, T. Kohno, and D. Wetherall (2009) Privacy scope: a precise information flow tracking system for finding application leaks. Ph.D. Thesis, Citeseer. Cited by: §1.1.