Recent advances in sensing, actuation, and communication technologies have allowed an unprecedented level of control over the behavior of our devices and an unprecedented fidelity of information about the state of our systems. This has found application in a wide variety of ‘smart’ decision making processes, including the management of parking spaces, the monitoring of water usage, and the energy-efficient operation of instrumented homes and offices (Shah and Yaqoob, 2016).
However, as the set of application domains for these ‘smart’ algorithms grow, we find ourselves in scenarios where multiple stakeholders care about the state of a shared resource. For example, consider the control of lights in an office space. At home, one can easily adjust the light settings in a non-problematic way, as they are dictator of their own home’s illumination. In contrast, for an office setting where multiple co-workers have different preferences about the intensity of lights, the definition of what ‘should’ happen is not as clear. Put another way, as we start to look at applications with a shared resource, we need to find ways to reconcile the opinions of multiple users.
Building on our previous example, we can think of a mechanism that asks each person: ‘How bright do you want the lights?’ Suppose there are two office occupants and our mechanism simply implements the average of the two votes. Furthermore, suppose your co-worker voted for 100% intensity, and you wanted the lights at 75% intensity. In this situation, you would vote for for 50% intensity, as the mechanism would implement the average, which would be your desired lighting level. This simple thought experiment reveals a crucial point: humans have an incentive to strategically report data to push mechanisms to their selfishly desired outcomes.
This problem is often studied in economics under the title of ‘mechanism design’. The Vickrey-Clarke-Groves (VCG) mechanism (Vickrey, 1961; Groves, 1973) is one of the most celebrated achievements in this area.111For more information about VCG mechanisms, we refer the reader to the wonderfully accessible (Nisan et al., 2007). VCG mechanisms have many desirable properties. First, VCG mechanisms implement the most socially beneficial outcome. Second, all participants are incentivized to truthfully reveal their preferences. Third, the mechanism can be designed such that every participant is better off participating in the mechanism, either due to the outcome chosen or due to some endogenously determined payment.
While the VCG mechanism has many desirable properties when all participants act as rational agents, the literature of real-world implementations outside the context of auctions is limited, as we will discuss further in Section 2. Furthermore, due to the application domain, minor modifications were required to implement VCG mechanisms in a real-time setting where users cannot walk away and the outside option is type-dependent; more details can be found in Section 3.
The rest of the paper is organized as follows. In Section 2, we discuss the existing literature most relevant to our work, and contextualize our contribution. In Section 3, we present the mathematical formalism for modeling the decision making processes of users and the formulation of the mechanism implemented by smartSDH. In Section 4, we outline our experimental setup. In Section 5, we present the statistical analysis of our experimental data. In Section 6, we discuss and interpret the results, as well as provide closing remarks summarizing our discoveries and potential avenues for future work.
2 Background and Related Work
In this section, we discuss the existing literature and works most closely related to ours. Broadly speaking, this literature can be broken down into two categories: works on building control and works on mechanism design.
2.1 Building control
Nearly 50% of the energy consumed in the U.S. is accounted for by residential and commercial buildings (McQuade, 2009), and well-designed algorithms for the control of lighting and heating-ventilation-air-conditioning systems in buildings promise significant benefits for the stability and efficiency of our power grid. Much of the research in building control focuses on finding algorithms that take into account the uncertainty about external factors and operating conditions, while satisfying the constraints introduced by user preferences (Ma, Anderson, and Borrelli, 2011; Aswani et al., 2012).
Recent work in smart building infrastructures that incorporate occupant preferences about thermal comfort (Karmann, Schiavon, and Arens, 2018), satisfaction/well-being (Frontczak et al., 2012), lighting comfort (Baron, Rea, and Daniels, 1992), acoustical quality (Ryherd and Wang, 2008), indoor air quality (Sundell et al., 2011), indoor environmental monitoring (Jin et al., 2018), privacy (Jia et al., 2017), and productivity (Wargocki et al., 2008), while simultaneously optimizing for energy efficiency and agile connectivity with the grid.
Prior work has attempted to correct the information imbalance by modeling occupant preferences by the means of a multi-agent system (Boman et al., 1998). An initial control policy is created from both the building manager’s and the occupants’ preferences. Then, a rule engine finds a compromise in the system, and iteratively performs a compromise. In another related work, Clearwater demonstrates an auction-based apparatus where each room in an office building makes a bid between the desired temperature and the actual temperature (Clearwater, 1995).
With regards to other building control scenarios, Ploennigs and Schumann carry out an experiment with sensors placed in buildings and use machine learning and statistical learning techniques with the information from the sensors to diagnose building operation problems related to energy usage and occupant comfort(Ploennigs and Schumann, 2017). Along a similar vein, Egedorf and Shaker develop a methodology using causal learning statistics to model occupants’ thermal discomfort in smart buildings using temperature sensors (Egedorf and Shaker, 2017). A more classical approach to understanding HVAC settings in a building using optimal control is explored in Wang and Ma’s work (Wang and Ma, 2008). Here, a variety of optimal control and optimization methods are analyzed in the context of energy efficiency and/or cost-efficient control of HVAC systems, many of which utilize variations of different mechanisms.
A common goal for those implementing building control is energy-efficiency. Chen et al. propose a smart software system that balances the requests from different stakeholders (owners, operators, etc.) and also considers the preferences and locations of individual occupants (Chen et al., 2009). The framework enables building control while reducing energy consumption and maximizing occupant comfort.
In contrast to the above methods for human-building interaction, we propose a different approach. At the core of our approach is the design of a mechanism that can find the shared conditions (e.g. lighting and HVAC) that fairly account for all occupants’ preferences, and provides rewards to those who are willing compromise on the shared conditions for their co-workers. Additionally, our mechanism can pass on some of the building manager’s energy-efficiency incentives onto the building occupants, so they can also experience the benefit of energy-efficiency programs. Our framework is centered around a real-time application of a modified VCG mechanism; occupants are asked for their preferences on the shared source, and this information is used to calculate the social optimum and allocate rewards. The rewards are calculated such that occupants can only benefit from the mechanism, and none of the occupants have any incentive to misrepresent their preferences.
2.2 Mechanism design
Game-theoretic models have been widely used to model selfish agent behavior in a wide variety of applications, ranging from traffic network flow allocation to smart grid optimization. There is one thing common in the core of these problems: each user is trying to maximize a personal utility function (which can be function of various internal and external parameters) subject to personal constraints. Mechanism design is an approach to strategically design incentives for these users towards an objective. Most game-theoretic analyses rely on the assumption that the utility functions are known a priori, which may not be a realistic assumption to make in real-world situations; this is especially true in the case of the energy industry where there are many ‘noise’ variables affecting how well a building can operate.
While VCG mechanisms have many nice theoretical properties, it is rare to see their supposed effect in the real-world integrated with decision-making. There are some existing works in the body of literature that contribute to the simulation and/or deployment of VCG mechanisms. For instance, the mechanism is applied to numerical examples of wholesale energy markets to achieve social optimality by incentivizing truth-telling (Xu and Low, 2017). Similarly, there exists compelling work on simulating the VCG mechanism results in optimal energy load scheduling where social welfare is also measured monetarily (Samadi, Schober, and Wong, 2011). With regards to human computation, crowdsourcing tasks and designing optimal pricing policies also require incentive-compatible mechanism design; strong theoretical guarantees are shown with regards to case studies done using Mechanical Turk (Goel, Nikzad, and Singla, 2014).
We would like to emphasize that to the best of our knowledge, our work is the first to use VCG mechanisms in a real-world building control application. It is also one of the first papers to implement VCG mechanisms in real-world settings outside of the classical domains of auctions. We hope that this literature develops more in the coming years, as new technologies will have to reconcile the opinions of many users, and an understanding of how humans interact with different mechanisms is crucial for these systems to operate as desired.
In this section, we briefly introduce the mathematical formalism for modeling the decision making processes of users, as well as the formulation of our implemented mechanism. In later sections, we’ll use data from our experiments to explore the faithfulness of these models.
Remark 3.1 (Application of VCG mechanisms to our setting).
The most common application of VCG mechanisms is in auction-like settings. In these settings, people submit bids to ‘win’ a good; alternatively, there could be a scenario where there are multiple goods to be auctioned off to more than one person. This is not the case here, and we briefly elucidate some of the finer nuances of our application at a higher level before presenting the mathematics.
The VCG mechanism can be applied to any setting with a set of mutually exclusive outcomes experienced by a set of agents. In auctions, the different outcomes are which bidder wins the good. In our application, the outcomes are the 3 possible light settings. In its full generality, the VCG mechanism simply takes into account user preferences and outputs a socially-optimal outcome after having reconciled these reported preferences. In smartSDH, users bid for an optimal light setting. The chosen light setting may not be the optimal light setting for every user, yet, due to the shared nature of the office space, only one light setting can be chosen (i.e. the 3 outcomes are mutually exclusive) and every user experiences the effect of the outcome.
3.1 User model
First, we formally outline the assumptions of our human models. We begin by introducing some notation. Let denote the finite set of participants, and let denote the finite set of outcomes. Recall that our mechanism will choose one of the outcomes in to implement. Let denote the number of participants, and denote the number of outcomes. Without any loss of generality, we’ll let the labels of users be .
In our smartSDH experiment, the number of outcomes , which corresponds to three different light settings: (‘Normal’, ‘Bright’, ‘Very Bright’). Additionally, in smartSDH, we have participants.
We are now ready to introduce our assumptions on our human decision-making model.
Assumption 3.2 (User model).
For each user and outcome , let denote the cost user incurs when the chosen outcome is .
We refer to the vector
. We refer to the vectoras the type of the user.
Each user makes decisions to maximize their utility function, which is parameterized by their type and takes as inputs an outcome-incentive bundle :
Our mechanism will choose an outcome and issue each user a payment . We wish to model people’s preferences not only across outcomes , but across outcome-incentive bundles . This allows us to compensate users for compromising on the outcome; for example, if we choose an outcome that is particularly distasteful to a user (i.e. is very large), then we can compensate by giving them a very large gift card (i.e. choosing to be very large as well). The interpretation is that when a user’s type is , the chosen outcome is , and the incentive payment received is , their utility is .
We note a few things about Assumption 3.2. First, the user’s type captures all the relevant information for their decision making process. Second, the utility function is quasilinear in the incentive, which implicitly means that every user’s utility is given in the same units: incentive points.
Additionally, we will use the notation to denote the types of all
users. As per common game-theory convention, we will use the notationto denote the types of all users other than user .
Assumption 3.3 (Informational structure).
We assume that each user knows their own type . Additionally, no one other than knows . This includes other users and the mechanism designer.
Assumption 3.3 sits at the core of what we wish to model. When asked about their preferences, we assume users will answer based on their desired outcome, not their true preferences. Our goal is to design a mechanism that can calculate the outcome that is most socially desirable for all participants, without access to information about the user types . In particular, our mechanism will ask users to report their types; we let their reported values be denoted . Our mechanism must decide payments and the outcome based only on these reported values .
3.2 Vickrey-Clarke-Groves mechanisms
Given the user models, we can analyze wheter mechanisms will achieve desired outcomes. In this subsection, we’ll define VCG mechanisms, outline the desirable properties these mechanisms have in theory, and discuss extensions required for our application.
First, let us define the socially desirable outcome.
Definition 3.4 (Social welfare).
The social welfare of an outcome for users of type is given by:
We define the social optima as the maximizers of the social welfare:
Intuitively, the social welfare function calculates the sum total of everyone’s utilities, excluding the incentive payments. In other words, the social welfare function assesses how good each outcome is for all participants.
Now, we’ll provide a definition of a class of mechanisms known as Vickrey-Clarke-Groves mechanisms, and outline some of their theoretically desirable properties as well as nuances in our experiment’s particular implementation.
Definition 3.5 (Vickrey-Clarke-Groves mechanisms).
Given the reported types , a Vickrey-Clarke-Groves mechanism chooses the outcome:
Additionally, for each user , it pays:
Here, is an arbitrary function which does not depend on the reported value of user .
VCG mechanisms choose the outcome that is socially optimal for the reported types. Furthermore, it issues payments to every user based on the utility of all other users from the chosen outcome. Intuitively, this means that once the payments are incorporated, the utility function of every user essentially becomes the social welfare function. Formally, if we plug in to the utility function of user , we get:
Proposition 3.6 (Incentive compatibility).
For every set of user types , and for every user and reported type , we have:
Incentive compatibility should be interpreted as follows: denotes the true types of all the users, and is a possible lie which user could report. The left-hand side of Equation (1) is the utility user gains when everyone is truthful. The right-hand side is the utility derives when everyone except truthfully responds. Equation (1) states that any such deviations will only lower user ’s utility. Thus, truthful reporting by all users forms a Nash equilibrium. Note that this utility function is still parameterized by the true type , as this determines the utility the user actually experiences.
Next, we cover another desirable property for VCG mechanisms: individual rationality. In the typical applications of the VCG mechanism, users can opt-out and choose an outside option. For example, in auctions, a participant can simply choose to not bid at all and walk away. They will receive no utility from receiving a good, and they will lose no utility from paying the auctioneer.
However, we note that in our smartSDH experiment, this is not as well-defined. Even if a participant does not report a vote to our mechanism, they must sit in their office space and experience the light chosen by our mechanism (see 3.1). This warrants a minor modification of the typical individual rationality constraint, since the outside option becomes type-dependent, which we outline here. We will see that, if we allow Assumption 3.9, this type-dependence presents no significant issue, due to Proposition 3.10.
Assumption 3.7 (Nominal outcome).
Without any mechanism in place, there is a nominal outcome that would occur. For each user, their outside option is , which is the utility they get from the nominal outcome and no awarded points.
Definition 3.8 (Ex-post individual rationality).
A VCG mechanism has ex-post individual rationality if for all user types and any user :
The interpretation is that the nominal outcome would be the chosen outcome in the absence of our mechanism; this serves as the ‘outside option’. When we enforce ex-post individual rationality, this means that every user is better off when the mechanism is implemented, as opposed to when the mechanism doesn’t exist at all.
It’s important to note that this is a deviation from typical applications of the VCG mechanism. In smartSDH, a user cannot ‘walk away’ from their shared office space; rather, we have to ensure that their utility increases as a result of our mechanism’s deployment. From a technical perspective, the important distinction is that the utility of the outside option depends on the user’s type; this is what necessitates modification at a theoretical level.
Assumption 3.9 (Bounded user utilities).
The types of all users are bounded between and , i.e. for all and .
Proposition 3.10 (Existence of an ex-post individually rational mechanism).
Using , the VCG mechanism is ex-post individually rational.
First, note that for any and any . Thus, . Next, note that since for all by assumption, we have . The inequality in Equation (2) follows.
In the smartSDH experiment, can be thought of as the baseline wage that every user is given for participating in the study.
Lastly, we outline the last modification needed to implement a VCG mechanism in our office space setting. Users can come and go at any point in time, and they can modify their reported preferences at any point in time as well. The modification needed from the classical VCG mechanism applications is the temporal aspect of this.
Essentially, this modification is quite simple: transform all the quantities discussed above into rates. Rather than interpreting as the cost user incurs when the outcome is , we interpret as the cost per hour user incurs. Similarly, if the VCG mechanism decides to pay user a payment of , they are paid points per hour. Then, any time a user enters, leaves, or modifies their vote, we treat that segment as one round of the mechanism, weighted by its duration.
More formally, suppose the set of users logged on and their reported preferences are constant on the time interval . Then, in that time interval, our mechanism chooses the outcome . Each user who is logged on in that time interval receives a reward for their participation during the time interval .
3.3 Incorporating incentives for energy efficiency
We note that this framework readily allows one to incorporate incentives for energy-efficient behavior. For example, if a building costs dollars to operate at setting for one hour, we can introduce another participant of type . This term now shows up in each participant’s utility function through the VCG mechanism, and if everyone gains more than in utility from choosing over , then the mechanism will choose over . Each participant can then internalize the incentives, directly experience its benefits, and, as a group, decide whether or not the difference in cost warrants a change in settings.
However, for our experiment itself, we did not incorporate any energy-efficiency incentives; our primary goal was to find the light settings that were the most preferred by all the occupants in the office space. This possible extension is discussed later on in greater detail in Section 6.
3.4 Contributions and goal of the investigation
The goal of the present investigation was to test the application of the modified VCG mechanism to a smart building control application, dubbed ‘smartSDH.’ As mentioned previously, a VCG mechanism selects the socially optimal outcome among a set of possible outcomes – in this case the preferred brightness of overhead office lights – and then issues payments to users based on the decisions of all other users. In this sense, it is a mechanism where being truthful is the best strategy for the individual and the group. Although VCG mechanisms have many desirable theoretical properties when all participants act as rational agents, real-world application of VCG mechanisms are limited. Therefore, the purpose of this experiment is to 1) assess the feasibility of our modified VCG mechanism in the context of smart building control in group settings and 2) evaluate the effect of smartSDH on light level satisfaction, incentive satisfaction, and energy consumption across time.
This study analyzes perceptions and behaviors of users in a shared office space who interacted with a VCG-operated smartSDH over a 5 month period. Participants interacted with smartSDH via a web portal, where they reported their preference for the brightness of the overhead office lights in real time so the VCG mechanism could determine the socially optimal light setting for the group. Users were allotted points according to the modified VCG mechanism. Whenever the total points earned crossed a threshold, a lottery was held for gift cards; over the 5-month period, we rewarded a total of $2,900 worth in gift cards. Additionally, to emphasize the non-competitive aspect of sharing an office space, whenever the total points earned crossed another threshold, we hosted a catered lunch for all the participants of smartSDH. Because adaptation to technology often evolves over time (DeSanctis and Poole, 1994), sometimes gradually and sometimes sporadically, we chose to evaluate the behavioral outcomes over the course of three distinct time periods (T1 = Wk1 - Wk7; T2 = Wk 8 - Wk15; T3 = Wk16 - Wk 22). In such a setting, three research questions become salient:
RQ1. What is the influence of using the VCG-operated smartSDH on light and incentive satisfaction across time?
RQ2. What is the influence of using the VCG-operated smartSDH on energy saving?
RQ3. What is the relationship between light level preferences and atmospheric conditions including humidity, temperature, pressure, and solar radiation?
Twenty-seven undergraduate students from a public research university in California, USA participated in the longitudinal study in exchange for prize items, which were allotted based on their performance in the point based system. The preliminary survey was optional, and out of the 27 participants, 18 responded. The majority of users were men (gender: 73% men, 22% women, 5% prefer not to respond) with 50% of users between the ages of 22-25, while the rest were above 26 years old. The majority of users (33%) had incomes between $35,000 and $40,000, and the rest had lower incomes. Before commencing the experiment, users were somewhat satisfied with the light conditions in the office, . As for ideal incentives, nearly everyone preferred gift cards from Amazon, iTunes, and Google Play (94%) over the other options, which were complimentary vouchers for drinks at a campus coffeeshop, or lotteries for big-ticket items such as Apple Watches, Fitbits, and EarPods.
Participants in the longitudinal study interacted with smartSDH in an open office space with cubicles on a university campus over a period of five months. The desks in the office space were divided into different lighting zones with a set of overhead lights serving as the primary source of illumination for each zone. The smartSDH operated independently for each zone. From the work hours of 9 AM to 5 PM local time, participants were only able to control their office lights through the smartSDH web portal. After work hours, the light switches returned to normal operation.
Each participant had access to the password-protected smartSDH web portal. In addition to allowing users to vote for their light settings in real time, the web portal also provided participants with visualizations of the state of the office space. Users could view their personal point totals, the currently implemented light setting, and the global progress to the individual and communal incentive thresholds. They could also see which occupants were present in their zone. Users were able to monitor their floor’s lighting lumen level and temperature in real-time, with a refresh interval of one second. A view of the designed portal can be seen in Appendix A.
To adjust the light conditions, participants first selected their preferred light setting from three available options: ‘Normal’, ‘Bright’, ‘Very Bright’, which corresponds to 33%, 67%, and 100% of the maximum possible illumination. Once participants selected their preferred light setting (e.g., ‘Bright’), the web portal requested follow-up information about the two settings not chosen (e.g., ‘Normal’ and ‘Very Bright’). Specifically, the web portal asked participants to indicate the extent to which they preferred their chosen light setting over each of the alternative settings (e.g., “How many points are you willing to pay to have the lights set to BRIGHT instead of NORMAL for one hour?”) from 0 to 100 points (see Appendix A for illustrations of the voting procedure). These two measures—preferred light setting and relative preference—contributed to the VCG mechanism’s chosen outcome. Every time a user logged on, logged off, or changed their reported preference, smartSDH calculated based on the reported values of all users who were currently logged on, and rewards each user ’s account with points , according to the calculations in Section 3. Users were able to modify their reported preferences at any time, however they were required to be present in the office to vote. Office presence was enforced via the browser’s geolocation data.
In addition to the points earned from the VCG mechanism, we also rewarded users for completing a repeatable survey about their experiences with smartSDH. This repeatable survey included Likert-scale questions about the incentives provided, the design of the web portal, the level of comfort participants experienced, their satisfaction with the current light setting, the level of awareness about energy-saving actions they could take, and their productivity in the office.
The points were converted to values by the users in two ways throughout the study. First, whenever the total points earned by all participants exceeded a threshold, we held a lottery for gift cards. The probability of one winning the lottery was proportional to the number of points in one’s account, and multiple gift cards were given each time the lottery threshold was reached. This was the individual incentive. Second, their points built toward a communal incentive. Whenever the total points earned by all participants exceeded the communal incentive threshold, we provided a catered lunch to participants.
The users interacted with a web portal where they submitted their preferences for the overhead lighting in the office. The Building Automation and Control (BACnet) protocol was used for communications between the web portal interface and the light actuators in the space. We used BACNet because it is the prominent protocol for HVAC applications.
Light setting preference. Participants selected one of three available settings: (‘Normal’, ‘Bright’, ‘Very Bright’), which corresponds to 33%, 67%, and 100% of the maximum possible illumination.
Relative preference. For the two settings not chosen, the web portal asked “How many points are you willing to pay to have the lights set to PREFERRED instead of ALTERNATIVE for one hour?” For each of the two alternatives, users provided an integer between 0 and 100. We assumed , where is as defined in creftypecap 3.9. As a quality-of-life feature, we also included a button that allowed users to set their vote to with one click.
Light level satisfaction. Two items measured participants’ satisfaction with their lighting conditions on a given day, “I am satisfied with today’s lighting conditions,” and “Today’s lighting conditions were uncomfortable.” Responses were recorded on a five-point Likert-type scale from 1 (strongly disagree) to 5 (strongly agree) ( = .91).
Incentive satisfaction. Two items measured participants’ satisfaction with the incentives provided on a given day, “I am happy with the current incentives provided,” and “The current incentives are not satisfactory.” Responses were recorded on a five-point Likert-type scale from 1 (strongly disagree) to 5 (strongly agree) ( = .86).
User interface satisfaction. Two items measured participants’ satisfaction with the web portal used to manipulate the smart lights on a given day, “I am satisfied with the current web interface,” and “The web portal leaves much to be desired.” Responses were recorded on a five-point Likert-type scale from 1 (strongly disagree) to 5 (strongly agree) ( = .78).
Energy consumption (% savings/time). We measured energy consumption in terms of percentage savings. That is, we calculated the implemented lighting over a baseline of 100% lighting. This is depicted in Figure 2.
Humidity (%). Atmospheric sensors in the office space measured relative humidity in percentage water vapor – where 100% corresponds to fully saturated air at dewpoint. atmospheric sensors placed in the room.
Temperature (deg F). The above-mentioned sensors measured the temperature. Another atmospheric condition captured by the aforementioned sensors.
Pressure (Hg). The sensors measured barometric pressure in terms of units of mercury (Hg). More precisely, a unit of Hg denotes the pressure exerted by a column of mercury 1 inch (25.4 mm) in height at the standard acceleration of gravity. Another atmospheric condition captured.
Solar radiation (). Also known as solar irradiance, and it is measured in terms of power per unit area (Watts per square meter, in this case).
Average weekly illumination intensity under the VCG-operated smartSDH is illustrated in Figure 2. Implementation of a VCG mechanism to smart light control shows promise for the stability and efficiency of our power grid.
Figure 2 shows the voting behavior of a user throughout the five-month period of the game. The y-axis, Reported Preferences, is the number of points the user is willing to pay to change the lights when it is at that setting. For instance, in this case, the user is willing to pay nothing to change the “Very Bright Lights” setting because it is their most preferred setting and would prefer that it remains at that setting. The shaded regions in the background show the implemented light setting during that time; blank regions indicate that the user was not logged in at the time or actively voting. In this instance, the user initially signals that they do not favor the dimmest light settings and medium light settings and the VCG mechanism implements the dimmest light. As time evolves, the user realizes they are more willing to accept the medium light setting since compromise has proven to output a brighter light overall, as seen in the short instances of blue and, eventually, red shading in the background.
5.1 Effect of smartSDH on light level and incentive satisfaction across time (RQ1)
Satisfaction with the light levels across time while controlling for user interface satisfaction was analyzed using a 3 (T1 vs. T2. vs. T3) within-subjects ANOVA. We observed a significant effect of time on light level satisfaction after controlling for user interface satisfaction, = 4.21, , . Participants reported greater satisfaction with light levels at T1 , than T2 , , and T3 , , . These results suggest that after an initial period of satisfaction with the light levels, participants became less satisfied with light levels (Figure 3).
Satisfaction with incentives across time while controlling for user interface satisfaction also was analyzed using a 3 (T1 vs. T2. vs. T3) within-subjects ANOVA. We observed a significant effect of time on incentive satisfaction after controlling for user interface satisfaction, , , . Participants reported greater satisfaction with light levels at T1 , than T2 , , and T3 , , . Despite the theoretical properties of VCG mechanisms, the experimental data suggest the VCG mechanism does not have an appreciable effect on light level and incentive satisfaction of users, shown in Figure 3.
5.2 Energy saving during smartSDH use over 5-month period (RQ2)
To evaluate the effect of the VCG-operated smartSDH on energy consumption, we compared the intensity of the overhead lights in the office space over the 5-month period of smartSDH use to the intensity of the overhead light during normal operation (i.e., 100% of the maximum possible illumination). Our results indicate that employing smartSDH in the office from 9AM to 5PM reduced energy consumption by 35.22% over the 5 month study period.
5.3 Relation between reported light setting preference and atmospheric conditions (RQ3)
) presents means, standard deviations, and bivariate correlations for atmospheric conditions and reported light setting preference. We observed a significant relationship between participants’ reported light setting preference and atmospheric data taken in real time including humidity, temperature, and solar radiation. Pressure did not significantly correlate with reported light setting preference, p = .292. We take these results to suggest that mechanisms for building control may require users to constantly report their preferences, as it would be difficult to build estimators of these preferences from externally observable factors.
In this study, we implemented a modified VCG mechanism, smartSDH, to determine the brightness of overhead lights in a shared office space. The goal of smartSDH was to determine the light setting that maximized the sum of everyone’s utility fairly. To do so, smartSDH would issue incentives as needed to promote truthful reporting of preferences, as well as ensure all users were better off with smartSDH than a nominal light setting along with no rewards. To this end, VCG mechanisms were an appropriate choice for satisfying many of our desiderata.
Our first research question dealt with the influence of smartSDH on light and incentive satisfaction. In theory, smartSDH should improve the satisfaction of participants with the lighting. Regarding the first hypothesis on measuring the influence of time on light and incentive satisfaction, quantitative results shown in Section 5 suggest that there is no appreciable increase in satisfaction. We see that in T1, there was greater satisfaction, on average, across all participants than compared to time periods T2 and T3. Additionally, participants were significantly more satisfied with incentives during T1; however, this satisfaction deteriorated in periods T2 and T3. This could be due to a variety of reasons. First, a few survey respondents stated that they were “too busy” to regularly report their preferences, which implies that the user interface for the portal was not as convenient as desired. So as time went on, people felt the task more burdensome resulting in a more sporadic voting pattern as time went on.
Secondly, there is evidence in the literature about “technology burnouts” (see Armel et al. (2013) and the references therein). These references discuss the issues with technology adoption, highlighting the fact that with the right system in place, it is possible for behavioral approaches such as the one in smartSDH to be much more effective in reducing energy consumption, which can improve the result for our second hypothesis on energy saving. This suggests that there was initially a novelty effect for our users, who were more active in the first time period. In the second and third time periods, users were generally less engaged and motivated to continue voting. In general, if the user population was more aware of appliance information (in our case, about the lights’ energy consumption), it would facilitate greater energy savings. For future work, this suggests more testing on how the user interface is to obtain better insights for design decisions as well as increasing users’ awareness on the mechanics of the sensors.
Our second research question dealt with the influence of smartSDH on energy consumption. We found that there is a significant reduction in energy usage. This is noteworthy because though the participants generally felt inconvenienced and dissatisfied with the implemented mechanism, their behaviors ended up effectively reducing overall energy usage. For future directions of the work, it might be useful to incorporate energy-saving incentives in the model as discussed in Section 3.3. This would allow users to internalize the externality – in other words, to shift the external cost to their own internally accrued cost. Moreover, technology adoption rates are generally higher for software solutions such as smart meters (analogous to the sensors used in smartSDH) since the hardware is generally installed by an external party such as the utility company. Since this has virtually no cost to the user, along with zero personal installation effort, there is much to be said about integrating smart software solutions to reduce energy consumption in an aggregate energy source.
Our last research question dealt with the relationship between the preferences of users and atmospheric conditions. Our results show that there is a significant relationship between participants’ votes and factors such as humidity, temperature, and solar radiation. However, there is not much qualitative information that we could obtain about why participants felt that these factors influenced them to vote in a particular manner. The data from this experiment suggests that estimating these preferences from environmental factors is not a viable avenue, because it indicates that mechanisms for smart building control may require occupants to consistently report their preferences. This is because these preferences cannot be accurately predicted from externally observable factors. In addition, the free-form qualitative survey also suggest that people experienced no appreciable increase in satisfaction as a result of the mechanism. We explore a few potential reasons for this, all of which are interesting avenues for future study.
We note that, due to limitations of this study, our geolocation methods required users to regularly sign on from their browser; otherwise, users could vote remotely or leave their computer on to earn points illicitly. We hope that future studies can explore methods to reduce the intrusiveness of these mechanisms.
Another possible reason why satisfaction levels were not significant may be because it may be that , as defined in Assumption 3.9, was set too low. This again was due to the limitations of our study. Our value for was determined by our experimental budget. That is, we set as a base rate for participation, and needed to bound how much money we would give out in rewards over the duration of the experiment. For smartSDH, our budget was over $100 per participant. In practical applications, it may be beneficial to relax the requirement that all users are better off due to the mechanism. Under this relaxation, some users may wind up paying money into the mechanism, but it would still implement the socially optimal outcome. This ties back in with the idea of “internalizing the externality” that may be caused due to differences in preferences of lighting. To implement this, there must be a way to force participation against a user’s will. We note that this is not likely to be viable in an academic research experiment.
Finally, we note that many qualitative aspects of our study suggest a status quo effect. Some participants told us during the catered events that they were happy with smartSDH because the lights were typically too bright. In contrast, some other participants insisted that no one would want lights so dim. Anecdotally, it seemed some participants were dissatisfied because they did not believe that the mechanism was implementing the social optimum. It is interesting to note how it is more socially acceptable to walk into a full office and turn up the lights than it is to walk into a full office and turn down the lights; these social contexts likely had a factor in the experience of participants of this study. We think examining the effect of the status quo when new IoT technologies are deployed in these settings is a very interesting direction for future research.
6.1 Concluding Remarks
In summary, the contributions of this paper are as follows:
The implementation of modified VCG mechanism to determine the brightness of the overhead lights for 27 participants over a 5-month period in their actual office space, giving out $2,900 in rewards. We emphasize that it is challenging to implement the VCG mechanism outside of typical auction settings; see LABEL:sec:mec_des.
Querying users’ perceptions of light levels, the received incentives, satisfaction with the user interface, and awareness of energy consumption patterns.
Testing the above-listed hypotheses (listed above). Although some factors had significant correlations with the reported preferences, the predictive value was generally very poor. This implies that mechanisms for building control may require users to constantly report their preferences, as it would be difficult to build estimators of these preferences from externally observable factors.
Outlining some barriers to the implementation of VCG mechanisms in IoT settings, and potential reasons our study did not achieve the expected gains in satisfaction.
Despite the theoretical properties of VCG mechanisms, the experimental data suggests that the VCG mechanism did not have an appreciable effect on the satisfaction or awareness of users in any of these user-perception categories. Our data suggests that rational agent models may require some modifications to capture how humans typically will interact with an IoT technology in the background of their work life. In particular, the preferences reported show some temporal variation that may be due to users learning their own preferences, which is in contrast with the typical rational agent model. Furthermore, there is a need to design the user experience to be as minimally intrusive as possible, and potentially relax the requirement that all users are better off with the existence of the mechanism.
Ioannis C. Konstantakopoulos was a Scholar of the Alexander S. Onassis Public Benefit Foundation. The authors would like to thank Chris Hsu, the applications programmer at CREST Laboratory, UC Berkeley, who developed and deployed the web portal application as well as the social game data pipeline architecture. We would like to thank Andrea Bajcsy for many helpful discussions on best practices for experiments with human behavior. We would also like to thank Claire Tomlin, without whom this experiment may never have been deployed. Additionally, we want to thank UC Berkeley Student Technology Fund for the Award of Student Technology Fund Initiative – Large-Scale Project (2018). Finally, we would like to thank the participants of smartSDH.
- Armel et al. (2013) Armel, K. C.; Gupta, A.; Shrimali, G.; Albert, A. 2013. Is disaggregation the holy grail of energy efficiency? The case of electricity. Energy Policy 52:213-234.
- Aswani et al. (2012) Aswani, A.; Master, N.; Taneja, J.; Smith, V.; Krioukov, A.; Culler, D.; and Tomlin, C. 2012. Identifying models of HVAC systems using semi-parametric regression. In Proc. of the American Control Conf., 3675–3680.
- Baron, Rea, and Daniels (1992) Baron, R. A.; Rea, M. S.; and Daniels, S. G. 1992. Effects of indoor lighting (illuminance and spectral distribution) on the performance of cognitive tasks and interpersonal behaviors: The potential mediating role of positive affect. Motivation and emotion 16(1):1–33.
- Boman et al. (1998) Boman, M.; Davidsson, P.; Skarmeas, N.; Clark, K.; and Gustavsson, R. 1998. Energy saving and added customer value in intelligent buildings. In Third Int. Conf. Practical Application of Intelligent Agents and Multi-Agent Technology, 505–517.
- Chen et al. (2009) Chen, H.; Chou, P.; Duri, S.; Lei, H.; and Reason, J. 2009. The design and implementation of a smart building control system. In 2009 IEEE International Conference on e-Business Engineering, 255–262. IEEE.
- Clearwater (1995) Clearwater, S. H. 1995. Auction-based control system for energy resource management in a building. US Patent 5,394,324.
- Cortina (1993) Cortina, J. M. 1993. What is coefficient alpha? an examination of theory and applications. Journal of applied psychology 78(1):98.
- DeSanctis and Poole (1994) DeSanctis, G., and Poole, M. S. 1994. Capturing the complexity in advanced technology use: Adaptive structuration theory. Organization Science 5(2), 121–147.
- Egedorf and Shaker (2017) Egedorf, S., and Shaker, H. R. 2017. Adverse condition and critical event prediction in cranfield multiphase flow facility. In 2017 International Conference on Sensing, Diagnostics, Prognostics, and Control (SDPC), 557–564. IEEE.
- Frontczak et al. (2012) Frontczak, M.; Schiavon, S.; Goins, J.; Arens, E.; Zhang, H.; and Wargocki, P. 2012. Quantitative relationships between occupant satisfaction and satisfaction aspects of indoor environmental quality and building design. Indoor air 22(2):119–131.
- Goel, Nikzad, and Singla (2014) Goel, G.; Nikzad, A.; and Singla, A. 2014. Mechanism design for crowdsourcing markets with heterogeneous tasks. In AAAI Conference on Human Computation and Crowdsourcing (HCOMP).
- Groves (1973) Groves, T. 1973. Incentives in teams. Econometrica 41(4):617–631.
- Jia et al. (2017) Jia, R.; Dong, R.; Sastry, S. S.; and Spanos, C. J. 2017. Privacy-enhanced architecture for occupancy-based hvac control. In 2017 ACM/IEEE 8th International Conference on Cyber-Physical Systems (ICCPS), 177–186.
- Jin et al. (2018) Jin, M.; Liu, S.; Schiavon, S.; and Spanos, C. 2018. Automated mobile sensing: Towards high-granularity agile indoor environmental quality monitoring. Building and Environment 127:268–276.
- Karmann, Schiavon, and Arens (2018) Karmann, C.; Schiavon, S.; and Arens, E. 2018. Percentage of commercial buildings showing at least 80% occupant satisfied with their thermal comfort.
- Ma, Anderson, and Borrelli (2011) Ma, Y.; Anderson, G.; and Borrelli, F. 2011. A distributed predictive control approach to building temperature regulation. In Proc. of the American Control Conf., 2089–2094.
- McQuade (2009) McQuade, J. 2009. A system approach to high performance buildings. United Technologies Corporation, Tech. Rep.
- Nisan et al. (2007) Nisan, N.; Roughgarden, T.; Tardos, E.; and Vazirani, V. V. 2007. Algorithmic Game Theory. Cambridge University Press.
- Ploennigs and Schumann (2017) Ploennigs, J., and Schumann, A. 2017. From semantic models to cognitive buildings. In AAAI.
- Ryherd and Wang (2008) Ryherd, E. E., and Wang, L. M. 2008. Implications of human performance and perception under tonal noise conditions on indoor noise criteria. The Journal of the Acoustical Society of America 124(1):218–226.
- Samadi, Schober, and Wong (2011) Samadi, P.; Schober, R.; and Wong, V. W. 2011. Optimal energy consumption scheduling using mechanism design for the future smart grid. In 2011 IEEE International Conference on Smart Grid Communications (SmartGridComm), 369–374. IEEE.
- Shah and Yaqoob (2016) Shah, S. H., and Yaqoob, I. 2016. A survey: Internet of Things (IOT) technologies, applications and challenges. In 2016 IEEE Smart Energy Grid Engineering (SEGE), 381–385. IEEE.
- Sundell et al. (2011) Sundell, J.; Levin, H.; Nazaroff, W. W.; Cain, W. S.; Fisk, W. J.; Grimsrud, D. T.; Gyntelberg, F.; Li, Y.; Persily, A.; Pickering, A.; et al. 2011. Ventilation rates and health: multidisciplinary review of the scientific literature. Indoor air 21(3):191–204.
- Vickrey (1961) Vickrey, W. 1961. Counterspeculation, auctions, competitive sealed tenders. J. Finance 16(1):8–37.
- Wang and Ma (2008) Wang, S., and Ma, Z. 2008. Supervisory and optimal control of building hvac systems: A review. HVAC&R Research 14(1):3–32.
- Wargocki et al. (2008) Wargocki, P.; Seppanen, O.; Andersson, J.; Boestra, A.; Clements-Croome, D.; Fitzner, K.; and Hanssen, S. O. 2008. Indoor climate and productivity in offices. how to integrate productivity in life cycle costs analysis of building services.
- Xu and Low (2017) Xu, Y., and Low, S. H. 2017. An efficient and incentive compatible mechanism for wholesale electricity markets. IEEE Transactions on Smart Grid 8(1):128–138.
Appendix A Tables and Figures
|Light setting preference||61.81||27.84||1.00|
|Temperature (deg F)||55.00||5.17||.017||.005|
|Solar radiation ()||249.21||370.72||.27||.001|
Note. r represents the zero-order correlation with light level preference. represents the corresponding -value for each correlation.