1 Related Work
In this section, we first introduce the main data-driven visualisation and visual analytics research in energy demand management problems, then discuss recent progress on spatiotemporal visual analytics.
Data-driven energy demand analysis research focuses on energy demand forecasting since the seminal concept proposed in 1984 . Amasyali and El-Gohary reviewed the data-driven building energy consumption prediction studies 
and reported, among the machine learning algorithms used for energy forecasting, 47% and 25% of the studies utilized ANN and SVM, respectively, to train their models. In contrast, only 4% of the studies utilized decision trees. 24% of the studies utilized other statistical algorithms such as multiple linear regression, least squares regression, and auto regressive integrated moving average. Some recent progress in time series data such as LSTM, RNN are also widely applied to energy demand forecasting tasks. For example, CNN-LSTM neural networks are used to learn spatial and temporal features to predict housing energy consumption.
Energy consumption pattern analysis also attracted research attention. For example, Hunt et al. model energy demand to analyse the trends and seasonal effects 14]. An association rule mining based quantitative analysis approach is designed for residential electricity consumption patterns analysis . Occupant behavior has a major impact upon energy demand. Frequent pattern mining of human behavioral variations such as the high uncertainty about the order of use, varying time of use, and increased or reduced frequency of use of appliances, are employed in the energy consumption decision patterns mining 16]. However, Markov chains are limitations in accurately capturing the coordinated behavior of occupants and are prone to over-fitting. Rich features related to the coordination of occupants’ activities are employed to compare occupant to nearest-neighbor behavior .
Visual Analytics for Energy. In the energy sector, prime visualisations such as graphs and bar charts are extensively used to provide comparable energy consumption data over time. The seminal interdisciplinary literature presents several power system visualisation techniques to help analyze the relationships between network power flows using techniques such as animation, contouring of bus and transmission line flow values, and interactive 3D data visualisation . Coincidence factor based heatmap visualisation is used to identify peak demand charges and avoid power outage . Calendar-like pixel visualisations, with color boosting to anomaly scores, integrated with spiral visualisation, line chart, and tree maps that are designed to detect anomalies in power consumption data . FigureEnergy is an interactive visualisation that allows users to annotate and manipulate a graphical representation of their electricity consumption data, and therefore make sense of their past energy usage by understanding when, how, and for what purpose, some amount of energy was used . Operational performance is integrated with building information modelling (BIM) as a visualisation dashboard to support the management of a building’s performance . Ambient and artistic visualisation for residential energy use feedback is explored, where Phyllotaxis design, Hive design, and Pinwheel design in energy use are discussed . Matches, Mismatches, and Methods for Multiple-View Workflows for Energy Portfolio analysis are discussed . Mosaic Groups mapping encoded by household energy use combines with geodemographics to enable a better understanding energy user types in the UK . GreenGrid is designed to explore the planning and monitoring of the Electricity Infrastructure. Geographic layout coming with a weighted network interface is designed to quickly identify where the system would be most likely to separate if an uncontrolled islanding event were to occur .
Spatiotemporal Visualisation. Spatiotemporal data visualisation has been extensively researched and applied in various domains [2, 9]. A majority of the studies are trajectories analysis [11, 20, 30]. Pattern extraction can be applied to obtain significant latent patterns from the movement data. Space-time cube representation is an information visualisation technique where spatiotemporal data points are mapped into a cube [19, 3]. AirVis, is designed to assist domain experts to efficiently capture and interpret the uncertain propagation patterns of air pollution based on graph visualisations 
. Multidimensional spatiotemporal data are modelled as tensors and then decomposed to extract the latent patterns for comparison and visual summarization
. Flow maps are employed to track the clustering behaviors and direction maps, drawn upon the orientation of vectors, can precisely identify the location of events. Extending the density difference model, Kim et al. proposed a gravity-based flow extraction model, which can effectively separate human movement from spatiotemporal data without using trajectory information . A population based vector field was proposed to visualise time-geographic demand momentum. By representing transport systems as vector fields that share the same time–space domain, demand can be projected onto the systems to visualize relationships between them .
2 Data and Task Abstraction
This section introduces the background of our study, describes the relevant concepts and datasets, and summarizes the requirements.
Problem Statement. In the energy demand management practice, experts or decision makers always wish to quantify energy consumption, to understand location differences, and to understand consumer preferences by consumption behaviour analysis. However, most of the published work in this area focuses on analysing the energy consumption fluctuations over time in a certain area , and that does not meet domain requirements.
To address this issue, we conducted intensive discussions with two experts in the energy sector and categorised energy demand into Temporal Energy Demand (TED), Spatial Energy Demand (SED), and Spatiotemporal Energy Demand-shift (ST-EDS). TED refers to energy consumption fluctuation over a period of time in a certain area. The topic is extensively discussed in the energy sector, and the Peak-load Shifting mentioned above in the introduction section is part of the temporal energy demand management problem; SED is defined as the geospatial demand distribution across a certain area at a specific moment. ST-EDS is an important yet unexplored research problem that is introduced in this paper. It is defined as the spatiotemporal changes in energy consumption for a certain area during different periods. The ST-EDS can help discover energy consumption behaviour patterns, and thus lead to better demand management and planning.
Summary of Analysis Tasks. We summarize the following analysis tasks, incorporating the feedback from experts, to support our exploratory analysis. They are: [T.1] Holistically analyse the temporal and spatial energy demand. A challenge for the experts to analyze the demand is the spatiotemporal nature of the energy consumption data. The entanglement of spatial and temporal distributions makes it difficult for energy experts to make and verify their hypotheses about the demand and its dynamic changes. In contrast to the vast traditional energy demand literature that discusses the two issues separately, an exploration that considers both temporal and spatial information comprehensively will have the potential to discover new knowledge. [T.2] Modelling the energy demand-shift through a suitable algorithm and support exploratory knowledge discovery. The geospatial dynamics of energy consumption reflect the behaviour of household occupants and play a pivotal role for better demand management and planning. However, there are very few studies that address the issue. It requires modelling the demand-shift mathematically, and devising a user-friendly interface to support exploratory knowledge discovery. [T.3] Quantify the preceding exploration results. Energy experts raised the requirement of quantifying the exploration results as numbers, is important for comparison and evaluation of the significance of the trends or patterns in the knowledge domain.
Design Rationales. Based on analysis tasks, we propose the following design rationales. D.1 Scalable, coordinated and appropriate temporal and spatial energy demand visualisation. This requires creating a compact visualisation in a form familiar to domain experts’ mental simulations and follows Shneiderman’s seeking Mantra  (T.1, T.2). D.2 Highly interactive design to reinforce understanding. An exploratory analysis requires intensive interactions to explore consumption data and to acquire knowledge about correlations between the various factors (T.1, T.2). D.3 Appropriate symbols and colour mapping for intuitive metaphors. Intuitive representation is an essential element of most visualisation systems. Carefully designed visual metaphors can help experts reduce the visual burden and improve their understanding of the real situation. Visual mapping of the discovered patterns can facilitate visual sense-making and help to interpret the human behaviour behind the energy consumption behaviour (T.3).
This section describes the method of the visual analytic system.
3.1 Overview of the Approach
Motivated by the domain requirements, we designed and implemented to support an exploratory analysis. Fig. 1 gives an overview of the approach. It consists of 1) front-end visual analytic interface, from which users may perform an exploratory analysis and define demand-shift tasks through interactions; 2) back-end server. It stores the data and performs demand-shift modeling based on the users’ setting.
The control panel (CP) is usually the start point for demand fluctuation exploration and demand-shift analysis task definitions. As the step (a) in Fig. 1 shows, users can directly explore the spatial demand for a certain area over a specific period (T.1), and define new demand-shift analysis tasks through interactions associated with the buttons on View CP.3 (see Fig. : Visual Exploration of Spatiotemporal Energy Demand). Three kinds of analysis tasks are supported: the demand-shift of the peak-valley period, of a regular-split period, and of multiple periods. The tasks will be modelled and computed at the back-end engine (T.2) as specified in Section 4.3. The results will be visually summarized in the index view DS. Users are able to choose the result of interest and view the details on View GV.
The energy data are usually multiple-grained time serial records with geospatial information of the consumers. We use the electrical energy demand as a representation in this paper, more forms of energy data such as district heating will be supported in our extended work. We collected the electrical household consumption data from the Pudong area in Shanghai for a whole year. The electrical energy consumption data are usually daily-grained with peak and valley detailed time serial data. Fig. 2 shows sample rows of the data in seven columns. Among them, (Positive Active Power) gives the total electrical consumption of the day; (Positive Active Power at Peak Time) gives the peak period (6:00 – 22:00), and (Positive Active Power at Valley Time) gives the valley period (22:00 – 6:00) energy consumption value of the day.
3.3 Spatiotemporal Demand–shift Modelling
As stated in Section 4.1, the users define the spatiotemporal demand-shift analysis tasks in the interface and send the settings to the backend engine for demand-shift modelling. In this section, we describe our potential flow-based modelling algorithm.
We observed that the geospatial energy demands change continuously over time, and thus the energy demand is a continuum occupying a simply-connected region in the time dimension with a irrotational characteristic. Inspired by the advancement of fluid dynamics and continuum mechanics, such a continuum can be modelled as a potential flow . Formally, the spatiotemporal demand-shift is defined as . The spatiotemporal demand–shift is modelled as the flow velocity , a vector field to describe the rate of demand change. It is defined as the gradient of the velocity potential that refers to the spatial energy demand change at the selected moments or periods.
In order to model the spatial energy demand, a kernel density estimation based approach is proposed to encode discrete household energy consumption into a continuous representation. Such an operation can efficiently generate a smoother vector field. In detail, it is modelled as, where the household locations, and the normalised energy demand to re-weight demand strength; bandwidth (or smoothing), a matrix, which is symmetrical and positive definite; the kernel function. We chose to use the gaussian kernel in this work. The spatial energy demand change can be modelled as the discrepancy between the estimated of different moments.
4 Visualisation and Interactions
In this section, we describe the visual interface and interaction of the system. As Fig. : Visual Exploration of Spatiotemporal Energy Demand shows, the system consists of four coordinated components. The panel view (CP) is usually the start point for temporal, spatial, and demand-shift analysis through highly interactions. As Fig. 1 shows, the demand-shift analysis task will be modelled and computed at the backend and visualise the results in the view GV and DS ( Fig : Visual Exploration of Spatiotemporal Energy Demand). In this section, we describe the key visual design details.
4.1 Control Panel
The control panel (View CP) displays the temporal energy demand (View CP.0) and provides the main functional interactions (View CP.2 and CP.3) to support spatial demand and demand-shift analysis.
In detail, the temporal energy demand view (CP.0) utilizes a stream graph to visualise the peak and valley energy demand. The user is able to toggle the auxiliary analysis line (yearly, quarterly, monthly average demand, and peak to valley ratio) in the View CP.2 to select the period of interest for further temporal analysis (T.1, T.2). The demand-shift (T.3) is started through the functional buttons in the view CP.3. After a user has a period of interest to analyse through the fore step, he or she can define the explore demand-shift tasks of different type of periods: peak-valley, regularly-split period, or multiple periods. The user can toggle the corresponding button and then select the period(s) of interest by brush operation on the View CP.0 and then toggle the compute button in View CP.3 to generate the results and list them in the Demand-shift visual index (view DS).
4.2 Geographical View and Demand-shift Visual Index
The geographical view GV enables users to view the spatial demand and demand-shift in the finest detail on a large map. It includes: 1) Spatial demand visualisation. A 3D hexagon visualisation (see Fig. 1 c.1) is designed for the energy demand exploration (T.1), where the height is proportional to the energy demand and the colour depth is proportional to the household number. 2) Demand-shift visualisation. It gives the spatiotemporal demand-shift computation result from the backend engine. The Demand-shift Visual Index (View DS) is designed to give a visual summary and glance of the demand-shift generated by the exploration task (T.2). The view DS also works as a visual index for the users to quickly locate the demand-shift of interest and view the details on the Geographical View GV.
We introduce four visual elements to represent the demand-shift, as shown in Fig. 3: 1) a flow map is used to visualise the demand-shift where the length is encoded as the rate of demand change () ; 2) a demand-shift window on the geographical view gives the coverage of analysis and the colour is encoded as the spatial energy demand change . Such window-shape design does not obscure the map and gives the necessary quantification information for analysis; 3) a demand-shift color legend gives the corresponding absolute value of spatial energy demand change in the grid area. We use absolute value in response to the task T.3 for quantitative analysis. 4) a demand-shift badge also encodes the spatial energy demand change . It has the same meaning as the demand-shift window. It gives a visual summary of the demand change of the grided areas on the visual index view. We use a solid grid instead of the frame design as it is much smaller on the visual index; as well, a solid design is more prominent.
4.3 Energy Demand Meter View
The energy demand meter view (View ED) gives the statistical overview of the energy demand and the distributions of the demand on a daily and household basis. The design of the meter is consistent with the charting used in the energy sector to give a quantified overview of the total, peak, valley, and average daily demand.
5 Case Study
The implemented system is evaluated by two energy experts. Expert A is a professor in Energy, who has over ten years of socioeconomic research in energy and Expert B is senior engineer working on the power system planning from the power supply company. We give a quick training on the core concepts and operations of the system. They work as a team to evaluate the system and we collected their feedback on the system usability, visual design, and limitations.
These experts started to explore the temporal energy demand fluctuation from the control panel, first observing an obvious bimodal pattern – a larger surge demands in the summer and a lower surge in the winter. This makes sense as people consume more electricity for cooling using air conditioners in the summer than heating in the winter in Shanghai. From the peak to total ratio curve, they observed: 1) A weekly periodic pattern which reflects the household behaviour pattern. 2) Much lower peak to total ratio in the summer than any other seasons. This means the power system is on full load and maintenance engineers should pay attention to any potential failures. Usually, the temperature in Shanghai is similar between May, June ( – ) and September, October ( – ), however, the experts found that the total energy demand in the later period is about 200,000 Kwh higher. The expert A explained that one of the causes is the psychological inertia and potential energy efficiency response people can be expected to take. The demand-shift attracted more of Expert B’s attention. From the demand-shift badge, he observed that the energy demands are unbalanced between regions. The area in the top-right corner (Sanlin town, a highly dense residential area) has the largest spatial demand change (more than 80,000 Kwh). He zoomed in to view the flow map to observe the rate and direction of demand change. The flow map visualises an interesting crowd energy demand behavior pattern – the arrows give the convergence/divergence hotspot or lines. This means that such an area may change the load strongly; expert B suggested that a proper energy efficiency response, facility operation, maintenance and planning should be considered. Regarding feedback, expert B confirmed that the demand-shift as well as the temporal and spatial demand exploration are useful for understanding consumer behaviour and future power system planning. Both of them liked the compact visual designs. Expert A said that training is necessary as the semantic meaning of the demand-shift windows might be ignored. He also suggested that the current system can be more powerful if it supported more forms of energy data. For example, supporting to analysis district heating demand, a promising neat solution for the supply of low-carbon heat, has an important practical significance, he complemented.
6 Conclusions and Future Work
We presented the initial development of an ongoing visualisation analytics study that explores the spatiotemporal energy demand of daily energy consumption. The main contributions at this stage include: characterising the problem of analyzing spatiotemporal energy demands and compiled visual exploration tasks, based on the iterative discussions with domain experts; a systematic visual analytic framework for temporal, spatial, spatiotemporal energy consumption behaviours is proposed; we developed a prototype system and had it evaluated by two experts. As we go forward, a more powerful potential flow algorithm will be created, which will generate better visualisation of spatiotemporal energy demands. The visual design will be modified to support multi-source and multi-granularity collected utilities data. Our collaborators plan to evaluate the system with more empirical data and use the prototype in several ongoing projects.
-  K. Amasyali and N. M. El-Gohary. A review of data-driven building energy consumption prediction studies. Renewable and Sustainable Energy Reviews, 81:1192 – 1205, 2018.
-  N. Andrienko, G. Andrienko, and P. Gatalsky. Exploratory spatio-temporal visualization: an analytical review. Journal of Visual Languages & Computing, 14(6):503 – 541, 2003. Visual Data Mining.
-  N. V. Andrienko, G. L. Andrienko, L. Barrett, M. Dostie, and S. P. Henzi. Space transformation for understanding group movement. IEEE Transactions on Visualization and Computer Graphics, 19:2169–2178, 2013.
M. L. Baptista, A. Fang, H. Prendinger, R. Prada, and Y. Yamaguchi.
Accurate household occupant behavior modeling based on data mining
Twenty-Eighth AAAI Conference on Artificial Intelligence, 2014.
-  C. K. Batchelor and G. Batchelor. An introduction to fluid dynamics. Cambridge university press, 2000.
-  R. Bhatia. Energy demand analysis in developing countries: a review. The Energy Journal, 8(Special Issue), 1987.
-  M. Brehmer, J. Ng, K. Tate, and T. Munzner. Matches, mismatches, and methods: Multiple-view workflows for energy portfolio analysis. IEEE transactions on visualization and computer graphics, 22(1):449–458, 2015.
-  E. Costanza, S. D. Ramchurn, and N. R. Jennings. Understanding domestic energy consumption through interactive visualisation: a field study. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, pages 216–225, 2012.
-  U. Demšar, K. Buchin, F. Cagnacci, K. Safi, B. Speckmann, N. Van de Weghe, D. Weiskopf, and R. Weibel. Analysis and visualisation of movement: an interdisciplinary review. Movement ecology, 3(1):5, 2015.
-  Z. Deng, D. Weng, J. Chen, R. P. Liu, Z. bin Wang, J. Bao, Y. Zheng, and Y. Wu. Airvis: Visual analytics of air pollution propagation. IEEE Transactions on Visualization and Computer Graphics, 26:800–810, 2019.
-  N. Ferreira, J. Poco, H. T. Vo, J. Freire, and C. T. Silva. Visual exploration of big spatio-temporal urban data: A study of new york city taxi trips. IEEE Transactions on Visualization and Computer Graphics, 19:2149–2158, 2013.
-  T. Gerrish, K. Ruikar, M. Cook, M. Johnson, M. Phillip, and C. Lowry. Bim application to building energy performance visualisation and management: Challenges and potential. Energy and Buildings, 144:218–228, 2017.
-  S. Goodwin and J. Dykes. Visualising variations in household energy consumption. In 2012 IEEE Conference on Visual Analytics Science and Technology (VAST), pages 217–218, Oct 2012.
-  H. Hino, H. Shen, N. Murata, S. Wakao, and Y. Hayashi. A versatile clustering method for electricity consumption pattern analysis in households. IEEE Transactions on Smart Grid, 4(2):1048–1057, June 2013.
-  L. C. Hunt, G. Judge, and Y. Ninomiya. Underlying trends and seasonality in uk energy demand: a sectoral analysis. Energy Economics, 25(1):93 – 118, 2003.
-  M. Kavgic, A. Mavrogianni, D. Mumovic, A. Summerfield, Z. Stevanovic, and M. Djurovic-Petrovic. A review of bottom-up building stock models for energy consumption in the residential sector. Building and Environment, 45(7):1683 – 1697, 2010.
-  S. Kim, S. Jeong, I. Woo, Y. Jang, R. Maciejewski, and D. S. Ebert. Data flow analysis and visualization for spatiotemporal statistical data without trajectory information. IEEE transactions on visualization and computer graphics, 24(3):1287–1300, 2017.
-  T.-Y. Kim and S.-B. Cho. Predicting residential energy consumption using cnn-lstm neural networks. Energy, 182:72 – 81, 2019.
-  P. O. Kristensson, N. Dahlback, D. Anundi, M. Bjornstad, H. Gillberg, J. Haraldsson, I. Martensson, M. Nordvall, and J. Stahl. An evaluation of space time cube representation of spatiotemporal patterns. IEEE Transactions on Visualization and Computer Graphics, 15(4):696–702, July 2009.
-  W. Lin, Y. Zhou, H. Xu, J. Yan, M. Xu, J. Wu, and Z. Liu. A tube-and-droplet-based approach for representing and analyzing motion trajectories. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39:1489–1503, 2017.
-  D. Liu, P. Xu, and L. Ren. Tpflow: Progressive partition and multidimensional pattern extraction for large-scale spatio-temporal data analysis. IEEE Transactions on Visualization and Computer Graphics, 25(1):1–11, Jan 2019.
-  K. Liu. Pattern Exploration from Citizen Geospatial Data. PhD thesis, Purdue University Graduate School, 2019.
-  X. Liu, W. Y. Yan, and J. Y. Chow. Time-geographic relationships between vector fields of activity patterns and transport systems. Journal of Transport Geography, 42:22 – 33, 2015.
-  N. R. C. U. P. on Energy Demand Analysis. Improving energy demand analysis. National Academies, 1984.
-  T. J. Overbye and J. D. Weber. New methods for the visualization of electric power system information. In IEEE Symposium on Information Visualization 2000. INFOVIS 2000. Proceedings, pages 131–16c. IEEE, 2000.
-  J. Rodgers and L. Bartram. Exploring ambient and artistic visualization for residential energy use feedback. IEEE transactions on visualization and computer graphics, 17(12):2489–2497, 2011.
-  S. Singh and A. Yassine. Mining energy consumption behavior patterns for households in smart grid. IEEE Transactions on Emerging Topics in Computing, 7(3):404–419, July 2019.
-  L. Suganthi and A. A. Samuel. Energy models for demand forecasting—a review. Renewable and sustainable energy reviews, 16(2):1223–1240, 2012.
-  Y. Sun, S. Wang, F. Xiao, and D. Gao. Peak load shifting control using different cold thermal energy storage facilities in commercial buildings: A review. Energy conversion and management, 71:101–114, 2013.
-  C. Tominski, H. Schumann, G. L. Andrienko, and N. V. Andrienko. Stacking-based visualization of trajectory attribute data. IEEE Transactions on Visualization and Computer Graphics, 18:2565–2574, 2012.
-  F. Wang, K. Li, N. Duić, Z. Mi, B.-M. Hodge, M. Shafie-khah, and J. P. Catalão. Association rule mining based quantitative analysis approach of household characteristics impacts on residential electricity consumption patterns. Energy conversion and management, 171:839–854, 2018.
-  P. C. Wong, K. Schneider, P. Mackey, H. Foote, G. Chin Jr., R. Guttromson, and J. Thomas. A novel visualization technique for electric power grid analytics. IEEE Transactions on Visualization and Computer Graphics, 15(3):410–423, May 2009.
-  I. Yarbrough, Q. Sun, D. Reeves, K. Hackman, R. Bennett, and D. Henshel. Visualizing building energy demand for building peak energy analysis. Energy and Buildings, 91:10 – 15, 2015.