Self-adaptive systems (SAS) shall adapt to changing contexts conditions to meet functional and non-functional requirements (de Lemos et al., 2017). For safety-critical systems such as medical applications, the performance and time constraints are domain requirements whose violation could lead to catastrophic failures, putting the user’s life at risk. Therefore, to guarantee the safe operation of this kind of system, a model checking approach is often adopted to provide the goals’ reachability values for every modeled path. Nevertheless, the uncertainty coming from the difficulty in predicting which context conditions the system will encounter, combined with the huge amount of possible configurations of a complex system, hinders the development of adequate adaptation policies at design time (Ramirez et al., 2012).
Through the analysis of runtime data obtained either by monitoring or simulating the SAS, learning approaches can be used to cope with this limitation by identifying the active context and modifying the adaptation space at runtime (Knauss et al., 2016). Thus, context changes and user profiles that were not anticipated at design time are addressed by learning new adaptation rules dynamically, or by modifying and improving existing rules (Sharifloo et al., 2016). However, the generation of all possible situations, exclusively at runtime, poses a risk to the system’s performance, reliability, and real-time constraints; this indicates the need of a comprehensive design-time assessment combined with an efficient runtime monitoring and validation.
Although there are some studies on assurance provision for SAS, their verification techniques that infer whether a software system complies with its requirements still neglect the context variability factor and the corresponding impact on real-time properties. Basically, the state-of-the-art approaches can be divided into three major categories (de Lemos et al., 2017): human driven (e.g., formal proof), system-driven (e.g., runtime verification), and hybrid (e.g., model checking). Despite the ability of hybrid methods to provide assurance at both on-line and off-line stages, there is a certain unpredictability concerning the influence of context variability on real-time properties that still needs to be mitigated. As a result, such unpredictability may hinder altogether the required management of “the continuous collection, analysis and synthesis of evidence that will form the core of the arguments that substantiate the provision of assurances” (de Lemos et al., 2017).
Our proposed work aims at supporting the provision of such evidence with special focus on real-time properties of SAS. The proposal relies on contextual goal modeling, taking goals as first-class citizen of the self-adaptation, and follows the MAPE-K reference model for SAS. Initially, the elicited requirements and domain knowledge are carried in a Contextual Goal Model (CGM). Before implementing the SAS, we build a formal model of its behavior and verify the real-time properties with UPPAAL, a modeling and verification tool for real-time systems (Bengtsson et al., 1995), to create the foundation upon which the SAS will be constructed. After the verification, we run a data-mining process on runtime data obtained from the SAS. The process applies a transductive transfer learning
transductive transfer learningsetting (Pan and Yang, 2010)
We experimentally evaluate our method on a simulated prototype of a Body Sensor Network system (BSN) (Pessoa et al., 2017). The experiment consists in analyzing the reachability of BSN goals by merging two different perspectives. The first one relies on the UPPAAL model-checking technique over the formal model of the BSN to extract the goals’ reachability values. Afterwards, the verified model of the BSN is implemented in OpenDaVINCI (Berger, 2017), a well-established distributed real-time platform, to simulate the BSN as a real-time SAS. The second one is based on the application of data-mining techniques, specifically classifiers, on the dataset obtained by simulation to discover how the system behaves in the presence of different contexts, particularly with respect to the real-time constraints of the BSN. The data generated by the simulated BSN is continuously stored in a database after each scheduling period and analyzed by the data-mining and learning process we propose.
The learning process has been shown to be useful in raising the system’s awareness towards operational contexts that might pose a threat at runtime to the real-time properties assured by the UPPAAL model checking. Therefore, it assists the development of an appropriate set of adaptation policies for the controller. In addition, the evaluation allowed us to simulate scenarios under varying numbers of active sensors, different modes of the controller, and varying health risks of patients. Moreover, the method has shown itself effective to provide optimization strategies for dynamic adaptation of the controller mode under adverse sensors battery conditions, which could be crucial for patients under critical health risk, still in accordance with the real-time properties verified off-line by model checking.
The rest of the paper is organized as follows. Section 2 provides a brief description of our running example. We present our methodology in Section 3 and evaluate our proposal through experiments in OpenDaVINCI in Section 4. Section 5 highlights major related work. Finally, Section 6 concludes along with future work.
2. Example: Body Sensor Network
To discuss our proposed methodology, we use the example of a Body Sensor Network (BSN) (Pessoa et al., 2017) throughout this paper. The main objective of the BSN is to keep track of a patient’s health status, continuously classifying it into low, moderate, or high risk and, in the case of any anomaly, to send an emergency signal to a central unit. The structure of the BSN is as follows: a few wireless sensors are connected to a person to monitor her vital signs, namely, an electrocardiogram sensor (ECG) for heart rate and electrocardiogram curve, an oximeter (SPO2) for blood oxidation and blood oxidation curve, and a temperature sensor (TEMP). Additionally, there may be a central node (Control Sensor) responsible for preprocessing the collected data, filtering redundancy, or translating communication protocols. Table 1 shows how the sensor values (and thus the context) relate to the patient’s health risk as specified by a domain expert.
|Sensor Information||Data Ranges|
|Oxygenation:||100 low 94 moderate 90 high 0|
|Pulse Rate:||high 120 low 80 high 0|
2.1. A Contextual Goal Model of the BSN
Modeling a SAS requires to take into consideration not only the requirements and means to achieve them, but also the contextual information that may be related to the system’s operation. For this purpose, we use a Contextual Goal Model (CGM) since it allows us to specify in a simple structure the stakeholders and high-level requirements, the ways to meet such requirements, and the environmental factors that can affect the quality and behavior of a system. Figure 1 shows an excerpt of the CGM for the BSN.
According to Ali et al. (Ali et al., 2010), a CGM is composed of: (i) actors such as humans or software that have goals and can decide autonomously on how to achieve these goals; (ii) goals as a useful abstraction to represent stakeholders’ needs and expectations, offering an intuitive way to elicit and analyze requirements; (iii) tasks as atomic parts that are responsible for the operationalization of a system goal, that is, an operational means to satisfy stakeholders’ needs; and (iv) contexts as partial states of the world that are relevant to a goal. A context is strongly related to goals since context changes may affect the goals of a stakeholder and the possible ways to satisfy the goals. Goals and tasks of a CGM can be refined into AND-decomposition (OR-decomposition), that is, a link that decomposes a goal/task into sub-goals/tasks, meaning that all (at least one) of the decomposed goals/tasks must be fulfilled/executed to satisfy its parent entity. The link between a goal and a task is called means-end, and indicates a means to fulfill a goal through the execution of a task.
According to Figure 1, the root goal of the actor BSN is “G1: Emergency is detected”. G1 is refined into “G2: Patient status is monitored” and “G3: Sampling rate is adjusted” by an AND decomposition. G2 is divided into two subgoals: “G4: Vital signs are monitored” and “G5: Vital signs are analysed”. Such goals are decomposed, within the boundary of the BSN actor, to finally reach executable tasks. The operation of the BSN is subject to three different context conditions. The first context (C1) is the aforementioned patient status, which may assume three possible values: low, moderate, and high risk. The second context (C2) is related to the controller’s mode, which may be on or off and adapts the sampling rate of the patients’ vitals depending on C1. The last context (C3) concerns the real-time mode of the sensor nodes’ scheduler that can be on or off. Thus, C2 and C3 determine whether the BSN has activated or not the controlled mode respectively the real-time mode.
Each combination of executable tasks might contain different conjunction of contexts, and each conjunction (i.e., context of a goal model variant (Ali et al., 2010)) shapes the system to fulfill a requirement at a different quality level. For this work, we use the three context conditions modeled in the CGM (Figure 1) as they impact the real-time properties (C3), quality (C2) and dependability attributes (C1).
2.2. BSN Architecture
The BSN requires a network of distributed devices responsible for the execution of the tasks defined in the CGM (see Figure 1). This network is defined by the architecture in Figure 2 and consists of (i) a Scheduler, (ii) a set of Sensor Nodes, and (iii) a BodyHub.
The scheduler realizes the deterministic execution of the other modules by dictating their execution sequence using a first-come first-served (FCFS) algorithm. Thus, the scheduler exclusively commands the BodyHub or an active sensor node to execute by sending fixed period release signals. Each sensor node is a self-adaptive device capable of capturing sensor signals, processing, storing, and eventually sending data through wireless communication. A wide range of configurations may be applied regarding its components to fulfill the requirements based on policies (e.g., if battery then activate controlled mode). Each sensor node operationalizes the tasks T1.1 (Collect sensor data) and T3 (Adjust the sampling rate) (cf. Figure 1), where the second one encompasses the following self-adaptive behavior when the controlled mode is activated: each node monitors its sensor data, analyzes the data to determine the patient’s health risk, plans a new sampling frequency based on the analysis, and finally executes the needed change. Changing the sampling frequency of a sensor node influences the reliability of the sensed data and the battery consumption of this node.
The BodyHub acts as an information centralizer for the data provided by the sensor nodes. It consumes, processes (e.g., store and fuse), and analyzes the data to decide and update the overall health status of the patient. Thus, it operationalizes the tasks T1.3 (Persist data) and T2 (Analyze vital signs) with its refinements (cf. Figure 1).
3. Our Approach for Assurance of Real-Time Sas
The software engineering process for building self-adaptive safety-critical systems (e.g., self-adaptive real-time systems) must follow a guideline with perpetual assurances of goals from design to run-time. Knowing that feedback loops supported by processes should provide the basis for managing, among other things, the continuous synthesis of evidence (de Lemos et al., 2017), we propose a feedback loop that combines off-line requirements elicitation and model checking with on-line data collection and data mining to provide the information that subsidize the provision of assurances. Hence, a means to guarantee the system goals fulfillment, both functional and non-functional, is using the knowledge obtained by the data mining to fine tune the adaptation policies towards the optimization of dependability attributes such as reliability, availability, or safety, as well as quality attributes like performance or energy-efficiency. The role of the data mining process is to discover and quantify the impact of operational contexts on predefined properties such as time constraints and quality attributes. Thus, adaptation policies can be developed to reconfigure the system in a way that respects the system’s properties independently of the changing environment. To structure this idea, we describe our method as an enhanced feedback loop in the following steps:
We start the method with a CGM as the specification of the stakeholders’ needs (see Figure 1);
The next step is to model the core SAS architecture elements as well as their behavior and conduct a model-checking process with UPPAAL to verify the correctness of the SAS, especially whether real-time properties are satisfied or not. Then, we devise the expected behavior of the SAS based on the analyzed context conditions;
After the verification stage, we implement the SAS and apply the concepts of transfer learning for the on-line assurance, in the sense that we learn from a simulation aiming at transferring the obtained knowledge to a real-world application. At this stage, we are concerned with the prototype implementation and its compliance with the properties verified by model checking while the next steps provide the on-line assurance;
We execute the prototype and collect the related runtime data such as the system’s resource consumption and the occurring contexts conditions. The runtime data is stored as snapshots of the SAS, that is, we collect all the relevant variables and their respective values in progressive moments along the execution;
In possession of the collected runtime data, we apply a set of data mining algorithms aiming at identifying hidden correlations between the system’s variables and the contexts and therefore, between combinations of contexts and the satisfaction of the system’s properties;
Closing the feedback loop, the learning mechanism supports the provision of assurance by allowing the fine tuning of the (or suggesting novel) adaptation policies taking runtime data into account that was not anticipated when initially developing the policies. In case of new policies, a refactoring of the verified model and system is performed to make sure the once assured properties still hold after the refactorings.
Next, we present such steps in further details.
3.1. From Goals to Model Checking
In our approach, the SAS is first modeled by taking into consideration the CGM leaf-tasks operationalization (see Figure 1) in conformity with the BSN architecture (see Figure 2). Each module of the architecture is then modeled as a timed automaton in UPPAAL (Behrmann et al., 2006), where each automaton represents a module template which may contain one or more instances (e.g., multiple sensor nodes use the same behavior template). In the BSN, the modules follow a First-Come-First-Served (FCFS) scheduling behavior fulfilling a basic life cycle represented by locations named according to the progress of the module behavior: wait, run and idle in which the run location is constrained by a clock and characterizes each architectural module. When the scheduling cycle is finished, it sets the guard condition done to true. Thus, our modeling strategy is to represent the life cycle of modules that denotes the progress of the modules’ behavior by different locations and guard conditions in the UPPAAL model for the verification of reachability properties.
For the sake of space, we provide details of the UPPAAL model for the BSN at GitHub111https://github.com/rdinizcal/SEAMS18/tree/master/uppaal. In the next subsection, we show how we map the goals of a SAS into properties for the UPPAAL models using our running example.
3.1.1. Properties Verification
We specify the properties of the model to be verified in Timed Computational Tree Logic (TCTL) (Henzinger et al., 1994), since its the UPPAAL language to verify the real-time properties of the formalized model. TCTL is a real-time variant of CTL aimed to express properties of timed automata. Like CTL, the model verification relies on state or path expressions regarding properties such as reachability, safety, or liveness. TCTL extends CTL with atomic clock constraints over the clocks, typically the set of clocks in the timed automaton under consideration. The TCTL model-checking problem is to check for a given timed automaton TA and TCTL formula whether TA (Baier and Katoen, 2008). In UPPAAL, the properties are specified with a subset of TCTL augmented with syntactical symbols such as and logical operators like and to include variables and time evaluations. Table 2 lists such basic expressions.
|E||there exists a path where eventually holds|
|A||for all paths always holds|
|E||there exists a path where always holds|
|A||for all paths will eventually hold|
|whenever holds will eventually hold|
|imply||holds if and only if the clock is within T|
|A not deadlock||checks for deadlocks|
|Goal||ID||Informal Description||Specification in TCTL|
|P1||The controlled system is deadlock free.|
|N/A||Whenever the scheduler cycle is completed,||imply|
|P2||implies that the bodyhub and the three|
|sensors have been executed.|
|G1||Whenever the patients’ health status is on high risk and an|
|emergency has been detected it implies that the observer’s clock|
|P3||is less or equal 250 (ms) and a single scheduler||imply|
|cycle has elapsed since last data acquisition.|
|G3||Whenever the sensornodes’ controller grants permission||&|
|P4||to execute and its on high risk, schedulers’ cycle may||imply|
|have passed since the last data acquisition.|
|Whenever the sensornodes’ controller grants permission||&|
|P5||to execute and its on moderate risk, schedulers’ cycles||& imply|
|may have passed since the last data acquisition.|
|Whenever the sensornodes’ controller grants permission||&|
|P6||to execute and its on low risk, schedulers’ cycles may||& imply|
|have passed since the last data acquisition.|
|G2||Whenever a sensor node has collected data,|
|P7||the bodyhub will eventually process it.|
|G4||Whether the sensornode has collected some data,|
|P8||eventually the bodyhub will persist it.|
|G5||Whenever a sensor node has sent data,|
|P9||the bodyhub will eventually process|
|low, moderate or high data.|
|Whether the bodyhub has processed some data, it|
|P10||eventually will detect a new patient health status.|
In our approach, we specify temporal logic formulae to verify the satisfiability of the goals modeled in the CGM (cf. Figure 1). The CGM root goal explicitly elicits the actor’s main goal that once satisfied assures the system’s correct behavior. Fulfilling hard goals requires its refinements by means of its AND- and OR-refinements into goals or tasks are satisfied. Taking into account the modeling strategy, the task’s behavior can be verified through reachability properties as the timed automata locations represent the progress and achievement of the task’s behavior. For example, T1.1 (Collect sensor data) with meaning “the sensor node will eventually collect sensor data”, also T2.2 (Detect patient health status) with as “the bodyhub will eventually detect the patient’s status”, and so on.
On the other hand, sequences of states comprising temporal relations need to be addressed in order to fulfill the CGM goals. These are achieved basically by combining task properties (reachable locations) in the UPPAAL models with invariants or path-like formulae. Table 3 describes the sufficient TCTL-like specifications to assure the correct system behavior of the BSN by means of the satisfiability of the goals modeled in the CGM (cf. Figure 1).
The safety property P1 is common to distributed systems and assures that the BSN model is deadlock free. The fairness property P2 assures that in a scheduling cycle all modules will be executed.
Regarding P3, it is noticed that goal verification is not trivial when it does not have a direct relation to task decompositions, specially when non-functional goals contributes to it, which is the case of the root goal G1 that demands the emergency detection within 250 ms (cf. Figure 1). This is addressed in UPPAAL through an observer automaton that records the time taken when high risk is acknowledged at the sensor node to its proper detection on the BodyHub.
To fulfill G3, the properties P4, P5, and P6 shall be assured as they represent the controller behavior in task T3 operationalization with respect to the frequency at which sensors data will be collected. Finally, the goals G4 and G5 are satisfied by assuring their means-end task executions. In particular, G5 guarantees through property P9 that the sensor data range follows the BSN operationalization data accordingly, that is, low, moderate or high. This is important to assure that the model does not have any data sent outside the range recognized by the BSN system. G2 is the fulfillment of property P7, which merely synthesizes the fulfillment of goals G4 and G5.
3.2. From Goals to the Data Mining Process
As object of the mining process, we apply the concept of transfer learning, that is, the process of using other sources to provide cheaper samples for accelerating model learning (Pan and Yang, 2010), for the provision of legitimate data that will cope with the context analysis process. The data mining process enables us to isolate the component behaviors that need to be analyzed. The scope of analysis scales as we go up in the CGM treelike structure, that is, encompassing more elements in a database record and embracing all TCTL properties. By these means, the outcome of the data mining process provides evidence for the assurance check in our methodology. Additionally, the data mining process for relevant context conditions (i.e., where the SAS operates) does not necessarily need to make a combinatorial exploration to every possible system state. Instead, it analyzes the relevant properties, following from those formally specified in UPPAAL, and the CGM tasks’ contexts under operation, which speeds up the learning process. As such, the enrichment of the goal model with causal relationship between context and the non-functional requirements fulfillment supports the anticipation of adaptation strategies and potentially mitigates runtime uncertainty. To bypass both time and space limitations, we aim at merging the context discovery potential by means of artificial intelligence over monitored data, typical of runtime approaches, with the perks of having a robust modeling process at design time. The core of our idea with respect to applying data mining techniques relies on the mining and analysis of the impact that contexts might have on the satisfaction of real-time properties for SAS.
The classification routine is the main technique we use in the scope of our work. It is a data mining technique that tackles the problem of identifying to which set of categories an observed fact belongs. It is done based on a training set of data containing observations (or instances) whose category membership is known, that is, it is an instance of supervised learning(Alpaydin, 2014). More specifically, we apply two classification methods in our approach: (i) JRip (Cohen, 1995), that creates rules for every class in the training set and then prunes these rules. The discovered knowledge in this class of algorithm is represented as IF-THEN prediction rules and are specially useful to define the operational thresholds of some resources. (ii) The J48 classifier algorithm, which implements a Decision Tree used to support the decision making process using the depth-first strategy (Quinlan, 1999)
. In our method, we benefit from the decision trees by showing: (i) if the behavior of the actual SAS conforms to the verified properties and (2) how the different contexts of operation are combined to achieve a given goal focusing on a target property value. In the case of a non-nominal classification, that is, when the target knowledge or prediction concerns to a numeric attribute, it is possible to replace the decision tree for other kinds of model trees, for instance, ones that work with linear regression such as M5P(Wang and Witten, 1997).
In a nutshell, we propose an analysis process using the aforementioned algorithms (J48 and JRip) to unveil operational contexts of the SAS that might influence the satisfaction of non-functional requirements and real-time properties and, at the same time, quantify such impacts on tasks and/or goals. Based on the CGM structure (cf. Figure 1) combined with the operationalization values of sensed information, we traverse the goal-tree visiting each CGM node (goal or task) and verify how the CGM nodes and the properties related to such nodes (vide Table 3) corroborate the design-time verification in a runtime environment. Such verification is supported by the data mining and analysis of the log generated by a SAS, in our case the BSN prototype in OpenDaVinci with its CGM goals (see Figure 1).
3.2.1. The Data Mining Process
We provide a fine-grained presentation of our data mining process with the algorithmic steps of Algorithm 1. The algorithm starts by traversing the CGM using a post-order depth-first search (DFS) in line 6. After running the SAS, for each node of the CGM, the portion of the log dataset, that is, particularly the variables related to the node’s subtree (line 7), is collected and processed, more specifically those variables encompassed by that node’s subtree. In lines 8 and 9 we get, respectively, the properties and the contexts of operation that are associated to that specific subtree. In possession of the dataset, we execute the JRip algorithm to extract the operationalization rules (line 10), and we apply the J48 to display the combination of such rules with the context observed at runtime (line 11). At last (line 14), the knowledge obtained from the data mining process is confronted with the UPPAAL properties previously defined and verified, allowing us to unveil any dissonance between the design/runtime model and the real execution. Moreover, the knowledge obtained by the process allows us to improve and fine tune the adaptation policies, maximizing a target attribute while observing the behavior of the whole system, either at design-time via simulation or at runtime. All this is possible through the identification of facts that are not noticeable by the model verification alone.
To illustrate how the data mining provides evidence for assurance, let us explain the process in a practical sense using the BSN as an example. Most of mobile health care applications depend upon batteries for several services. Therefore, the predictability of the duration of a battery cycle is paramount in medical applications to guarantee that the devices’ availability are always observed. Considering that the probability of triggering an emergency signal is directly proportional to the patient health risk and to guarantee the safety of a high-risk patient, it requires a continuous monitoring of the patient’s status, that is, a high sampling frequency of the patient vital signs. On the other hand, for a low-risk patient, a sporadic monitoring is sufficient to guarantee the patient’s safety. Therefore, the higher the monitoring frequency the higher the energy consumption of the device processing the sensed vital data. Our BSN’s controller is responsible for managing the frequency of sampling rate of the patient’s vital signs, according to her health risk status. For instance, when the controller context is present (i.e., the controller is activated), the period of data acquisition could be retarded for low and moderate risks in a factor of ten and five times, respectively. The importance of the controller policy can be seen in Figure3, which shows how frequently the BSN sensors in our prototype are required to perform, that is, obtaining the patient’s vital signs and categorizing the patient’s health risk status, under the presence or absence of the controller. So the issue lies on defining suitable parameters for the controller policy that is supported by an efficient energy consumption, but on the other hand still satisfies the verified properties of the SAS.
The knowledge about the overall energy consumption in face of a controller’s context variability (context C2 in the subtree of goal G3, Figure 1), can be enhanced by our data mining process to find out a precise outcome that could not be obtained merely by a model checking process. For example, as an outcome of our data mining process, Figure 4 presents a decision tree of the energy consumption in energy units (e.u.) per health risk status of a patient on the BSN prototype. Such values in the decision tree nodes were obtained from monitoring the battery consumption from the BSN prototype implemented in OpenDaVINCI and, aiming at the normalization of the analysis for the user profile simulated. As such, our data mining process helps on quantifying and classifying the battery consumption per health risk status of a patient in the controlled context mode.
Therefore, after learning from our data mining process, it is possible to draw a relationship between consumption per risk status for a specific user profile. Through the rules generated by JRip and the decision trees provided by J48, it is possible to define ranges of operation in which energy will be drained at a predictable rate. However, one should note there were no checked properties regarding the energy consumption in the CGM, nor in the properties verified in UPPAAL. Nevertheless, our data mining process was able to provide means to support the assurance process regarding the controller mode of operation. For example, since the controller mode of operation plays an important role on the energy consumption in our BSN, our data mining process could support the creation of adaptation policies to dynamically adjust the sampling frequency. These policies are used by the sensor node’s controller to optimize the energy consumption in a specific context (battery level) without violating its verified properties.
In the next section, we further present the outcomes of the experimentation of our learning-based approach to support assurance for real-time SAS on the BSN prototype developed in OpenDaVINCI.
4. Experiments in Opendavinci
The BSN prototype was developed in OpenDaVINCI (Berger, 2017), that stands for Open Source Development Architecture for Virtual, Networked, and Cyber-Physical System Infrastructures. OpenDaVINCI is ideal for networked cyber-physical applications, as it enables TCP, UDP, and serial port communication. The platform permits real-time scheduling for distributed software architectures. Working as a middleware responsible for data- or time-triggering software modules, it deals with message distribution on publish/subscribe communication architecture. Moreover, it supports real-time operations under a real-time Linux system. Therefore, it became a natural choice to the development of the prototyped real-time BSN.
We evaluate our approach by means of a Goal-Question-Metric (GQM) methodology (van Solingen et al., 2002). The questions that are relevant to the evaluation of the present work are divided into three major experiments that analyze: (i) the impact of the number of BSN sensor nodes on the satisfaction of real-time constraints, (ii) the impact of patients health risk diagnosis strategies on the satisfaction of real-time constraints, and (iii) the impact of the controller context on the energy consumption of the BSN. Table 4 details these questions.
We have adopted two different time metrics to illustrate our results. The first one is , which represents the time difference between two consecutive measurements of a given sensor node. In terms of TCTL specification, the metric is described as . The second time metric is , referring to the time taken to detect an emergency after a patient’s health status is identified as high risk. Transcribing to TCTL, the metric is represented as , that is, the time difference between the sending of measured data by a sensor node and the processing of the data by the BodyHub.
|Goal 1: Property Refinement through Data Mining Process|
|1 Are the scheduling period (property P2) and emergency detection time constraint (property P3) respected while varying operational contexts such as the number of active sensors?||(s), (s).|
|2 How does the data mining process support the maximization of the trustworthiness of a measured data while respecting a tight scheduling window?||
(s), Paired t-test.
|3 How does the learning method assist the unveiling of sensitive runtime aspects and guide the fine tuning of the BSN’s adaptation policies?||Average battery consumption (%) for patient risk status.|
The presented experiments were executed under the following configuration: CPU 4x Intel(R) Core(TM) i7-5500U CPU @2.40GHz, 8075MB RAM, Ubuntu 16.04.3 LTS, Kernel Linux 4.4.86-rt99 (x86_64) with SMP PREEMPT RT, GNU C Compiler version 5.4.0 20160609, hard drive ATA Corsair Force LE.
We have divided the evaluation of our work into three major experiments, each one related to a question described in Table 4. All the artifacts related to these experiments are available at GitHub222https://github.com/rdinizcal/SEAMS18.
We should note that sensors are prone to failure so that we applied in our experiments two different strategies to confirm the vital sign status sent by the sensors. In the first strategy, namely 3 of last 5, the system verifies whether the same vital sign status is confirmed in at least three out of the last five reads of the same sensor. Only in this case the data is sent to the central unit. In the second strategy, namely Replication, each sensor node comprises 5 redundant sensors and the system takes the measurement of the majority of the replicated sensors as a valid reading.
Q1: Are the scheduling period (property P2) and emergency detection time constraint (property P3) respected while varying operational contexts such as the number of active sensors?
First of all we need to characterize the time constraints related to the BSN’s real-time properties. The first time constraint we analyze is the scheduling cycle period, whose violation potentially leads to the violation of the fairness property of the system (P2). According to this property, all the sensor nodes have to be executed within the time window of 100ms, which is the default scheduling period of OpenDaVINCI working at 10Hz. We say that the scheduling period constraint is respected if the time difference between two consecutive measurements of a given sensor node is less than (or equal) to 100ms (). The second time constraint refers to the property P3 presented in Table 3. This property is related to the goal G1, displayed in Figure 1, concerning the time threshold of 250ms in which an emergency shall be detected after a patient’s health status is identified as high risk. In order to satisfy this constraint, the time difference between the sending of a measurement by a sensor node and the processing of such reading by the BodyHub is less than (or equal to) 250ms (). We should also note that the experiments have been made with different numbers of sensor nodes (1, 5, 10, and 20 sensor nodes) to investigate the consequences when increasing them. As such, each sensor node would contain different number of sensors, depending on the confirmation strategy: in the 3 of last 5 it would contain only one sensor, while in the Replication it would contain 5 redundant sensors.
We have verified for the first time constraint that all the mean values are below 100ms either using the 3-of-last-5 and Replication strategies, except of the scenarios with 20 sensor nodes, in which the values for both strategies are around 123ms. To satisfy all configurations and time constraints, the upper bound limit is determined by the execution with 20 sensor nodes and applying Replication strategy, that is, approximately 126ms. Therefore, the scheduling period constraint (P2) was not fully satisfied for any configuration as it is depicted in Figure 5. To summarize, the increase of the number of sensor nodes is a sensitive factor for the satisfaction of the real-time constraints. The execution of the system with more than 20 sensor nodes hinders the monitoring/processing of the patient’s vital signs and is potentially the cause of data loss, jeopardizing the integrity and availability of the BSN during the scheduling routine.
For the second time constraint that refers to the emergency detection, we have noted that the property P3 was respected in every scenario with exception of some outliers observed in scenarios with 3 sensor nodes running with3-of-last-5 strategy, as well as 3 and 5 sensor nodes using Replication. The evaluated scenarios and their respective values are shown in Figure 6.
The data mining analysis has exposed some BSN aspects that, from a model checking perspective, do not impact the system behavior, but at runtime pose a threat to the fulfillment of real-time requirements. The method allowed us to spot the sources of unexpected changes that can only be verified through runtime data, such as the variability in the amount of active sensors, confirmation strategy, or the controller mode. The data mining process also pointed out that part of the measurements are received by the BodyHub after the scheduling cycle period closes. Although the OpenDaVINCI implementation was able to store the messages in a buffer and process all of them afterwards, validating property P9, we cannot guarantee that the same would happen in a real-world scenario. We can only infer that as we employ more sensor nodes, more measurements tend to violate the scheduling window, potentially putting at risk the fairness property of the system (P2) as well as P9.
Q2: How does the data mining process support the maximization of the trustworthiness of a measured data while respecting a tight scheduling window?
In real-world scenario, sensors are considered failure-prone. This experiment aims at verifying how sensors data validation strategies scale in a system with tight real-time constraints and how the data mining can assist the process. We have applied a pairwise t-test to compare two population means ( where we have two samples in which observations in one sample (3 of last 5 strategy) can be paired with observations in the other sample (Replication strategy). We can use the results from our sample emergency detection to draw conclusions about the impact of changing the strategy in general.
To calculate the confidence interval for the true mean difference, at a 95% confidence interval the true mean difference is:. The result confirms that the difference in strategies is not statistically significant, since the interval (-0.08284, 0.02484) includes 0.
The J48 method with its generated decision trees assisted us in defining the processing time of each sensor while sensors with higher computation times can be executed later in the scheduling sequence. Figure 7 shows how the execution queue of the sensors can be sorted based on the processing period. The decision tree is useful to assist the debugging process of a SAS, finding, for instance, the modules that are potential sources of failures. In the BSN case, it helped us to spot bottlenecks in the patient monitoring and possible data loss from sensors. This enables the development of a more dependable SAS by fine tuning and hence obtaining robust adaptation rules.
Q3: How does the learning method assist the unveiling of sensitive runtime aspects and guide the fine tuning of the BSN’s adaptation policies?
Defining adaptation policies at design time with respect to aspects that can only be known at runtime is a challenging task for engineering a SAS. Since the sampling rate variable is highly dependent on the user profile (e.g., health status), the data mining process stands out on fine tuning the related policies assisted by runtime data. By introducing a multivariate multiple regression technique into our process, we are able to estimate the duration in which a patient stays in a health risk status before it changes to another one (see Figure8). Moreover, it enables the prediction of energy consumption based on the current patient status and duration in such a status. As we have mentioned before, the patient health risk state is directly related to the battery consumption. Therefore, by merging the knowledge obtained via data mining, we could predict for how long a patient can be reliably monitored given the patient’s current health risk and the remaining battery charge.
Based on the output of the data mining process, we devised a dynamic adaptation policy for the controlled mode to balance the trade-off between the energy consumption and safety assurance. The battery charge was divided into three categories: good (50%-100%), medium (15%-49%), and critical (15%). Basically, for a conservative monitoring policy, the system keeps track of the minimum duration in which a patient’s health risk status remains unchanged for each status (low, moderate and high). Such a period will be the parameter to adjust the sampling rate in each situation. When the battery charge reaches the medium level, the adaptation planner takes the minimum duration in which a patient stayed in a given risk during the measurement lifetime, and set this duration as the sampling rate for that risk status. For instance, if an individual patient stays on average 3 hours in moderate risk, but the minimum duration measured for such status was 30 minutes, the latter will be the new sampling rate for this risk state. This policy will minimize the energy consumption and at the same time maintain the assurance of the real-time constraints. Finally, when the system notices that it will not be able to guarantee the next measurement due to insufficient charge, that is, the sampling period is greater than the estimated working time for current charge level, it enters in an energy saving profile and adapts the replication strategy. Instead of taking the majority of five readings, it shuts down two sensors and take the best of three measurements in order to save even more battery.
Figure 9 shows the progression of BSN’s energy consumption over time for a patient’s monitoring considering (i) a non-controlled scenario, (ii) a controlled scenario with a static adaptive policy, and (iii) a controlled scenario with a dynamic adaptive policy, that is, the default policy enhanced with the knowledge obtained by the data mining process. We have noticed that the battery, in the policy supported by the learning process, lasts over three times longer in comparison to the other contexts. During this time, we were able to reliably identify all the patients’ statuses and their variations without violating the real-time constraints.
Closing the feedback loop, the UPPAAL model was updated as well to confirm whether the properties described in Table 3 still hold after the changes introduced in the adaptation policy. The obtained results were encouraging since we were able to identify some runtime aspects that are directly related to the satisfaction of the real-time constraints and to the optimization of quality attributes such as energy efficiency. Our method stood out in identifying such aspects and manage them to improve the dependability of a system in a cost-effective manner. Previously, such knowledge could only be identified at runtime via a reactive approach. With our method, we have access to anticipated information that assists the (i) validation of design-time properties, (ii) development and refinement of adequate adaptation policies, and (iii) the assurance in decision making process, even at design time through simulating a SAS prototype.
4.4. Threats to validity
Construct validity – Our input data relied on a reported sound case study (BSN) and its published and available data. In addition, our process aligns goals, model checking, prototyping and data mining following a sound procedure. Despite all the care we took to avoid the generation of unrealistic data, further study must be done to verify the applicability in a real-world scenario.
Internal validity – Our approach showed itself efficient to adequately evaluate our approach. However, unveiling all the contexts involved in a system’s operation is inherently NP-complete, which could represent a threat to the overall assurance of the system.
External validity – Although our approach is not tailored to be domain specific, we do reckon the limitation of the evaluation since it was applied in the specific case of the BSN. Further evaluation must be performed to evaluate the generalization capability.
5. Related Work
Among the approaches that aim at assurances for self-adaptive systems through design-time verification and validation, Cámara and Lemos (Cámara and de Lemos, 2012) define an approach that relies on the notion of stimulation and probabilistic model-checking to provide levels of confidence regarding service delivery, with focus on the resilience property. Still on model-checking, de la Iglesia and Weyns (de la Iglesia and Weyns, 2013) extend an agent-based mobile learning application with a self-adaptation layer. The authors also developed a set of formally specified MAPE-K templates that encode design expertise for a family of self-adaptive systems (de la Iglesia and Weyns, 2015). In the domain of advanced distributed embedded real-time systems, Giese et al. (Giese and Schäfer, 2013) propose MechatronicUML, a model-driven development approach which supports the modeling and verification of safety guarantees for SAS at runtime. Calinescu et al. (Calinescu et al., 2017) have recently proposed the end-to-end ENTRUST methodology through systematic stages to provide assurance evidence, cases and arguments for the controller platform at design and runtime for SAS. In our work we increment the assurance processes with model checking by means of a learning-based approach to verify and validate real-time properties while accounting for context variability, including adverse conditions, at runtime.
Regarding runtime models in SAS, Chen et al. (Chen et al., 2014) propose the combination of requirements-driven self-adaptation and architecture-based self-adaptation to reconfigure component-based architecture models using incremental and generative model transformations for complex architectural adaptations. Vrbaski et al. (Vrbaski et al., 2012) propose a work that leverages goal models as runtime entities and integrates them into modeling, simulation, and execution environment of context-aware systems. As a complement to the requirements-driven self-adaptation approach proposed by Qian et al. (Qian et al., 2015), that combines goal reasoning and case-based reasoning, our approach explores how the combination of contextual variables, systems’ configurations and non-functional requirements affects the selection of the adaptation solution. In our approach, we similarly rely on the modeling structure of the contextual model. But most of all, we rely on a feedback loop to keep a verification model always up-to-date, accounting for the real-time properties, as well as others, aligned with the dynamic aspects of the system.
In the field of machine learning and data analysis to support the modeling and adaptation of self-adaptive systems, Sharifloo et al. (Sharifloo et al., 2016) propose a solution for design-time uncertainty, particularly in the realm of Dynamic Software Product Lines (DSPL), by proposing a feedback approach through an adaptive system model that combines learning of adaptation rules with evolution of the DSPL configuration space. Knauss et al. (Knauss et al., 2016) propose a framework to provide adaptation of contextual requirements at runtime. Our work, on the other hand, supports the provision of evidence for SAS assurance still at design time. To this purpose, we consider the impact of contexts combinations in real-time constraints and dependability attributes, and use such knowledge to improve the system’s adaptation. Aiming at the enhancement of the learning process, Jamshidi et al. (Jamshidi et al., 2017) define a cost model that transform the traditional view of model learning into a multi-objective problem, considering model accuracy and measurements effort. Similar to our approach, their objective is to apply the concept of transfer learning to improve model predictions in SAS. Still in the domain of large scale distributed systems, Schmid et al. (Schmid et al., 2017), tackle the difficulty in developing a complete and accurate model for SAS at design time by proposing a method where the system model consists only of the essential input and output parameter. Our method also benefits from the ability of classification methods particularly by (1) discovering hidden patterns of operation in presence of different contexts of execution, and (2) supporting with evidences that the model-checked properties hold for the running system.
Regarding the assurance of real-time properties, Zeller and Prehofer (Zeller and Prehofer, 2012) deepen the study of time constraints for runtime adaptation analyzing two approaches for finding solutions in the resulting search space for adaptations, one based on planning algorithms and the other based on constraint solving. In our work we verify the applicability of data mining techniques to assist us in defining time constraints without sacrificing the performance in the contexts of operation. For systems with strict time constraints but where accuracy is not a major concern, statistical model checking could be an alternative (Younes and Simmons, 2006; Legay et al., 2010). Initial research that uses statistical techniques at runtime for providing guarantees in self-adaptive systems is reported by Weyns and Iftikhar in (Weyns and Iftikhar, 2016). In our work, we go one step further on exploring context conditions and their implications on the real-time properties through the data mining, since we believe in domains such as the BSN, vital and accurate information require a more thorough perspective of analysis to make more evident whether the SAS do hold the properties even under adverse conditions.
6. Conclusion and Future Work
In this paper, we proposed an integrated method that applies the concepts of data mining and analysis over the runtime data of a SAS, that supports the formulation of hypothesis concerning the impact of context variability in non-functional requirements and time constraints. Using such analysis to feedback the approach allows us to validate and refine the properties collected at design time via a model verification process. Hence, we are able to reduce the gap between the design- and runtime models, reducing the uncertainty in the adaptation process. In our evaluation based on the published information of the BSN, we were able to model the variability observed in sensed data and perceive a significant impact on the outcomes with respect to real-time constraints, sensor nodes scheduling, controllers actuation, and energy profiling. For future work, we plan to provide means to seamlessly integrate the steps of our approach as well as to explore unsupervised data mining approaches to be able of dealing with more complex contexts variations.
Acknowledgements.This work has been partially funded by CAPES/PROCAD under Grant No. 183794, and Arthur Rodrigues’ CAPES/DS scholarship.
- Ali et al. (2010) Raian Ali, Fabiano Dalpiaz, and Paolo Giorgini. 2010. A goal-based framework for contextual requirements modeling and analysis. Requir. Eng. 15, 4 (2010), 439–458.
- Alpaydin (2014) E. Alpaydin. 2014. Introduction to Machine Learning. MIT Press.
- Baier and Katoen (2008) Christel Baier and Joost-Pieter Katoen. 2008. Principles of Model Checking (Representation and Mind Series). The MIT Press.
- Behrmann et al. (2006) Gerd Behrmann, Alexandre David, and Kim G. Larsen. 2006. A Tutorial on Uppaal 4.0. (2006).
- Bengtsson et al. (1995) Johan Bengtsson, Kim G. Larsen, Fredrik Larsson, Paul Pettersson, and Wang Yi. 1995. Uppaal — a Tool Suite for Automatic Verification of Real–Time Systems. In Proc. of Workshop on Verification and Control of Hybrid Systems III (Lecture Notes in Computer Science). Springer–Verlag, 232–243.
- Berger (2017) Christian Berger. 2017. OpenDaVINCI. (2017). https://github.com/se-research/OpenDaVINCI
- Calinescu et al. (2017) Radu Calinescu, Simos Gerasimou, Ibrahim Habli, M. Usman Iftikhar, Tim Kelly, and Danny Weyns. 2017. Engineering Trustworthy Self-Adaptive Software with Dynamic Assurance Cases. IEEE Transaction on Software Engineering PP, 99 (2017).
- Cámara and de Lemos (2012) Javier Cámara and Rogério de Lemos. 2012. Evaluation of resilience in self-adaptive systems using probabilistic model-checking. In 7th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS. 53–62.
- Chen et al. (2014) Bihuan Chen, Xin Peng, Yijun Yu, Bashar Nuseibeh, and Wenyun Zhao. 2014. Self-adaptation through incremental generative model transformations at runtime. In 36th International Conference on Software Engineering, ICSE ’14, Hyderabad, India - May 31 - June 07, 2014. 676–687.
- Cohen (1995) William W. Cohen. 1995. Fast Effective Rule Induction. In Machine Learning, Proceedings of the Twelfth International Conference on Machine Learning, Tahoe City, California, USA, July 9-12, 1995. 115–123.
- de la Iglesia and Weyns (2013) Didac Gil de la Iglesia and Danny Weyns. 2013. Guaranteeing robustness in a mobile learning application using formally verified MAPE loops. In Proceedings of the 8th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS. 83–92.
- de la Iglesia and Weyns (2015) Didac Gil de la Iglesia and Danny Weyns. 2015. MAPE-K Formal Templates to Rigorously Design Behaviors for Self-Adaptive Systems. TAAS 10, 3 (2015), 15:1–15:31.
- de Lemos et al. (2017) Rogério de Lemos, David Garlan, Carlo Ghezzi, Holger Giese, Jesper Andersson, Marin Litoiu, Bradley Schmerl, and Danny Weyns et al. 2017. Software Engineering for Self-Adaptive Systems: Research Challenges in the Provision of Assurances. In Software Engineering for Self-Adaptive Systems III, Rogério de Lemos, David Garlan, Carlo Ghezzi, and Holger Giese (Eds.). Vol. 9640. Springer.
- Giese and Schäfer (2013) Holger Giese and Wilhelm Schäfer. 2013. Model-Driven Development of Safe Self-optimizing Mechatronic Systems with MechatronicUML. In Assurances for Self-Adaptive Systems - Principles, Models, and Techniques. 152–186.
- Henzinger et al. (1994) T.A. Henzinger, X. Nicollin, J. Sifakis, and S. Yovine. 1994. Symbolic Model Checking for Real-Time Systems. Information and Computation 111, 2 (1994), 193 – 244.
- Jamshidi et al. (2017) Pooyan Jamshidi, Miguel Velez, Christian Kästner, Norbert Siegmund, and Prasad Kawthekar. 2017. Transfer Learning for Improving Model Predictions in Highly Configurable Software. In 12th IEEE/ACM International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS. 31–41.
- Knauss et al. (2016) Alessia Knauss, Daniela Damian, Xavier Franch, Angela Rook, Hausi A. Müller, and Alex Thomo. 2016. ACon: A learning-based approach to deal with uncertainty in contextual requirements at runtime. Information & Software Technology 70 (2016), 85–99.
- Legay et al. (2010) Axel Legay, Benoît Delahaye, and Saddek Bensalem. 2010. Statistical Model Checking: An Overview. In RV (Lecture Notes in Computer Science), Vol. 6418. Springer, 122–135.
- Pan and Yang (2010) S. J. Pan and Q. Yang. 2010. A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering 22, 10 (Oct 2010), 1345–1359.
- Pessoa et al. (2017) Leonardo Pessoa, Paula Fernandes, Thiago Castro, Vander Alves, Genaína Nunes Rodrigues, and Hervaldo Carvalho. 2017. Building reliable and maintainable Dynamic Software Product Lines: An investigation in the Body Sensor Network domain. Information & Software Technology 86 (2017), 54–70.
- Qian et al. (2015) Wenyi Qian, Xin Peng, Bihuan Chen, John Mylopoulos, Huanhuan Wang, and Wenyun Zhao. 2015. Rationalism with a dose of empiricism: combining goal reasoning and case-based reasoning for self-adaptive software systems. Requir. Eng. 20, 3 (2015), 233–252.
- Quinlan (1999) J. Ross Quinlan. 1999. Simplifying decision trees. Int. J. Hum.-Comput. Stud. 51, 2 (1999), 497–510.
- Ramirez et al. (2012) Andres J. Ramirez, Adam C. Jensen, and Betty H. C. Cheng. 2012. A taxonomy of uncertainty for dynamically adaptive systems. In 7th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS. 99–108.
- Schmid et al. (2017) Sanny Schmid, Ilias Gerostathopoulos, Christian Prehofer, and Tomás Bures. 2017. Self-Adaptation Based on Big Data Analytics: A Model Problem and Tool. In 12th IEEE/ACM International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS. 102–108.
- Sharifloo et al. (2016) Amir Molzam Sharifloo, Andreas Metzger, Clément Quinton, Luciano Baresi, and Klaus Pohl. 2016. Learning and evolution in dynamic software product lines. In Proceedings of the 11th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS. 158–164.
- van Solingen et al. (2002) Rini van Solingen, Vic Basili, Gianluigi Caldiera, and H. Dieter Rombach. 2002. Goal Question Metric (GQM) Approach. John Wiley & Sons, Inc.
- Vrbaski et al. (2012) Mira Vrbaski, Gunter Mussbacher, Dorina C. Petriu, and Daniel Amyot. 2012. Goal models as run-time entities in context-aware systems. In Proceedings of the 7th Workshop on Models@run.time, Innsbruck, Austria, October 02, 2012. 3–8.
- Wang and Witten (1997) Y. Wang and I. H. Witten. 1997. Induction of model trees for predicting continuous classes. In Poster papers of the 9th European Conference on Machine Learning. Springer.
- Weyns et al. (2016) Danny Weyns, Nelly Bencomo, Radu Calinescu, Javier Cámara, Carlo Ghezzi, Vincenzo M Grassi, Lars Grunske, Paola Inverardi, Jean-Marc Jézéquel, Sam Malek, Raffaela Mirandola, Marco Mori, and Giordano Tamburrelli. 2016. Perpetual Assurances for Self-Adaptive Systems.
- Weyns and Iftikhar (2016) Danny Weyns and M. Usman Iftikhar. 2016. Model-Based Simulation at Runtime for Self-Adaptive Systems. In ICAC. IEEE Computer Society, 364–373.
- Younes and Simmons (2006) Håkan L. S. Younes and Reid G. Simmons. 2006. Statistical probabilistic model checking with a focus on time-bounded properties. Inf. Comput. 204, 9 (2006), 1368–1409.
- Zeller and Prehofer (2012) Marc Zeller and Christian Prehofer. 2012. Timing constraints for runtime adaptation in real-time, networked embedded systems. In 7th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS. 73–82.