Rethinking System Health Management

03/10/2019 ∙ by Edward Balaban, et al. ∙ 0

Health management of complex dynamic systems has traditionally evolved separately from automated control, planning, and scheduling (generally referred to in the paper as decision making). A goal of Integrated System Health Management has been to enable coordination between system health management and decision making, although successful practical implementations have remained limited. This paper proposes that, rather than being treated as connected, yet distinct entities, system health management and decision making should be unified in their formulations. Enabled by advances in modeling and computing, we argue that the unified approach will increase a system's operational effectiveness and may also lead to a lower overall system complexity. We overview the prevalent system health management methodology and illustrate its limitations through numerical examples. We then describe the proposed unification approach and show how it accommodates the typical system health management concepts.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

System Health Management (SHM) has evolved from simple red-line alarms and human-initiated responses to a discipline that often includes sophisticated modeling and automated fault recovery recommendations [Aaseng2001]. The main goal of modern SHM has been defined as the preservation of a system’s ability to function as intended [Rasmussen2008, Johnson and Day2011]. While in this paper we apply the term SHM to the operational phase of a system’s lifespan, in other contexts the term may also encompass design-time considerations [Johnson and Day2011]. The actual achievement of a system’s operational objectives, on the other hand, is under the purview of the fields of control, planning, and scheduling. In this paper, we will generally refer to all of the processes aimed at accomplishing operational objectives as decision making (DM), while using the more specialized terms where necessary.

Historically, the field of SHM has developed separately from DM. The typical SHM functions (monitoring, fault detection, diagnosis, mitigation, and recovery) were originally handled by human operators–and similarly for DM [Aaseng2001, Ogata2010]. As simple automated control was being introduced for DM, automated fault monitors and alarms reduced operator workload on the SHM side. Gradually, more sophisticated control techniques were developed for DM [Ogata2010], while automated emergency responses started handling some of the off-nominal system health conditions [Rasmussen2008]. Capable automated planning and scheduling tools eventually became available for performing more strategic DM [Ghallab, Nau, and Traverso2016]. On the SHM side, automated fault diagnosis was added, in some cases coupled with failure prediction (i.e., prognostic) algorithms [Lu and Saeks1979], as well as with recovery procedures [Avizienis1976]. Still, the two sides largely remained separated. When the concept of Integrated System Health Management became formalized, most interpretations of integrated encompassed some interactions between DM and SHM, although practical implementations of such interactions have been limited [Figueroa and Walker2018].

This paper makes the following claims:

  1. For actively controlled systems, prognostics is not meaningful; for uncontrolled systems prognostics may only be meaningful under a specific set of conditions;

  2. DM should be unified with SHM for optimality and there is a path for doing so;

  3. Automated emergency response should be done separately from unified DM/SHM, in order to provide performance guarantees and dissimilar redundancy.

For the second claim, we intend to show a path towards DM/SHM unification that builds on the latest developments in state space modeling, planning, and control. While in the past limitations in computing hardware and algorithms would have made unified DM/SHM difficult, we believe that the advances of the recent years make it an attainable goal. We also believe that this unified approach is applicable to a broad spectrum of DM, from traditional deterministic planners to complex algorithms computing long-horizon action policies in the presence of uncertainty. We use the term unification to emphasize the idea of DM and SHM being done within the same framework, rather than the two being integrated as separate subsystems exchanging information.

Some initial progress towards DM/SHM unification can be found in the earlier work by Balaban2013a Balaban2013a, Balaban2018 Balaban2018, and Johnson2010 (Johnson2010,Johnson2011a). System health information was also incorporated into DM by others [Bethke, How, and Vian2008, Ure et al.2013, Agha-mohammadi et al.2014]. This paper aims to introduce a systematic view on such integration, discuss its benefits, and illustrate how current SHM concepts map into the proposed approach without a loss of functionality or generality.

The paper first defines the categories of systems that are of interest to this study (Section 2) and then overviews the prevailing approach to SHM (Section 3). The first claim (on prognostics) is discussed in Section 4. Section 5 discusses the second claim, concerning DM/SHM integration and its benefits, as well as the rationale for the third claim (that DM/SHM should be separate from automated emergency response). Section 6 concludes.

2 Systems of interest

In the discussion to follow, we consider both uncontrolled and controlled systems. For our purposes, the uncontrolled systems category includes not only those systems types for which control is not available or required, but also those operating on predefined control sequences, such as industrial robots performing the same sets of operations over extended periods of time. Also included are system types that can be considered uncontrolled within some time interval of interest (decision horizon). Systems in this category may have degradation modes that affect the system’s performance within its expected useful life span. The rate of degradation is influenced by internal (e.g., chemical decomposition) and external factors (e.g., temperature of the operating environment). In addition to aforementioned industrial robots, examples of uncontrolled system types include bridges, buildings, electronic components, and certain types of rotating machinery, such as electrical power generators.

The controlled systems category covers all other system types, including dynamically controlled systems operating in uncertain environments, where the current state cannot be fully observed and non-determinism is present in control action outcomes. Degradation processes are influenced not only by the same kinds of internal and external factors as for the uncontrolled systems, but also by the control actions.

Most of the discussion to follow is applicable to both categories, although controlled systems would, naturally, benefit more from active DM/SHM. In describing the systems, we adopt the notation from the field of decision making under uncertainty [Kaelbling, Littman, and Cassandra1998].

A system state

can be a scalar or a vector belonging to a state space

. A system action initiates state transitions, with denoting the space of all available actions ( may be state-dependent). A transition model

describes the probability of transitioning to a particular state

as a result of taking action from state . A reward model describes a positive reward obtained or a negative cost incurred as a result of taking action from state . Terminal states form a subset . Terminal states may include both failure and goal states. Transitions from a terminal state are only allowed back to itself.

If there is state uncertainty, belief states are used instead of regular states (also referred to as beliefs). A belief

is a probability distribution over

, with denoting the space of all beliefs. Observations (e.g., sensor readings) can help with

belief estimation

and updating. Like a state, an observation can be a vector quantity. An observation model describes the probability of getting an observation upon transition to state as a result of action .

The general function of decision making is to select actions. While in some systems we are only concerned with selecting a single at a given time , decision making problems often involve selecting a sequence of actions. In a strictly deterministic system, an entire sequence of actions (a plan) can be selected ahead of time. In systems with action outcome uncertainty, however, a fixed plan can quickly become obsolete. Instead, a policy needs to be selected that prescribes which action should be taken in any state. If a policy is selected that, in expectation, optimizes a desired metric (e.g., maximizes cumulative reward), it is referred to as an optimal policy and denoted .

Throughout the paper, we use a robotic exploration rover operating on the surface of the Moon as a running example of a complex controlled system. The rover is solar-powered and stores electrical energy in a rechargeable battery.

3 System Health Management

A typical contemporary SHM integration approach is shown in Figure 1. A DM subsystem generates an action , aimed at achieving the operational objectives. The plant executes and an observation is generated and relayed to both DM and SHM. DM computes on the basis of , while SHM analyzes for indications of faults (defined here as system states considered to be off-nominal) and, if any are detected, issues a mitigation or recovery command either directly to the plant or as a recommendation to the DM subsystem [Valasek2012].

Figure 1: A typical system architecture with SHM
Figure 2: A typical contemporary SHM architecture

Figure 2 goes into more detail on the SHM subsystem. The fault detection module corresponds to the traditional red-line monitors detecting threshold-crossing events of sensor values, represented on the diagram by a Boolean fault detection function . If a fault is detected (), fault isolation and diagnosis (or identification) is performed, generating a vector of fault descriptors [Daigle and Roychoudhury2010]. Each fault descriptor typically identifies a component, its fault mode, and its fault parameters [Daigle and Roychoudhury2010]. There are diagnostic systems that also include an estimated fault probability in the descriptor [Narasimhan and Brownston2007]. If the uncertainty of the diagnostic results is deemed too high (e.g., consists of only low-probability elements), uncertainty management is sometimes performed in order to obtain a better estimate of the current system condition [Lopez and Sarigul-Klijn2010].

Some recent SHM implementations then pass to a prognostic module [Roychoudhury and Daigle2011]. In the SHM context, the intended goal of the prognostic module is to predict, at time (here ), whether and when faults will lead to system failure (defined as inability to perform its assigned function) within the window of a prediction horizon (the terms prediction horizon and decision horizon are equivalent for our purposes). In prognostics literature, the time of failure is commonly synonymous with the term end of [useful] life (EOL). Equivalently, the goal of prognostics can be defined as predicting the system’s remaining useful life (RUL). In Figure 2, the prognostic prediction is represented as a probability distribution for . Uncertainty management is sometimes also prescribed following prognostic analysis, meant to improve the prediction if the confidence in it is insufficiently high [Wang, Youn, and Hu2012]. Note that if a prognostic module is part of an SHM sequence, the term Prognostics and Health Management (PHM) is used by some instead of SHM in order to emphasize the role prognostics is playing in managing the system’s lifecycle.

Finally, and are passed to the fault mitigation and recovery component to select an action from the action set , in order to mitigate or recover from faults in . As part of this process, operational constraints may be set for those faulty components that cannot be restored to nominal health. If functional redundancy exists for such components, their further use may be avoided.

The overall limitations of the current SHM approach are discussed in Section 5, where an approach that unifies DM and SHM is then proposed. The next section, however, focuses on the prognostic component and discusses why it is not meaningful for actively controlled systems and is challenging to implement in a useful manner for uncontrolled systems.

4 Prognostics

A general definition of prognostics is that of a process predicting the time of occurrence of an event [Daigle, Sankararaman, and Kulkarni2015]. Using notation from Section 2, if (where ) is an event threshold function, then is the nearest predicted time of ( is the state at time ). If the state evolution trajectory is non-deterministic, then is computed instead. If states cannot be directly observed, is computed. As defined, prognostics is only meaningful in a specific set of circumstances and next we use two examples to illustrate why.

4.1 Uncontrolled systems

There are two main desirable, interrelated attributes for a prognostic system: (1) low uncertainty in EOL estimation, so that a decision about mitigation or recovery actions can be made with confidence, and (2) the ability to make a prediction far enough in advance for the actions to be successfully executed. For uncontrolled systems, this means that prognostics is primarily useful for systems with long lifetimes, low process uncertainty, or both. To illustrate why this is the case, we start with a simple uncontrolled system example:

Figure 3: Uncontrolled system prognostics (Example 1)
Example 1 ().

At time , the health state of a system is (states are scalar). According to a deterministic model, the nominal health degradation rate is constant at , where is the prediction time step, selected as the minimum time interval within which a change in the system’s health is expected to be detectable. A stochastic model predicts the probability of the nominal degradation rate within any time step as and the probability of a higher degradation rate () as . Assume .

The objective for both models is to predict EOL, i.e., the smallest for which . For this example, the prediction uncertainty is defined as , i.e., the absolute difference between the expected EOL values computed by the two models at a prediction time . A requirement is set on the maximum EOL prediction uncertainty as .

For this example, let us assume that the health state is fully observable and define , the fraction of full health remaining at . In Figure 3, a prediction is shown to be made at , with , , and . Since


In the last equation, we substitute the value for the nominal degradation rate in order to focus on the effects of the degradation rate uncertainty. As can be seen in Figure 3, EOL is reached by both models within the prediction horizon. However, from Equation 1, .

For the requirement on to be satisfied, either (health fraction remaining), (the probability of deviations from the nominal degradation rate), (the magnitude of deviations), or some combination of them needs to be reduced. If and are kept the same, with we can get . However, now needs to be , with only left until failure (RUL). RUL corresponds to the time available to either replace/repair the uncontrolled system or initiate an emergency response. For a quickly degrading system with , RUL would be only , which is likely enough time for an emergency response, but not for repair or replacement. In practice, in uncontrolled systems where handling a fast-developing degradation process is important, estimating is unlikely to bring tangible benefits. For instance, if pressure starts building quickly in a fuel tank of an ascending crewed rocket, the launch abort system (emergency response) is likely to be activated by the exceedance of a predefined pressure limit or pressure increase rate (i.e., functions of fault detection). Computing whether the tank breach will occur in seconds or seconds will not materially influence that response. For uncontrolled systems with mid-range degradation rates (minutes, hours, days), extrapolation of the degradation function may be able to serve as part of a caution and warning mechanism.

What follows is that if degradation uncertainty is relatively high or varies significantly over time, a short prediction horizon (compared to the overall system lifetime) may be necessary to limit the uncertainty propagation and result in a usable . In this case, systems with longer lifetimes are more suitable for applying prognostics. For example, if a bridge failure can be predicted years in advance with an accuracy of year, that can still be a useful prediction.

However, while many uncontrolled systems can be classified as systems with long lifetimes, there exists a number of fundamental practical difficulties in performing effective prognostics for them. One of the primary issues stems directly from the typically long (often decades) lifetimes. In order to establish trust in the degradation models, they need to be adequately tested using long-term observations from “normal use” degradation or observations from properly formulated accelerated testing. With useful (from the statistical point of view) “normal use” degradation data sets being rare for many long-life system types

[Heng et al.2009], accelerated degradation efforts are common. If an accelerated degradation regime is proposed, however, what needs to be clearly demonstrated is that:

  1. The regime can be used as a substitute for real-life degradation processes. For instance, while Rigamonti2016 Rigamonti2016 and Celaya2011a Celaya2011a use thermal and electrical overstress to quickly degrade electrical capacitors and predict the time of their failure (by using an empirical equation), their work does not extend to making a connection to real-life degradation, which takes place at lower temperatures and voltage/current levels. Similar issues are highlighted by Dao2006 Dao2006 for composite materials, where mechanical, thermal, and chemical processes result in complex interactions during aging.

  2. There is a known mapping from the accelerated timeline to the unaccelerated timeline. Oh2015 Oh2015 note in an overview of condition monitoring and prognostics of insulated gate bipolar transistors that while numerous fatigue models have been constructed that predict cycles-to-failure under repetitive cycle loading, they are not designed to predict RUL under realistic usage conditions. Although use of various fatigue analysis models, e.g., Paris1961 Paris1961, has been proposed for estimating RUL on the basis of stress cycles, their accuracy has proven difficult to confirm.

There are subfields of prognostics where accelerated aging regimes may be viable, such as in aircraft metallic structures or rotating machinery (where mechanical degradation factors could be assumed dominant). However, the issue of high uncertainty of degradation trajectories still arises, even under the same test conditions [Virkler, Hillberry, and Goel1979, Meng1994]. Finite element modeling may help alleviate degradation trajectory uncertainty in specific cases, albeit at a significant computational cost [Heng et al.2009].

Some of the other challenges with effective prognostics for uncontrolled systems include the accuracy of estimating the actual state of health, the effects of fault interactions, and the effects of system maintenance actions [Heng et al.2009].

If these challenges are successfully overcome and the failure mechanisms of a component are understood well enough to develop useful degradation models, a different question then arises: should the design or usage of the component be changed to mitigate these mechanisms? While in some cases this may not be feasible, in others it may be the simplest and most reliable way of improving safety and maintenance requirements [Bathias and Pineau2013]. A redesign or change in usage would, on the other hand, make the degradation models obsolete. The next tier of degradation modes would then need to be analyzed and modeled, possibly followed by another redesign. Thus analysis intended for the development of degradation (prognostics) models instead becomes part of the design cycle.

For those uncontrolled systems that are, in fact, suitable for health management based on prognostics, the action space is typically limited to: (a) no action, (b) replacement, or (c) repair to a nominal operating condition. Even so, we still propose that predictive analysis for these systems needs to be driven by decision making requirements. For instance, if domain knowledge informs that variability in system behavior beyond some health index value is too great for choosing actions with sufficiently high confidence, then EOL can be redefined as and system dynamics beyond need to be neither modeled nor computed during prediction, potentially freeing computing resources for estimating system behavior up to with more accuracy.

4.2 Controlled systems

As soon as dynamic control is introduced into the system and uncertainty is taken into account, prognostics, as defined above, and the PHM version of the process in Figure 2 are no longer meaningful–for two key reasons. First, not having the knowledge, at , of the future system actions, a PHM algorithm will either (a) have to rely on some precomputed plan to obtain for its predictive analysis (a plan that can quickly become obsolete due to outcome uncertainty) or (b) have to use a random policy (which can, for instance, result in less optimal actions being considered as probable as the more optimal ones). A random policy is also likely to result in a greater state uncertainty throughout the interval. Second, the oft-proposed strategy of rerunning prognostic analysis after an action, so that new information can be taken into account [Tang et al.2011], may not help. Once a suboptimal execution branch has been committed to, it may remain suboptimal regardless of future decisions. The following example provides an illustration of these issues:

Example 2 ().

A rover needs to traverse an area with no sunlight, going around a large crater from waypoint to the closest suitable recharge location at . The battery charge at is .

There are three possible levels of terrain difficulty: difficult (requiring per drive segment), moderate ( per segment), and easy ( per segment). All drive segments are the same length. Probabilities of terrain types in different regions are shown in Figure 4.

The rover can go to the left, , or to the right, (left and right are relative to the diagram). If going to the right, it can decide to detour around a smaller crater (easy terrain with ) instead of going directly .

Figure 4: PHM vs. DM for a controlled system (Example 2)
Figure 5: Execution scenarios in Example 2

A decision support PHM algorithm, running a sufficiently large number of simulations, would consider two possible execution scenarios along the left route: (L1) , and (L2) , ( is the total energy consumed in a scenario). The expected energy consumption along the left route can then be computed as .

The algorithm would then consider four possible execution scenarios along the right route (assuming uniform random choice of action at ): (R1) , ; (R2) , ; (R3) , ; and (R4) , . Then . With , the PHM algorithm commits to the left path. Note that in many cases prognostics algorithms generate even less information to support action selection than what was done here (typically only).

A DM algorithm capable of sequential reasoning under uncertainty [Kochenderfer2015] would compute in the same manner as the PHM algorithm, as there are no actions needing to be selected along the left route after . On the right side, however, the DM algorithm can make an informed choice at , based on observations made along . This implies having only two possible execution scenarios: (R1) if the terrain is observed as difficult, the detour through is taken, and (R2) if the terrain is observed as moderate, the rover goes directly to . For R1, , . For R2, , . The expected energy use is thus . With , the algorithm chooses the right path.

Now let us assume that the true terrain condition both on the left and the right sides of the crater is difficult. The left path () will require to traverse, therefore a rover relying on the PHM algorithm will fall short and will not reach . A rover relying on the DM algorithm will expend only (scenario R1), arriving at with in reserve.

It may be suggested that the issues with the PHM approach could be eliminated if access to a precomputed operational policy is provided. However, even if such a policy was available, that would still be insufficient. If, at time , is computed using , then is taken on the basis of , could immediately become invalid unless .

5 Unifying DM and SHM

This section discusses the benefits of a unified DM/SHM approach and outlines some of the key implementation details.

5.1 Separated vs. unified DM/SHM

The next example illustrates how unification can be helpful in balancing system health needs and operational objectives:

Figure 6: Separated vs. unified DM/SHM (Example 3)
Figure 7: Execution scenarios in Example 3
Example 3 ().

The rover starts at in Zone 1 (Figure 6) with in the battery (out of the capacity). The solar panels can charge the battery at a rate of . Sunlight in Zone 1 will last for another 12 hours.

Two actions are available in the initial state at (Figure 7): skip charging () and charge to full (). In Figure 7, the unitless numbers are Watt-hours of energy going in or out of the battery. Time (in hours) at each state is denoted as ’’.

The rover needs to perform a 2-hour stationary science activity at and be able to arrive at

, the next recharge point. The prior probability of the activity at

needing to be redone (a 2-hour delay) is . Science payload power consumption is , resulting in a net-positive () power flow.

The driving times from to and from to are hours, with the average drive train power consumption of , resulting in a net-negative () power flow.

If operating without sunlight, a heater needs to be used to keep batteries and electronics warm, thus resulting in a net-negative power flow of for driving and for stationary science activities.

According to the general SHM policy of restoring health (battery charge, in this case) to nominal, the action chosen at is and the battery is recharged to full . After the science activity at is completed, assessment is made that it needs to be repeated. The 2-hour delay means that the entirety of segment needs to be done without sunlight, resulting in a complete battery depletion before is reached, with a deficit of .

In computing a unified policy, however, where SHM actions are considered in the context of the overall mission, all four scenarios depicted in Figure 7 would play a role. The expected amount of battery charge remaining if is chosen would be . For : . Action (no recharge) would be chosen and, with the two-hour delay at , the rover would arrive at with still remaining.

The example illustrates the first benefit of unifying DM and SHM: the ability to naturally take operational objectives and constraints into account when making a system health recovery decision. The next example illustrates how DM, on the other hand, can benefit from unified action spaces and access to health-related models:

Figure 8: Unified action spaces (Example 4)
Example 4 ().

The rover is traveling from to (flat terrain) when a decision is made at point A to make a detour and attempt data collection at a scientifically valuable , requiring a 6-hour climb up a steep hill (Figure 8). The 1-hour data collection activity at must be completed before the loss of illumination there in hours.

After completing the 2-hour climb up to point ( of the way up), it is observed that the internal temperature of one of the drive motors has risen to from the nominal of . At there is a significant risk of permanent damage and failure of the motor.

If the SHM system on the rover consists of the traditional fault detection, diagnosis, and mitigation/recovery components only, it may diagnose the fault to be increased friction, mark the component (motor) as faulty, and set a constraint on terrain types to either flat or downhill. It would then invoke the recommended action for this fault from : stop and cool down (until ).

If a prognostic component is present, it may predict that at the current temperature increase rate, the RUL of the motor is 1 hour (with 4 hours of climb remaining to reach ). The same mitigation action (stop and cool down) would be initiated and the same constraint on the current to the motor (and, thus, on the incline angle) may be set. After returns to nominal (which happens to take 1 hour), control is returned to DM. Based on the new constraints and the motor marked as faulty, DM would command the rover to abort the detour, return to , and resume the drive to .

If, however, DM had stop and cool down as part of its action space () and updated the state variables for the affected motor with the newly computed heat-up and cool down rates, an operational policy could be computed that optimizes the duration of driving and cool down intervals and allows the rover to reach in time. For instance, if the rover drives for two hours, then stops for an hour to cool down the motor, it can still reach in hours. With the science activity taking 1 hour, there would still be an hour in reserve before the loss of sunlight at .

5.2 Unification approach

In proposing the unified DM/SHM approach, we rely heavily on utility theory [Fishburn1970]. The following, in our opinion, are the key ingredients for a successful unification: (1) a state-based system modeling framework and (2) a utility (value) function capturing the operational preferences for the system. A utility function (denoted as ) captures, numerically, our preferences over the space of possible outcomes. For instance, we may assign a higher utility value to a rover state where a desired scientific location has been reached. Utility can be defined for system states or for state-action pairs. If, in some system state , the outcome of an action (a particular state ) is not guaranteed, then the expected utility of the tuple is


In a strictly deterministic system, where a plan up to a horizon can be provided ahead of time, the expected utility of relative to the plan is:


For systems with action outcome uncertainty, the expected utility associated with executing a policy for steps from state can be computed recursively as


where is the discount factor that is sometimes used to bias towards earlier rewards, particularly in infinite horizon problems (). The optimal utility for a state can then also be computed recursively using


which for becomes the Bellman equation [Bellman1957]. Knowing the optimal utility function, we can derive an optimal policy:


In problems with state uncertainty, beliefs can take the place of states in equations 46 [Kaelbling, Littman, and Cassandra1998]. The Markov property is assumed, meaning that does not depend on the sequence of transitions that led to [Kemeny and Snell1983].

Now that the foundational concepts have been described, we will focus on those most relevant to the proposed unification: states and the reward function . States can be vector quantities (Section 2). For real-world problems, the relevant elements of the operating environment are sometimes included in the state vector, either explicitly or implicitly [Ragi and Chong2014]. For instance, the rover state vector would certainly need to include the rover’s and coordinates, but may also include time . These three elements allow us to indirectly access other information about the environment, e.g., solar illumination, ambient temperature, communications coverage, and terrain properties.

Similarly, health-related elements can be included in the same state vector. For the rover, the battery charge would likely be in it already for operational purposes. Adding battery temperature, however, would allow for better reasoning about the state of battery health, when combined with information on ambient temperature, terrain, and recharge current. Thus, including even a few health-related elements in the state vector may have a multiplicative effect on the amount of information available. The resulting size of the state vector may also end up being smaller than the sum of separately maintained DM and SHM state vectors, as redundant elements are eliminated.

The reward function encodes the costs and the rewards of being in a particular state or taking a particular action in a state and can also be thought of as a local utility function. For many realistic problems with multi-variable state representations, the reward function needs to combine costs or rewards pertinent to state components. Several approaches have been proposed [Keeney and Raiffa1993], with additive decomposition being an effective option in many cases. However it is implemented, the key property of the function is that by mapping multiple variables to a single number, it allows us to compute or and translate a potentially complex DM formulation into an abstract utility maximization problem.

As an example of such mapping, consider two different rover states: , where the rate of battery discharge is higher than nominal due to a parasitic load in the electrical system and , where the rover is healthy, but is traversing difficult terrain, thus also leading to a higher rate of battery discharge. The utilities and may end up being approximately equal, however, given the similar impact of the two issues on future rewards. Policies computed for , , and their follow-on states may be similar also and, for instance, could result in more frequent recharge stops.

One notable consequence of health-related components being integrated into a common state vector is that from the computational point of view the concepts of fault and failure become somewhat superfluous. If subsets or are defined for the system, the framework described above will not do anything differently for them. The only essential subset of is (the terminal states). Failure states may be part of if they result in termination of system operations; however, goal (success states) are members of also. The only difference between them is in their values. As long as a component fault or a failure does not lead to a transition to a terminal state, actions that maximize the expected value of that state will be selected (which, as it happens, implements the “fail operational” philosophy).

Figure 9: The main loop of health-aware decision making

We will refer to this unified approach as health-aware decision making (HADM). The rest of the major SHM concepts are incorporated into the new approach as follows. Fault detection and diagnostics are subsumed in belief estimation and updating, although these operations are, of course, used for nominal belief states as well. Uncertainty management can now be purposefully incorporated into the decision making process by either augmenting with information gathering actions [Spaan, Veiga, and Lima2015], evaluated in the same context as other types of actions, or by continuing to improve or estimates until a desired level of confidence in them is reached [Browne and Powley2012]. For actively controlled systems, predictive simulations are simply an integral part of calculation, where serves as a one-step “prognostic” function (with degradation models, if any). Whereas prognostic algorithms applied to controlled systems are limited in their predictive ability due to the lack of knowledge about future actions, here the calculation process is an exploration of possible execution scenarios, thus combining or estimation with sequential action selection.

The overall HADM operational loop (assuming state and outcome uncertainty) can be seen in Figure 9. Once the initial belief is estimated at time , either an offline policy is referenced or an online policy is computed to determine (best action). The action is executed by the plant, transitioning to a new (hidden) state, and generating an observation . The observation is then used to update (typically through some form of Bayesian updating) and the process repeats until a terminal state is believed to be reached.

A detailed discussion of the actual algorithms that can implement the proposed approach is left for future publications, however examples of DM algorithms that can deal with state or outcome uncertainty are provided by Kochenderfer2015 Kochenderfer2015. In systems where both state and outcome uncertainty are not a factor, a variety of traditional state space planning algorithms [Ghallab, Nau, and Traverso2016] would not need any modification to produce plans for spaces of state vectors that include health-related components.

5.3 Emergency response vs. health management

For realistic complex systems operating in the presence of state and outcome uncertainty, , , and are likely to be infinitely large (with , although still potentially large). The problem of finding exact optimal policies in such cases is PSPACE-complete [Papadimitriou and Tsitsiklis1987]. Approximate solution methods typically work online, constructing for the current belief based on beliefs reachable from within the decision horizon [Browne and Powley2012]. They also typically perform targeted sampling from , , and , thus optimality guarantees can be harder to provide. We, therefore, propose the following:

  • That system emergency response (SER) be defined as an automated or semi-automated process that is invoked to maximize the likelihood of preserving the system’s integrity, regardless of the effect on operational goals (i.e., a different objective from HADM).

  • That emergency response policy be computed separately from the HADM policy.

In the space domain, an example of SER is putting a spacecraft in a safe mode until the emergency is resolved [Rasmussen2008]. In aviation, it could be executing recommendations of a collision avoidance system [Kochenderfer2015]. Introduction of a separate SER system would likely require the introduction of , an additional subset which defines for which states is invoked versus . Once again, however, will not necessarily only contain system fault and failure states. For instance, states where environmental conditions warrant emergency response (e.g., solar activity interrupting teleoperation of the rover) would be included as well. Since the scope of the SER problem is likely to be much narrower than that of HADM, it opens up the possibility of computing, verifying, and validating offline [Kochenderfer2015].

As sensors, computing capabilities, and DM algorithms improve further, the fraction of the system’s state space that is under the purview of SER will decrease. Still, we foresee the need for a “safety net” SER to be there for safety-critical functions and thus advocate SER independence. SER would cover the situations where the primary HADM system may not be able to produce a suitable solution in a desired amount of time, serving as an equivalent of human reflexive responses triggered by important stimuli versus the more deliberative cognitive functionality of the brain. It could also provide dissimilar redundancy for critical regions of , essentially implementing the Swiss Cheese Model [Reason1990], which advocates multiple different layers of protection for important functions.

5.4 Remarks on computational complexity

A counter-argument to the unified approach can be made that it increases the computational complexity by operating on larger , , and spaces. While this is indeed a concern, algorithms have been developed in the recent years that can effectively accommodate much of the complexity, even for problems with state and outcome uncertainty, by approximating optimal policies online [Silver and Veness2010, Browne and Powley2012]. Secondly, some of the complexity is removed from HADM into an independent SER component, making the former problem easier. Finally, as noted above, unifying the SHM and DM elements may also result in reduction of overlapping model variables.

6 Conclusions

This paper reexamines the prevalent approach to performing system health management and makes the case for why keeping system health management functions separate from decision making functions can be inefficient and/or ineffective. We also present the case for why prognostics is only meaningful in a limited set of circumstances and, even then, needs to be driven by decision making requirements.

We then explain the rationale for unifying (not just integrating) system health management with decision making and outline an approach for accomplishing that. We also propose keeping emergency response functionality separate to guarantee timely response, provide dissimilar redundancy, and allow for offline computation, validation, and verification of emergency response policies.

We believe that the proposed unification approach will improve the performance of both decision making and system health management, while potentially simplifying informational architectures, reducing sensor suite sizes, and combining modeling efforts. The approach is scalable with respect to system complexity and the types of uncertainties present. It also puts a variety of existing and emerging computational methods at the disposal of system designers.


  • [Aaseng2001] Aaseng, G. B. 2001. Blueprint for an integrated vehicle health management system. In Digital Avionics Systems Conference.
  • [Agha-mohammadi et al.2014] Agha-mohammadi, A.; Ure, N. K.; How, J. P.; and Vian, J. 2014. Health aware stochastic planning for persistent package delivery missions using quadrotors. In IEEE/RSJ International Conference on Intelligent Robots and Systems.
  • [Avizienis1976] Avizienis, A. 1976. Fault-tolerant systems. IEEE Transactions on Computers 25(12).
  • [Balaban and Alonso2013] Balaban, E., and Alonso, J. J. 2013. A modeling framework for prognostic decision making and its application to UAV mission planning. In Annual Conference of the Prognostics and Health Management Society.
  • [Balaban et al.2018] Balaban, E.; Arnon, T.; Shirley, M. H.; Brisson, S. F.; and Gao, A. 2018. A system health aware POMDP framework for planetary rover traverse evaluation and refinement. In AIAA SciTech Forum.
  • [Bathias and Pineau2013] Bathias, C., and Pineau, A. 2013. Fatigue of Materials and Structures: Fundamentals. Wiley.
  • [Bellman1957] Bellman, R. 1957. A Markovian decision process. Journal of Mathematics and Mechanics (6):679–684.
  • [Bethke, How, and Vian2008] Bethke, B.; How, J. P.; and Vian, J. 2008. Group health management of UAV teams with applications to persistent surveillance. In American Control Conference.
  • [Browne and Powley2012] Browne, C., and Powley, E. 2012. A survey of Monte Carlo Tree Search methods. IEEE Transactions on Computational Intelligence and AI in Games 4(1):1–49.
  • [Celaya et al.2011] Celaya, J.; Kulkarni, C. S.; Biswas, G.; and Saha, S. 2011. A model-based prognostics methodology for electrolytic capacitors based on electrical overstress accelerated aging. In Annual Conference of the Prognostics and Health Management Society.
  • [Daigle and Roychoudhury2010] Daigle, M. J., and Roychoudhury, I. 2010. Qualitative event-based diagnosis: Case study on the Second International Diagnostic Competition. In International Workshop on the Principles of Diagnosis.
  • [Daigle, Sankararaman, and Kulkarni2015] Daigle, M. J.; Sankararaman, S.; and Kulkarni, C. S. 2015. Stochastic prediction of remaining driving time and distance for a planetary rover. In IEEE Aerospace Conference.
  • [Dao et al.2006] Dao, B.; Hodgkin, J.; Krstina, J.; Mardel, J.; and Tian, W. 2006. Accelerated aging versus realistic aging in aerospace composite materials. Journal of Applied Polymer Science 102:4291–4303.
  • [Figueroa and Walker2018] Figueroa, F., and Walker, M. G. 2018. Integrated System Health Management (ISHM) and autonomy. In AIAA SciTech Forum.
  • [Fishburn1970] Fishburn, P. C. 1970. Utility Theory for Decision Making. Research Analysis Corporation.
  • [Ghallab, Nau, and Traverso2016] Ghallab, M.; Nau, D.; and Traverso, P. 2016. Automated Planning and Acting. Cambridge University Press.
  • [Heng et al.2009] Heng, A.; Zhang, S.; Tan, A. C. C.; and Mathew, J. 2009. Rotating machinery prognostics: State of the art, challenges and opportunities. Mechanical Systems and Signal Processing 23(3):724–739.
  • [Johnson and Day2010] Johnson, S. B., and Day, J. C. 2010. Conceptual framework for a fault management design methodology. In AIAA Infotech@Aerospace Conference.
  • [Johnson and Day2011] Johnson, S. B., and Day, J. C. 2011. System health management theory and design strategies. In AIAA Infotech@Aerospace Conference.
  • [Kaelbling, Littman, and Cassandra1998] Kaelbling, L. P.; Littman, M. L.; and Cassandra, A. R. 1998. Planning and acting in partially observable stochastic domains. Artificial Intelligence 101(1-2):99–134.
  • [Keeney and Raiffa1993] Keeney, R. L., and Raiffa, H. 1993. Decisions with Multiple Objectives: Preferences and Value Tradeoffs. Cambridge University Press.
  • [Kemeny and Snell1983] Kemeny, J., and Snell, J. 1983.

    Finite Markov Chains

  • [Kochenderfer2015] Kochenderfer, M. J. 2015. Decision Making Under Uncertainty: Theory and Application. MIT Press.
  • [Lopez and Sarigul-Klijn2010] Lopez, I., and Sarigul-Klijn, N. 2010. A review of uncertainty in flight vehicle structural damage monitoring, diagnosis and control: Challenges and opportunities. Progress in Aerospace Sciences 46(7):247–273.
  • [Lu and Saeks1979] Lu, K., and Saeks, R. 1979. Failure prediction for an on-line maintenance system in a Poisson shock environment. IEEE Transactions on Systems, Man, and Cybernetics 9(6):356–362.
  • [Meng1994] Meng, H.-C. 1994. Wear modeling: evaluation and categorization of wear models. Ph.D. Dissertation, University of Michigan.
  • [Narasimhan and Brownston2007] Narasimhan, S., and Brownston, L. 2007. HyDE - a general framework for stochastic and hybrid model-based diagnosis. In International Workshop on the Principles of Diagnosis.
  • [Ogata2010] Ogata, K. 2010. Modern Control Engineering. Pearson.
  • [Oh et al.2015] Oh, H.; Han, B.; McCluskey, P.; Han, C.; and Youn, B. D. 2015. Physics-of-failure, condition monitoring, and prognostics of insulated gate bipolar transistor modules: A review. IEEE Transactions on Power Electronics 30(5):2413–2426.
  • [Papadimitriou and Tsitsiklis1987] Papadimitriou, C., and Tsitsiklis, J. 1987.

    The complexity of Markov decision processes.

    Mathematics of Operations Research 12(3):441–450.
  • [Paris, Gomez, and Anderson1961] Paris, P. C.; Gomez, M. P.; and Anderson, W. E. 1961. A rational analytic theory of fatigue. The Trend in Engineering 13(1):9–14.
  • [Ragi and Chong2014] Ragi, S., and Chong, E. K. P. 2014. UAV path planning in a dynamic environment via partially observable Markov decision process. IEEE Transactions on Aerospace and Electronic Systems 49(4):2397–2412.
  • [Rasmussen2008] Rasmussen, R. D. 2008. GN&C fault protection fundamentals. In AAS Guidance and Control Conference.
  • [Reason1990] Reason, J. 1990. Human Error. Cambridge University Press.
  • [Rigamonti et al.2016] Rigamonti, M.; Baraldi, P.; Zio, E.; Astigarraga, D.; and Galarza, A. 2016. Particle filter-based prognostics for an electrolytic capacitor working in variable operating conditions. IEEE Transactions on Power Electronics 31(2):1567–1575.
  • [Roychoudhury and Daigle2011] Roychoudhury, I., and Daigle, M. J. 2011. An integrated model-based diagnostic and prognostic framework. In International Workshop on the Principles of Diagnosis.
  • [Silver and Veness2010] Silver, D., and Veness, J. 2010. Monte-Carlo planning in large POMDPs. In Advances In Neural Information Processing Systems.
  • [Spaan, Veiga, and Lima2015] Spaan, M. T. J.; Veiga, T. S.; and Lima, P. U. 2015. Decision-theoretic planning under uncertainty with information rewards for active cooperative perception. Autonomous Agents and Multi-Agent Systems 29:1157–1185.
  • [Tang et al.2011] Tang, L.; Hettler, E.; Zhang, B.; and Decastro, J. 2011. A testbed for real-time autonomous vehicle PHM and contingency management applications. In Annual Conference of the Prognostics and Health Management Society.
  • [Ure et al.2013] Ure, N. K.; Chowdhary, G.; How, J. P.; Vavrina, M. A.; and Vian, J. 2013. Health aware planning under uncertainty for UAV missions with heterogeneous teams. In European Control Conference.
  • [Valasek2012] Valasek, J. 2012. Advances in Intelligent and Autonomous Aerospace Systems. AIAA.
  • [Virkler, Hillberry, and Goel1979] Virkler, D. A.; Hillberry, B.; and Goel, P. K. 1979. The statistical nature of fatigue crack propagation. Journal of Engineering Materials and Technology 101(2):148–153.
  • [Wang, Youn, and Hu2012] Wang, P.; Youn, B. D.; and Hu, C. 2012. A generic probabilistic framework for structural health prognostics and uncertainty management. Mechanical Systems and Signal Processing 28:622–637.