A Generally Applicable, Highly Scalable Measurement Computation and Optimization Approach to Sequential Model-Based Diagnosis

11/15/2017 ∙ by Patrick Rodler, et al. ∙ Alpen-Adria-Universität 0

Model-Based Diagnosis deals with the identification of the real cause of a system's malfunction based on a formal system model and observations of the system behavior. When a malfunction is detected, there is usually not enough information available to pinpoint the real cause and one needs to discriminate between multiple fault hypotheses (called diagnoses). To this end, Sequential Diagnosis approaches ask an oracle for additional system measurements. This work presents strategies for (optimal) measurement selection in model-based sequential diagnosis. In particular, assuming a set of leading diagnoses being given, we show how queries (sets of measurements) can be computed and optimized along two dimensions: expected number of queries and cost per query. By means of a suitable decoupling of two optimizations and a clever search space reduction the computations are done without any inference engine calls. For the full search space, we give a method requiring only a polynomial number of inferences and show how query properties can be guaranteed which existing methods do not provide. Evaluation results using real-world problems indicate that the new method computes (virtually) optimal queries instantly independently of the size and complexity of the considered diagnosis problems and outperforms equally general methods not exploiting the proposed theory by orders of magnitude.



There are no comments yet.


page 11

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Model-based diagnosis (MBD) is a widely applied approach to finding explanations for unexpected behavior of observed systems such as hardware (Reiter, 1987; Dressler and Struss, 1996), software (Stumptner and Wotawa, 1999; Mateis et al., 2000; Steinbauer et al., 2005), knowledge bases (Parsia et al., 2005; Kalyanpur, 2006; Shchekotykhin et al., 2012; Rodler, 2015), discrete event systems (Darwiche and Provan, 1996; Pencolé and Cordier, 2005), feature models (White et al., 2010) and user interfaces (Felfernig et al., 2009). MBD assumes a formal system model and a set of relevant possibly faulty system components (e.g. lines of code, gates in a circuit). The model includes descriptions of the interrelation between the components (e.g. wires between gates), descriptions of the components’ nominal behavior (e.g. relation between inputs and outputs of a gate) and other relevant knowledge (e.g. axioms of Boolean logic). An MBD problem arises if observations (e.g. sensor readings, system outputs) of the system’s behavior differ from predictions based on the system model. In this case, the set of observations is inconsistent with the system model under the assumption that all system components are exhibiting a nominal behavior. The sought solution to an MBD problem is a diagnosis pinpointing the faulty components causing the observed system failure. Normally, however, due to initially insufficient observations, this fault localization is ambiguous and multiple possible diagnoses exist.

Sequential Diagnosis methods (de Kleer and Williams, 1987; Pietersma et al., 2005; Feldman et al., 2010; Siddiqi and Huang, 2011; Shchekotykhin et al., 2012) address this issue. These collect additional information by generating a sequence of queries and assume available some oracle providing answers to these queries. Depending on the MBD application domain, queries can be, for instance, measurements (e.g. probes in a circuit), system tests (observations about the system’s behavior upon new system inputs), questions to a domain expert (e.g. to a doctor when debugging a medical knowledge base) or component inspections (e.g. checking the battery of a car). Likewise, the instantiation of the oracle might be, for instance, an electrical engineer performing probes using a voltmeter, an IDE running software tests or a car mechanic inspecting components of a vehicle. If queries are chosen properly, each query’s answer eliminates some diagnoses and thus reduces the diagnostic uncertainty (pruning of the space of possible diagnoses). As query answering is normally costly, the goal of sequential diagnosis is to minimize the diagnostic cost in terms of, e.g., time, manpower or equipment required to achieve a diagnostic goal

, e.g., the extraction of a diagnosis with a probability above some threshold or the isolation of a single remaining diagnosis (which then corresponds to the

actual diagnosis, i.e. the actual cause of the system failure).

Figure 1: Schematic view on a generic sequential diagnosis system. The area shaded in violet shows the part of the system optimized by the approach in this work. The red arrow emphasizes that (expensive) reasoner calls have to be minimized.

A generic sequential diagnosis system is illustrated by Fig. 1. It gets the inputs sd (system description), comps (system components), obs (initial observations), meas (additional observations / performed measurements), which altogether make up a diagnosis problem instance (DPI), and possibly some fault information (e.g. in terms of failure probabilities of system components). The usual workflow (see numbers in Fig. 1) followed by such a system involves the (1) computation of a (feasible) set of diagnoses by a diagnosis engine using the DPI and fault information, (2) computation of a set of query candidates by a query generation module based on the given diagnoses, (3) selection of the best query from the given candidates, (4) answering of this query by the interacting oracle, (5+6) addition of the returned query along with its answer to the DPI in terms of new measurements (meas). The diagnosis engine uses these new measurements to perform various updates (e.g. pruning of the diagnoses space, adapting the fault information). If the diagnostic goal is not accomplished, the entire process starts anew from (1). Otherwise, the best diagnosis is output. The focus of this work lies on the optimization of steps (2) and (3) in terms of both efficiency and output quality (see violet shaded area in Fig. 1).

Note, the steps (1) and (2) draw on a logical reasoner. Since logical reasoning is one of the main sources of complexity in sequential diagnosis, the amount of reasoning should be ideally as minimal as possible, indicated by the red arrow in Fig. 1. Basically, there are two different reasoning paradigms sequential diagnosis systems might use, glass-box and black-box. Glass-box approaches directly integrate reasoning with diagnoses finding with the goal of achieving better performance. To this end the internals of the reasoner are suitably modified or, respectively, reasoners are complemented by additional services, e.g., bookkeeping in an ATMS (de Kleer, 1986). One example (de Kleer and Williams, 1987) is the storing of (minimal) environments (sets of logical sentences sufficient) for entailments predicted by the system model. These are leveraged to compute so-called nogood sets (de Kleer, 1986), i.e. environments for entailments inconsistent with observations. The latter can be directly used for diagnoses construction. Glass-box approaches are therefore dependent on the particular (modified) reasoner and thus on the particular logic for which the reasoner is sound and complete. Black-box approaches use the reasoner as an oracle for answering consistency or entailment queries. The reasoner is used as-is without requiring any alterations to its implementation or any supplements. Consequently, these approaches are independent of the logic used for describing the system model and of the particular reasoner employed, and can benefit from latest improvements of reasoning algorithms. For instance, black-box approaches can switch to reasoners specialized in a certain sublanguage (e.g. polynomial-time reasoner ELK (Kazakov et al., 2014) for OWL EL (Krötzsch, 2010)) of a logic (e.g. OWL 2 (Grau et al., 2008) where reasoning is N2EXPTIME-complete) “for free” in a simple plug-in fashion if the system description is formalized in this sublanguage.

First, while glass-box approaches in many cases offer some performance gain over black-box approaches, this gain was shown to be not that significant – in most cases the time cost of both paradigms lay within the same order of magnitude – in extensive evaluations carried out by (Horridge, 2011) using Description Logics (Baader et al., 2007) of reasoning complexity ranging from polynomial to N2EXPTIME-complete. Black-box approaches even outperformed glass-box approaches in a significant number of cases, witnessed in similar experiments conducted by (Kalyanpur, 2006). When using bookkeeping methods, the information stored by these might grow exponentially with the problem size (Schiex and Verfaillie, 1994). Moreover, switching to more efficient reasoners (e.g., for fragments of a logic, see above) is not (easily) possible for glass-box approaches. Second, system descriptions (sd) in MBD might use a wide range of different knowledge representation formalisms such as First-Order Logic fragments, Propositional Logic, Horn clauses, equations, constraints, Description Logics or OWL. For these reasons we present a logics- and reasoner-independent black-box approach to sequential diagnosis which is appropriate for all monotonic and decidable knowledge representation languages. This preserves a maximal generality of our approach and makes it broadly applicable across different MBD application domains.

Because the problem of optimal query selection111Also known as Optimal Test Sequencing Problem (Pattipati and Alexandridis, 1990) or

Optimal Decision Tree Problem

(Hyafil and Rivest, 1976). is NP-complete (Hyafil and Rivest, 1976), sequential diagnosis approaches have to bear on a trade-off between query optimality and computational complexity. Therefore, it is current practice to rely on myopic (usually one-step lookahead) methods to guide diagnoses discrimination (de Kleer and Williams, 1987; Feldman et al., 2010; Gonzalez-Sanchez et al., 2011; Shchekotykhin et al., 2012; Rodler et al., 2013). Empirical (de Kleer et al., 1992b; Shchekotykhin et al., 2012; Rodler et al., 2013) and theoretical (Pattipati and Alexandridis, 1990)

evaluations have evidenced that such heuristic methods in many cases deliver reasonable and in some scenarios even (nearly) optimal results. Moreover, query selection based on a multi-step lookahead is computationally prohibitive due to the involved expensive model-based reasoning (cf. Sec. 

5). In common with the above-mentioned approaches we model the query selection heuristic as a query selection measure assigning a real-value to each query based on its quality (regarding diagnoses discrimination). One popular such measure is entropy (de Kleer and Williams, 1987), which favors queries with a maximal expected information gain or, equivalently, a maximal expected reduction of the diagnostic uncertainty. The goal of any such measure is the minimization of the number of queries required until achieving the appointed diagnostic goal.

Whereas sequential diagnosis approaches usually incorporate the optimization of a query selection measure , they often do not optimize the query (answering) cost such as the time required to perform measurements (Heckerman et al., 1995). We model this cost by a query cost measure , a function allocating a real-valued cost to each query. The approach suggested in this work is devised to compute optimized queries along the and axes at each (query selection) step in the sequential diagnosis process while minimizing the required computational resources. More concretely, the contributions of this work are the following:

Contributions. We present a novel query optimization method that is generally applicable to any MBD problem in the sense of (de Kleer and Williams, 1987; Reiter, 1987) and

  1. defines a query as a set of First-Order Logic sentences and thus generalizes the measurement notion of (de Kleer and Williams, 1987; Reiter, 1987),

  2. given a set of leading diagnoses (de Kleer and Williams, 1989), allows the two-dimensional optimization of the next query in terms of the expected number of subsequent queries (measure ) and query cost (measure ),

  3. for an aptly refined (yet exponential) query search space, finds – without any reasoner calls – the globally optimal query w.r.t. measure that globally optimizes measure ,222The term globally optimal has its standard meaning (cf. (Luenberger and Ye, 2015, p. 184)) and emphasizes that the optimum over all queries in the respective query search space is meant.

  4. for the full query search space, finds – with a polynomial number of reasoner calls – the (under reasonable assumptions) globally optimal query w.r.t.  that includes, if possible, only “cost-preferred” sentences (e.g. those answerable using built-in sensors),

  5. guarantees the proposal of queries that discriminate between all leading diagnoses and that unambiguously identify the actual diagnosis.


  1. we show that any MBD problem can be reduced to a Knowledge Base Debugging (KBD) problem (Shchekotykhin et al., 2012; Rodler, 2015). This result establishes a formal relationship between these two paradigms, shows the greater generality of the latter and enables the transferral of findings in the KBD domain to the MBD domain.

In a nutshell, the presented query optimization method can be subdivided into three phases, P1, P2 and P3. In the first place, P1 optimizes the next query’s discrimination properties (e.g. the expected information gain) based on the criteria imposed by the given QSM , realized by a heuristic backtracking search. Then, as a first option, P2 computes an optimal query regarding the given QCM by running a uniform-cost hitting set tree search over a suitable (and explicitly given) set of partial leading diagnoses. This is done in a way meets exactly the optimal discrimination properties determined in P1. P2 explores the largest possible query search space that can be handled without any reasoner calls in a complete way. The output suggests the inspection of the system component(s) that is least expensive for the oracle (QCM ) among all those that yield the highest information (QSM ). As a second option and alternative to P2, P3 performs a two-step optimization consisting of a first generalization of the addressed search space and a subsequent divide-and-conquer exploration of this search space focused on cost-preferred measurements. P3 returns a cost-optimal query (w.r.t. some QCM ) complying with the optimal discrimination properties fixed in P1. may include measurements of arbitrary type, depending on priorly definable requirements.

Roughly, the efficiency of the novel approach is possible by the recognition that the optimizations of and can be decoupled and by using logical monotonicity as well as the inherent (already inferred) information in the (-minimal) leading diagnoses. The latter is leveraged to achieve a retention of costly reasoner calls until the final query computation stage (P3), and hence to reduce them to a minimum. In particular, the method is inexpensive as it

  1. avoids the generation and examination of unnecessary (non-discriminating) or duplicate query candidates,

  2. actually computes only the single

    best query by its ability to estimate a query’s quality without computing it, and

  3. guarantees soundness and completeness w.r.t. an exponential query search space independently of the properties and output of a reasoner.

Modern sequential diagnosis methods like (de Kleer and Williams, 1987) and its derivatives (Feldman et al., 2010; Shchekotykhin et al., 2012; Rodler et al., 2013) do not meet all properties (a) – (c). The black-box approaches among them extensively call a reasoner in order to compute a query. As we show in our evaluations, the presented method can save an exponential overhead compared to these approaches.

Moreover, we emphasize that our approach can also deal with problems where the query space is implicit, i.e. all possible system measurements cannot be enumerated in polynomial time in the size of the system model. E.g., in a digital circuit all measurement points (and hence the possible queries) are given explicitly by the circuit’s wires which can be directly extracted from the system description (sd). In, e.g., knowledge-based problems, by contrast, the possible measurements, i.e. questions to an expert, must be (expensively) inferred and are not efficiently enumerable. In fact, we show that for problems involving implicit queries, approaches not using the proposed theory might be drastically incomplete and hence might miss optimal queries.

Finally, by the generality of our query notion, our method explores a more complex search space than (de Kleer and Williams, 1987; de Kleer and Raiman, 1993), thereby guaranteeing property (5) above.

Organization. The rest of this work is organized as follows. Sec. 2 provides theoretical foundations needed in later sections. In particular, it gives a short introduction on Model-Based Diagnosis (MBD) in Sec. 2.1, on Knowledge Base Debugging (KBD) in Sec. 2.2 and formally proves that each MBD problem can be reduced to a KBD problem in Sec. 2.3. Henceforth, the work focuses w.l.o.g. just on KBD. Basics on Sequential Diagnosis including important definitions, the formal characterization of the addressed problem, and a generic algorithm to solve this problem are treated in Sec. 2.4. The main part of the paper starts with Sec. 3, where we first formalize the measurement selection problem (Sec. 3.1) and then discuss the proposed novel algorithm to solve this problem (Sec. 3.2). The presentation of our method is subdivided into a first part attempting to give the reader a prior intuition, motivation and overview of the later introduced theoretical concepts (Sec. 3.2.1), and three further parts, one dedicated to each phase (P1, P2 and P3) of the new algorithm (Sec. 3.2.2, 3.2.3 and 3.2.5). Besides an extensively exemplified expansion of the relevant theory, each phase description includes a complexity analysis. A formal specification of the computed solution’s properties for P1+P2 is given in Sec. 3.2.4 and for P3 in Sec. 3.2.6. Finally, Sec. 3.2.7 recapitulates the entire approach by means of a detailed example. Sec. 4 includes the description of our experimental evaluations in order to complement the theoretical findings of Sec. 3.2. The experimental settings are explicated in Sec. 4.1, whereas the experimental results are discussed in Sec. 4.2. Subsequently, there is a section on related work (Sec. 5) before we conclude with Sec. 6. Appendix A comprises all proofs that are not given in the text. Appendix B provides a table including all important symbols used in the text along with their meaning.

2 Preliminaries

In this section, we revise the general theory of Model-Based Diagnosis (MBD) proposed by (Reiter, 1987), define the knowledge base debugging framework (KBD) we will use to formalize MBD problems in this work, and demonstrate that KBD is a generalization of MBD.

2.1 Model-Based Diagnosis

We briefly review the classical model-based diagnosis (MBD) problem described by (Reiter, 1987). At first, we characterize a system, e.g. a digital circuit, a car or some software, which is the subject of a diagnosis task:

Definition 1 (System).

A system is a tuple where sd, the system description, is a set of First-Order Logic sentences, and comps, the system components, is a finite set of constants .

The distinguished unary “abnormal” predicate ab is used in sd to model the expected behavior of components . Let us denote the First-Order Logic sentence describing this expected behavior of by and let . The latter subsumes a statement of the form “if is nominal (not abnormal), then its behavior is ” for each system component . Any behavior different from implies that is at fault, i.e.  holds. But, an abnormal component does not necessarily manifest a faulty behavior in each situation (weak fault model (de Kleer et al., 1992a; Feldman et al., 2009)), e.g. for an or-gate stuck at 1 faulty behavior can only be observed if both inputs are 0. Further, sd might include general axioms describing the system domain or descriptions of the interplay between the system components. Let us call the set of these general axioms . So, .

The behavior of a system assuming all components working correctly is captured by the description . Note, this description is equal to .

A diagnosis problem arises when the observed system behavior – represented by a finite set of First-Order Logic sentences obs – differs from the expected system behavior. Formally, this means that . For instance, in circuit diagnosis obs might be the observation of the system inputs and outputs.

There are usually multiple different hypotheses (diagnoses) that explain the discrepancy between observed and predicted system behavior. Discrimination between these hypotheses can then be accomplished by means of additional observations meas called measurements (Reiter, 1987; de Kleer and Williams, 1987). Each measurement in the set of measurements meas is a set of First-Order Logic sentences (Reiter, 1987) describing additional knowledge about the actual system behavior, e.g. whether a particular wire in a faulty circuit is high or low. Usually new measurements are conducted and added to meas until some diagnostic goal is achieved, e.g. the presence of just a single or one highly probable remaining hypothesis. Each added measurement , if chosen properly, will invalidate some hypotheses. Throughout this paper we assume stationary health (Feldman et al., 2010), i.e. that one and the same (faulty) behavior can be constantly reproduced for each during system diagnosis.

Formalized, these notions lead to the definitions of an MBD diagnosis problem instance (MBD-DPI) and of an MBD-diagnosis.

Definition 2 (Mbd-Dpi).

Let obs (system observations) be a finite set of First-Order Logic sentences, meas (measurements) be a finite set including finite sets of First-Order Logic sentences, and let be a system. Then the tuple is an MBD diagnosis problem instance (MBD-DPI).

Definition 3.

Let be an MBD-DPI and denote the union of all . Then for denotes the behavior description of the system

  • [noitemsep,topsep=5pt]

  • under the current state of knowledge given by the in terms of obs and meas, and

  • under the assumption that all components in are faulty and all components in are healthy.

Definition 4 (MBD-Diagnosis).

Let be an MBD-DPI. Then is an MBD-diagnosis for iff is consistent ( explains obs and meas). An MBD-diagnosis for is called minimal iff there is no MBD-diagnosis for such that .

In many practical applications there are multiple (minimal) MBD-diagnoses for a given MBD-DPI. Without additional information about the system, one cannot conjecture a unique diagnosis. The idea is then to perform measurements in order to discriminate between competing (minimal) MBD-diagnoses until a sufficient degree of diagnostic certainty (the specified diagnostic goal ) is reached. This is the problem addressed by Sequential MBD and can be stated as follows:

Problem 1 (Sequential MBD).


Given: An MBD-DPI , and a diagnostic goal .

Find: and , where is a set of new measurements such that is a minimal MBD-diagnosis for the MBD-DPI , and satisfies .

Remark 1  Due to the intractability of the computation of the entire set of minimal diagnoses (Bylander et al., 1991), both the measurement selection and the decision whether a diagnostic goal is satisfied for some diagnosis is usually made by using a (computationally feasible) set of leading minimal diagnoses (de Kleer and Williams, 1989). acts as an approximation of all minimal diagnoses for the given DPI and usually comprises the most probable minimal (de Kleer and Williams, 1989) or minimum-cardinality (Feldman et al., 2010) diagnoses for a DPI. Given a set of leading minimal diagnoses for , examples for the specification of are is the only minimal diagnosis for (de Kleer and Raiman, 1993), exceeds some predefined probability threshold ”, e.g.  (de Kleer and Williams, 1987; Shchekotykhin et al., 2012) or has times the probability of all other elements in ”. Note that the goal represents a maximally strict requirement on the final diagnostic result as it requires the verification of the invalidity of all but the correct minimal diagnosis (we call a diagnostic goal more strict than a diagnostic goal if is satisfied earlier in any diagnostic session than ). The specification of (constants in) depends on the seriousness of misdiagnosis, e.g. higher probability thresholds signify higher criticality.∎

In general, the size of the search space for minimal MBD-diagnoses for , is in . A useful concept to restrict this search space is the one of an MBD-conflict (Reiter, 1987; de Kleer and Williams, 1987), a set of components whose elements cannot all be healthy given obs and meas:

Definition 5 (MBD-Conflict).

Let be an MBD-DPI. Then is an MBD-conflict for iff is inconsistent. An MBD-conflict for is called minimal iff there is no MBD-conflict for such that .

Definition 6 (Hitting Set).

Let be a collection of sets. Then is called a hitting set of iff and for all . A hitting set of is minimal iff there is no hitting set of such that .

The following result (Reiter, 1987) can be used to determine MBD-diagnoses through the computation of MBD-conflicts:

Theorem 1.

A (minimal) MBD-diagnosis for a DPI is a (minimal) hitting set of all minimal MBD-conflicts for this DPI.

circuit inputs (from top to bottom) 1 0 1 circuit outputs (from top to bottom) 1 0
Figure 2: MBD Example due to (Reiter, 1987) from the domain of circuit diagnosis.

Example 1  Let us revisit the circuit diagnosis example given in (Reiter, 1987) shown in Fig. 2. The first step towards diagnosing the circuit using MBD is to formulate the problem as an MBD-DPI. The result is given by Tab. 2 and explained next.

The circuit, i.e. the system to be diagnosed, includes five gates (xor-gates), (and-gates) and (or-gate), which are at the same time the system components comps of interest. The system description consists of a knowledge base describing the behavior of each gate given it is working properly, e.g. for gate , includes the sentence . Besides, sd includes a knowledge base describing which gate-terminals are connected by wires, e.g. the wire connecting to is defined by the sentence . For simplicity we omit the explicit statement of additional general domain knowledge in such as axioms for Boolean algebra or axioms restricting wires to only either or values. The observations are given by the system inputs and outputs (see the table in Fig. 2). Finally, since there are no already performed measurements, the set meas is empty.

Assuming all components are healthy, i.e. all gates function properly, we find out that is inconsistent (cf. Def. 3). That is, the assumption of no faulty components conflicts with the observations obs made. E.g., if and manifest nominal behavior, we can deduce that the output which contradicts the observation sentence . Supposing either of the components , to be nominal, we can no longer deduce (or any other sentence contradicting obs). Therefore, is a minimal MBD-conflict (cf. Def. 5). Similarly, we find that is the only other minimal MBD-conflict for . Computing minimal hitting sets of all minimal MBD-conflicts , we obtain three minimal MBD-diagnoses , and .

Let the diagnostic goal be the achievement of complete diagnostic certainty, i.e. to single out the correct minimal MBD-diagnosis. The goal of the MBD-problem is then to find new measurements such that there is a single minimal diagnosis for , . Let the first measurement be the observation of the terminal , and let the value of it be . Then, is still a minimal MBD-diagnosis for , since the abnormality of explains both obs and meas. Moreover, all other MBD-diagnoses for must contain (since its faultiness is the only explanation for meas) and thus be supersets of . Hence, is the only minimal MBD-diagnosis for and thus the actually faulty component in this scenario is (under the assumption that a -minimal set of components is broken). This fact could be derived by conducting only one measurement.∎

obs 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 comps for meas
Table 1: MBD-DPI obtained from circuit diagnosis problem in Fig. 2.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 consistency min KBD-conflicts min KBD-diagnoses
Table 2: KBD-DPI obtained from MBD-DPI from Tab. 2.

2.2 Knowledge Base Debugging

In this section we revisit the knowledge base debugging (KBD) problem (Friedrich and Shchekotykhin, 2005; Shchekotykhin et al., 2012; Rodler, 2015) which we will use subsequently as a generalized reformulation of Reiter’s original MBD problem described above. Besides offering some notational conveniences, KBD allows users to specify negative measurements (or test cases) (Felfernig et al., 2004a). Contrary to (positive) measurements as characterized above, negative measurements state properties that must not hold. In other words, any diagnosis must fulfill that – under its assumption – the system description together with the observations and positive measurements does not entail any negative measurement. Additionally, it is possible in KBD to postulate stronger logical properties apart from consistency. For example, when debugging an ontology (i.e. a system where comps are ontology axioms) one might want the assumption of a diagnosis to yield a coherent (Schlobach et al., 2007; Parsia et al., 2005)

system description (repaired ontology), i.e. one without unsatisfiable classes. In First-Order Logic terms (using logic programming notation), an

unsatisfiable class in a KB is an -ary predicate such that where . That is, coherency means that every predicate in can have some instance without yielding an inconsistency.

Another possible use case for the adoption of (logical) requirements such as coherency is the fault localization in flawed (e.g. inconsistent) system models used for MBD. For instance, a model (which is itself a KB) used to describe the circuit in Fig. 2 might include an unsatisfiable class (which essentially makes the model inconsistent after the creation of, e.g., the sentence declaring as an xor-gate). The reason for this incoherency might be that includes the sentences and (where the system modeler forgot to include ) as well as sentences stating that no instance can be of more than one type of gate. That is, KBD (with the coherency requirement) could be used in such scenario to repair the model thus enabling a sound diagnostic process.

2.2.1 The Used Notation

Let denote some formal knowledge representation language. We will call logical sentences over and a set of logical sentences a knowledge base (KB) over . Sentences in will sometimes be referred to as axioms. We denote by the semantic entailment relation for the logic and we write to state that is a logical consequence of the KB . For brevity, we will write for two KBs and to denote that for all and to state that for some .

Given a collection of sets , we use and to denote the union and intersection, respectively, of all elements in . Further, Tab. 7 (see Appendix B) summarizes the meaning of other formalisms used in the paper (many of them introduced at some later point).

2.2.2 Assumptions

The KBD techniques described in this work are applicable to any knowledge representation formalism which is Tarskian, i.e. for which the semantic entailment relation is monotonic, idempotent and extensive (Tarski, 1983; Ribeiro, 2012) and for which reasoning procedures for deciding consistency of a KB over are available.

Definition 7.

The relation is called

  • [noitemsep, topsep=5pt]

  • monotonic iff whenever then
    (i.e. adding new sentences to a KB cannot invalidate any entailments of the KB)

  • idempotent iff and implies
    (i.e. adding entailed sentences to a KB does not yield new entailments of the KB)

  • extensive iff for all
    (i.e. each KB entails all sentences it comprises).

In the following, “sentence” will always mean “logical sentence”. We will omit the index for brevity when referring to sentences or KBs, tacitly assuming that any sentence or KB we speak of is formulated over some (fixed) language where meets the conditions given above.

Examples of logics that comply with these requirements include, but are not restricted to Propositional Logic, Datalog (Ceri et al., 1989), (decidable fragments of) First-Order Predicate Logic, The Web Ontology Language (OWL (Patel-Schneider et al., 2004), OWL 2 (Grau et al., 2008; Motik et al., 2009)), sublanguages thereof such as the OWL 2 EL Profile (with polynomial time reasoning complexity (Kazakov et al., 2014)), Boolean or linear equations and various Description Logics (Baader et al., 2007) and constraint languages.

2.2.3 Definitions and Properties

We next state the KBD problem and give some important definitions and properties (discussed in detail in (Rodler, 2015)).

The inputs to a KB debugging problem can be characterized as follows: Given is a KB to be repaired and a KB (background knowledge). All sentences in are considered correct and all sentences in are considered potentially faulty. does not meet postulated requirements (where consistency is a least requirement333We assume consistency a minimal requirement to a solution KB provided by a debugging system, as inconsistency makes a KB completely useless from the semantic point of view.) or does not feature desired semantic properties, called test cases. Positive test cases (aggregated in the set ) correspond to necessary entailments and negative test cases (aggregated in the set ) represent necessary non-entailments of the correct (repaired) KB (together with the background KB ). Each test case and is a set of sentences. The meaning of a positive test case is that the union of the repaired KB and must entail each sentence (or the conjunction of sentences) in , whereas a negative test case signalizes that some sentence (or the conjunction of sentences) in must not be entailed by this union.

The described inputs to the KB debugging problem are captured by the notion of a KBD diagnosis problem instance (KBD-DPI):

Definition 8 (Kbd-Dpi).


  • [noitemsep,topsep=5pt]

  • be a KB,

  • be sets including sets of sentences,

  • be a set of (logical) requirements,

  • be a KB such that and satisfies all requirements , and

  • the cardinality of all sets , , , be finite.

Then we call the tuple a KBD diagnosis problem instance (KBD-DPI).

1 2 3 4 5 6 7 8 9 consistency
Table 3: Running example KBD-DPI over Propositional Logic.
min KBD-conflict explanation min KBD-diagnosis explanation Theorem 3 Theorem 3 Theorem 3 Theorem 3 Theorem 3 Theorem 3
Table 4: Minimal KBD-conflicts and KBD-diagnoses for the KBD-DPI in Tab. 4.

Example 2  An example of a Propositional Logic KBD-DPI is depicted by Tab. 4. will serve as a running example throughout this paper. It includes a KB with seven axioms , a background KB with two axioms , one singleton positive test case and three singleton negative test cases . There is one requirement in imposed on the correct (repaired) KB. It is easy to verify that the standalone KB is consistent, i.e. satisfies all , and that . Hence, indeed constitutes a KBD-DPI as per Def. 8. ∎

A solution (KB) for a DPI is characterized as follows:

Definition 9 (Solution KB).

Let be a KBD-DPI. Then a KB is called solution KB w.r.t.  iff all the following conditions hold:


A solution KB w.r.t.  is called maximal iff there is no solution KB w.r.t.  such that (i.e.  has a set-maximal intersection with among all solution KBs).

Usually, observing the Principle of Parsimony (Reiter, 1987), maximal solution KBs will be preferred to non-maximal ones since they result from the input KB through the modification of a minimal set of axioms.

Example 3  For the KBD-DPI given by Tab. 4, is not a solution KB w.r.t.  since, e.g.  clearly which is a positive test case and therefore has to be entailed. Another reason why is not a solution KB w.r.t.  is that , which is a negative test case and hence must not be an entailment. This is straightforward since imply , and and thus clearly .

On the other hand, is clearly a solution KB w.r.t.  as is obviously consistent (satisfies all ), does entail and does not entail any . However, is not a maximal solution KB since, e.g.  can be added to without resulting in the violation of any of the Equations (1) – (3). Note that also e.g.  for arbitrary finite is a solution KB, albeit not a maximal one, although it has no axioms in common with and includes an arbitrary number of axioms not occurring in . However, to maintain a maximum amount of the knowledge specified in the KB of interest, one will usually prefer minimally invasive modifications (i.e. maximal solution KBs) while repairing faults in .

Maximal solution KBs w.r.t. the given DPI are, e.g.  (resulting from the deletion of from and the addition of ) or (resulting from the deletion of from and the addition of ). That these KBs constitute solution KBs can be verified by checking the three conditions named by Def. 9. Indeed, adding an additional axiom in to any of the two KBs leads to the entailment of a negative test case . That is, no solution KB can contain a proper superset of the axioms from that are contained in any of the two solution KBs and . Hence, both are maximal.∎

Remark 2  There are generally infinitely many (maximal) solution KBs resulting from the deletion of one and the same set of axioms from the original KB . This stems from the fact that there are infinitely many (semantically equivalent) syntactical variants of any set of suitable sentences that can be added to in order for Eq. (2) to be satisfied. One reason for this is that there are infinitely many tautologies that might be included in these sentences, another reason is that sentences can be equivalently rewritten, e.g. .∎

In terms of our running example, this circumstance can be illustrated as follows:

Example 4  Consider again in Tab. 4 and assume is deleted from . Then one solution KB constructible from is given in the last example. To determine the maximal solution KB from , the most straightforward way of adding just all sentences occurring in positive test cases in has been chosen in this case. Other maximal solution KBs obtainable from adding sentences to are, e.g.  (which differs syntactically, but not semantically from ) and (which differs both syntactically and semantically from yielding the entailment which is not implied by ).∎

Despite generally multiple semantically different solution KBs, the diagnostic evidence of a DPI in terms of positive test cases does not justify the inclusion of sentences (semantically) different from (cf. (Friedrich and Shchekotykhin, 2005; Shchekotykhin et al., 2012)). Since we are moreover interested in only one instance of a solution KB resulting from for each , we define as the canonical solution KB for w.r.t.  iff is a solution KB w.r.t. .

A KBD-diagnosis is defined in terms of the axioms that must be deleted from the KB of a DPI in order to construct a solution KB w.r.t. this DPI. In particular, the deletion of from targets the fulfillment of Equations (1) and (3) such that can be added to the resulting modified KB without introducing any new violations of (1) or (3).

Definition 10 (KBD-Diagnosis).

Let be a KBD-DPI. A set of sentences is called a KBD-diagnosis w.r.t.  iff is a solution KB w.r.t.  (i.e.  meets Equations (1) – (3)). A KBD-diagnosis w.r.t.  is

  • [noitemsep,topsep=5pt]

  • minimal iff there is no such that is a KBD-diagnosis w.r.t. 

  • a minimum cardinality KBD-diagnosis w.r.t.  iff there is no KBD-diagnosis w.r.t.  such that .

We will write to state that is a KBD-diagnosis w.r.t.  and to state that is a minimal KBD-diagnosis w.r.t. .

Remark 3  Since trivially satisfies (2) due to the inclusion of , is a KBD-diagnosis w.r.t.  iff satisfies (1) and (3).∎

The next theorem captures the relationship between maximal canonical solution KBs and minimal KBD-diagnoses w.r.t. a DPI. In fact, it tells us that we can concentrate only on the computation of minimal KBD-diagnoses in order to find all maximal canonical solution KBs.

Theorem 2.

Let be a KBD-DPI. Then the set of all maximal canonical solution KBs w.r.t.  is given by .

In a completely analogous way as MBD-conflicts provide an effective mechanism for focusing the search for MBD-diagnoses, we can exploit KBD-conflicts for KBD-diagnoses calculation. Simply put, a (minimal) KBD-conflict is a (minimal) per se faulty subset of the original KB , i.e. one source causing the faultiness of in the context of . For a KBD-conflict there is no extension that yields a solution KB. Instead, such an extension is only possible after deleting appropriate axioms from the KBD-conflict.

Definition 11 (KBD-Conflict).

Let be a KBD-DPI. A set of formulas is called a KBD-conflict w.r.t.  iff is not a solution KB w.r.t.  (i.e.  violates at least one of the Equations (1) – (3)). A KBD-conflict w.r.t.  is minimal iff there is no such that is a KBD-conflict w.r.t. .

Theorem 3.

(Friedrich and Shchekotykhin, 2005, Prop. 2) Let be a KBD-DPI. Then a (minimal) KBD-diagnosis w.r.t.  is a (minimal) hitting set of all minimal conflicts w.r.t. .

Proposition 1.

(Rodler, 2015, Prop. 3.4) Let be a KBD-DPI. Then a KBD-diagnosis w.r.t.  exists iff satisfies all and for all .

Example 5  Tab. 4 gives a list of all minimal KBD-conflicts w.r.t. our running example . Let us briefly reflect why these are KBD-conflicts (cf. third col. of Tab. 4). Recall Ex. 9, where we explained why is a KBD-conflict (violation of ). is minimal since, first, it is consistent, i.e. satisfies all , and does not entail any of the negative test cases . So, by logical monotonicity no proper subset of can violate , or . Second, the elimination of any axiom from