# Conditions for Hierarchical Supervisory Control under Partial Observation

The fundamental problem in hierarchical supervisory control under partial observation is to find conditions preserving observability between the original (low-level) and the abstracted (high-level) plants. Two conditions for observable specifications were identified in the literature – observation consistency (OC) and local observation consistency (LOC). However, the decidability of OC and LOC were left open. We show that both OC and LOC are decidable for regular systems. We further show that these conditions do not guarantee that supremal (normal or relatively observable) sublanguages computed on the low level and on the high level always coincide. To solve the issue, we suggest a new condition – modified observation consistency – and show that under this condition, the supremal normal sublanguages are preserved between the levels, while the supremal relatively observable high-level sublanguage is at least as good as the supremal relatively observable low-level sublanguage, i.e., the high-level solution may be even better than the low-level solution.

## Authors

• 2 publications
• 4 publications
• ### Hierarchical visuomotor control of humanoids

We aim to build complex humanoid agents that integrate perception, motor...
11/23/2018 ∙ by Josh Merel, et al. ∙ 0

• ### Reinforcement Control with Hierarchical Backpropagated Adaptive Critics

Present incremental learning methods are limited in the ability to achie...
12/08/2015 ∙ by John W. Jameson, et al. ∙ 0

• ### Supervisory Control of Probabilistic Discrete Event Systems under Partial Observation

The supervisory control of probabilistic discrete event systems (PDESs) ...
05/17/2018 ∙ by Weilin Deng, et al. ∙ 0

• ### Plan-Recognition-Driven Attention Modeling for Visual Recognition

Human visual recognition of activities or external agents involves an in...
12/02/2018 ∙ by Yantian Zha, et al. ∙ 0

• ### Automatic Inference of High-Level Network Intents by Mining Forwarding Patterns

There is a semantic gap between the high-level intents of network operat...
02/06/2020 ∙ by Ali Kheradmand, et al. ∙ 0

• ### Abstracting Probabilistic Relational Models

Abstraction is a powerful idea widely used in science, to model, reason ...
10/04/2018 ∙ by Vaishak Belle, et al. ∙ 0

• ### High-low level support vector regression prediction approach (HL-SVR) for data modeling with input parameters of unequal sample sizes

Support vector regression (SVR) has been widely used to reduce the high ...
05/31/2019 ∙ by Maolin Shi, et al. ∙ 0

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Organizing systems into hierarchical structures is a common engineering practice used in manufacturing, robotics, or artificial intelligence to overcome the combinatorial state explosion problem. Hierarchical supervisory control of discrete-event systems (DES) was introduced by

ZhongW1990 as a two-level vertical decomposition of the system. The low-level plant modeling the system behavior is restricted by a high-level specification, and the aim is to synthesize a nonblocking and optimal supervisor based on the high-level abstraction of the plant in such a way that it can be used for a low-level implementation. They identified a sufficient condition to achieve the goal. ZhongW1990b extended the framework to hierarchical coordination control and developed an abstract hierarchical supervisory control theory. WongW96a applied the theory to the Brandin-Wonham framework of timed DES. KS extended hierarchical supervisory control to decentralized systems, and SB11 found weaker sufficient conditions for maximal permissiveness of high-level supervisors with complete observations. Recently, BaierM15 generalized hierarchical supervisory control to the Büchi framework, where the plant and the specification are represented by -languages.

Motivated by abstractions of hybrid systems to DES, HubbardC02 developed a hierarchical control theory for DES based on state aggregation, and TorricoC2002 investigated a hierarchical control approach where the low level is in the Ramadge-Wonham framework and the high level is obtained by state aggregation. Here, the high-level events are subsets of low-level events, and advanced control structures are used to synthesize a controller. Furthermore, CunhaC07 proposed hierarchical supervisory control for DES where the low level is in the Ramadge-Wonham framework and the high level is represented by systems with flexible marking, in order to simplify the modeling of the high level. NgoS14; NgoS18 investigated hierarchical control for Moore automata and for timed DES, and SakakibaraU2018 considered concurrent DES modeled by Mealy automata.

fekri2009 first considered hierarchical supervisory control of partially observed DES. They used Moore automata models and defined controllable and observable events based on vocalization. Hence, they need a specific definition of the low-level supervisor. Furthermore, their approach is monolithic, while ours allows distributed synthesis using the standard synchronous composition of the plant with the supervisor.

In this paper, we adapt the classical hierarchical supervisory control of DES in the Ramadge-Wonham framework, where the systems are modeled as DFAs and the abstraction is modeled as a natural projection, i.e., the behavior of the high-level plant is the projection of the behavior of the low-level plant to the high-level alphabet. The problem is then as follows. Given a low-level plant over an alphabet modeling the system behavior and a high-level specification language over a high-level alphabet . The low-level plant is abstracted to the high-level plant describing the high-level behavior. The aim is to synthesize a nonblocking and optimal supervisor on the high level in such a way that it can be used for a construction of a low-level supervisor that is nonblocking and optimal wrt the specification .

To achieve the goal for fully observed DES, important concepts have been developed in the literature, including the observer property of WW96, output control consistency (OCC) of ZhongW1990, and local control consistency (LCC) of SB11. These concepts are sufficient for the high-level synthesis of a nonblocking and optimal supervisor to have a low-level implementation.

However, the conditions are not sufficient for partially observed DES. The sufficient condition of KM10 requires that all observable events must be high-level events, which is a very restrictive assumption. Therefore, cdc-ecc2011 investigated weaker and less restrictive conditions, and introduced two concepts – local observation consistency (LOC) and observation consistency (OC). The latter ensures a certain consistency between observations on the high level and the low level, and the former is an extension of the observer property to partial observation. The paper shows that, for observable specifications, projections that satisfy OC, LOC, LCC, and that are observers are suitable for the nonblocking least restrictive hierarchical supervisory control under partial observation. The fundamental question whether the properties of OC and LOC are decidable is left open.

In this paper, we first show that checking OC and LOC properties is decidable for systems with regular behaviors and that the problems are actually PSpace-complete (Theorems 4 and 5).

Then we show that OC and LOC are not sufficient to preserve optimality for non-observable specifications. These are specifications, for which a suitable supremal sublanguage (normal or relatively observable) needs to be computed. We show that OC and LOC do not guarantee that the supremal normal (relatively observable) low-level sublanguage coincides with the composition of the plant and the supremal normal (relatively observable) high-level sublanguage (Example 6).

For normality, we suggest a condition of modified observation consistency (MOC) and show that it preserves optimality, i.e., the supremal normal sublanguages are preserved between the levels (Definition 6 and Theorem 6.1). Then we discuss two special cases often considered in the literature: (i) the case where all observable events are also high-level events, and (ii) the case where all high-level events are also observable. Our new results generalize the previously known results.

For relative observability, we show that MOC ensures that the high-level solution is at least as good as the low-level solution (Theorem 6.2). In particular, the low-level implementation of the high-level solution may be better than what we can obtain directly on the low level (Example 6.2). This observation makes relative observability an interesting and suitable notion for hierarchical supervisory control.

Finally, the newly suggested condition of MOC is stronger than OC of cdc-ecc2011 as shown in Lemma 6. Moreover, similarly as OC, the MOC condition is structural only wrt the plant. We discuss the complexity of MOC in Theorem 6.2, and show that it is compositional in Theorem 7.

All the missing proofs can be found in the appendix.

## 2 Preliminaries and Definitions

We assume that the reader is familiar with the basics of supervisory control, see CL08. For a set , denotes the cardinality of . For an alphabet (finite nonempty set) , denotes the set of all finite strings over ; the empty string is denoted by . The alphabet is partitioned into controllable events and uncontrollable events as well as into observable events and unobservable events . A language is a subset of . For a language , the prefix closure ; is prefix-closed if .

A (natural) projection , where are alphabets, is a homomorphism for concatenation defined so that for , and for . The action of on is to remove all events from that are not in . The inverse image of under is the set . These definitions can naturally be extended to languages.

A nondeterministic finite automaton (NFA) is a quintuple , where is a finite set of states, is an input alphabet, is a set of initial states, is a set of marked states, and is the transition function that can be extended to the domain in the usual way. The automaton is deterministic (DFA) if , and for every state and every event . The language generated by is the set , and the language marked by is the set . By definition, , and is prefix-closed. If , then is nonblocking.

Let , be languages. The parallel composition of and is the language , where is a projection, for ; see CL08 for a definition for automata. For two DFAs and , . Languages and are synchronously nonconflicting if .

Let be a DFA over an alphabet . A language is controllable wrt and the set of uncontrollable events if ; is observable wrt , the set of observable events with being the corresponding projection, and the set of controllable events if for all with and for every , if , , and , then . Algorithms to verify controllability and observability can be found in CL08.

It is known that there is no supremal observable sublanguage. Therefore, stronger properties, such as normality of LinWon88 or relative observability of CaiZW15, are used for specifications that are not observable. Language is normal wrt and the projection if . Relative observability has recently been introduced by CaiZW15 and further studied by AlvesCB17 as a condition weaker than normality and stronger than observability. Let be languages. Language is relatively observable wrt , , and (or simply -observable) if for all strings with and for every , whenever , , and , then . For , the definition coincides with observability.

A decision problem is a yes-no question. A decision problem is decidable

if there exists an algorithm that solves the problem. Complexity theory classifies decidable problems to classes based on the time or space an algorithm needs to solve the problem. The complexity class we consider in this paper is

PSpace, denoting all problems solvable by a deterministic polynomial-space algorithm. A decision problem is PSpace-complete if the problem belongs to PSpace (membership) and every problem from PSpace can be reduced to the problem by a polynomial-time algorithm (hardness). It is unknown whether PSpace-complete problems can be solved in polynomial time.

## 3 Principles of Hierarchical Control

In the sequel, we use the following notation for projections and abstractions, see the commutative diagram in Fig. 1. Let be the low-level alphabet, the high-level alphabet, and the set of observable events. Let be the projection corresponding to system’s partial observation, the projection corresponding to the high-level abstraction, and and the corresponding observations and abstractions.

We now state the hierarchical supervisory control problem for partially observed DES.

Let be a low-level plant over an alphabet , and let be a high-level specification over an alphabet . The abstracted high-level plant is defined over the alphabet so that and . The aim of hierarchical supervisory control is to determine, based on the high-level plant and the specification , without using the low-level plant , a nonblocking low-level supervisor such that .

cdc-ecc2011 identified sufficient conditions (observation consistency and local observation consistency) on the low-level plant for which observability of wrt is equivalent to observability of wrt the high-level plant .

A prefix-closed language is observation consistent (OC) wrt projections , , and if for all strings such that , there are such that , , and . Intuitively, any two strings of the high-level plant with the same observation have corresponding strings with the same observation in the low-level plant.

A prefix-closed language is locally observation consistent (LOC) wrt projections and and the set of controllable events if for all strings and all events such that and , there exist low-level strings such that and . Intuitively, continuing two observationally equivalent high-level strings by the same controllable event, the corresponding low-level observationally equivalent strings can be continued by this same event in the original plant in the future (after possible empty low-level strings with the same observations). LOC can be seen as a specialization of the observer property and LCC for partially observed DES.

Besides observability, Problem 3 further requires the preservation of controllability between the levels. It has been previously achieved by the conditions of -observer of WW96 and output control consistency of ZhongW1990, or its weaker variant, local control consistency of SB11. Formally, projection is an -observer for a nonblocking plant over if for all strings and , if is a prefix of , then there exists such that and . We say that is locally control consistent (LCC) for a string if for all such that , either there is no such that or there is such that . We call LCC for a language if is LCC for every .

Notice that the conditions are structural and hold for any specification once the plant is fixed. The following result formulates a solution to Problem 3.

[cdc-ecc2011] Let be a nonblocking DFA over , and let be a (high-level) specification. Let be LCC for and , and an -observer. Let be OC wrt , , and , and LOC wrt , , and . Then is controllable wrt and , and observable wrt , , and if and only if is controllable wrt and , and observable wrt , , and .

Theorem 1 allows to verify the existence of a supervisor realizing a high-level specification for a given system , under the aforementioned properties, based on the abstraction . Namely, if there is a nonblocking supervisor such that , then there is a nonblocking supervisor such that . In particular, a DFA realization of such that can be used to implement the supervisor in the form .

Considering only observability, the following results hold. [cdc-ecc2011] Let be a nonblocking DFA over , and let be a specification. Assume that is OC wrt , , and , that and are synchronously nonconflicting, and that is LOC wrt , , and . Then is observable wrt , , and if and only if is observable wrt , , and .

If all controllable events are observable, observability is equivalent to normality, and OC is sufficient to preserve observability.

[cdc-ecc2011] Let be a nonblocking DFA, and let be a specification. If is OC wrt , , and , and and are synchronously nonconflicting, then is normal wrt and if and only if is normal wrt and .

We now show that a result similar to Theorem 1 does not hold for relative observability without additional assumptions; namely, if is -observable, then is not necessarily -observable. Let , over , and over be prefix-closed languages, and hence synchronously nonconflicting. Let . It can be verified that is OC and LOC, and that is -observable wrt , and hence observable. However, is not -observable, since , , and , but (but is observable by Theorem 1).

## 4 Verification of Observation Consistency

In this section, we show that the verification of OC is PSpace-complete, and hence decidable, for systems modeled by finite automata. The same problem for LOC is treated in the next section.

Verifying OC for systems modeled by NFAs is PSpace-complete. To prove membership in PSpace, we generalize the parallel composition to a set of synchronizing events. Let be an alphabet, and let be languages of NFAs and , respectively. Let be a set of synchronizing events. The parallel composition of and synchronized on the events of is denoted by and defined as the language of the NFA

 G1∥Σ′G2=(Q1×Q2,(Σ∪{ε})×(Σ∪{ε}),δ,I1×I2,F1×F2),

where the alphabet is a set of pairs based on the synchronization of events in . There are two categories of pairs to construct, corresponding to (a) events in , and (b) events in . For every , we have the pair , and for every , we have two pairs and . The transition function is defined on these event pairs as follows:

• for , ;

• for , and ;

• undefined otherwise.

For simplicity, a sequence of event pairs, , is written as a pair of the concatenated components . Then we can say that the language consists of pairs of strings of the form , where and coincide on the letters of , that is, for the projection .

Let be a prefix-closed language, and let and be the respective observation and high-level alphabets. We show that is OC wrt , , and if and only if

 Q(L)∥Σhi∩ΣoQ(L)⊆Q(L∥ΣoL),

where, for an event , . Membership in PSpace then follows, since we can express , as well as , as NFAs, and the inclusion of two NFAs can be verified in PSpace, see ClementeM19.

The intuition behind the equivalence is to couple all strings with the same high-level observations, which are exactly the pairs , and to verify that for every such pair there are strings with the same observations, which are exactly the pairs , that are abstracted to the pair , that is, they satisfy .

The rest of the proof can be found in the appendix.

By a slight modification of the proof, it can be shown that the problem is not easier for DFAs, that is, it remains PSpace-hard even for DFA models. We leave this proof for the full version.

## 5 Verification of Local Observation Consistency

In this section, we study decidability and complexity of LOC. As in the case of OC, the problem is not easier for DFA models. The proof is again left for the full version. A proof sketch of the following theorem can be found in the appendix.

Verification of LOC for systems modeled by NFAs is PSpace-complete.

## 6 Preservation of Supremality

Problem 3 requires that the specification language is achievable by the supervisor, i.e., is observable. However, this is not always the case. If is not observable, a common approach is to find a suitable sublanguage of that is observable. Since there is no supremal observable sublanguage, the supremal normal sublanguage or the supremal relatively observable sublanguage is computed instead. The problem is now formulated as follows.

Given a low-level plant over and a high-level specification over . The abstracted high-level plant over is defined so that and . The aim is to determine a maximally permissive nonblocking supervisor such that using the abstraction . That is, if a maximally permissive nonblocking supervisor exists for the abstracted plant such that , then a maximally permissive nonblocking supervisor exists such that .

Compared to Corollary 1 saying that under the OC condition the specification is normal if and only if is normal, the following example shows that OC is not sufficient to preserver normality (relative observability) if the supremal normal (relatively observable) sublanguage of the specification is a strict sublanguage of . The problem is that it is not true that every supremal normal (relatively observable) sublanguage of is of the form for some convenient language , and hence there may be no that would be the supremal normal sublanguage of .

Before stating the example, we introduce the following notation. For a prefix-closed language and a specification , we write (resp. ) to denote the supremal normal (resp. the supremal relatively observable) sublanguage of wrt and the corresponding set of observable events.

Let with and , and let To show that is OC, notice that and , and hence we have two cases: (i) and , and (ii) and . Case (i) is trivial because we can choose and , which clearly satisfies OC. For case (ii), we choose and . Then, , , and . Thus, is OC.

To compute the supremal normal sublanguages, we use the formula of brandt stating that , for prefix-closed languages , and we obtain the following: , , and . This gives that On the other hand, , , and , which gives that showing that OC is not a sufficient condition to preserve supremal normal sublanguages.

Inspecting further the example, the reader may verify that the computed supremal normal sublanguages coincide with the supremal relatively observable sublanguages for the choice of . Therefore, the example also illustrates that OC is neither a sufficient condition to preserve supremal relatively observable sublanguages.

To preserver the properties for supremal sublanguages, we modify the condition of OC by fixing one of the components.

A prefix-closed language is modified observation consistent (MOC) wrt projections , , and if for every and every such that , there exists such that and .

MOC is a stronger property than OC. Indeed, if is MOC, then for any with , we have that for some , and hence there exists such that and , which shows that is OC. This proves the following observation. MOC implies OC.

### 6.1 Normality

We now show that MOC guarantees the preservation of normality for supremal sublanguages.

Let be a nonblocking DFA, and let be a specification. If is MOC wrt , , and , and and are synchronously nonconflicting, then

 supN(K∥Lm(G),L(G))=supN(K,Q(L(G)))∥Lm(G).

(): Since is normal wrt and , Corollary 1 implies that is normal wrt and . The implication that normality of implies normality of in Corollary 1 holds without any assumptions. Therefore, .

(): Let be normal wrt and , that is, . Then, . We show that is normal wrt and , i.e., that . To do this, let and be such that , that is, . We show that . By MOC, there exists such that and , i.e., , and hence , which shows normality of .

Two special cases are often considered in the literature: (i) , and (ii) . We show that both imply MOC, and hence OC. Consequently, Theorem 6.1 strengthens the result of KM10 showing that for any prefix-closed languages and , if , then .

First, assume that . Then , since is an identity. Let and be such that . Consider any with ; such exists because . Then, , which was to be shown.

Second, assume that . Then, is an identity, and hence for any and satisfying , we have , i.e., we can chose in the definition of MOC.

### 6.2 Relative Observability

We now show that an analogy of Theorem 6.1 does not hold for relative observability. In particular, the inclusion

 supRO(K∥Lm(G),L(G))⊇supRO(K,Q(L(G)))∥Lm(G)

does not hold in general as shown in the following example.

Let the low-level plant and the high-level specification be defined by automata in Fig. 2.

Let and . Then is shown in Fig. 3 as well as . There, the reader can also see the supremal relatively observable sublanguage of wrt , , and , which obviously does not include .

By Theorem 1, is always observable. It is thus an interesting question under which conditions the opposite inclusion holds. In other words, under which conditions is the low-level implementation of the high-level supervisor at least as good as the low-level supervisor? We now show that MOC is such a condition.

Let be a nonblocking DFA over and a specification. If is MOC wrt , , and , and and are synchronously nonconflicting, then

 supRO(K∥Lm(G),L(G))⊆supRO(K,Q(L(G)))∥Lm(G).

Let . Since , . We now show that is relatively observable wrt , , and . To this end, let be such that , and let be such that , , and . We have to show that . To this aim, let be such that . Since and , MOC implies that there is such that and . Then for some . Since , we have that and . From and the synchronous nonconflictingness of and , we conclude that . Altogether, , , , and . Then, relative observability of wrt , , and implies that . Hence, .

Notice that the plant in Example 6.2 does not satisfy MOC, and hence MOC is not a necessary condition in Theorem 6.2.

A proof of the following result can be found in the appendix. Verifying MOC for NFAs is PSpace-complete.

Similarly as for OC, the verification of MOC is not easier for DFA models. We provide a proof of PSpace-hardness for DFAs in the full version.

## 7 Modularity

Let be a modular DES. For simplicity, we write to denote and . Similarly for and .

In addition to the high-level alphabet and the set of observable events , we have the local alphabets , . The intersection of the alphabets is denoted by adding two corresponding subscripts, e.g., denotes the locally observable events of , and denotes the high-level observable events. The various projections are denoted as shown in Fig. 4.

We further assume that the high-level alphabet contains all shared events, i.e., , where is the set of all events shared by two or more components. In addition, we assume that the modular components agree on the controllability and observability status of the shared events, which is a standard assumption in hierarchical decentralized control.

We now show that if all the local languages satisfy MOC, the their parallel composition also satisfies MOC.

Assume that each shared event is high level and observable, i.e., . If, for , is MOC wrt , , and , then is MOC wrt , , and .

## 8 Conclusion

We have completed the missing results in hierarchical supervisory control under partial observation. The regular behavior of the systems is essential for decidability of OC, MOC, and LOC. In the full version, we show that if slightly more expressive one-turn deterministic pushdown systems are used, the properties are undecidable. Deterministic pushdown systes have been discussed in supervisory control in the context of controllability and synthesis as a generalization of system models for which the synthesis is still possible.

## Appendix A Proofs

### a.1 PSpace-hardness proof of Theorem 4

We first show that if is OC, then the inclusion holds. To this end, assume that . By the definition of , and coincide on the letters of , i.e., . Since is OC, there are such that , , and . However, implies that , and and imply that , which shows the inclusion.

On the other hand, assume that the inclusion holds. We show that is OC. To this end, assume that are such that . By the definition of , we obtain that . Since the inclusion holds, we have , which means that there is a pair such that . Since , strings and belong to and coincide on the letters from , i.e., , which was to be shown.

To show PSpace-hardness, we reduce the problem of deciding universality for NFAs with all states marked, see KaoRS09. Such NFAs recognize exactly prefix-closed languages. The problem asks, given an NFA over with all states marked, whether the language . To , we construct an NFA such that . It is not difficult to construct from in polynomial time by adding a new initial state that goes to the initial state of under the sequence and that has a self-loop under every event from after , and by adding a new state reachable under having a self-loop under . Let the abstraction remove , and the observation remove , that is, and . Then . We now show that is OC if and only if is universal.

If is universal, then any two different strings with