Refining HTN Methods via Task Insertion with Preferences

11/29/2019
by   Zhanhao Xiao, et al.
0

Hierarchical Task Network (HTN) planning is showing its power in real-world planning. Although domain experts have partial hierarchical domain knowledge, it is time-consuming to specify all HTN methods, leaving them incomplete. On the other hand, traditional HTN learning approaches focus only on declarative goals, omitting the hierarchical domain knowledge. In this paper, we propose a novel learning framework to refine HTN methods via task insertion with completely preserving the original methods. As it is difficult to identify incomplete methods without designating declarative goals for compound tasks, we introduce the notion of prioritized preference to capture the incompleteness possibility of methods. Specifically, the framework first computes the preferred completion profile w.r.t. the prioritized preference to refine the incomplete methods. Then it finds the minimal set of refined methods via a method substitution operation. Experimental analysis demonstrates that our approach is effective, especially in solving new HTN planning instances.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

11/30/2011

Task Interaction in an HTN Planner

Hierarchical Task Network (HTN) planning uses task decomposition to plan...
07/01/2019

Incomplete Preferences in Single-Peaked Electorates

Incomplete preferences are likely to arise in real-world preference aggr...
01/12/2011

Planning with Partial Preference Models

Current work in planning with preferences assume that the user's prefere...
03/15/2021

Cloth Manipulation Planning on Basis of Mesh Representations with Incomplete Domain Knowledge and Voxel-to-Mesh Estimation

We consider the problem of open-goal planning for robotic cloth manipula...
11/18/2020

Domain Concretization from Examples: Addressing Missing Domain Knowledge via Robust Planning

The assumption of complete domain knowledge is not warranted for robot p...
06/02/2010

Learning Probabilistic Hierarchical Task Networks to Capture User Preferences

We propose automatically learning probabilistic Hierarchical Task Networ...
04/16/2018

Heuristic Approaches for Goal Recognition in Incomplete Domain Models

Recent approaches to goal recognition have progressively relaxed the ass...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

Introduction

Hierarchical task network (HTN) planning techniques [6] are increasingly used in a number of real-world applications [16, 5]. In the real-world logistics domain, such as Amazon and DHL Global Logistics, the shipment of packages is arranged via decomposition into a more detailed shipment arrangement in a top-down way according to the predefined HTN methods. In practice, there exist a vast number of cases occurring, such as the delay caused by the weather, leading that it is difficult and time-consuming for humans to find all complete methods for all actions. This suggests that it is important to learn complete methods.

Normally, domain experts have partially hierarchical domain knowledge, which possibly is not sufficient to cover all desirable solutions [15]. On one hand, with partially hierarchical domain knowledge, a planner may fail to obtain a solution via decomposition according to the given methods. The main reason lies in that the given method set is incomplete, which includes at least an incomplete method lacking subtasks. On the other hand, the hierarchical domain knowledge comes from the experience and investigation of domain experts, which normally are supposed to be necessary. However, the traditional approaches to learning HTN methods, such as [12], only concentrate on declarative goals of compound tasks and omit hierarchically procedural knowledge obtained from domain experts.Indeed, these procedural knowledge cannot be expressed by only operator structures (action models), which can be concluded from [11, 13]. Therefore, in this paper, we focus on refining HTN methods and keeping the original hierarchical domain knowledge from domain experts.

Figure 1: An example of a decomposition tree from incomplete methods (the parameters of some actions are hidden). The initial task is decomposed into a sequence of primitive tasks (the black leaves) according to the original methods. But when plane1 is not in airport A, the sequence is not executable. It becomes executable if arranging plane 1 to airport A before loading the package, which implies that should be considered as a subtask of .

To tackle the method incompleteness, Geier and Bercher (geier2011decidability) proposed a hybrid planning formalization, HTN planning with task insertion (TIHTN planning), to allow generating plans via decomposing tasks according to the methods but also inserting tasks from outside the given methods. Example 1 shows a plan with task insertion (called TIHTN plan) for an HTN problem with incomplete methods. Actually, the TIHTN plan offers a reference to refine the methods by adding the inserted tasks. For example, the method of can be refined by adding as its subtask. It allows us to absolutely preserve the original domain knowledge to refine the incomplete methods, further to assist domain experts to improve the HTN domain.

Example 1.

Consider an example in the logistics domain, suppose every task has only one method and a decomposition tree is shown in Figure 1. The initial task is to ship a package from city A to city B and it has a method: to ship the package from the warehouse to the airport by truck, from city A to city B by plane and from the airport to the shop by truck. But in case that the plane is not in the airport of city A, the air transportation task cannot be accomplished, neither can the initial task. When arranging the plane to airport A, , is done before loading to the plane, an executable plan is found.

Whereas, even if a TIHTN plan is found, without designating declarative goals for compound tasks, it is still difficult to identify incomplete methods – because an inserted task can be considered as a missed subtask of different methods. For example, the inserted task can also be used to refine the method of . In practice, the missing of subtasks happens more likely on certain methods than on some other methods. It motivates us to introduce the notion of prioritized preference on methods to capture the incompleteness possibility of methods.

Our contributions are listed as follows. First, we introduce the notion of completion profiles to refine HTN methods from TIHTN plans. Second, we propose a framework MethodRefine to refine HTN methods with completely preserving the original methods. Specifically, we first compute the preferred completion profile w.r.t. the prioritized preference to refine incomplete methods and then propose a method substitution operation to obtain the preferred set of refined methods. Third, by taking experiments on three well-known planning domains, we compare our approach with different method incompleteness against the classical HTN learning approach, HTN-MAKER, on the ability of solving new instances in the same domain and show that our approach is more effective.

Related Work

Besides HTN-MAKER, there are a number of HTN learning approaches [17, 25] which only focus on declarative goals and omit hierarchically procedural knowledge. Specially, they require annotated preconditions and effects of compound tasks and only consider the declarative goals like classical planning, so they require a complete executable plan as input. Whereas, in many domains, it is difficult to verify the correctness of the task annotations. Besides, these approaches restrict the tasks in the methods to be totally ordered, while we allow them to be partially ordered. Last but not least, comparing with these approaches, we also consider the prioritized preference on the learned methods.

Another related work is [9] which proposes an approach to construct and maintain hierarchical task models from a set of annotated examples provided by domain experts. Similar to the annotated tasks, obtaining these annotated examples is difficult and needs a lot of human effort. Our work also is related to the works on learning the precondition of HTN methods [14, 23], which takes the hierarchical relationships between tasks, the action models, and a complete description of the intermediate states as input. The similar work also includes [19] and [21], which used means-end analysis to learn structures and preconditions of the input plans. The precondition and effect of primitive actions can also be learned in [24]. All these approaches of learning method precondition require a complete method set as input.

For TIHTN planning, Geier and Bercher (geier2011decidability) first addressed the decidability of propositional TIHTN planning. Later Alford et al. (alford2015tighttihtn) proved that propositional TIHTN planning is EXPTIME-complete and proposed an acyclically searching approach to compute TIHTN plans, which provides a route to refine incomplete HTN methods.

Definitions

We adapt the definitions of lifted HTN planning [2]. First, we define a function-free first order language from a set of variables and a finite set of predicates and constants. We take parts of variables in as task symbols to identify tasks. A state is any subset of ground atoms in and the finite set of states are denoted by . In HTN planning, actions (or task names), noted

, are syntactically first-order atoms, which are classified into two categories: the actions the agent can execute directly are called

primitive actions or operators, noted , while the rest are called compound actions, noted . Every primitive action is a tuple where consists of a predicate out of and a list of variables, called its name; is a first-order logic formula, called its precondition; and are two conflict-free sets of atoms called its positive and negative effect. Then we define a state-transition function : is defined if is applicable in (i.e., ); . A sequence of primitive actions is executable in a state iff there is a state sequence such that .

Given a set , we use to denote the set of all sequences over and use to denote the cardinality of . For its subset and a function , its restriction to is . For a binary relation , we define its restriction to by .

Task networks. A task network is a tuple where is a set of tasks, is a non-empty set of ordering constraints over and labels every task with an action. Every task is associated to an action and the ordering constraints restrict the execution order of tasks. A task is called primitive if is primitive (otherwise called compound), and called ground if is ground. A task network is called primitive if it contains only primitive tasks, and called ground if it contains only ground tasks.

HTN methods. Compound actions cannot be directly executed and need to be decomposed into a task network according to HTN methods. An HTN method consists of a compound action (called head) and a task network whose inner tasks are called subtasks. Generally, an HTN method includes variables, which can be grounded as actions are grounded. Note that a compound action may have more than one HTN method.

Intuitively, decomposition is done by selecting a compound task, adding its subtask network and replacing it. The constraints about the decomposed task are propagated to its subtasks: the tasks before are before all its subtasks and the tasks after are after all its subtasks.

A task network is a grounding of another task network if there exists a bijection such that is a grounding of and for all , .

HTN problems. An HTN planning domain is a tuple where is a set of HTN methods and . We call a pair an instance where is a ground initial state and is a ground initial task. An HTN problem is a tuple .

Solutions. A solution to an HTN problem is a valid decomposition tree w.r.t.  and we say is solved under and is satisfied by .

In different literature, the solution to the HTN problem has different forms: mostly a plan (such as [6]), a primitive task network (such as [4]) and a list of decomposition trees (such as [25]). In this paper, we consider a solution to the HTN problem as a decomposition tree rooted in the initial task .

A decomposition tree (DT) is a tuple where is a tree, with nodes and with directed edges mapping each node to an ordered list of its children; is a set of constraints over ; function links tasks and actions; function labels every inner node with an HTN method.

We use to denote the transitive closure of and the order defined by . We say is a predecessor of if . Dually, we also say is a successor of . According to , we say the sequence constituted by the ground leaf nodes of is its plan, denoted by .

Definition 1 (Valid DTs).

A DT is valid w.r.t. an HTN problem iff its plan is executable in and its root is and for every inner node where , it satisfies:

  1. ;

  2. is a grounding of ;

  3. if then for every , ;

  4. if then for every , ;

  5. there are no such that and .

Example 2 (Example 1 cont.).

If plane1 is already at airport A in , the DT drawn with black arrows in Figure 1 is a solution to the HTN problem with a plan load;drive;unload;load;fly;unload;load;drive;unload.

Refining Methods via Task Insertion

In this paper, we focus on the HTN problem with an incomplete method set, where there is no valid decomposition tree w.r.t. the problem. In other words, there is no executable plan obtained only by applying methods. By allowing inserting tasks, [10] proposes a hybrid planning formalization, TIHTN planning. For an HTN problem , we say a primitive action sequence is it TIHTN plan, if is executable in and there is a valid DT w.r.t. whose is not required to be executable in satisfying is a sub-sequence of .

Example 3 (Example 2 cont.).

If plane1 is not at airport A in , the DT in Example 2 is not valid as its plan is not executable in . While load;drive;unload;fly;load;fly;unload;load;drive;unload is a TIHTN plan to the problem.

Refining Methods and Completing DTs

Actually, the inserted tasks in the TIHTN plan are subtask candidates: they provide clues for refining the original methods by adding them as subtasks. Then, based on a TIHTN plan, we propose the completion profile to refine methods and complete decomposition trees.

Suppose the TIHTN planner outputs a plan and its corresponding decomposition tree , we use to denote all the inserted tasks in . The TIHTN plan actually is an ordering of primitive tasks and we extend the relation of by considering the execution order of primitive actions in . To get the compound tasks, we use to denote the inner nodes of . Next, we show how to link these inserted tasks with the inner nodes of the decomposition tree to generate a new decomposition tree.

Definition 2.

We define a completion profile as a function , such that for every inserted task there is not a primitive task where either both and , or and .

Intuitively, every inserted task is associated with a compound task as its subtask. Every inserted task is restricted to be performed before the predecessors and after the successors of its corresponding compound task.

Next, we define how to refine a method by inserting tasks. A completion profile leads to a set of refined methods by adding the relevant inserted tasks into the original methods. Formally, for a completion profile and ,we use to denote all inserted tasks associated with . The inserted subtasks with the original subtasks of compose a new subtask network, written by , where which labels the inserted task with ground primitive actions. Suppose is the method of , i.e., , for every constant in the inserted primitve actions which occurs in the ground actions associated with the children of or , we replace it with its corresponding variable in the unground action in and update the function to . Then we define the refined method of as w.r.t. . We use to denote the set of refined methods from the completion profile .

Example 4 (Example 3 cont.).

We have a completion profile where . The refined method of the original method is where

  • ;

  • ;

  • ,
    , etc.

The completion profile actually completes the DT: the inserted tasks are connected with their corresponding inner nodes as their children. When we add new nodes into the DT, the integrity of ordering constraints will be destroyed. To avoid that, we define an operator to complete the ordering constraints. Formally, for a tree , we define its closure on the ordering constraint as , given by:

Intuitively, the closure operation completes the ordering constraints about the children which should be inherited from their parent.

Next, we define the completion of DT by completion profile w.r.t. TIHTN plan as , which is given by:

where is the set of the inner nodes associated by .

The procedure of completing a DT consists of first connecting the inserted tasks with the inner nodes, then completing the ordering constraints and finally updating the method applied as the refined method. The DT being completed will satisfy the instance:

Proposition 1.

Given an HTN problem , let be one of its TIHTN plans, be its corresponding DT and be one of their completion profiles. Then the completed DT satisfies under the new domain .

Proof.

First, we show that is a valid DT w.r.t. . For every node in with , i) the function is not reduced, so ; ii) the edges between the task and its inserted tasks are added, so the task network induced by its children is a grounding of ; iii) guarantees that all ordering constraints of are propagated to the inserted tasks and only introduces the ordering constraints among the subtasks in the same method, so conditions 3. and 4. are satisfied; iv) as the completion profile guarantees that no contradict pair about is introduced, condition 5. is satisfied.

Without removing nodes, the root of is still . As the plan is the TIHTN plan executable in , satisfies the instance . ∎

When an HTN problem has incomplete methods, the completion profile offers a way to improve the HTN domain:

Theorem 1.

If an HTN problem has a TIHTN plan but no solution, then there is a completion profile where the HTN problem is solvable.

Proof.

Straightforward. ∎

Prioritized Preferences

To formalize the phenomenon that the missing of subtasks happens more likely on some methods than other methods, we consider a prioritization on the methods.

Given a method set , we define a prioritization as a partition on it: where . Intuitively, the HTN methods in have a higher priority to be refined than those in if . We further consider the prioritized preference in terms of cardinality: for , if there is some such that and that , then we write . We say is strictly preferred over w.r.t. , written by , if and .

Generally, the prioritization comes from the confidences of domain experts on methods: the method believed to lack subtasks more likely to have a higher priority to be refined. In particular, there exists a class of HTN domains where actions can be stratified according to the decomposition hierarchy [7, 3]. We found an interesting phenomenon that the more detailed tasks are more sensitive to these situations and more easily to be thoughtless. In this case, we assume that an action is more abstract than its subtasks and we consider a preference in terms of a stratum-based prioritization: the more abstract actions have a lower priority to be refined.

Preferred Completion Profiles

Generally, we hope to find a completion profile changing the original methods minimally under the prioritized preference.

We first define some notations: for a refined method , we use to denote its original method . For a refined method set , we use to denote all the original methods of the refined methods in , i.e., . Note that several completions may be associated with the same HTN method. For two HTN methods and , if , we say and are homologous.

Definition 3.

Given a TIHTN plan and its corresponding DT, a completion profile is preferred w.r.t. preference if there is not a completion profile s.t. .

Intuitively, the preferred completion profile refines methods minimally under the prioritized preference.

Next, we will show how to find the preferred completion profile, as shown in Algorithm 1. First, we consider all inserted tasks in the plan as unlabelled (line 1). Then we scan all inner nodes from the nodes with a method of higher priority to the nodes with a method of lower priority (line 2-3). Next, for an inner node, we find the set of candidate subtasks from the inserted tasks, which do not violate the ordering constraints if they were inserted as its subtasks (line 5). More specifically, for the inner node , the inserted tasks which are executed between the last task required to be executed ahead of and the first task required to be after , are allowed to be added as subtasks of . According to the total order ‘’ in the DT, we define the subtasks candidate set of as the set of the unlabelled inserted tasks between the last predecessor of and the first successor of . Finally, we associate all tasks in the subtask candidate set to (line 5) and label them as subtasks (line 6). When all inserted tasks are labeled, it returns a preferred completion profile. It must terminate and the worst case is that the inserted tasks are associated with the root task.

Algorithm 1 only scans the inner nodes of the DT once and searching the subtask candidate set can be done in linear time, so the algorithm terminates in polynomial time.

input : A TIHTN plan , its DT and a prioritization on
output : A completion profile
1 ;
2 for  to  do
3       for each s.t.  do
4             if  then
5                   for every , set ;
6                   ;
7                  
8            
9      
return
Algorithm 1

Actually, to find a preferred completion profile, we only need to scan the inner nodes in the DT according to the preference and link appropriate inserted tasks with inner nodes, which can be done in polynomial time.

Refining Methods from Instances

As stated above, we focus on keeping the original methods coming from domain experts and consider adding the refined methods into the original domain. For an HTN domain and a method set , we use to denote the resulting domain by adding into . An HTN method refining problem is defined as a tuple where is an HTN domain and is a set of instances.

However, an excess of methods introduced may slow down problem-solving significantly, as there are excessive choices to decompose tasks. So, we hope the refined methods to be as minimal as possible. Then we define a solution of the HTN method refining problem is a set of refined methods which should satisfy:

  • all instances in the set are solvable under ;

  • the refined method set is as minimal as possible;

  • the refined methods in have as few inserted subtasks as possible.

Given a set of HTN instances, we first compute a TIHTN plan and a DT for each instance and then compute the preferred completion profile. Indeed, the completion profiles for various instances induce many different refined methods with the same head which possibly handel the same situation.Such refined methods are redundant because they can be replaced by other methods.

To compute the minimal set, we define a method substitution operation for a decomposition tree. More specifically, for a DT and two homologous refined methods and , we use to denote the resulting DT obtained by replacing every subtree induced by with a subtree by and completing the closure of the ordering constraints. Then for an instance , if satisfies it and is executable in , then also satisfies it. In other words, for this instance, the refined method is replaceable by .

Then we generalize the method substitution operation into sets: given a DT set and two refined method sets , we use to denote the set of the DTs that substitutes every method in with certain method in . If each resulting DT still satisfies the corresponding instance, we say is replaceable by . Given a set of refined methods, for the minimality, we need to compute the minimal subset of which are not replaceable by any strict subset of .

Indeed, it is difficult to find the minimal method set under the replaceability relation between refined methods, as all subsets of the refined method set require to be considered. Fortunately, the prioritized preference indicates what methods should be refined in a higher priority. The prioritized preference can be extended to the refined methods: if . Then we seek for the sub-optimal solution by computing the minimal subset w.r.t. the replaceability relation under the refined methods with the same priority, which reduces the searching space significantly.

Next, we give an algorithm to refine methods for a set of HTN problems and a given prioritization, as shown in Algorithm 2. The framework consists of two main components: the first iteration for refining methods (line 2-8) and the second iteration for reducing refined methods (line 9-11).

We developed the TIHTN planning approach in [2] (noted HPlan). In order to pursue as few inserted subtasks as possible in refined methods, we exploited a breadth-first strategy to search inserted tasks.

In the first iteration, it first invokes HPlan to compute TIHTN plan and the corresponding DTs (line 3) and then computes preferred completion profiles (line 4) by Complete. According to these completion profiles, it completes these DTs and constructs a set of refined methods.

In the second iteration, we use a greedy strategy to find the minimal set: the refined methods with lower priority are reduced first, which is the opposite against the procedure of searching the preferred completion profile. Here we use to denote the refined methods in with the priority . The algorithm enumerates the power set of and computes the minimal subset of w.r.t. the replaceability relation under and the union of the minimal subsets with lower priorities.

input : An HTN domain , an instance set and a prioritization on
output : A set of refined methods
1 ;      ;
2 for each in  do
3       compute a plan and DT ;
4       ;
5       complete the DT to by ;
6       ;
7       construct a new method set from ;
8       ;
9      
10for  to  do
11       compute the minimal subset w.r.t. the replaceability relation under ;
12       ;
13      
return
Algorithm 2

In fact, due to the greedy strategy, our approach cannot guarantee criterion 2 and 3, but must satisfy criterion 1:

Theorem 2.

Suppose is the method set refined by MethodRefine, if every instance in has a TIHTN plan under the domain , then it is also solvable under the domain .

Proof.

As every instance has a TIHTN plan, by Proposition 1, there exists a set of DTs , each of which satisfies each instance w.r.t. the domain where is a method set obtained via completion profiles. As in the second iteration, the algorithm keeps the satisfiability of the instances, is replaceable by and each DT in satisfies its corresponding instance in . Thus, every instance is solvable w.r.t. the new domain . ∎

(a) The Logistics Domain
(b) The Satellite Domain
(c) The Blocks-World Domain
Figure 2: Experimental Results on the Solving Rate of MethodRefine with Different Domain Incompleteness and HTN-MAKER
(a) The Logistics Domain
(b) The Satellite Domain
(c) The Blocks-World Domain
Figure 3: Experimental Results on the Solving Rate of MethodRefine with Different Preferences

Experimental Analysis

In this section, we evaluate MethodRefine111Available in https://github.com/sysulic/MethodRefine in three well-known planning domains comparing with HTN-MAKER222http://www.cse.lehigh.edu/InSyTe/HTN-MAKER/ on the ability of solving new instances.

We consider the three domains which HTN-MAKER uses: Logistics, Satellite, and Blocks-world. We first get the problem generators from International Planning Competition website333http://ipc02.icaps-conference.org/ and randomly generate 100 instances for each domain and take 50 instances as the training set and 50 instances as the testing set. We run MethodRefine and HTN-MAKER with 50 instances growingly as input and obtain different learned method sets from these two approaches. A testing instance is considered as solved, if its goal is achieved by a plan computed under the learned HTN method set via an HTN planner. For HTN-MAKER we use the HTN planner from their website and for our approach MethodRefine we still use our TIHTN planner with forbidding task insertion. Experiments are run on the 3.00 GHz Intel i5-7400 with 8 GB RAM with a cutoff time of one hour. In order to check if an instance is solved, we add a verifying action whose precondition is the goal and whose effect is empty in the last subtask of the initial task. The learning performance is measured via the proportion of the solved instances on the testing instances, which is called solving rate.

Experimental results on comparing different domain incompleteness. First, we evaluate the influence of the different incompleteness of the given method sets on the solving rate. To simulate the incomplete method set as the input of MethodRefine, we take the HTN domain descriptions in the website444https://www.cs.umd.edu/projects/shop/ of SHOP2 HTN planner, and remove different sets of subtasks from these domains. Then we consider three removal cases: 1) remove one primitive task from each method (if exists), with meaning the high completeness, noted by MR-H; 2) remove two primitive tasks from each method (if exists), noted by MR-M, with meaning the middle completeness; 3) remove one more compound task in some method of MR-L, noted by MR-L, with meaning the low completeness. Taking the method set shown in Figure 1 as example, for MR-H, we remove the first drive and the first fly in the methods cityShip and airShip, respectively, while for MR-M, we remove all drive and fly in the methods. For MR-L, the first cityShip is additionally removed from the method of ship based on the MR-M setting. As these domain are stratifiable, we use the stratum-based prioritized preference as the input of MethodRefine.

The experimental results are shown in Figure 2. It demonstrates that the more complete the domain is, the better the learning performance is. Generally, the solving rate increases along with the training set growing. For the Logistics and Satellite domains, in the settings of MR-H and MR-M, MethodRefine learns the necessary methods to solve all testing problems from a few instances. It is because the structure of these two domains is relatively straightforward and the DTs still can be constructed by the incomplete method sets. In the MR-L setting, the compound action removed in the Logistics Domain, cityShip, contains more arguments, making the learned methods become more case-specific, which cannot contribute to other instances.

MR-H eventually learns 2 methods which already effectively solve all testing instances on the three domains. While MR-M and MR-L learn more than 10 methods in the Blocks-World domain but still fail to achieve the full solving rate.

Experimental results on comparing MethodRefine against HTN-MAKER. From Figure 2, we observe that HTN-MAKER learns methods less slowly than MethodRefine with MR-H setting. Comparing with the other two settings, HTN-MAKER is superior on the Blocks-World domain but inferior on the Logistics and Satellite domain. In the Satellite domain, after training a number of instances, HTN-MAKER exceeds memory limitation when solving some instances, causing a sudden drop in its curve. It is because HTN-MAKER learns a method which causes an infinitely recursive decomposition. It never occurs in our approach because we only learn methods from acyclic decomposition trees. But they are solved when suitable methods are learned with the training set growing. HTN-MAKER finally learns 87, 23, 92 methods in the Logistics, Satellite, Blocks-World domain, respectively. Comparing with our approach, it learns more methods but many of which are redundant.

Experimental results on the performances of MethodRefine with different preferences. To evaluate our assumption on the stratum-based prioritized preference, we also compare it against its opposite prioritization where tasks are inserted in as abstract compound tasks as possible (‘MR-M-Abs’), and the case with no preference where any completion profile is allowed (‘MR-M-rand’). We choose the domains with the middle completeness.

From the experimental results (Figure 3), our stratum-based prioritized preference outperforms the other two cases. It is because the completion profile associates the inserted tasks to a more abstract task and it generates a more case-specific method which may not suit other instances.

Discussion and Conclusion

Without declarative goals, we suppose that in the original method set, every compound action at least has a method to decompose. Our approach can also accept a declarative goal: we can trivially introduce a compound action of achieving the goal which is decomposed into a “verifying” action whose precondition is the goal and whose effect is empty.

To sum up, we present a framework to help domain experts to improve HTN domains by refining methods. The experiment results demonstrate that our approach outperforms the method learning approach, HTN-MAKER, given an appropriately incomplete method set as input. It is also illustrated that the stratum-based prioritized preference is effective to solve new instances.

In this paper, we assume primitive action models are available. In the future, it would be interesting to remove this assumption and study learning HTN methods and action models simultaneously from history data [26, 1]. Also, in this paper we only focus on keeping the procedural knowledge from domain experts and it would be a promising avenue to jointly consider partial annotations of tasks or state constraints together [22, 18]. In what follows, we are going to study refining methods directly from raw data, such as texts [8] and to explore plan recognition techniques with incomplete action models [27] to improve HTN domains.

References

  • [1] D. Aineto, S. J. Celorrio, and E. Onaindia (2019) Learning action models with minimal observability. Artif. Intell. 275, pp. 104–137. External Links: Link, Document Cited by: Discussion and Conclusion.
  • [2] R. Alford, P. Bercher, and D. W. Aha (2015) Tight bounds for HTN planning with task insertion. In

    Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI-15)

    ,
    pp. 1502–1508. Cited by: Definitions, Refining Methods from Instances.
  • [3] R. Alford, V. Shivashankar, U. Kuter, and D. S. Nau (2012) HTN problem spaces: structure, algorithms, termination. In Proceedings of the 5th Annual Symposium on Combinatorial Search, (SOCS-12), pp. 2–9. Cited by: Prioritized Preferences.
  • [4] G. Behnke, D. Höller, and S. Biundo (2017) This is a solution! (… but is it though?) - verifying solutions of hierarchical planning problems. In Proceedings of the 27th International Conference on Automated Planning and Scheduling (ICAPS), pp. 20–28. External Links: Link Cited by: Definitions.
  • [5] G. Behnke, M. R. G. Schiller, M. Kraus, P. Bercher, M. Schmautz, M. Dorna, M. Dambier, W. Minker, B. Glimm, and S. Biundo (2019) Alice in DIY wonderland or: instructing novice users on how to use tools in DIY projects. AI Communication 32 (1), pp. 31–57. External Links: Link, Document Cited by: Introduction.
  • [6] K. Erol, J. Hendler, and D. S. Nau (1994) HTN planning: complexity and expressivity. In Proceedings of the 12th AAAI Conference on Artificial Intelligence (AAAI-94), pp. 1123–1128. Cited by: Introduction, Definitions.
  • [7] K. Erol, J. Hendler, and D. S. Nau (1996) Complexity results for HTN planning. Annals of Mathematics and Artificial Intelligence 18 (1), pp. 69–93. Cited by: Prioritized Preferences.
  • [8] W. Feng, H. H. Zhuo, and S. Kambhampati (2018)

    Extracting action sequences from texts based on deep reinforcement learning

    .
    See DBLP:conf/ijcai/2018, pp. 4064–4070. External Links: Link, Document Cited by: Discussion and Conclusion.
  • [9] A. Garland, K. Ryall, and C. Rich (2001) Learning hierarchical task models by defining and refining examples. See 20, pp. 44–51. External Links: Link, Document Cited by: Related Work.
  • [10] T. Geier and P. Bercher (2011) On the decidability of HTN planning with task insertion. In IJCAI, pp. 1955–1961. Cited by: Refining Methods via Task Insertion.
  • [11] R. P. Goldman (2009) A semantics for HTN methods. In Proceedings of the 19th International Conference on Automated Planning and Scheduling (ICAPS), External Links: Link Cited by: Introduction.
  • [12] C. Hogg, H. Muñoz-Avila, and U. Kuter (2008)

    HTN-MAKER: learning HTNs with minimal additional knowledge engineering required

    .
    In Proceedings of the 23rd AAAI Conference on Artificial Intelligence, (AAAI-08), pp. 950–956. External Links: Link Cited by: Introduction.
  • [13] D. Höller, G. Behnke, P. Bercher, and S. Biundo (2014) Language classification of hierarchical planning problems.. In ECAI-14, pp. 447–452. Cited by: Introduction.
  • [14] O. Ilghami, H. Muñoz-Avila, D. S. Nau, and D. W. Aha (2005) Learning approximate preconditions for methods in hierarchical plans. In

    Proceedings of the 22nd International Conference on Machine Learning (ICML-05)

    ,
    pp. 337–344. External Links: Link, Document Cited by: Related Work.
  • [15] S. Kambhampati, A. Mali, and B. Srivastava (1998) Hybrid planning for partially hierarchical domains. In AAAI/IAAI, pp. 882–888. Cited by: Introduction.
  • [16] N. Lin, U. Kuter, and E. Sirin (2008) Web service composition with user preferences. In Proceedings of European Semantic Web Conference (EWSC), pp. 629–643. Cited by: Introduction.
  • [17] D. Lotinac and A. Jonsson (2016) Constructing hierarchical task models using invariance analysis. In Proceedings of the 22nd European Conference on Artificial Intelligence (ECAI-16), pp. 1274–1282. External Links: Link, Document Cited by: Related Work.
  • [18] H. Muñoz-Avila, D. Dannenhauer, and N. Reifsnyder (2019) Is everything going according to plan? expectations in goal reasoning agents. See DBLP:conf/aaai/2019, pp. 9823–9829. External Links: Link, Document Cited by: Discussion and Conclusion.
  • [19] N. Nejati, P. Langley, and T. Könik (2006) Learning hierarchical task networks by observation. In Proceedings of the Twenty-Third International Conference on Machine Learning, (ICML-06), pp. 665–672. Cited by: Related Work.
  • [20] (2001) Proceedings of the first international conference on knowledge capture (K-CAP 2001), october 21-23, 2001, victoria, bc, canada. ACM. External Links: ISBN 1-58113-380-4 Cited by: 9.
  • [21] C. Reddy and P. Tadepalli (1997) Learning goal-decomposition rules using exercises. In Proceedings of the 14th International Conference on Machine Learning (ICML-97), pp. 278–286. Cited by: Related Work.
  • [22] Z. Xiao, A. Herzig, L. Perrussel, H. Wan, and X. Su (2017) Hierarchical task network planning with task insertion and state constraints. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI), pp. 4463–4469. External Links: Document, Link Cited by: Discussion and Conclusion.
  • [23] K. Xu and H. Muñoz-Avila (2005) A domain-independent system for case-based task decomposition without domain theories. In Proceedings of the 20th National Conference on Artificial Intelligence (AAAI-05), pp. 234–240. External Links: Link Cited by: Related Work.
  • [24] H. H. Zhuo, D. H. Hu, C. Hogg, Q. Yang, and H. Muñoz-Avila (2009) Learning HTN method preconditions and action models from partial observations. In Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI-09), pp. 1804–1810. External Links: Link Cited by: Related Work.
  • [25] H. H. Zhuo, H. Muñoz-Avila, and Q. Yang (2014) Learning hierarchical task network domains from partially observed plan traces. Journal of Artificial Intelligence 212, pp. 134–157. Cited by: Related Work, Definitions.
  • [26] H. H. Zhuo and Q. Yang (2014)

    Action-model acquisition for planning via transfer learning

    .
    Artif. Intell. 212, pp. 80–103. External Links: Link, Document Cited by: Discussion and Conclusion.
  • [27] H. H. Zhuo (2019) Recognizing multi-agent plans when action models and team plans are both incomplete. ACM TIST 10 (3), pp. 30:1–30:24. External Links: Link, Document Cited by: Discussion and Conclusion.