Policy Recognition in the Abstract Hidden Markov Model

by   H. H. Bui, et al.

In this paper, we present a method for recognising an agent's behaviour in dynamic, noisy, uncertain domains, and across multiple levels of abstraction. We term this problem on-line plan recognition under uncertainty and view it generally as probabilistic inference on the stochastic process representing the execution of the agent's plan. Our contributions in this paper are twofold. In terms of probabilistic inference, we introduce the Abstract Hidden Markov Model (AHMM), a novel type of stochastic processes, provide its dynamic Bayesian network (DBN) structure and analyse the properties of this network. We then describe an application of the Rao-Blackwellised Particle Filter to the AHMM which allows us to construct an efficient, hybrid inference method for this model. In terms of plan recognition, we propose a novel plan recognition framework based on the AHMM as the plan execution model. The Rao-Blackwellised hybrid inference for AHMM can take advantage of the independence properties inherent in a model of plan execution, leading to an algorithm for online probabilistic plan recognition that scales well with the number of levels in the plan hierarchy. This illustrates that while stochastic models for plan execution can be complex, they exhibit special structures which, if exploited, can lead to efficient plan recognition algorithms. We demonstrate the usefulness of the AHMM framework via a behaviour recognition system in a complex spatial environment using distributed video surveillance data.


page 1

page 2

page 3

page 4


Probabilistic State-Dependent Grammars for Plan Recognition

Techniques for plan recognition under uncertainty require a stochastic m...

A New Model of Plan Recognition

We present a new abductive, probabilistic theory of plan recognition. Th...

Parallel tempering as a mechanism for facilitating inference in hierarchical hidden Markov models

The study of animal behavioural states inferred through hidden Markov mo...

Planning with Abstract Learned Models While Learning Transferable Subtasks

We introduce an algorithm for model-based hierarchical reinforcement lea...

Monitoring Teams by Overhearing: A Multi-Agent Plan-Recognition Approach

Recent years are seeing an increasing need for on-line monitoring of tea...

Approximately Optimal Monitoring of Plan Preconditions

Monitoring plan preconditions can allow for replanning when a preconditi...

Session Analysis using Plan Recognition

This paper presents preliminary results of our work with a major financi...

Please sign up or login with your details

Forgot password? Click here to reset