Reasoning with Scene Graphs for Robot Planning under Partial Observability

by   Saeid Amiri, et al.

Robot planning in partially observable domains is difficult, because a robot needs to estimate the current state and plan actions at the same time. When the domain includes many objects, reasoning about the objects and their relationships makes robot planning even more difficult. In this paper, we develop an algorithm called scene analysis for robot planning (SARP) that enables robots to reason with visual contextual information toward achieving long-term goals under uncertainty. SARP constructs scene graphs, a factored representation of objects and their relations, using images captured from different positions, and reasons with them to enable context-aware robot planning under partial observability. Experiments have been conducted using multiple 3D environments in simulation, and a dataset collected by a real robot. In comparison to standard robot planning and scene analysis methods, in a target search domain, SARP improves both efficiency and accuracy in task completion. Supplementary material can be found at


page 6

page 7


Deep Affordance Foresight: Planning Through What Can Be Done in the Future

Planning in realistic environments requires searching in large planning ...

POMDP Manipulation Planning under Object Composition Uncertainty

Manipulating unknown objects in a cluttered environment is difficult bec...

Waiting Tables as a Robot Planning Problem

We present how we formalize the waiting tables task in a restaurant as a...

Specifying and achieving goals in open uncertain robot-manipulation domains

This paper describes an integrated solution to the problem of describing...

TASKOGRAPHY: Evaluating robot task planning over large 3D scene graphs

3D scene graphs (3DSGs) are an emerging description; unifying symbolic, ...

FRIDA: A Collaborative Robot Painter with a Differentiable, Real2Sim2Real Planning Environment

Painting is an artistic process of rendering visual content that achieve...

Statler: State-Maintaining Language Models for Embodied Reasoning

Large language models (LLMs) provide a promising tool that enable robots...

Please sign up or login with your details

Forgot password? Click here to reset