Foundation Model based Open Vocabulary Task Planning and Executive System for General Purpose Service Robots

by   Yoshiki Obinata, et al.

This paper describes a strategy for implementing a robotic system capable of performing General Purpose Service Robot (GPSR) tasks in robocup@home. The GPSR task is that a real robot hears a variety of commands in spoken language and executes a task in a daily life environment. To achieve the task, we integrate foundation models based inference system and a state machine task executable. The foundation models plan the task and detect objects with open vocabulary, and a state machine task executable manages each robot's actions. This system works stable, and we took first place in the RoboCup@home Japan Open 2022's GPSR with 130 points, more than 85 points ahead of the other teams.


page 2

page 6

page 8


Desiderata for Planning Systems in General-Purpose Service Robots

General-purpose service robots are expected to undertake a broad range o...

Deliberative and Conceptual Inference in Service Robots

Service robots need to reason to support people in daily life situations...

Autonomous Planning Based on Spatial Concepts to Tidy Up Home Environments with Service Robots

Tidy-up tasks by service robots in home environments are challenging in ...

Modular Robots: extending the capabilities of one robot

For a robot to be perfect and enter the everyday life of humans,like com...

Specifying and achieving goals in open uncertain robot-manipulation domains

This paper describes an integrated solution to the problem of describing...

Pave the Way to Grasp Anything: Transferring Foundation Models for Universal Pick-Place Robots

Improving the generalization capabilities of general-purpose robotic age...

Digital Twin System for Home Service Robot Based on Motion Simulation

In order to improve the task execution capability of home service robot,...

Please sign up or login with your details

Forgot password? Click here to reset