Scene Editing as Teleoperation: A Case Study in 6DoF Kit Assembly

10/09/2021
by   Shubham Agrawal, et al.
0

Studies in robot teleoperation have been centered around action specifications – from continuous joint control to discrete end-effector pose control. However, these robot-centric interfaces often require skilled operators with extensive robotics expertise. To make teleoperation accessible to non-expert users, we propose the framework "Scene Editing as Teleoperation" (SEaT), where the key idea is to transform the traditional "robot-centric" interface into a "scene-centric" interface – instead of controlling the robot, users focus on specifying the task's goal by manipulating digital twins of the real-world objects. As a result, a user can perform teleoperation without any expert knowledge of the robot hardware. To achieve this goal, we utilize a category-agnostic scene-completion algorithm that translates the real-world workspace (with unknown objects) into a manipulable virtual scene representation and an action-snapping algorithm that refines the user input before generating the robot's action plan. To train the algorithms, we procedurally generated a large-scale, diverse kit-assembly dataset that contains object-kit pairs that mimic real-world object-kitting tasks. Our experiments in simulation and on a real-world system demonstrate that our framework improves both the efficiency and success rate for 6DoF kit-assembly tasks. A user study demonstrates that SEaT framework participants achieve a higher task success rate and report a lower subjective workload compared to an alternative robot-centric interface. Video can be found at https://www.youtube.com/watch?v=-NdR3mkPbQQ .

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 6

research
08/02/2023

Virtual Reality Based Robot Teleoperation via Human-Scene Interaction

Robot teleoperation gains great success in various situations, including...
research
07/05/2023

FOCUS: Object-Centric World Models for Robotics Manipulation

Understanding the world in terms of objects and the possible interplays ...
research
04/08/2023

Pick2Place: Task-aware 6DoF Grasp Estimation via Object-Centric Perspective Affordance

The choice of a grasp plays a critical role in the success of downstream...
research
06/26/2019

From Multi-modal Property Dataset to Robot-centric Conceptual Knowledge About Household Objects

Tool-use applications in robotics require conceptual knowledge about obj...
research
07/09/2023

HA-ViD: A Human Assembly Video Dataset for Comprehensive Assembly Knowledge Understanding

Understanding comprehensive assembly knowledge from videos is critical f...
research
10/08/2019

Object-centric Forward Modeling for Model Predictive Control

We present an approach to learn an object-centric forward model, and sho...
research
03/16/2021

Behavior-Tree-Based Person Search for Symbiotic Autonomous Mobile Robot Tasks

We consider the problem of people search by a mobile social robot in cas...

Please sign up or login with your details

Forgot password? Click here to reset