DURableVS: Data-efficient Unsupervised Recalibrating Visual Servoing via online learning in a structured generative model

02/08/2022
by   Nishad Gothoskar, et al.
0

Visual servoing enables robotic systems to perform accurate closed-loop control, which is required in many applications. However, existing methods either require precise calibration of the robot kinematic model and cameras or use neural architectures that require large amounts of data to train. In this work, we present a method for unsupervised learning of visual servoing that does not require any prior calibration and is extremely data-efficient. Our key insight is that visual servoing does not depend on identifying the veridical kinematic and camera parameters, but instead only on an accurate generative model of image feature observations from the joint positions of the robot. We demonstrate that with our model architecture and learning algorithm, we can consistently learn accurate models from less than 50 training samples (which amounts to less than 1 min of unsupervised data collection), and that such data-efficient learning is not possible with standard neural architectures. Further, we show that by using the generative model in the loop and learning online, we can enable a robotic system to recover from calibration errors and to detect and quickly adapt to possibly unexpected changes in the robot-camera system (e.g. bumped camera, new objects).

READ FULL TEXT
research
03/10/2020

Learning a generative model for robot control using visual feedback

We introduce a novel formulation for incorporating visual feedback in co...
research
02/11/2021

Robotic Tool Tracking under Partially Visible Kinematic Chain: A Unified Approach

Anytime a robot manipulator is controlled via visual feedback, the trans...
research
04/17/2023

Applications of Uncalibrated Image Based Visual Servoing in Micro- and Macroscale Robotics

We present a robust markerless image based visual servoing method that e...
research
05/18/2020

A Visual Kinematics Calibration Method for Manipulator Based on Nonlinear Optimization

The traditional kinematic calibration method for manipulators requires p...
research
02/21/2020

Learning Precise 3D Manipulation from Multiple Uncalibrated Cameras

In this work, we present an effective multi-view approach to closed-loop...
research
10/21/2008

Astronomical imaging: The theory of everything

We are developing automated systems to provide homogeneous calibration m...
research
01/25/2017

Learning from Label Proportions in Brain-Computer Interfaces: Online Unsupervised Learning with Guarantees

Objective: Using traditional approaches, a Brain-Computer Interface (BCI...

Please sign up or login with your details

Forgot password? Click here to reset