DURableVS: Data-efficient Unsupervised Recalibrating Visual Servoing via online learning in a structured generative model

by   Nishad Gothoskar, et al.

Visual servoing enables robotic systems to perform accurate closed-loop control, which is required in many applications. However, existing methods either require precise calibration of the robot kinematic model and cameras or use neural architectures that require large amounts of data to train. In this work, we present a method for unsupervised learning of visual servoing that does not require any prior calibration and is extremely data-efficient. Our key insight is that visual servoing does not depend on identifying the veridical kinematic and camera parameters, but instead only on an accurate generative model of image feature observations from the joint positions of the robot. We demonstrate that with our model architecture and learning algorithm, we can consistently learn accurate models from less than 50 training samples (which amounts to less than 1 min of unsupervised data collection), and that such data-efficient learning is not possible with standard neural architectures. Further, we show that by using the generative model in the loop and learning online, we can enable a robotic system to recover from calibration errors and to detect and quickly adapt to possibly unexpected changes in the robot-camera system (e.g. bumped camera, new objects).


Learning a generative model for robot control using visual feedback

We introduce a novel formulation for incorporating visual feedback in co...

Robotic Tool Tracking under Partially Visible Kinematic Chain: A Unified Approach

Anytime a robot manipulator is controlled via visual feedback, the trans...

Applications of Uncalibrated Image Based Visual Servoing in Micro- and Macroscale Robotics

We present a robust markerless image based visual servoing method that e...

A Visual Kinematics Calibration Method for Manipulator Based on Nonlinear Optimization

The traditional kinematic calibration method for manipulators requires p...

Learning Precise 3D Manipulation from Multiple Uncalibrated Cameras

In this work, we present an effective multi-view approach to closed-loop...

Astronomical imaging: The theory of everything

We are developing automated systems to provide homogeneous calibration m...

Learning from Label Proportions in Brain-Computer Interfaces: Online Unsupervised Learning with Guarantees

Objective: Using traditional approaches, a Brain-Computer Interface (BCI...

Please sign up or login with your details

Forgot password? Click here to reset