FlowControl: Optical Flow Based Visual Servoing

07/01/2020
by   Max Argus, et al.
0

One-shot imitation is the vision of robot programming from a single demonstration, rather than by tedious construction of computer code. We present a practical method for realizing one-shot imitation for manipulation tasks, exploiting modern learning-based optical flow to perform real-time visual servoing. Our approach, which we call FlowControl, continuously tracks a demonstration video, using a specified foreground mask to attend to an object of interest. Using RGB-D observations, FlowControl requires no 3D object models, and is easy to set up. FlowControl inherits great robustness to visual appearance from decades of work in optical flow. We exhibit FlowControl on a range of problems, including ones requiring very precise motions, and ones requiring the ability to generalize.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 6

research
05/08/2021

PCA Event-Based Otical Flow for Visual Odometry

With the advent of neuromorphic vision sensors such as event-based camer...
research
10/16/2021

Learning Cloth Folding Tasks with Refined Flow Based Spatio-Temporal Graphs

Cloth folding is a widespread domestic task that is seemingly performed ...
research
12/02/2021

Dimensions of Motion: Learning to Predict a Subspace of Optical Flow from a Single Image

We introduce the problem of predicting, from a single video frame, a low...
research
11/18/2018

Optical Flow Based Online Moving Foreground Analysis

Obtained by moving object detection, the foreground mask result is unsha...
research
01/18/2023

Real-Time Viewport-Aware Optical Flow Estimation in 360-degree Videos for Visually-Induced Motion Sickness Mitigation

Visually-induced motion sickness (VIMS), a side effect of illusionary mo...
research
07/11/2019

Graph-Structured Visual Imitation

We cast visual imitation as a visual correspondence problem. Our robotic...
research
07/29/2021

Using Visual Anomaly Detection for Task Execution Monitoring

Execution monitoring is essential for robots to detect and respond to fa...

Please sign up or login with your details

Forgot password? Click here to reset