Multi-Objective Convolutional Neural Networks for Robot Localisation and 3D Position Estimation in 2D Camera Images

04/06/2018
by   Justinas Miseikis, et al.
0

The field of collaborative robotics and human-robot interaction often focuses on the prediction of human behaviour, while assuming the information about the robot setup and configuration being known. This is often the case with fixed setups, which have all the sensors fixed and calibrated in relation to the rest of the system. However, it becomes a limiting factor when the system needs to be reconfigured or moved. We present a deep learning approach, which aims to solve this issue. Our method learns to identify and precisely localise the robot in 2D camera images, so having a fixed setup is no longer a requirement and a camera can be moved. In addition, our approach identifies the robot type and estimates the 3D position of the robot base in the camera image as well as 3D positions of each of the robot joints. Learning is done by using a multi-objective convolutional neural network with four previously mentioned objectives simultaneously using a combined loss function. The multi-objective approach makes the system more flexible and efficient by reusing some of the same features and diversifying for each objective in lower layers. A fully trained system shows promising results in providing an accurate mask of where the robot is located and an estimate of its base and joint positions in 3D. We compare the results to our previous approach of using cascaded convolutional neural networks.

READ FULL TEXT

page 2

page 3

page 4

page 6

research
01/06/2018

Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and Cascaded Convolutional Neural Networks

Many works in collaborative robotics and human-robot interaction focuses...
research
02/15/2019

Two-Stage Transfer Learning for Heterogeneous Robot Detection and 3D Joint Position Estimation in a 2D Camera Image using CNN

Collaborative robots are becoming more common on factory floors as well ...
research
05/30/2018

Transfer Learning for Unseen Robot Detection and Joint Estimation on a Multi-Objective Convolutional Neural Network

A significant problem of using deep learning techniques is the limited a...
research
06/05/2020

Anticipatory Human-Robot Collaboration via Multi-Objective Trajectory Optimization

We address the problem of adapting robot trajectories to improve safety,...
research
11/04/2020

Toward Force Estimation in Robot-Assisted Surgery using Deep Learning with Vision and Robot State

Knowledge of interaction forces during teleoperated robot-assisted surge...
research
01/25/2022

Poisson's CDF applied to Flexible Skylines

The evolution of skyline and ranking queries has created new archetypes ...

Please sign up or login with your details

Forgot password? Click here to reset