Bonnet: An Open-Source Training and Deployment Framework for Semantic Segmentation in Robotics using CNNs

02/25/2018
by   Andres Milioto, et al.
0

The ability to interpret a scene is an important capability for a robot that is supposed to interact with its environment. The knowledge of what is in front of the robot is, for example, key to navigation, manipulation, or planning. Semantic segmentation labels each pixel of an image with a class label and thus provides a detailed semantic annotation of the surroundings to the robot. Convolutional neural networks (CNNs) became popular methods for addressing this type of problem. The available software for training and the integration of CNNs in real robots, however, is quite fragmented and difficult to use for non-experts, despite the availability of several high-quality open-source frameworks for neural network implementation and training. In this paper, we propose a novel framework called Bonnet, which addresses this fragmentation problem. It provides a modular approach to simplify the training of a semantic segmentation CNN independently of the used dataset and the intended task. Furthermore, we also address the deployment on a real robotic platform. Thus, we do not propose a new CNN approach in this paper. Instead, we provide a stable and easy-to-use tool to make this technology more approachable in the context of autonomous systems. In this sense, we aim at closing a gap between computer vision research and its use in robotics research. We provide an open-source framework for training and deployment. The training interface is implemented in Python using TensorFlow and the deployment interface provides a C++ library that can be easily integrated in an existing robotics codebase, a ROS node, and two standalone applications for label prediction in images and videos.

READ FULL TEXT
research
02/20/2019

An efficient solution for semantic segmentation: ShuffleNet V2 with atrous separable convolutions

Assigning a label to each pixel in an image, namely semantic segmentatio...
research
05/26/2023

SSSegmenation: An Open Source Supervised Semantic Segmentation Toolbox Based on PyTorch

This paper presents SSSegmenation, which is an open source supervised se...
research
09/14/2022

Timor Python: A Toolbox for Industrial Modular Robotics

Development of controllers, novel robot kinematics, and learning-based a...
research
04/12/2023

Few Shot Semantic Segmentation: a review of methodologies and open challenges

Semantic segmentation assigns category labels to each pixel in an image,...
research
02/29/2016

Modular Tracking Framework: A Unified Approach to Registration based Tracking

This paper presents a modular, extensible and highly efficient open sour...
research
08/25/2021

Semantic Scene Segmentation for Robotics Applications

Semantic scene segmentation plays a critical role in a wide range of rob...
research
10/22/2021

IVS3D: An Open Source Framework for Intelligent Video Sampling and Preprocessing to Facilitate 3D Reconstruction

The creation of detailed 3D models is relevant for a wide range of appli...

Please sign up or login with your details

Forgot password? Click here to reset