Log In Sign Up

Task-relevant Representation Learning for Networked Robotic Perception

by   Manabu Nakanoya, et al.

Today, even the most compute-and-power constrained robots can measure complex, high data-rate video and LIDAR sensory streams. Often, such robots, ranging from low-power drones to space and subterranean rovers, need to transmit high-bitrate sensory data to a remote compute server if they are uncertain or cannot scalably run complex perception or mapping tasks locally. However, today's representations for sensory data are mostly designed for human, not robotic, perception and thus often waste precious compute or wireless network resources to transmit unimportant parts of a scene that are unnecessary for a high-level robotic task. This paper presents an algorithm to learn task-relevant representations of sensory data that are co-designed with a pre-trained robotic perception model's ultimate objective. Our algorithm aggressively compresses robotic sensory data by up to 11x more than competing methods. Further, it achieves high accuracy and robust generalization on diverse tasks including Mars terrain classification with low-power deep learning accelerators, neural motion planning, and environmental timeseries classification.


page 1

page 2

page 3

page 4

page 5

page 7

page 8

page 9


Network Offloading Policies for Cloud Robotics: a Learning-based Approach

Today's robotic systems are increasingly turning to computationally expe...

Sampling Training Data for Continual Learning Between Robots and the Cloud

Today's robotic fleets are increasingly measuring high-volume video and ...

MMLatch: Bottom-up Top-down Fusion for Multimodal Sentiment Analysis

Current deep learning approaches for multimodal fusion rely on bottom-up...

Guided Imitation of Task and Motion Planning

While modern policy optimization methods can do complex manipulation fro...

Automated Generation of Reactive Programs from Human Demonstration for Orchestration of Robot Behaviors

Social robots or collaborative robots that have to interact with people ...

Efficient State Representation Learning for Dynamic Robotic Scenarios

While the rapid progress of deep learning fuels end-to-end reinforcement...

Body Models in Humans and Robots

Neurocognitive models of higher-level somatosensory processing have emph...