Trust Your Robots! Predictive Uncertainty Estimation of Neural Networks with Sparse Gaussian Processes

09/20/2021

∙

This paper presents a probabilistic framework to obtain both reliable and fast uncertainty estimates for predictions with Deep Neural Networks (DNNs). Our main contribution is a practical and principled combination of DNNs with sparse Gaussian Processes (GPs). We prove theoretically that DNNs can be seen as a special case of sparse GPs, namely mixtures of GP experts (MoE-GP), and we devise a learning algorithm that brings the derived theory into practice. In experiments from two different robotic tasks – inverse dynamics of a manipulator and object detection on a micro-aerial vehicle (MAV) – we show the effectiveness of our approach in terms of predictive uncertainty, improved scalability, and run-time efficiency on a Jetson TX2. We thus argue that our approach can pave the way towards reliable and fast robot learning systems with uncertainty awareness.

READ FULL TEXT

Trust Your Robots! Predictive Uncertainty Estimation of Neural Networks with Sparse Gaussian Processes

Approximate Inference Turns Deep Networks into Gaussian Processes

Vecchia Gaussian Process Ensembles on Internal Representations of Deep Neural Networks

Structure and Distribution Metric for Quantifying the Quality of Uncertainty: Assessing Gaussian Processes, Deep Neural Nets, and Deep Neural Operators for Regression

Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks

Learning robot inverse dynamics using sparse online Gaussian process with forgetting mechanism

Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions

Semi-described and semi-supervised learning with Gaussian processes

Trust Your Robots! Predictive Uncertainty Estimation of Neural Networks with Sparse Gaussian Processes

Related Research

Approximate Inference Turns Deep Networks into Gaussian Processes

Vecchia Gaussian Process Ensembles on Internal Representations of Deep Neural Networks

Structure and Distribution Metric for Quantifying the Quality of Uncertainty: Assessing Gaussian Processes, Deep Neural Nets, and Deep Neural Operators for Regression

Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks

Learning robot inverse dynamics using sparse online Gaussian process with forgetting mechanism

Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions

Semi-described and semi-supervised learning with Gaussian processes