On Mean Absolute Error for Deep Neural Network Based Vector-to-Vector Regression

by   Jun Qi, et al.

In this paper, we exploit the properties of mean absolute error (MAE) as a loss function for the deep neural network (DNN) based vector-to-vector regression. The goal of this work is two-fold: (i) presenting performance bounds of MAE, and (ii) demonstrating new properties of MAE that make it more appropriate than mean squared error (MSE) as a loss function for DNN based vector-to-vector regression. First, we show that a generalized upper-bound for DNN-based vector- to-vector regression can be ensured by leveraging the known Lipschitz continuity property of MAE. Next, we derive a new generalized upper bound in the presence of additive noise. Finally, in contrast to conventional MSE commonly adopted to approximate Gaussian errors for regression, we show that MAE can be interpreted as an error modeled by Laplacian distribution. Speech enhancement experiments are conducted to corroborate our proposed theorems and validate the performance advantages of MAE over MSE for DNN based regression.



There are no comments yet.


page 1

page 2

page 3

page 4


Analyzing Upper Bounds on Mean Absolute Errors for Deep Neural Network Based Vector-to-Vector Regression

In this paper, we show that, in vector-to-vector regression utilizing de...

Leveraging IoT and Weather Conditions to Estimate the Riders Waiting for the Bus Transit on Campus

The communication technology revolution in this era has increased the us...

Iterative Error Decimation for Syndrome-Based Neural Network Decoders

In this letter, we introduce a new syndrome-based decoder where a deep n...

Improving evidential deep learning via multi-task learning

The Evidential regression network (ENet) estimates a continuous target a...

On the Equivalence between Objective Intelligibility and Mean-Squared Error for Deep Neural Network based Speech Enhancement

Although speech enhancement algorithms based on deep neural networks (DN...

`Basic' Generalization Error Bounds for Least Squares Regression with Well-specified Models

This note examines the behavior of generalization capabilities - as defi...

Estimation and Inference with Trees and Forests in High Dimensions

We analyze the finite sample mean squared error (MSE) performance of reg...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.