Post-hoc Calibration of Neural Networks

06/23/2020

∙

Calibration of neural networks is a critical aspect to consider when incorporating machine learning models in real-world decision-making systems where the confidence of decisions are equally important as the decisions themselves. In recent years, there is a surge of research on neural network calibration and the majority of the works can be categorized into post-hoc calibration methods, defined as methods that learn an additional function to calibrate an already trained base network. In this work, we intend to understand the post-hoc calibration methods from a theoretical point of view. Especially, it is known that minimizing Negative Log-Likelihood (NLL) will lead to a calibrated network on the training set if the global optimum is attained (Bishop, 1994). Nevertheless, it is not clear learning an additional function in a post-hoc manner would lead to calibration in the theoretical sense. To this end, we prove that even though the base network (f) does not lead to the global optimum of NLL, by adding additional layers (g) and minimizing NLL by optimizing the parameters of g one can obtain a calibrated network g ∘ f. This not only provides a less stringent condition to obtain a calibrated network but also provides a theoretical justification of post-hoc calibration methods. Our experiments on various image classification benchmarks confirm the theory.

READ FULL TEXT VIEW PDF

1 Introduction

In this paper we consider the problem of calibration of neural networks, or classification functions in general. This problem has been considered in the context of Support Vector Machines

platt1999probabilistic

, but has recently been considered in the context of Convolutional Neural Networks (CNNs)

guo2017calibration. In this case, a CNN used for classification takes an input

x \in D_{X}

, belonging to one of

n

classes, and outputs a vector

f (x)

I R^{n}

, where the

k

-th component,

f_{k} (x)

is often interpreted as a probability that input

x

belongs to class

k

. If this value is to represent probabilities accurately, then we require that

f_{k} (x) = P (k | f (x))

. In this case, the classifier

f

is said to be calibrated, or multi-class calibrated. ¹¹1 In many papers, e.g. kull2019beyond and calibration metrics, e.g. ECE naeini2015obtaining a slightly different condition known as classwise-calibration is preferred:

f_{k} (x) = P (k | f_{k} (x))

A well-known condition (bishop1994mixture) for a classifier to be calibrated is that it minimizes the cross-entropy cost function, over all functions $f : D_{X} \to Δ^{n - 1}$ , where $Δ^{n - 1}$ is the standard probability simplex (see definition below). If the absolute minimum is attained, it is true that $f_{k} (x) = P (k | x)$ . However, this condition is rarely satisfied, since $D_{X}$

may be a very large space (for instance a set of images, of very high dimension) and the task of finding the absolute (or even a local) minimum of the loss is difficult: it requires the network to have sufficient capacity, and also that the network manages to find the optimal value through training. To fulfil this requirement, two networks that reach different minima of the loss function cannot both be calibrated. However, the requirement that a network be calibrated is separate from that of finding the optimal classifier.

In this paper, it is shown that a far less stringent condition is sufficient for the network to be calibrated; we say that the network is optimal with respect to calibration or calibration-optimal provided no adjustment of the output of the network in the output space can improve the calibration (see definition LABEL:def:calibration-optimal). This is a far simpler problem, since it requires that a function between far smaller-dimensional spaces should be optimal.

Achieving optimality with respect to calibration can be achieved in either of two ways:

by addition of extra layers at the end of the network and post-hoc training on a hold-out calibration set to minimize the cross-entropy cost function. The extra layers (which we call $g$
-layers) can be added before or after the usual softmax layer. Since the output space of the network is of small dimension (compared to the input of the whole network), optimization of the loss by training the
$g$ -layers is a far easier task;
by training the network with the small $g$ -layers in place from the beginning, on the training set. Since the $g$ -layers are small, and near the end of the network, the expectation is that they will become trained far more easily and more quickly than the rest of the network. A final training of the $g$ -layers alone on a small hold-out calibration set should also be applied.

We conduct experiments on various image classification datasets by learning a small fully-connected network for the $g$ -layers on a hold-out calibration set and evaluate on an unseen test set. Our experiments confirm the theory that if the calibration set and the test set are statistically similar, our method outperforms existing post-hoc calibration methods while retaining the original accuracy.

2 Preliminaries

We consider a pair of joint random variables,

(X, K)

. Random variable

X

should take values in some domain

D_{X}

for instance a set of images, and

K

takes values in a finite class set

K = {1, 2, \dots, n}

. The variable

n

will refer always to the number of labels, and

k

denotes an element of the class set.

We shall be concerned with a (measurable) function $f : D_{X} \to D_{Z} = I R^{m}$ , and random variable $Z$ defined by $Z = f (X)$ . Note that $D_{Z}$ is the same as $I R^{m}$ , but we shall usually use the notation $D_{Z}$ to remind us that it is the range of function $f$ . The distribution of the random variable $X$ induces the distribution for the random variable $Z = f (X)$ . The symbol $z$ will always represent $f (x)$ where $x$ is a value of random variable $X$ . The notation $x \sim X$ means that $x$ is a value sampled from the random variable $X$ . The situation we have in mind is that $f$ is the function implemented by a (convolutional) neural network. A notation $P (\cdot)$ (with upper-case $P$ ) always refers to probability, whereas a lower case $p$

represents a probability distribution. We use the notation

P (k | z)

for brevity to mean

P (K = k | Z = z)

A common way of doing classification, given $n$ classes, is that the neural net is terminated with a layer represented by a function $q : I R^{m} \to I R^{n}$ (where typically $m = n$ , but this is not required), taking value $q (z) = (q_{1} (z), \dots, q_{n} (z))$ in $I R^{n}$ , and satifying $q_{k} (z) > 0$ and $\sum_{k = 1}^{n} q_{k} (z) = 1$ . The set of such vectors $q (z)$ satisfying these conditions is called the standard probability simplex, $Δ^{n - 1}$ , or simply the standard (open) simplex. This is an $n - 1$ dimensional subset of $I R^{n}$ . An example of such a function $q$ is the softmax function defined by $q_{k} (z) = exp (z_{k}) / \sum_{j = 1}^{n} exp (z_{j})$ .

Thus, the function implemented by a neural net is $q \circ f$ , where $f : D_{X} \to D_{Z} = I R^{m}$ , and $q : I R^{m} \to Δ^{n - 1}$ . The function $q$ will be called the activation in this paper. The notation $q \circ f$ represents the composition of the two functions $f$ and $q$ . One is tempted to declare (or hope) that $q_{k} (z) = P (k | z)$ , in other words that the neural network outputs the correct conditional class probabilities given the network output. At least it is assumed that the most probable class assignment is equal to $\operatornamewithlimitsargmaxk∈Kqk(z)$ . It will be investigated how justified these assumptions are. Clearly, since $f$ can be any function, this is not going to be true in general.

2.1 Loss

Given a pair $(x, k) \in D_{X} \times K$ , the negative-log loss is given by $L (q \circ f, x, k) = - log (q_{k} (f (x))) .$ The expected loss over the distribution given by the random variables $(X, K)$ is

\begin{matrix} L (q \circ f, X, K) & = E_{(x, k) \sim (X, K)} L (q \circ f, x, k) = - E_{(x, k) \sim (X, K)} log (q_{k} (f (x))) . \end{matrix}

(1)

We cannot know the complete distribution of the random variables $(X, K)$ in a real situation, however, if the distributions are represented by data pairs $D = {(x_{i}, k_{i})_{i = 1}^{N}}$ sampled from the distribution of $(X, K)$ , then the expected loss is approximated by the empirical loss

L (q \circ f, D)

\approx - E_{(x, k) \sim D} log (q_{k} (f (x)) = - N \sum i = 1 log (q_{k_{i}} (f (x_{i})) .

(2)

The training process of the neural network is intended to find the function $f^{*}$ that minimizes the loss. Thus $f∗=\operatornamewithlimitsargminf:DX→IRmL(q∘f,D) .$

3 Calibration

According to the theory (see bishop1994mixture), if the trained network is $f∗=\operatornamewithlimitsargminf:DX→Δn−1L(f,X,K)$ , where $L (f, X, K)$ is the loss given in (1) then the network (function $f^{*}$ ) is calibrated, in the sense that $f_{k}^{*} (x) = P (k | x)$ , as stated in the following theorem.

Consider joint random variables $(X, K)$ , taking values in $D_{X}$ and $K$ respectively, where $D_{X}$ is some Cartesian space. Let $f : D_{X} \to Δ^{n - 1}$ be a function. Define the loss $L (f, X, K) = - E_{(x, k) \sim (X, K)} log (f_{k} (x))$ . If $f=\operatornamewithlimitsargmin^f:DX→Δn−1L(^f,X,K)$ then $P (k | x) = f_{k} (x) .$

This theorem is a simple corollary of the following slight generalization, which is proved in the supplementary material. (The theorem is stated in terms of a function $q$ , rather than $f$ for convenience later.)

Post-hoc Calibration of Neural Networks

Authors

Post-hoc Uncertainty Calibration for Domain Drift Scenarios

Classifier Calibration: How to assess and improve predicted class probabilities: a survey

Meta-Cal: Well-controlled Post-hoc Calibration by Ranking

Heterogeneous Calibration: A post-hoc model-agnostic framework for improved generalization

Quantile Regularization: Towards Implicit Calibration of Regression Models

Post-hoc loss-calibration for Bayesian neural networks

The Logic Traps in Evaluating Post-hoc Interpretations

1 Introduction

2 Preliminaries

2.1 Loss

3 Calibration

Post-hoc Calibration of Neural Networks

Authors

Related Research

Post-hoc Uncertainty Calibration for Domain Drift Scenarios

Classifier Calibration: How to assess and improve predicted class probabilities: a survey

Meta-Cal: Well-controlled Post-hoc Calibration by Ranking

Heterogeneous Calibration: A post-hoc model-agnostic framework for improved generalization

Quantile Regularization: Towards Implicit Calibration of Regression Models

Post-hoc loss-calibration for Bayesian neural networks

The Logic Traps in Evaluating Post-hoc Interpretations

This week in AI

1 Introduction

2 Preliminaries

2.1 Loss

3 Calibration