Background Invariance Testing According to Semantic Proximity

by   Zukang Liao, et al.

In many applications, machine learned (ML) models are required to hold some invariance qualities, such as rotation, size, intensity, and background invariance. Unlike many types of variance, the variants of background scenes cannot be ordered easily, which makes it difficult to analyze the robustness and biases of the models concerned. In this work, we present a technical solution for ordering background scenes according to their semantic proximity to a target image that contains a foreground object being tested. We make use of the results of object recognition as the semantic description of each image, and construct an ontology for storing knowledge about relationships among different objects using association analysis. This ontology enables (i) efficient and meaningful search for background scenes of different semantic distances to a target image, (ii) quantitative control of the distribution and sparsity of the sampled background scenes, and (iii) quality assurance using visual representations of invariance testing results (referred to as variance matrices). In this paper, we also report the training of an ML4ML assessor to evaluate the invariance quality of ML models automatically.


page 4

page 5

page 6

page 7


ML4ML: Automated Invariance Testing for Machine Learning Models

In machine learning workflows, determining invariance qualities of a mod...

Zero-shot object prediction using semantic scene knowledge

This work focuses on the semantic relations between scenes and objects f...

What can we learn about CNNs from a large scale controlled object dataset?

Tolerance to image variations (e.g. translation, scale, pose, illuminati...

Instance Segmentation based Semantic Matting for Compositing Applications

Image compositing is a key step in film making and image editing that ai...

Noise or Signal: The Role of Image Backgrounds in Object Recognition

We assess the tendency of state-of-the-art object recognition models to ...

Scalable Object-Oriented Sequential Generative Models

The main limitation of previous approaches to unsupervised sequential ob...

Please sign up or login with your details

Forgot password? Click here to reset