Test-time Adaptation vs. Training-time Generalization: A Case Study in Human Instance Segmentation using Keypoints Estimation

by   Kambiz Azarian, et al.

We consider the problem of improving the human instance segmentation mask quality for a given test image using keypoints estimation. We compare two alternative approaches. The first approach is a test-time adaptation (TTA) method, where we allow test-time modification of the segmentation network's weights using a single unlabeled test image. In this approach, we do not assume test-time access to the labeled source dataset. More specifically, our TTA method consists of using the keypoints estimates as pseudo labels and backpropagating them to adjust the backbone weights. The second approach is a training-time generalization (TTG) method, where we permit offline access to the labeled source dataset but not the test-time modification of weights. Furthermore, we do not assume the availability of any images from or knowledge about the target domain. Our TTG method consists of augmenting the backbone features with those generated by the keypoints head and feeding the aggregate vector to the mask head. Through a comprehensive set of ablations, we evaluate both approaches and identify several factors limiting the TTA gains. In particular, we show that in the absence of a significant domain shift, TTA may hurt and TTG show only a small gain in performance, whereas for a large domain shift, TTA gains are smaller and dependent on the heuristics used, while TTG gains are larger and robust to architectural choices.


page 1

page 3

page 4

page 5

page 6

page 8


Unsupervised Adaptation of Semantic Segmentation Models without Source Data

We consider the novel problem of unsupervised domain adaptation of sourc...

STFAR: Improving Object Detection Robustness at Test-Time by Self-Training with Feature Alignment Regularization

Domain adaptation helps generalizing object detection models to target d...

SITA: Single Image Test-time Adaptation

In Test-time Adaptation (TTA), given a model trained on some source data...

Learning Instance-Specific Adaptation for Cross-Domain Segmentation

We propose a test-time adaptation method for cross-domain image segmenta...

Unseen Object Instance Segmentation with Fully Test-time RGB-D Embeddings Adaptation

Segmenting unseen objects is a crucial ability for the robot since it ma...

CD-TTA: Compound Domain Test-time Adaptation for Semantic Segmentation

Test-time adaptation (TTA) has attracted significant attention due to it...

How Shift Equivariance Impacts Metric Learning for Instance Segmentation

Metric learning has received conflicting assessments concerning its suit...

Please sign up or login with your details

Forgot password? Click here to reset