Self-Supervised Convolutional Visual Prompts

03/01/2023

∙

by   Yun-Yun Tsai, et al.

∙

0

∙

share

Machine learning models often fail on out-of-distribution (OOD) samples. Visual prompts emerge as a light-weight adaptation method in input space for large-scale vision models. Existing vision prompts optimize a high-dimensional additive vector and require labeled data on training. However, we find this paradigm fails on test-time adaptation when labeled data is unavailable, where the high-dimensional visual prompt overfits to the self-supervised objective. We present convolutional visual prompts for test-time adaptation without labels. Our convolutional prompt is structured and requires fewer trainable parameters (less than 1 experiments on a wide variety of OOD recognition tasks show that our approach is effective, improving robustness by up to 5.87 model architectures.