Self-Supervised Convolutional Visual Prompts

03/01/2023
by   Yun-Yun Tsai, et al.
0

Machine learning models often fail on out-of-distribution (OOD) samples. Visual prompts emerge as a light-weight adaptation method in input space for large-scale vision models. Existing vision prompts optimize a high-dimensional additive vector and require labeled data on training. However, we find this paradigm fails on test-time adaptation when labeled data is unavailable, where the high-dimensional visual prompt overfits to the self-supervised objective. We present convolutional visual prompts for test-time adaptation without labels. Our convolutional prompt is structured and requires fewer trainable parameters (less than 1 experiments on a wide variety of OOD recognition tasks show that our approach is effective, improving robustness by up to 5.87 model architectures.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset