EXTERN: Leveraging Endo-Temporal Regularization for Black-box Video Domain Adaptation

08/10/2022
by   Yuecong Xu, et al.
0

To enable video models to be applied seamlessly across video tasks in different environments, various Video Unsupervised Domain Adaptation (VUDA) methods have been proposed to improve the robustness and transferability of video models. Despite improvements made in model robustness, these VUDA methods require access to both source data and source model parameters for adaptation, raising serious data privacy and model portability issues. To cope with the above concerns, this paper firstly formulates Black-box Video Domain Adaptation (BVDA) as a more realistic yet challenging scenario where the source video model is provided only as a black-box predictor. While a few methods for Black-box Domain Adaptation (BDA) are proposed in image domain, these methods cannot apply to video domain since video modality has more complicated temporal features that are harder to align. To address BVDA, we propose a novel Endo and eXo-TEmporal Regularized Network (EXTERN) by applying mask-to-mix strategies and video-tailored regularizations: endo-temporal regularization and exo-temporal regularization, performed across both clip and temporal features, while distilling knowledge from the predictions obtained from the black-box predictor. Empirical results demonstrate the state-of-the-art performance of EXTERN across various cross-domain closed-set and partial-set action recognition benchmarks, which even surpassed most existing video domain adaptation methods with source data accessibility.

READ FULL TEXT
research
03/09/2022

Learning Temporal Consistency for Source-Free Video Domain Adaptation

Video-based Unsupervised Domain Adaptation (VUDA) methods improve the ro...
research
03/18/2023

Augmenting and Aligning Snippets for Few-Shot Video Domain Adaptation

For video models to be transferred and applied seamlessly across video t...
research
08/25/2023

Black-box Unsupervised Domain Adaptation with Bi-directional Atkinson-Shiffrin Memory

Black-box unsupervised domain adaptation (UDA) learns with source predic...
research
03/07/2023

Bootstrap The Original Latent: Learning a Private Model from a Black-box Model

In this paper, considering the balance of data/model privacy of model ow...
research
07/30/2022

Simplex Clustering via sBeta with Applications to Online Adjustment of Black-Box Predictions

We explore clustering the softmax predictions of deep neural networks an...
research
08/02/2023

Curriculum Guided Domain Adaptation in the Dark

Addressing the rising concerns of privacy and security, domain adaptatio...
research
08/17/2023

The Unreasonable Effectiveness of Large Language-Vision Models for Source-free Video Domain Adaptation

Source-Free Video Unsupervised Domain Adaptation (SFVUDA) methods consis...

Please sign up or login with your details

Forgot password? Click here to reset