On the Hardness of Robustness Transfer: A Perspective from Rademacher Complexity over Symmetric Difference Hypothesis Space

by   Yuyang Deng, et al.

Recent studies demonstrated that the adversarially robust learning under ℓ_∞ attack is harder to generalize to different domains than standard domain adaptation. How to transfer robustness across different domains has been a key question in domain adaptation field. To investigate the fundamental difficulty behind adversarially robust domain adaptation (or robustness transfer), we propose to analyze a key complexity measure that controls the cross-domain generalization: the adversarial Rademacher complexity over symmetric difference hypothesis space ℋΔℋ. For linear models, we show that adversarial version of this complexity is always greater than the non-adversarial one, which reveals the intrinsic hardness of adversarially robust domain adaptation. We also establish upper bounds on this complexity measure. Then we extend them to the ReLU neural network class by upper bounding the adversarial Rademacher complexity in the binary classification setting. Finally, even though the robust domain adaptation is provably harder, we do find positive relation between robust learning and standard domain adaptation. We explain how adversarial training helps domain adaptation in terms of standard risk. We believe our results initiate the study of the generalization theory of adversarially robust domain adaptation, and could shed lights on distributed adversarially robust learning from heterogeneous sources, e.g., federated learning scenario.


page 1

page 2

page 3

page 4


Robustified Domain Adaptation

Unsupervised domain adaptation (UDA) is widely used to transfer a model ...

Predicting the Success of Domain Adaptation in Text Similarity

Transfer learning methods, and in particular domain adaptation, help exp...

On Localized Discrepancy for Domain Adaptation

We propose the discrepancy-based generalization theories for unsupervise...

Generalization Bounds for Representative Domain Adaptation

In this paper, we propose a novel framework to analyze the theoretical p...

Tackling Partial Domain Adaptation with Self-Supervision

Domain adaptation approaches have shown promising results in reducing th...

Adversarial Bipartite Graph Learning for Video Domain Adaptation

Domain adaptation techniques, which focus on adapting models between dis...

Domain Adaptation with L2 constraints for classifying images from different endoscope systems

This paper proposes a method for domain adaptation that extends the maxi...

Please sign up or login with your details

Forgot password? Click here to reset