Using Human Perception to Regularize Transfer Learning

11/15/2022
by   Justin Dulay, et al.
0

Recent trends in the machine learning community show that models with fidelity toward human perceptual measurements perform strongly on vision tasks. Likewise, human behavioral measurements have been used to regularize model performance. But can we transfer latent knowledge gained from this across different learning objectives? In this work, we introduce PERCEP-TL (Perceptual Transfer Learning), a methodology for improving transfer learning with the regularization power of psychophysical labels in models. We demonstrate which models are affected the most by perceptual transfer learning and find that models with high behavioral fidelity – including vision transformers – improve the most from this regularization by as much as 1.9% Top@1 accuracy points. These findings suggest that biologically inspired learning agents can benefit from human behavioral measurements as regularizers and psychophysical learned representations can be transferred to independent evaluation tasks.

READ FULL TEXT

page 2

page 13

research
05/31/2019

Augmenting Transfer Learning with Semantic Reasoning

Transfer learning aims at building robust prediction models by transferr...
research
02/08/2023

A Systematic Performance Analysis of Deep Perceptual Loss Networks Breaks Transfer Learning Conventions

Deep perceptual loss is a type of loss function in computer vision that ...
research
07/05/2022

Guiding Machine Perception with Psychophysics

Gustav Fechner's 1860 delineation of psychophysics, the measurement of s...
research
11/30/2020

TransMIA: Membership Inference Attacks Using Transfer Shadow Training

Transfer learning has been widely studied and gained increasing populari...
research
07/11/2020

Enhanced Behavioral Cloning Based self-driving Car Using Transfer Learning

With the growing phase of artificial intelligence and autonomous learnin...
research
09/21/2022

Modeling Perceptual Loudness of Piano Tone: Theory and Applications

The relationship between perceptual loudness and physical attributes of ...
research
12/16/2019

Prospect Theory for Human-Centric Communications

Entering the 5G/6G era, the core concept of human-centric communications...

Please sign up or login with your details

Forgot password? Click here to reset