Visual Knowledge Tracing

07/20/2022
by   Neehar Kondapaneni, et al.
15

Each year, thousands of people learn new visual categorization tasks – radiologists learn to recognize tumors, birdwatchers learn to distinguish similar species, and crowd workers learn how to annotate valuable data for applications like autonomous driving. As humans learn, their brain updates the visual features it extracts and attend to, which ultimately informs their final classification decisions. In this work, we propose a novel task of tracing the evolving classification behavior of human learners as they engage in challenging visual classification tasks. We propose models that jointly extract the visual features used by learners as well as predicting the classification functions they utilize. We collect three challenging new datasets from real human learners in order to evaluate the performance of different visual knowledge tracing methods. Our results show that our recurrent models are able to predict the classification behavior of human learners on three challenging medical image and species identification tasks.

READ FULL TEXT
research
09/01/2021

Atoms of recognition in human and computer vision

Discovering the visual features and representations used by the brain to...
research
01/07/2022

Incremental Knowledge Tracing from Multiple Schools

Knowledge tracing is the task of predicting a learner's future performan...
research
11/28/2022

Pitfalls of Conditional Batch Normalization for Contextual Multi-Modal Learning

Humans have perfected the art of learning from multiple modalities throu...
research
12/05/2018

Visual Attention for Behavioral Cloning in Autonomous Driving

The goal of our work is to use visual attention to enhance autonomous dr...
research
06/18/2022

Attention-based Dynamic Subspace Learners for Medical Image Analysis

Learning similarity is a key aspect in medical image analysis, particula...
research
10/09/2020

Large-scale randomized experiment reveals machine learning helps people learn and remember more effectively

Machine learning has typically focused on developing models and algorith...
research
06/06/2020

Visual Prediction of Priors for Articulated Object Interaction

Exploration in novel settings can be challenging without prior experienc...

Please sign up or login with your details

Forgot password? Click here to reset