Categorical Perception: A Groundwork for Deep Learning

12/10/2020
by   Laurent Bonnasse-Gahot, et al.
0

Classification is one of the major tasks that deep learning is successfully tackling. Categorization is also a fundamental cognitive ability. A well-known perceptual consequence of categorization in humans and other animals, called categorical perception, is characterized by a within-category compression and a between-category separation: two items, close in input space, are perceived closer if they belong to the same category than if they belong to different categories. Elaborating on experimental and theoretical results in cognitive science, here we study categorical effects in artificial neural networks. Our formal and numerical analysis provides insights into the geometry of the neural representation in deep layers, with expansion of space near category boundaries and contraction far from category boundaries. We investigate categorical representation by using two complementary approaches: one mimics experiments in psychophysics and cognitive neuroscience by means of morphed continua between stimuli of different categories, while the other introduces a categoricality index that quantifies the separability of the classes at the population level (a given layer in the neural network). We show on both shallow and deep neural networks that category learning automatically induces categorical perception. We further show that the deeper a layer, the stronger the categorical effects. An important outcome of our analysis is to provide a coherent and unifying view of the efficacy of different heuristic practices of the dropout regularization technique. Our views, which find echoes in the neuroscience literature, insist on the differential role of noise as a function of the level of representation and in the course of learning: noise injected in the hidden layers gets structured according to the organization of the categories, more variability being allowed within a category than across classes.

READ FULL TEXT
research
05/11/2018

Learning-induced categorical perception in a neural network model

In human cognition, the expansion of perceived between-category distance...
research
05/19/2018

Capturing human category representations by sampling in deep feature spaces

Understanding how people represent categories is a core problem in cogni...
research
05/08/2022

Dynamic categories, dynamic operads: From deep learning to prediction markets

Natural organized systems adapt to internal and external pressures and t...
research
03/13/2023

Cartesian Coherent Differential Categories

We extend to general Cartesian categories the idea of Coherent Different...
research
03/27/2023

Measuring Categorical Perception in Color-Coded Scatterplots

Scatterplots commonly use color to encode categorical data. However, as ...
research
12/22/2021

MC-DGCNN: A Novel DNN Architecture for Multi-Category Point Set Classification

Point set classification aims to build a representation learning model t...
research
12/18/2022

A Layered Architecture for Universal Causality

We propose a layered hierarchical architecture called UCLA (Universal Ca...

Please sign up or login with your details

Forgot password? Click here to reset