Achievable Rates for Pattern Recognition

09/08/2005
by   M. Brandon Westover, et al.
0

Biological and machine pattern recognition systems face a common challenge: Given sensory data about an unknown object, classify the object by comparing the sensory data with a library of internal representations stored in memory. In many cases of interest, the number of patterns to be discriminated and the richness of the raw data force recognition systems to internally represent memory and sensory information in a compressed format. However, these representations must preserve enough information to accommodate the variability and complexity of the environment, or else recognition will be unreliable. Thus, there is an intrinsic tradeoff between the amount of resources devoted to data representation and the complexity of the environment in which a recognition system may reliably operate. In this paper we describe a general mathematical model for pattern recognition systems subject to resource constraints, and show how the aforementioned resource-complexity tradeoff can be characterized in terms of three rates related to number of bits available for representing memory and sensory data, and the number of patterns populating a given statistical environment. We prove single-letter information theoretic bounds governing the achievable rates, and illustrate the theory by analyzing the elementary cases where the pattern data is either binary or Gaussian.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/16/2000

Oracle Complexity and Nontransitivity in Pattern Recognition

Different mathematical models of recognition processes are known. In the...
research
12/20/2007

Pattern Recognition System Design with Linear Encoding for Discrete Patterns

In this paper, designs and analyses of compressive recognition systems a...
research
01/10/2013

Application of Hopfield Network to Saccades

Human eye movement mechanisms (saccades) are very useful for scene analy...
research
04/07/2016

A robust autoassociative memory with coupled networks of Kuramoto-type oscillators

Uncertain recognition success, unfavorable scaling of connection complex...
research
11/28/2019

Neural networks with redundant representation: detecting the undetectable

We consider a three-layer Sejnowski machine and show that features learn...
research
09/22/2006

Problem Evolution: A new approach to problem solving systems

In this paper we present a novel tool to evaluate problem solving system...
research
05/10/2010

On Macroscopic Complexity and Perceptual Coding

The theoretical limits of 'lossy' data compression algorithms are consid...

Please sign up or login with your details

Forgot password? Click here to reset