Distributed Representations

What is Distributed Representation?

Distributed representation describes the same data features across multiple scalable and interdependent layers. Each layer defines the information with the same level of accuracy, but adjusted for the level of scale. These layers are learned concurrently but in a non-linear fashion. This mimics human logic in a neural network, since each concept can be accessed by more than one neuron firing and each neuron can represent more than one concept.

Distributed vs Local Representation

In non-distributed or local representation, each possible value has a unique representation slot, which requires a lot of memory to process a large database than the distributed approach. For example, if analyzing the features of new automobile sales in the US during a given year, you would need a large databank to track all the relevant details. Whereas with the distributed approach, you could store all that data with just a few memory units:

Vehicle class (1= Large SUV, 0.1 = Compact, etc…), Brand, Price, Location, etc…