Thermodynamics of Encoding and Encoders

11/12/2021
by   Yang Tian, et al.
0

Non-isolated systems have diverse coupling relations with the external environment. These relations generate complex thermodynamics and information transmission between the system and its environment. The framework depicted in the current research attempts to glance at the critical role of the internal orders inside the non-isolated system in shaping the information thermodynamics coupling. We characterize the coupling as a generalized encoding process, where the system acts as an information thermodynamics encoder to encode the external information based on thermodynamics. We formalize the encoding process in the context of the nonequilibrium second law of thermodynamics, revealing an intrinsic difference in information thermodynamics characteristics between information thermodynamics encoders with and without internal correlations. During the information encoding process of an external source 𝖸, specific sub-systems in an encoder 𝖷 with internal correlations can exceed the information thermodynamics bound on (𝖷,𝖸) and encode more information than system 𝖷 works as a whole. We computationally verify this theoretical finding in an Ising model with a random external field and a neural data set of the human brain during visual perception and recognition. Our analysis demonstrates that the stronger internal correlation inside these systems implies a higher possibility for specific sub-systems to encode more information than the global one. These findings may suggest a new perspective in studying information thermodynamics in diverse physical and biological systems.

READ FULL TEXT

page 4

page 5

page 7

page 10

page 17

page 22

page 23

page 24

research
12/08/2022

Adaptive Neural Network Backstepping Control Method for Aerial Manipulator Based on Variable Inertia Parameter Modeling

For the aerial manipulator that performs aerial work tasks, the actual o...
research
02/18/2016

Encoding Data for HTM Systems

Hierarchical Temporal Memory (HTM) is a biologically inspired machine in...
research
12/13/2021

Differential EEG Characteristics during Working Memory Encoding and Re-encoding

Many studies have discussed the difference in brain activity related to ...
research
05/28/2023

Plug-and-Play Document Modules for Pre-trained Models

Large-scale pre-trained models (PTMs) have been widely used in document-...
research
02/10/2022

Decomposing neural networks as mappings of correlation functions

Understanding the functional principles of information processing in dee...
research
03/16/2022

CUE Vectors: Modular Training of Language Models Conditioned on Diverse Contextual Signals

We propose a framework to modularize the training of neural language mod...
research
05/19/2023

Artificial intelligence moral agent as Adam Smith's impartial spectator

Adam Smith developed a version of moral philosophy where better decision...

Please sign up or login with your details

Forgot password? Click here to reset