1 Introduction
The correlation of biological and noisesolicited CMOS artificial spiking neurons between small distinct populations of neurons is modulated and compared by controlling the number of connections and of synapses. The exploration of neuronal microcircuits buldyrev2000description ; jones2000microcolumns ; haeusler2007statistical ; haeusler2009motif ; rinkus2010cortical ; markram2011innate ; potjans2014cell ; opris2014prefrontal ; wang2016brain as Lego building blocks at the basis of complex architectures is not only supported by experimental evidence sadovsky2014mouse ; ocker2015self but also substantiated by mathematical models xie2016brain .
The study of biological neurons at few neurons scale represents an unavoidable intermediate step towards the comprehension and the control of building blocks of efficiently engineered artificial neural networks. Minimal circuits of two neurons bengtsson2013cross , including key ingredients like activation with noise zeldenrust2013modulation have been recently explored.
The major role of noise as neural computation resource has been recognized long ago in both biological burton1992event ; lukashin1996modeling ; nozaki1999effects ; natschlager2005dynamics ; brascamp2006time ; ecker2014there ; orlandi2013noise ; vukovic2015robust ; audhkhasi2016noise and artificial hinton1983optimal ; ackley1985learning ; lizeth2012impact ; prati2016noise ; prati2016atomic neural networks. Experiments on a random network of spiking neurons show that a significant amount of knowledge is stored stochastically inside the nervous system habenschuss2013stochastic and applications of noise in the learning process inside a spiking neural network (SNN) have been discussed maass2014noise .
Networks of stochastic spiking neurons have been used for solving constraint satisfaction problems where noise is believed to play a key role in the problem solving ability of the human brain maass2015spike ; jonke2016solving ; kappel2015network ; mostafa2015event . For instance, the individual activity of neurons are correlated in the isolated regions of auditory cortex to which the neuron belongs nir2007coupling . The correlation among distinct groups of neurons (islands for brevity from now on) are directly proportional to the number of interneuron connections and interneuronal correlation. Furthermore, synchronization among neuronal populations in motor cortical regions where they communicate through interneurons, may play an important role in cognitive motor processes riehle1997spike . Propagation of spiking activity amongst different neuronal modules has been reviewed in kumar2010spiking Refinements of a natural communication stimulus is permitted by the correlated neural activity in fish metzen2016neural . A mechanism by which a neural circuits effectively shape their signal and noise in concert has been demonstrated, enabling the minimization of the corruption of signal by noise and by enhancing the speed of information transmission zylberberg2016direction .
Previously, we separately explored the island sizedependence of synchronized activity in living neuronal networks yamamoto2016size and the Gaussian whitenoise to enhance the transmission of spiking activity along a linear chain of artificial neurons based on discrete components, and its interspike interval (ISI) by varying the amplitude of the noise prati2016noise .
CMOS artificial neurons cassidy2013design ; sharifipoor2012analog ; yasukawa2016real represent the ideal platform to emulate biologically plausible neurons operating in speed mode and implementing universal functionalities speculated for Lego block microcircuits and build engineered artificial neural networks.
Here we exploit the effects of electronic noise in a network of simulated CMOS artificial spiking neurons in order to reverse engineer the features of a SNN of real neurons attached to a silicon patch. Figure 1 graphically represents an overview of our research framework. With the goal of observing the change of correlation rate of spiking activity of neurons among different islands of neurons by adding interconnection neurons, we performed experiments with real in vitro neurons, as the first part of this research. Spiking activity of four islands of randomly connected spiking neurons with sufficiently interconnected neurons has been investigated. We observed that by increasing the number of connections between islands the correlation of spiking activities dramatically rises up. In the second part, we aim to the exploitation of a nature inspired physical effect such an electric noise (by adopting the same network topology of the first part) on silicon neurons. It is therefore divided as follows: we initially describe the network elements (a CMOS neuron operating in speed mode, both excitatory and inhibitory silicon synapse and two compact noise generators for both white and pink noise), briefly. Such components are combined to build up a network of spiking neuron ready to investigate the effects of noise on it. The noise response of the silicon neuron to different noise spectra is discussed. The silicon neuron microcircuits are utilized to investigate the correlation of noise assisted spiking activity of the four island topology. As key results, we demonstrate how correlation of spiking activity among isolated subnetworks ignited by distinct background noise is progressively increased by adding interconnection elements between neurons of different subnetworks, and more efficiently by adopting multiple synapses ensuring firing to more input neurons, by achieving the artificial microcircuit counterpart of the biological network results.
2 Spontaneous firing activity of group of biological neurons
Biological neurons, which are obtained from animal brains kaech2006culturing or stem cell differentiation shi2012directed , can be cultivated in vitro under a defined physicochemical condition. Neurons are adherent cells, and hence an appropriate scaffold is essential for their growth. A number of microfabrication methods, including semiconductor lithography or soft lithography, has been utilized to engineer the scaffold to control the growth area at singlecell or multicellular scales wheeler2010designing ; aebersold2016brains ; yamamoto2016size ; yamamoto2016unidirectional ; kono2016live .
After approximately a week in culture, neuronal networks begin to form synaptic connections and spontaneously generate bursting activity that is highly synchronized within the network yamamoto2016size ; orlandi2013noise . We previously showed using micropatterned rat cortical neurons that neuronal correlation in the spontaneous bursting activity decreases with the size of the network yamamoto2016size . In this section, we use a similar culture to show that correlation between a group of living neurons can be regulated by altering the number of neurite guidance pathways that bridge neuron islands.
The materials and methods used in fabricating micropatterned substrates, culturing primary neurons, and recording spontaneous neural activity have been described previously yamamoto2016size ; kono2016live . Briefly, and methoxy (polyethleneoxy)propyltrimethoxysilane were used as cellpermissive and nonpermissive layers, respectively, and were patterned on a glass coverslip using electronbeam lithography. The micropatterns used in the current experiment consisted of a set of four square islands of and wide lines that interconnect the island. Three geometries with different number of interconnecting lines were compared, i.e., nobond and triplebond structures which have zero and three lines between a pair of islands.
Primary neurons were obtained from embryonic rat cortices, plated on the micropatterned coverslips, and cultured for 10 days. Then, the cells were loaded with the fluorescence calcium indicator Fluo4 AM (Molecular Probes), and their spontaneous activity was measured by calcium imaging. Images were obtained every for 6 or 9 min using an inverted microscope (Nikon Eclipse TE300) equipped with a objective lens (numerical aperture, 0.75) and a cooledCCD camera (Hamamatsu OrcaER). The image sequences were later analyzed offline using the ImageJ (NIH) and customwritten Perl programs yamamoto2016size .
Representative phasecontrast micrographs of living neuronal networks grown on the micropatterned substrate are shown in Figure 1B and 1C . Apparently, neurons adhered and grew neurites only on the permissive domains. Very little nonspecific adhesion or neurite growth was observed on the nonpermissive area. The number of cells in each network was counted from the images and were evaluated to be 148, 143, and 154 cells for the nobond, singlebond, and triplebond structures, respectively.
Next, spontaneous neural activity was measured by fluorescence calcium imaging, and the effect of changing the number of interconnections between neuronal islands was analyzed by evaluating the Pearson correlation coefficient. Correlation coefficient of neuron pairs A and B, is calculated using the Pearson correlation coefficient equation described in Appendix section. The correlation coefficient matrix is calculated for three different situations and it is reported in Figure 2. In the nobond structure, the correlation of the synchronized bursting activity was high only for neurons belonging to the same island and were otherwise nearly zero. By connecting the islands of biological neurons by an individual connection, the correlation among different islands is switched on. Finally, in the triplebond structure, the correlation is high across nearly all neuron pairs.
The results indicate that the neural correlation can be controlled using micropatterned substrates by changing the number of interconnections. This is in agreement with a previous work which showed that the likelihood of an activity being transferred between two neuronal populations increased with the number of interconnecting microtunnels in a microfluidic device pan2015vitro . Detailed statistical analysis of the structuredependent modulation in the activity pattern is in progress and will be reported elsewhere soon.
The spontaneous activity in biological neuronal networks is triggered by spontaneous release of either synaptic vesicles or ion channel stochasticity which temporally induce fluctuation of the neuronal membrane potential. It is interesting from an engineering point of view that the up to 80% of the metabolic energy consumed in the brain seems to be used in maintaining the spontaneous activity raichle2006brain . Their possible role in neural computation has been suggested to include learning burton1992event , event directionality lukashin1996modeling , stochastic resonance nozaki1999effects , management of binocular rivalry brascamp2006time , and coherence orlandi2013noise , just to mention some examples, but full understanding of the functional role of such an energyconsuming background activity awaits further research.
3 Exploiting injected noise in networks of silicon neurons
In this section we simulate the implementation of network of CMOS spiking neurons and the effects of injecting noise. Noise is naturally present in CMOS devices, from white and pink () noise nemirovsky20011 , to telegraph noise prati2007microwave , to even noise prati2016band . Therefore, the most straightforward strategy consists of amplifying such natural noise and to exploit it in the circuit by injection in the selected nodes. Selected topology of the network is inspired by the experiments performed on biological neurons. HudgkinHuxley (HH) spiking neurons hodgkin1952quantitative are designed and simulated in a 0.35 CMOS technology sarpeshkar1992refractory . Figure 2(a) depicts the structure of the silicon neuron designed in subthreshold regime where the sodium and potassium conductance channels are modeled. Neurons are designed to be able to spike three orders of magnitude faster than real neurons. Speed mode allows fast data processing more importantly it takes advantage of scaling of the size of the circuit. The silicon neurons are mutually connected by means of CMOS excitatory and inhibitory synapses (see Figure 2(b) and 2(c) respectively). With the aim of applying noise to SNNs, a voltage to current converter (Figure 2(d)) has been developed in order to deliver a zero mean white and pink current noise to the neurons. In the following, such basic elements of the network are discussed.
Fastmode (HH) Silicon Neuron. Circuit of Figure 2(b) realizes a sodiumpotassium conductance based model of spiking neurons biased in subthreshold regime. Capacitor stands for the membrane capacitance. When an input current stimulates the capacitance, it starts to be charged and consequently membrane potential, increases. Therefore transistor M1 turns on and sinks current . The current then is mirrored to the transistor M4 and creates the sodium activation current, which builds up the upswing of the action potential. Simultaneously, is also copied to the transistor M5 and as a result of the so current , the capacitor get charged and activates the transistor M6, ergo it sinks the current . Therefore, is discharged and drops to its resting potential. Threshold of activation of the sodium conductance is set by . It controls the threshold of firing of the neuron. Pulse width of the spike is controlled by through . Refractory period of the neuron is set by set by the voltage activating the transistor M7. The neuron is designed such that it fires three orders of magnitude faster than the biological neurons. In such way, respecting the working principles of a real neuron, one can acquire more spikes in shorter simulation times and enables fast processing. The size of a single CMOS neuron is therefore reduced to , in contrast to previous implementations such as in serrano2006neuromorphic (more examples inindiveri2011neuromorphic ).
Excitatory Silicon Synapse. Biological synapses are modeled by an exponentially decaying time course, having different time constants for different types of synapses destexhe1998kinetic . Therefore, a firstorder differential equation of the type:
(1) 
where stands for the output current, and represents the input stimulus to the synapse, can be implemented in order to model synaptic transmission destexhe1998kinetic . We use a currentmode lowpass filter in order to model the synapses. We design the synapse to be compatible with the silicon neuron. Therefore the circuit has much greater bandwidth in comparison to that of biological synapses. A differentialpair Integrator (DPI) circuit chicca2014neuromorphic shown in Figure 2(b) is designed as an excitatory synapse. Voltagespikes from the presynaptic neurons arrive at the input transistor convert to a current, . By applying translinear principle in which, in the log domain circuits, the sum of transistor voltages can be replaced by the multiplication of their currents gilbert1996translinear , and by writing the currentvoltage relationship of the capacitor , the dynamics of the circuit can be presented as:
(2) 
where the time constant is in which represents the synapse capacitor, is the subthreshold slope factor and stands for the thermal voltage. Note that by adjusting the value of the synapse capacitor, , and one can control the time constant of the synapse.
Inhibitory Silicon Synapse. The inhibitory synapse consists of complementary version of the excitatory synapse shown in Figure 2(b). Figure 2(c) represents the inhibitory synapse circuit. Working principle of such circuit is similar to that of excitatory synapse. All nMOS transistors are replaced with pMOSs and vice versa. At the input of the inhibitory synapse, a CMOS inverter circuit consisting of the transistors indicated as QP0 and QN0, is used to invert the upcoming voltagespikes from the presynaptic neuron. Eventually, an inhibiting current pulse is generated at the output of the circuit once the synapse get stimulated from a presynaptic neuron.
CMOS Voltage to Current Converter. CMOS VI converter shown in Figure 2(d), is designed in order to inject noise signal of a silicon resistor to the silicon neuron. We use a resistor with larger value in the range of k and amplify the current in order to deliver the required amplitude of the noise. An integrator (negativefeedback inverting structured integrator using opamp UAmp1) together with a capacitor, C2, is utilized for current amplification purpose. If we connect the capacitor, C2, directly to the neuron, the structure will provide a current amplification with the ratio (C2/C1) and it is AC coupled. The main disadvantage is that the output current depends on the input voltage of the neuron therefore the system is not a true current source. Moreover, we need a zeromean thermal noise to be applied to the neuron. As a result, we add the second stage by utilizing a modified Howland current source structure instruments2008comprehensive shown in the red box in Figure 2(d). At the output of a current source, we need a large impedance; by taking size into account, capacitors are optimum choices for implementation of large impedances. Therefore, instead of the resistors in the Howland’s solution we used capacitors with very small values. This implies the need to provide a DC pass for the current in the feedback loop of opamp UAmp2. Note that also in the DC regime, the output impedance must be high enough in order to have a good current source. Therefore we used the channel resistor of unbiased small NMOSs, QM1 and QM2 in order to provide very high value impedance and at the same time occupying small silicon area for delivering a DC pass for the feedback current. Such circuit is able to deliver a tuneable zeromean white or pinknoise to the neuron.
4 Spontaneous spiking activity of an individual silicon neuron stimulated by noise
We now turn to the response of a single silicon neuron to injected input noise to address stochastic nature of the spiking activity, the interspike interval (ISI) distribution of the spiking activity and the relationship between input current noise amplitude and the spiking rate of the neuron, respectively. Two transient simulations with the duration of 50 ms on the silicon neuron are performed using Cadence software and Spectre simulator where Gaussian white and pinknoise are injected to the neuron, respectively. The rms amplitude of the current noise in both cases is equally set to 1.5 . Such noise value causes the neuron to generate adequate spike rate in order to perform proper ISI and intertrain intervals (ITI) analysis. As a result, one can explore the features of the responses of the neuron to different noise spectrum signals and compare how the interspike intervals ISI and ITI are affected. Figure 4A shows the power spectrum of the current noises employed to carry the simulation tuned so to cover the operating frequency of the silicon neuron (1MHz). Figure 4C depicts the spiking activity of the silicon neuron in both simulations.
ISI distribution of the spiking activity of a neuron helps quantifying the electrical properties of the neuron such as its refractory period, meanfiring rate and randomness of the spiking pattern, and therefore of the network stein2005neuronal . Figure 4D describes how the number of spike events varies upon increase of the amplitude of the input noise. Normally, the silicon neuron fires spikes with an amplitude of 2.5V when a deterministic current stimulates it. For detection of a spike, a threshold voltage above which the counter counts one spike has been determined. Figure 4D includes the trend of the number of detected spikes versus injected input noise for different spike detection thresholds. Counting stabilizes for detection threshold voltage of 1 V. For a threshold less than 1V, random fluctuations would be considered as spike events which lead to faulty information. Figure 4E shows the ISI histogram for the 50 ms transient simulation of the silicon neuron, by applying a zeromean Gaussian whitenoise to its input. The simulation is repeated for 4 different amplitudes of the injected input whitenoise ranging from 350 to 600 . By increasing the amplitude of the noise the histogram is denser and shifted to the shorter time intervals. The vertical line shows the natural period of spike activity of the neuron in case of a constant input current, with amplitude equivalent to the noise rms values injected in the simulations. A high number of spike events lay on the left side of their corresponding no–noise simulation results. Notice that the cross point of each histogram with its corresponding no–noise condition happens approximately at the same spike rate. ISI and ITI of the spiking activities of the silicon neuron between the case of white and pinknoise stimuli are compared in Figure 4E and 4F. The flicker noise makes a longer tail on the histogram than the whitenoise in agreement with other reports sobie2011neuron . Furthermore, the histogram in case of whitenoise is much denser than the one in the case of pinknoise. This implies that whitenoise stimulus increases the excitability of a neuron within its natural refractory period with a higher rate in comparison to the pinknoise stimulus. In addition to analysis on the ISI distributions, we also plot the ITI histograms to see how different sources of noise affect the propagation of a train of spike events. For both white and pinknoise stimuli, the distribution of spiking activity is approximately concurrent. For the simulation of the correlation among the islands discussed in the next section, we adopt white noise who proves more effective to induce excitation of the neurons from the ISI point of view.
5 Controlling the correlation of spiking activity of neuronal islands by tuning the interconnections
With the aim of emulating and reverse engineering the test system topology constituted of four neuronal islands of biological neurons, we designed and simulated a network of spiking neurons comprising 64 silicon neurons and 1024 DPI synapses including a background whitenoise. The case of an individual island comprising 16 silicon neurons and 256 synapses is first described. As anticipated, individual islands are stimulated with whitenoise. Next, the increase of correlation of spiking activity of the four islands of neurons is controlled by tuning the number of the equivalent of interneurons connecting distinct neuronal islands, and the number of synapses excited by the same output interneuron. The observations are quantitatively accounted for by employing correlation coefficient matrix of stochastic events (Appendix for methods).
5.1 Simulation of Islands of Silicon Neurons with Internal LowDensity Connectivity
Figure 5A represents a network of 16 silicon neurons and 256 reconfigurable synapses. In order to be consistent with networks of typical hybrid systems of real neurons on artificial solid state structures, we adopted a ratio between inhibitory and excitatory synapses in the range 5% to 10%. In our case such ratio is set to 8%. The output of each neuron is connected to 16 synapses presented in a row in order to be able to connect to other 15 neurons and itself. The output of a neuron is connected directly to the input of each synapse. The output of the synapse is connected to the input nodes of the next neurons through a nMOS switch. By choosing whether the switch is closed or open, one can connect arbitrarily any neuron to the others. A Gaussian whitenoise is applied through the VI converter to each neuron. A single source of noise (noise of an onchip resistor) distributes the noise to all neurons of an island in order to emulate local potential fluctuations affecting similarly all the neurons within the same neuronal island. Each island is solicited by a distinct noise generator, to emulate distinct regions with no common local potential fluctuations.
A random network topology (Figure 5C) is configured on the 16neuron network. We define four islands of such 16neuron SNN as it is shown in Figure 5B. Each island of neurons consist of different network topologies. In each island, 92% of the synapses are excitatory and 8% inhibitory which emulates a reasonable average ratio typical of realistic biological networks of spiking neurons.
In Figure 5B, circles and hexagons represent neurons and synapses respectively. The green synapses are activated. For instance, in the row number 4 (R4) of the Island 1 (top left), the 10th synapse is green, which indicates that neuron N4 stimulates neuron N10 by means of that synapse.
In the first island (top left) we implement a lowdensity connection in a random topology; on the contrary, in the second island (top right), we implement more connections between neurons, in order to demonstrate the influence of the variation of number of connections of each island on the correlation of the spiking activities.
We adopt the same random network topology on the third (bottom left) and forth (bottom right) islands, with the purpose of observing the variation of spiking activity of neurons when we inject uncorrelated sources of noise to the same network topology. As said, all neurons in each island are connected to a single source of noise.
Our aim is to characterize the influence of spike trains generated by distinct regions and to observe how the correlation among spiking activity is created thanks to the propagation of the signaling on the surrounding network of the receiving neuron. The correlation propagates thanks to equivalent of interneurons and synchronization of spiking activity is rapidly achieved when few interconnective synapses are employed.
In order to investigate the variation of correlation between the spiking activities in different islands we perform simulations by using the network represented in Figure 5B.
In the first configuration, the islands of neurons are not mutually connected. By applying uncorrelated currentmode Gaussian whitenoise with a power of 200 , to the islands for a simulation time of 240 μs, we obtain the spiking activity of the neurons represented in Figure 5D. As a result, we obviously observe highly correlated firing activity only from neurons of the same island.
In the second simulated configuration, islands of neurons are connected in a ring topology through interconnected synapses, such that the island 1 is connected to the island 2 by means of 8 interneurons via unidirectional synapses, in the same way, the island 2 is connected to island 4 , the 4 to the 3, and the 3 to the island 1. In such ring topology we achieved the spiking activity of the neurons represented in Figure 5E, which looks correlated.
In order to assess quantitatively the correlation, we explore the correlation coefficient matrix. Figure 6 illustrates the correlation coefficient matrices of spiking activity of islands of silicon neurons in the mentioned simulations. In matrices, the correlation spectrum colorbar represents uncorrelated regions with blue and highly correlated regions with dark red.
When the islands are not connected (Figure 6A), as a result of applying the same noise to individual islands and of their mutual connections, selfspiking activity of islands is highly correlated while, like in the case of biological neurons, neuron activity of different islands is not correlated.
When islands are connected to each other (Figure 6B) by means of interconnective synapses, activity of four islands coincides and therefore we observe universal highly correlated regions within the networks. We noticed that, in order to observe similar correlation to the biological case with only 1 connectons, 8 neurontoneuron connections are needed, suggesting that each neurite connects to more neurons within the same island. The condition of multiple synapses associated to the same neuron is developed in the next Section.
5.2 Simulation of Islands of neurons with multiple synapses
In order to achieve higher correlation between the island by adding only few neuron connections, like in the biological network of four islands, the simulated systems is modified in order to make the microconnectome of the artificial interneurons more effective.
Figure 7A to 7D, represents the correlation matrix of the activity of the neurons in cases of isolated islands, which refer to the connections represented in 7E to 7H. The case AE () provides the background of the correlation when there are no connection. The case BF () represents a limiting case when one neuron solicits all the neurons in the next islands. It is worth noting that even if all the neurons of the next island are solicited, the correlation does not saturate among distinct islands. The case CG with three connections of kind () shows much stronger correlation even if the synapses to input neurons on the next islands are cut of a half, thanks to the triple excitation connection. Such condition provides a result which is closer to the 3N connection of biological islands, namely a good correlation with only three connections, if compared to the individual pointtopoint connections discussed previously. Finally the case DH of three () connected neurons represents again a limiting case where all the neurons of the next islands are fed, which exhibits even higher (but not saturated despite the connection to all the neurons of the next island) correlation. The lack of saturation may be explained by the strong hypothesis set by injecting uncorrelated noise to the four islands, which in turn may become a limiting boundary condition to the system, and does not account for independent potential fluctuations of the interneurons. Such findings suggest that in the biological islands the spontaneous connections of the interneurons with the neurons of the islands, in spite of the process and the methods, are very likely and very efficient. As a finel remark about the level of smaller correlation among the islands of the silicon network compared to biological network, there is some effect also ascribed to smaller populations of the silicon islands, and to higher degree of connections between interneurons and the islands.
6 Conclusions
To conclude, we compared a network of four randomly connected biological and silicon islands, equipped of additional interneurons to enable communication between distinct islands, and therefore push spontaneous firing into higher correlation. We proposed an original and robust implementation of artificial silicon neurons by designing compact and low power consuming electronic CMOS microcircuits, including both pink and white noise generators. We employed noise to activate the spontaneous firing activity of the neurons and we characterized the statistics of the response of individual neurons. Finally, the simulations of a ring of four noise–activated silicon islands and its comparison with its twin made of real neurons enabled us to reverse engineer the nature of the connections formed between the real neurons of the biological system. The results suggest that several synapses of each interneuron employed to connect distinct islands are required to grant the high correlation experimentally observed.
Acknowledgments
E.P. acknowledges JSPS Fellowship, the Hokkaido University and the Short Term Mobility Program 2016 of CNR.
Appendix
Correlation coefficient matrix calculation
We aim to mathematically characterize the influence of spike trains generated by the neurons of each island and observe how the correlation among spiking activity is created. For this purpose we use Pearson correlation coefficient of two random variables which is a linear measure of their dependence
stuart1968advanced . If each of the variables has N samples, the Pearson correlation coefficient is defined as:(3) 
where and
are the mean and standard deviation of
, respectively, and and are the mean and standard deviation of . Correlation coefficient can be written in terms of covariance of and :(4) 
The correlation coefficient matrix of two random variables for all pairs of variables as the following:
(5) 
and are directly correlated to themselves thus the diagonal entries of the matrix are 1. The matrix is symmetric such that
(6) 
Since spiking activities of the neurons inside the 64 neuron network are random variables we can simply build their correlation coefficient matrix in order to find their dependence, quantitatively. For calculation of the correlation coefficient matrix of the spiking activity of neurons we utilize MATLAB function corrcoef(X). It returns the matrix of correlation coefficients for X, where the columns of X represent random variables and the rows represent the samples.
References
 (1) S. Buldyrev, L. Cruz, T. GomezIsla, E. GomezTortosa, S. Havlin, R. Le, H. Stanley, B. Urbanc, B. Hyman, Description of microcolumnar ensembles in association cortex and their disruption in alzheimer and lewy body dementias, Proceedings of the National Academy of Sciences 97 (10) (2000) 5039–5043.
 (2) E. G. Jones, Microcolumns in the cerebral cortex, Proceedings of the National Academy of Sciences 97 (10) (2000) 5019–5021.
 (3) S. Haeusler, W. Maass, A statistical analysis of informationprocessing properties of laminaspecific cortical microcircuit models, Cerebral cortex 17 (1) (2007) 149–162.
 (4) S. Haeusler, K. Schuch, W. Maass, Motif distribution, dynamical properties, and computational performance of two databased cortical microcircuit templates, Journal of PhysiologyParis 103 (1) (2009) 73–87.
 (5) G. J. Rinkus, A cortical sparse distributed coding model linking miniand macrocolumnscale functionality, Frontiers in neuroanatomy 4 (17).
 (6) H. Markram, R. Perin, Innate neural assemblies for lego memory, Frontiers in neural circuits 5 (2011) 6.
 (7) T. C. Potjans, M. Diesmann, The celltype specific cortical microcircuit: relating structure and activity in a fullscale spiking network model, Cerebral Cortex 24 (3) (2014) 785–806.
 (8) I. Opris, M. F. Casanova, Prefrontal cortical minicolumn: from executive control to disrupted cognitive processing, Brain 137 (7) (2014) 1863–1875.
 (9) X.J. Wang, H. Kennedy, Brain structure and dynamics across scales: in search of rules, Current opinion in neurobiology 37 (2016) 92–98.
 (10) A. J. Sadovsky, J. N. MacLean, Mouse visual neocortex supports multiple stereotyped patterns of microcircuit activity, The Journal of Neuroscience 34 (23) (2014) 7769–7777.
 (11) G. K. Ocker, A. LitwinKumar, B. Doiron, Selforganization of microcircuits in networks of spiking neurons with plastic synapses, PLoS Comput Biol 11 (8) (2015) e1004458.

(12)
K. Xie, G. E. Fox, J. Liu, C. Lyu, J. C. Lee, H. Kuang, S. Jacobs, M. Li,
T. Liu, S. Song, J. Z. Tsien,
Brain
computation is organized via poweroftwobased permutation logic, Frontiers
in Systems Neuroscience 10 (2016) 95.
doi:10.3389/fnsys.2016.00095.
URL http://journal.frontiersin.org/article/10.3389/fnsys.2016.00095  (13) F. Bengtsson, P. Geborek, H. Jörntell, Crosscorrelations between pairs of neurons in cerebellar cortex in vivo, Neural Networks 47 (2013) 88–94.
 (14) F. Zeldenrust, W. J. Wadman, Modulation of spike and burst rate in a minimal neuronal circuit with feedforward inhibition, Neural Networks 40 (2013) 1–17.
 (15) R. M. Burton, G. J. Mpitsos, Eventdependent control of noise enhances learning in neural networks, Neural Networks 5 (4) (1992) 627–637.

(16)
A. V. Lukashin, G. L. Wilcox, A. P. Georgopoulos, Modeling of directional operations in the motor cortex: a noisy network of spiking neurons is trained to generate a neuralvector trajectory, Neural Networks 9 (3) (1996) 397–410.
 (17) D. Nozaki, D. J. Mar, P. Grigg, J. J. Collins, Effects of colored noise on stochastic resonance in sensory neurons, Physical Review Letters 82 (11) (1999) 2402.
 (18) T. Natschläger, W. Maass, Dynamics of information and emergent computation in generic neural microcircuit models, Neural Networks 18 (10) (2005) 1301–1308.
 (19) J. W. Brascamp, R. Van Ee, A. J. Noest, R. H. Jacobs, A. V. van den Berg, The time course of binocular rivalry reveals a fundamental role of noise, Journal of vision 6 (11) (2006) 8–8.
 (20) A. S. Ecker, A. S. Tolias, Is there signal in the noise?, Nature 201 (2014) 4.
 (21) J. G. Orlandi, J. Soriano, E. AlvarezLacalle, S. Teller, J. Casademunt, Noise focusing and the emergence of coherent activity in neuronal cultures, Nature Physics 9 (9) (2013) 582–590.
 (22) N. Vuković, Z. Miljković, Robust sequential learning of feedforward neural networks in the presence of heavytailed noise, Neural Networks 63 (2015) 31–47.

(23)
K. Audhkhasi, O. Osoba, B. Kosko, Noiseenhanced convolutional neural networks, Neural Networks 78 (2016) 15–23.

(24)
G. E. Hinton, T. J. Sejnowski, Optimal perceptual inference, in: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, Citeseer, 1983, pp. 448–453.

(25)
D. H. Ackley, G. E. Hinton, T. J. Sejnowski, A learning algorithm for boltzmann machines, Cognitive science 9 (1) (1985) 147–169.
 (26) G.C. Lizeth, T. Asai, M. Motomura, Impact of noise on spike transmission through serially connected electrical fitzhughnagumo circuits with subthreshold and suprathreshold interconductances, Journal of Signal Processing 16 (6) (2012) 503–509.
 (27) E. Prati, E. Giussani, G. Ferrari, T. Asai, Noiseassisted transmission of spikes in maeda–makino artificial neuron arrays, International Journal of Parallel, Emergent and Distributed Systems (2016) 1–9.
 (28) E. Prati, Atomic scale nanoelectronics for quantum neuromorphic devices: comparing different materials, International Journal of Nanotechnology 13 (7) (2016) 509–523.
 (29) S. Habenschuss, Z. Jonke, W. Maass, Stochastic computations in cortical microcircuit models, PLoS Comput Biol 9 (11) (2013) e1003311.
 (30) W. Maass, Noise as a resource for computation and learning in networks of spiking neurons, Proceedings of the IEEE 102 (5) (2014) 860–880.
 (31) P. A. Merolla, J. V. Arthur, R. AlvarezIcaza, A. S. Cassidy, J. Sawada, F. Akopyan, B. L. Jackson, N. Imam, C. Guo, Y. Nakamura, et al., A million spikingneuron integrated circuit with a scalable communication network and interface, Science 345 (6197) (2014) 668–673.
 (32) W. Maass, To spike or not to spike: That is the question, Proceedings of the IEEE 103 (12) (2015) 2219–2224.
 (33) Z. Jonke, S. Habenschuss, W. Maass, Solving constraint satisfaction problems with networks of spiking neurons, Frontiers in Neuroscience 10.

(34)
D. Kappel, S. Habenschuss, R. Legenstein, W. Maass, Network plasticity as bayesian inference, PLoS Comput Biol 11 (11) (2015) e1004485.
 (35) H. Mostafa, L. K. Müller, G. Indiveri, An eventbased architecture for solving constraint satisfaction problems, Nature communications 6.
 (36) Y. Nir, L. Fisch, R. Mukamel, H. GelbardSagiv, A. Arieli, I. Fried, R. Malach, Coupling between neuronal firing rate, gamma lfp, and bold fmri is related to interneuronal correlations, Current Biology 17 (15) (2007) 1275–1285.
 (37) A. Riehle, S. Grün, M. Diesmann, A. Aertsen, Spike synchronization and rate modulation differentially involved in motor cortical function, Science 278 (5345) (1997) 1950–1953.
 (38) A. Kumar, S. Rotter, A. Aertsen, Spiking activity propagation in neuronal networks: reconciling different perspectives on neural coding, Nature reviews neuroscience 11 (9) (2010) 615–627.
 (39) M. G. Metzen, V. Hofmann, M. J. Chacron, Neural correlations enable invariant coding and perception of natural stimuli in weakly electric fish, eLife 5 (2016) e12993.
 (40) J. Zylberberg, J. Cafaro, M. H. Turner, E. SheaBrown, F. Rieke, Directionselective circuits shape noise to ensure a precise population code, Neuron 89 (2) (2016) 369–383.
 (41) H. Yamamoto, S. Kubota, Y. Chida, M. Morita, S. Moriya, H. Akima, S. Sato, A. HiranoIwata, T. Tanii, M. Niwano, Sizedependent regulation of synchronized activity in living neuronal networks, Physical Review E 94 (1) (2016) 012407.
 (42) A. S. Cassidy, J. Georgiou, A. G. Andreou, Design of silicon brains in the nanocmos era: Spiking neurons, learning synapses and neural architecture optimization, Neural Networks 45 (2013) 4–26.
 (43) O. Sharifipoor, A. Ahmadi, An analog implementation of biologically plausible neurons using CCII building blocks, Neural Networks 36 (2012) 129–135.
 (44) S. Yasukawa, H. Okuno, K. Ishii, T. Yagi, Realtime object tracking based on scaleinvariant features employing bioinspired hardware, Neural Networks 81 (2016) 29–38.
 (45) S. Kaech, G. Banker, Culturing hippocampal neurons, Nature protocols 1 (5) (2006) 2406–2415.
 (46) Y. Shi, P. Kirwan, F. J. Livesey, Directed differentiation of human pluripotent stem cells to cerebral cortex neurons and neural networks, Nature protocols 7 (10) (2012) 1836–1846.
 (47) B. C. Wheeler, G. J. Brewer, Designing neural networks in culture, Proceedings of the IEEE 98 (3) (2010) 398–406.
 (48) M. J. Aebersold, H. Dermutz, C. Forró, S. Weydert, G. ThompsonSteckel, J. Vörös, L. Demkó, “brains on a chip”: Towards engineered neural networks, TrAC Trends in Analytical Chemistry 78 (2016) 60–69.
 (49) H. Yamamoto, R. Matsumura, H. Takaoki, S. Katsurabayashi, A. HiranoIwata, M. Niwano, Unidirectional signal propagation in primary neurons micropatterned at a singlecell resolution, Applied Physics Letters 109 (4) (2016) 043703.
 (50) S. Kono, H. Yamamoto, T. Kushida, A. HiranoIwata, M. Niwano, T. Tanii, Livecell, labelfree identification of gabaergic and nongabaergic neurons in primary cortical cultures using micropatterned surface, PloS one 11 (8) (2016) e0160987.
 (51) L. Pan, S. Alagapan, E. Franca, S. S. Leondopulos, T. B. DeMarse, G. J. Brewer, B. C. Wheeler, An in vitro method to manipulate the direction and functional strength between neural populations, Frontiers in neural circuits 9.
 (52) M. E. Raichle, The brain’s dark energy, Science 314 (5803) (2006) 1249.
 (53) Y. Nemirovsky, I. Brouk, C. G. Jakobson, 1/f noise in cmos transistors for analog applications, IEEE transactions on Electron Devices 48 (5) (2001) 921–927.
 (54) E. Prati, M. Fanciulli, A. Calderoni, G. Ferrari, M. Sampietro, Microwave irradiation effects on random telegraph signal in a mosfet, Physics Letters A 370 (5) (2007) 491–493.
 (55) E. Prati, K. Kumagai, M. Hori, T. Shinada, Band transport across a chain of dopant sites in silicon over micron distances and high temperatures, Scientific reports 6.
 (56) A. L. Hodgkin, A. F. Huxley, A quantitative description of membrane current and its application to conduction and excitation in nerve, The Journal of physiology 117 (4) (1952) 500.
 (57) R. Sarpeshkar, L. Watts, C. Mead, Refractory neuron circuits.
 (58) E. Chicca, F. Stefanini, C. Bartolozzi, G. Indiveri, Neuromorphic electronic circuits for building autonomous cognitive systems, Proceedings of the IEEE 102 (9) (2014) 1367–1388.
 (59) T. Instruments, A comprehensive study of the howland current pump, AN1515 A.
 (60) R. SerranoGotarredona, T. SerranoGotarredona, A. AcostaJiménez, B. LinaresBarranco, A neuromorphic corticallayer microchip for spikebased event processing vision systems, IEEE Transactions on Circuits and Systems I: Regular Papers 53 (12) (2006) 2548–2566.
 (61) G. Indiveri, B. LinaresBarranco, T. J. Hamilton, A. Van Schaik, R. EtienneCummings, T. Delbruck, S.C. Liu, P. Dudek, P. Häfliger, S. Renaud, et al., Neuromorphic silicon neuron circuits, Frontiers in neuroscience 5 (2011) 73.
 (62) A. Destexhe, Z. F. Mainen, T. J. Sejnowski, Kinetic models of synaptic transmission, Methods in neuronal modeling 2 (1998) 1–25.
 (63) B. Gilbert, Translinear circuits: an historical overview, Analog Integrated Circuits and Signal Processing 9 (2) (1996) 95–118.
 (64) R. B. Stein, E. R. Gossen, K. E. Jones, Neuronal variability: noise or part of the signal?, Nature Reviews Neuroscience 6 (5) (2005) 389–397.
 (65) C. Sobie, A. Babul, R. de Sousa, Neuron dynamics in the presence of 1/f noise, Physical Review E 83 (5) (2011) 051912.
 (66) A. Stuart, M. G. Kendall, et al., The advanced theory of statistics, Charles Griffin, 1968.
Comments
There are no comments yet.