Biological Mechanisms for Learning: A Computational Model of Olfactory Learning in the Manduca sexta Moth, with Applications to Neural Nets

02/08/2018
by   Charles B. Delahunt, et al.
University of Washington
0

The insect olfactory system, which includes the antennal lobe (AL), mushroom body (MB), and ancillary structures, is a relatively simple neural system capable of learning. Its structural features, which are widespread in biological neural systems, process olfactory stimuli through a cascade of networks where large dimension shifts occur from stage to stage and where sparsity and randomness play a critical role in coding. Learning is partly enabled by a neuromodulatory reward mechanism of octopamine stimulation of the AL, whose increased activity induces rewiring of the MB through Hebbian plasticity. Enforced sparsity in the MB focuses Hebbian growth on neurons that are the most important for the representation of the learned odor. Based upon current biophysical knowledge, we have constructed an end-to-end computational model of the Manduca sexta moth olfactory system which includes the interaction of the AL and MB under octopamine stimulation. Our model is able to robustly learn new odors, and our simulations of integrate-and-fire neurons match the statistical features of in-vivo firing rate data. From a biological perspective, the model provides a valuable tool for examining the role of neuromodulators, like octopamine, in learning, and gives insight into critical interactions between sparsity, Hebbian growth, and stimulation during learning. Our simulations also inform predictions about structural details of the olfactory system that are not currently well-characterized. From a machine learning perspective, the model yields bio-inspired mechanisms that are potentially useful in constructing neural nets for rapid learning from very few samples. These mechanisms include high-noise layers, sparse layers as noise filters, and a biologically-plausible optimization method to train the network based on octopamine stimulation, sparse layers, and Hebbian growth.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 4

02/15/2018

Putting a bug in ML: The moth olfactory network learns to read MNIST

We seek to (i) characterize the learning architectures exploited in biol...
08/23/2018

Insect cyborgs: Biological feature generators improve machine learning accuracy on limited data

Despite many successes, machine learning (ML) methods such as neural net...
10/15/2021

Evolving spiking neuron cellular automata and networks to emulate in vitro neuronal activity

Neuro-inspired models and systems have great potential for applications ...
09/27/2019

BEAN: Interpretable Representation Learning with Biologically-Enhanced Artificial Neuronal Assembly Regularization

Deep neural networks (DNNs) are known for extracting good representation...
10/07/2019

Biologically-Inspired Spatial Neural Networks

We introduce bio-inspired artificial neural networks consisting of neuro...
03/18/2017

Non-Associative Learning Representation in the Nervous System of the Nematode Caenorhabditis elegans

Caenorhabditis elegans (C. elegans) illustrated remarkable behavioral pl...
06/14/2020

Emergent Properties of Foveated Perceptual Systems

We introduce foveated perceptual systems, inspired by human biological s...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction


Learning is a vital function of biological neural networks, yet the underlying biomechanical mechanisms responsible for robust and rapid learning are not well understood. The insect olfactory network, and the moth’s olfactory network (MON) in particular (e.g. the

Manduca sexta moth), is a comparatively simple biological neural network capable of learning [1, 2], and makes an ideal model organism for characterizing the mechanics of learning. It is amenable to interrogation through experimental neural recordings of key, well-understood structural components including the antenna lobe (AL) [3] and mushroom body (MB) [4]. In addition, the AL-MB contain many structural motifs that are widespread in biological neural systems. These motifs include: (i) the use of neuromodulators (octopamine and dopamine) in learning [5], (ii) a cascading networks structure [6], (iii) large changes in dimensionality (i.e. numbers of neurons) between networks [7], (iv) sparse encodings of data in high-dimensional networks [8], (v) random connections [9], and (vi) the presence of noisy signals [10]. Bio-inspired design principles suggest that each of the features has high value to the olfactory system. The mechanism of octopamine/dopamine release during learning is of particular interest, since it is not well-understood how this stimulation promotes the construction of new sparse codes in the MB.

In this work, we build a computational model of the MON that is closely aligned with both the known biophysics of the moth AL-MB and in vivo neural firing rate data, and that includes the dynamics of octopamine stimulation. There exist computational neural network models of the insect MB [11, 12, 13, 14, 15] inspired by but not closely tied to particular organisms (indeed, [11]

points out the advantages of this more general approach). Our goal here is to analyze the interactions of AL, MB, and octopamine during associative learning. To this end, we model the architecture and neural dynamics of the whole system: AL, MB, octopamine stimulation, synaptic growth, odor projection onto the AL, and an extrinsic (readout) neuron downstream. We model neurons as “integrate-and-fire” units

[16]. In addition, we tether the model closely to the known biophysics of the M. sexta moth. This allows us to calibrate the model’s behavior to in vivo neural firing rate data from moth AL neurons, collected during odor and octopamine stimulation.

We thus create a full, end-to-end neural network model (hereafter “Network Model”) that demonstrates robust learning behavior while also tightly matching the structure and behavior of a real biological system. This approach has three advantages: (i) we can meaningfully compare Network Model simulation output to experimental data in order to tune model parameters, (ii) simulation results can map back to the original biological system in order to render meaningful insights, and (iii) each structural and dynamical feature of the system can demonstrate its relevance to the task of learning in a neural net. In silico experiments with the Network Model allow us to abstract out critical features in the moth’s toolkit, study how these features enable learning in a neural network, and thus derive bio-inspired insight into the mathematical framework that enables rapid and robust learning. Specifically, our experiments elucidate mechanisms for fast learning from noisy data that rely on cascaded networks, sparsity, and Hebbian plasticity.

These mechanisms have potential applications to engineered neural nets. NNs have emerged as a dominant mathematical paradigm for characterizing neural processing and learning. This is not surprising given that they were inspired by the Nobel prize winning work of Hubel and Wiesel on the primary visual cortex of cats [17]. These seminal experiments showed that networks of neurons were organized in hierarchical layers of cells for processing visual stimulus. The first mathematical model of a neural network, the Neocognitron in 1980 [18], had many of the characteristic features of today’s deep neural networks (DNNs). However, many of the biological motifs listed above are largely absent from engineered NNs, whose principles and building blocks are biologically implausible even as DNNs have achieved great success [19, 20]

. For example, the AL-MB interaction with octopamine and Hebbian plasticity operates in a fundamentally different manner than the backpropagation optimization used in DNNs. We seek to characterize a set of biological elements, a “biological toolkit”, that can be assembled into NNs that operate on fundamentally different principles than standard NNs and that are capable of rapid and robust learning from very few training samples, an ability common in biological NNs that remains challenging for today’s DNNs.

A brief summary of the AL-MB network: It is organized as a feed-forward cascade of five distinct networks, as well as a reward mechanism [21, 22]. Roughly 30,000 noisy chemical receptor neurons (RNs) detect odor and send signals to the Antenna Lobe (AL) [6]. The AL acts as a pre-amplifier, providing gain control and sharpening of odor representations [23]. It contains roughly 60 isolated units (glomeruli), each focused on a single odor stimuli feature [21]. Glomeruli laterally inhibit each other, and project odor codes to the Mushroom Body (MB). AL neurons are noisy [10]. The MB contains about 4000 Kenyon Cells (KCs). These fire sparsely and encode odor signatures as memories [24, 4, 8]. MB sparsity is enforced by global inhibition from the Lateral Horn [25]. Extrinsic Neurons (ENs), numbering 10’s, are believed to be “readout neurons” that interpret the KC codes [26, 27]. In response to reward (sugar at the proboscis), a large neuron sprays octopamine globally over the AL and MB, causing generalized stimulation of neurons. Learning does not occur without this octopamine input [28, 1]. The connections into the KCs (ALKCs) and out of the KCs (KCsENs) are plastic during learning [29, 6]. Figure 1 gives a system schematic (A) along with typical firing rate (FR) timecourses (from simulation) for neurons in each network (B). More network details are given in Methods.

Figure 1: AL-MB overview. A: System schematic: Chemical sensors (RNs) excite a noisy pre-amp network (AL), which feeds forward to a plastic sparse memory layer (MB), which excites readout (decision) neurons (ENs). Green lines show excitatory connections, red lines show inhibitory connections (LH inhibition of the MB is global). Light blue ovals show plastic synaptic connections into and out of the MB.
B: Neuron timecourse outputs from each network (typical simulation) with time axes aligned vertically. Timecourses are aligned horizontally with their regions-of-origin in the schematic. The AL timecourse shows all responses within 2.5 std dev of mean spontaneous rate as medium blue. Responses outside this envelope are yellow-red (excited) or dark blue (inhibited). MB responses are shown as binary (active/silent). Timecourse events are as follows: (1) A period of no stimulus. All regions are silent. (2) Two odor stimuli are delivered, 3 stimulations each. AL, MB, and ENs display odor-specific responses. (3) A period of control octopamine, ie without odor or Hebbian training. AL response is varied, MB and EN are silent. (4) The system is trained (octopamine injected) on the first odor. All regions respond strongly. (5) A period of no stimulus. All regions are silent, as in (1). (6) The stimuli are re-applied. The AL returns to its pre-trained activity since it is not plastic. In contrast, the MB and EN are now more responsive to the trained odor, while response to the untrained odor is unchanged. Green dotted line in the EN represents a hypothetical “action” threshold. The moth has learned to respond to the trained odor.

2 Results


We first show the calibration of our Network Model to in vivo data. We then describe neural behaviors of the Network Model during learning and give results of learning experiments. Finally, we give results of experiments on MB sparsity.

2.1 Calibration of Model


The Network Model was calibrated to behave in a statistically similar way to three sets of in vivo data measuring projection neuron (PN) firing rate (FR) activity in the AL (See Methods for details): (i) PN spike counts with odor but without octopamine: 129 units with FR1 spike/sec, (ii) PN spike counts with odor and with octopamine: 180 units with FR1 spike/sec, and (iii) PN spike counts with odor, with and without octopamine: 52 units with FR1 spike/sec. Due to the limited number of experimental units, only qualitative comparisons of the model and experiment could be made. Excessive tuning of the model parameters would have served only to overfit the particular data, rather than matching true PN behavior distributions or, more importantly, the general learning behavior of the moth. Fig 2 shows the close match of typical Network Model PN statistics to in vivo PN statistics based on mean (

) and variance (

) of spontaneous FRs and odor responses, both without and with octopamine (details of metrics are given in Methods). Importantly, Fig 2 shows significant octopamine-modulated increase in PN FR activity in the Network Model, consistent with in vivo experiments involving octopamine stimulation.

There is limited experimental data measuring the FR activity of Kenyon cells (KC) in the MB, and no data to our knowledge measuring KC in response to octopamine stimulation. However, we note that the behavior of KCs during the application of octopamine to the AL, either with or without odor, is not an artifact of parameter tuning. Rather, it follows from the tuning the AL to match in vivo data. Specifically, PN FRs at baseline (with no odor or octopamine), with odor alone, with octopamine alone, and with odor and octopamine, are all determined by calibration of the model to in vivo data. KCs respond only to PNs and to inhibition from the LH (See Fig 1). Calibrating the KC baseline response in the absence of octopamine to in vivo data in [30] fixes the feed-forward connections from PNs. Assumed in this model, due to lack of biophysical evidence, is that octopamine has no direct effect on KC FRs. Thus KC behavior with octopamine is fully determined once the model is tuned to PN data. This completes the calibration process of our model parameters. As Fig 2 shows, the model agrees well with in vivo experiment.

There are no bulk data, to our knowledge, measuring EN firing rates in response to odors and/or octopamine. However, calibrating EN response is not necessary to demonstrate an ability to learn. The key marker is post-training increase in EN response.

Figure 2: In vivo firing rate data and model calibration: Comparison of PN firing rate activity from in vivo data and simulations. A: Histograms and CDFs of in vivo data and simulations. Col a: mean spontaneous FRs . Col b: of spontaneous FRs, a measure of noisiness of a PN. Col c: odor response, measured as distance from in units. Distance implies a strong activation/inhibition. Col d: odor response during octopamine, in units distance from . Note that PN responses are broadened (i.e. more PNs are strongly activated or inhibited). The dotted line in the CDF inset is the same as the CDF of the odor response without octopamine, to show the broadening towards both extremes. Col e: change in mean spontaneous FRs due to octopamine, measured in units distance from (non-octopamine) . Some PNs are excited, some are inhibited.
B: Activity of PNs indexed by increasing spontaneous FR. Blue lines = mean spontaneous FRs (cf col 1). Shaded regions = envelopes (cf col 2). Solid red dots = odor response FRs (cf col 3). Hollow red dots = odor response FRs during octopamine (cf col 4). Red lines show the change in odor response FRs due to octopamine (cf broadened response). Black stars (*) = spontaneous FRs during octopamine (cf col 5). In panel A cols 3, 4, 5, the x-axes are expressed in units of , while in panel B the y-axis measures raw spikes/sec FR.

2.2 Learning Experiments: PN and KC Behaviors


PN activity in the AL, and KC activity in the MB, from typical Network Model simulations are shown in Fig 1 as heatmaps, evolved over the time course of a simulation in which the system was exposed to two different odors and trained on one of them. The AL is stimulated with octopamine during training. Each row of the heatmap represents a distinct PN or KC as it evolves in time (left to right columns of heat map). All the timescales are aligned.

PNs

In the AL heatmap, the light blue region corresponds to PN FRs within 2.5 of their respective mean spontaneous FRs , warm colors correspond to very high FRs, and dark blues correspond to strongly inhibited FRs. The simulations demonstrate a number of key PN behaviors, including (i) absent odor, PN FRs stay within their noise envelopes (by definition), (ii) the two odors have distinct excitation/inhibition signatures on PNs, (iii) octopamine alone (without odor) results in more PNs being excited beyond their usual noise envelopes, and also results in some PNs being inhibited beyond their usual envelopes, (iv) octopamine and odor, applied together, result in an overall excitation of PNs, and (v) the AL behavior returns to baseline after octopamine is withdrawn, since AL connection weights are not plastic. This last observation is in contrast to what occurs in the MB.

KCs

In the MB, the KCs fire sparsely due to global inhibition from the Lateral Horn. The only plastic connections in the AL-MB system involve the KCs: Between PNs (and QNs) and KCs ( in Methods Section); and between KCs and extrinsic readout neurons (ENs) ( in Methods Section). Thus the KC odor signatures are modulated with training. Black regions indicate FRs 1 spike/sec, white regions indicate FRs 1 spike/sec. The white regions have been dilated to make the sparsely-firing KCs easier to see.

The simulations demonstrate a number of key KC behaviors, including (i) the baseline KC FR response absent any odor is essentially zero, (ii) the two odors excite distinct sets of KCs with varying consistency from noise trial to noise trial, (iii) for a given odor, some KCs fire reliably in response to an odor stimulation and some fire only occasionally, (iv) when subject to octopamine but no odor, KCs are unresponsive, a benefit during learning since it prevents environmental noise from being encoded as meaningful, (v) when subject to octopamine and odor, KCs respond strongly to the odor with high trial-to-trial consistency, and (vi) the global inhibition from the LH controls the level of sparseness in the KCs, both their silence absent any odor (with or without octopamine), and their sparse firing in response to odors.

Statistics of KC responses to odors pre-, during, and post-training are shown in Fig 3. Naive moths have low KC response to odors, in both percentage of KCs activated and their consistency of response to odor stimulations (Fig 3, blue dots and curves). During training octopamine induces high KC response, in both percentage and consistency (Fig 3, red dots and curves). After octopamine is withdrawn, KC response is lower than during training, but remains higher than naive levels in both percentage and consistency (Fig 3, green dots and curves) for the trained odor only. Thus the newly-learned importance of the trained odor is encoded as broader and stronger KC responses by means of strengthened synaptic connections.

EN (readout neuron) activity is also shown over time at the bottom of Fig 1. Learning is evidenced by the increased EN response to the trained odor even after octopamine has been withdrawn, due to Hebbian growth of synaptic connections into and out of the MB.

The FR activity of the PNs in the AL, the KCs in the MB, and the ENs, as illustrated in Figs. 1 and 3, demonstrate the entire learning process that occurs under the influence of octopamine stimulation. Without octopamine learning does not occur. Interestingly, although the AL does not itself experience plasticity changes, it is the AL’s increased FR activity (induced by octopamine) which enables permanent rewiring of the MB via Hebbian plastic updates.

Figure 3: KC responses to odor during training: KCs respond sparsely to odor pre- and post-training, ie absent octopamine (blue and green dots and curves). Octopamine induces transient increased responsivity (red dots and curves). Training results in permanent increases in response to the trained odor, but no increase in response to control odor (green dots and curves).
A: KC response to an odor before, during, and after training. x-axis: indexed KCs (500 shown). y-axis: consistency of response (in %). The plots are for odor 1 as the trained odor (ie same data as panel B). Blue = pre-training (no octopamine). Red = during training (with octopamine); note the heightened transient response. Green = post-training (no octopamine). There is a permanent increase in the number of KCs that respond to the trained odor.
B: Response rate vs. percentage of active KCs for trained and control odors before, during, and after training.
x-axis: percentage of KCs responding at the given rate. y-axis: consistency of response (in %). Blue = pre-training. Red = during octopamine (transient). Green = post-training. The LH plot shows odor 1 as the reinforced odor. The scatterplots in (A) correspond to the three curves in this plot. Note that the permanent KC response curve shifts up and to the right (bluegreen) in the trained odor, ie more KCs respond to the odor (right shift) and they respond more consistently (upward shift). The RH plot shows odor 2 as a control. The control’s permanent KC response curve does not shift.
C: As (B) above, but in this experiment odor 1 is now the control (LH plot), and odor 2 is reinforced (RH plot). In this case, the response curve of odor 2 (reinforced) shifts to the right (bluegreen), while the response curve of odor 1 (control) is unchanged.

2.3 Learning Experiments: EN Behavior


A key finding of this paper is that the AL-MB model demonstrates robust learning behavior. Here “learning” is defined as rewiring the system so that the reinforced odor yields a significantly stronger response in the readout neuron (EN) post-training, relative to naive (i.e. pre-training) response to that odor, and also relative to the post-training responses to control odors.

Learning Experiment Structure

Moths were randomly generated from a fixed parameter template, which included randomly-assigned input maps (odorAL) for four odors. The odors projected broadly onto the AL, each odor hitting 20 out of 60 glomeruli. As a result, their projections onto the AL overlapped substantially. On average, a given odor uniquely projected onto about 6 glomeruli, and shared the other 14 glomeruli with other odors. Each generated moth was put through a series of training experiments, with each run in the series structured as follows:

  1. The moth first received a series of stimulations from each odor, to establish a baseline (naive) EN response. The stimulations were 0.2 seconds long and separated by gaps of several seconds.

  2. The moth was trained on one of the odors for 1 to 4 sessions (one session = 5 odor stimulations), by applying odor and octopamine concurrently. The MB plastic weights were updated according to a Hebbian rule.

  3. Upon completion of training, the four odors were each again applied as a series of odor stimulations, to establish post-training EN response.

For each {odor, #sessions} pair, this experiment was conducted 11 times (i.e. 11 noise realizations), for a total of 176 experiments on each moth. These results were aggregated to assess the particular moth’s learning response.

Learning Experiment Results

As a general rule, Network Model moths consistently demonstrated strong learning behavior: Training increased EN response to the trained odor well beyond naive levels, and also much more than it affected EN response to control odors. Fig 4 summarizes the changes in EN responses in a typical experiment on a moth with four odors. Panel A shows a typical noise realization timecourse, where one odor was reinforced with octopamine and the other three odors were controls. Panel B shows the statistics of EN response modulation, according to {odor, #sessions} pairs.

For ease of interpretation, the moth shown in Fig 4 had naive EN responses of roughly equal magnitude for all four odors. When naive EN response magnitudes were highly uneven ( 3x), robust learning still occurred, but the interpretation of the results is more complex due to scaling issues. A typical experiment using a moth with odor responses of highly unequal magnitude is shown in Fig 8.

Points of interest (EN responses to learning)

  1. Because EN response is driven solely by feed-forward signals from KCs, ENs had response 0 in the absence of odor, with or without octopamine, as expected (since KCs are silent absent any odor). Thus Hebbian growth during training did not increase EN baseline (no-odor) response.

  2. The EN response to odor + octopamine was always very strong, as seen in Fig 4 (A), where EN responses to odor + octopamine extend above the top of the figure. Note that this effect follows automatically from the calibration of the Network Model to in vivo data. Its functional value to the moth is addressed in the Discussion.

  3. Training consistently increased the EN response to the reinforced odor much more than EN response to control odors, measured as percentage of naive odor response.

    Since the Network Model did not include a Hebbian decay dynamic (for simplicity, absent clear evidence), this was the key indicator of robust learning. That is, focused learning was expressed by substantially higher increase in EN response to reinforced vs control odors. We assume that an added Hebbian decay term would have knocked smaller increases back, thus returning control odor responses to baseline.

Results of ANOVA analysis for differential effects of training on reinforced vs unreinforced odors shows that when naive odor EN response magnitudes were similar (within 3x of each other) p-values were consistently 0.01. ANOVA analysis results are given in Section 5.3.

Figure 4: Effect of training on EN FRs: A:

Typical timecourse of EN responses from an experiment with a single moth. First, 16 stimulations of each odor were delivered, to establish naive odor responses. Note EN response variability due to noise in the system, especially in the AL. Next, the moth was trained on the first (blue) odor trained over 2 sessions (10 stimulations), by delivering odor and octopamine concurrently This timecourse corresponds to the {odor, #sessions} pair in the first column in panel B, at index 2 on the x-axis. Octopamine was then withdrawn, and the four odors were again delivered in series of stimulations, to establish post-training changes in EN response. The long green line represents a hypothetical trigger threshold, such that EN response

threshold would induce a distinct behavior.
B: EN response changes due to training, aggregated results with 11 noise realizations for each {odor, #sessions} pair. Each column shows results of training a given odor, color coded: blue, purple, red, green. x-axis = number of training sessions.
First row: The y-axis measures percent change in EN FR. The line shows mean percent change. The error bars show 1, 2 stds.
Second row: The y-axis measures percent changes in EN response, relative to the trained odor (ie subtracting the trained odor’s change from all odors). This shows how far each control odor lags behind the trained odor. The line shows mean percent lag. The error bars show 1, 2 stds.

2.4 MB Sparsity Experiments


Projection into a high-dimensional, sparse layer is a common motif in biological neural systems [31, 32]. To explore the role of MB sparsity during learning, we ran Network Model experiments that varied the level of generalized inhibition imposed on the MB (the lateral horn, LH, controls MB sparsity level). Each experiment set a certain level of LH inhibition, then ran simulations (see Methods) that trained moths on one odor with 15 odor stimulations and left one control odor untrained. EN responses to both trained and control odors were recorded, as well as the percentage of KCs active in response to odor.

Too little damping from the LH resulted in a high percentage of KCs being active (low sparsity). This regime gave consistent EN responses to odor. But it also caused EN responses to both control odor and noise to increase significantly during training, reducing the contrast between EN responses to trained and control odors and also increasing spontaneous EN noise.

Too much damping resulted in a very low percentage of KCs being active (high sparsity). This ensured that training gains were focused on the trained odor while EN response to control odors and noise were not boosted. However, in this regime EN responses to all odors, both pre- and post-training, were generally unreliable because too few KCs were activated.

Thus sparseness in the high-dimensional MB fulfilled a vital role in the Network Model’s learning system. LH inhibition of the MB had an optimal regime that balanced opposing demands, viz for reliable odor response and for well-targeted Hebbian growth, such that on one hand EN odor responses were reliable, and on the other hand training gains were focused on the trained odor only. Timecourses illustrating this effect are seen in Fig 5 (A). Fig 5 (B) shows how this trade-off varied with MB sparsity, by plotting two figures-of-merit:

(1)

and

(2)

trained odor, = mean post-training EN response to control odor.

Figure 5: Effects of sparsity on learning and EN reliability Results for a typical experiment on a moth with two odors. A: EN responses timecourses for two odors, at varying levels of KC activation (a, b: 1%. c, d: 5 to 15%. e, f: 20 to 45%. Order of events: 3 stimulations of each odor as baseline, train on first odor (only one session shown), then 3 stimulations each post-training. At very sparse levels (a, b) training is focused but odor response is not reliable. At low sparsity levels (e, f) training is unfocused, boosting EN response to control odor and to background noise. B: Two Figures of Merit (FoMs) plotted against MB sparsity. Low KC activation (high sparsity) correlates with well-focused learning, but low odor response SNR. High KC activation (low sparsity) correlates with poorly-focused learning, but high odor response SNR. The FoMs are each normalized for easier plotting. y-axis: Blue data: , a measure of odor EN response SNR, where = EN odor response. Red data: , a measure of learning focus, where = mean EN post-training response to reinforced odor; = mean EN post-training response to control odor (values are thresholded at 1 for plotting). A high value indicates that increases in EN response due to training were focused on the trained odor; low values indicate that irrelevant signal () was also boosted by training. The points are experimental data, the curves are cubic fits. Vertical green lines indicate the 5 - 15% sparsity region, typical in biological neural systems.

3 Discussion


Our discussion centers around observations and implications of our Network Model experiments, which offer insights into the moth olfactory network and how it learns, including: (i) predictions about aspects of the AL-MB still unclear in the literature, (ii) the role of sparse layers, (iii) the role of octopamine, (iv) the value of randomness, and (v) the value of noise.

In addition, we consider these insights in the context of machine learning algorithms.

3.1 Predictions re details of AL-MB structure


Model simulations enable us to make predictions about some unresolved aspects of the moth’s AL-MB system. Some examples:

Do LNs inhibit PNs and LNs as well as RNs

In the AL, LNs have a net inhibitory effect on PNs [33, 34], but the exact means to this end are not clear. In particular, while LNs are known to inhibit RNs [33], it is less clear whether or to what degree LNs also directly inhibit PNs and LNs. Efforts to calibrate our Network Model to in vivo data indicate that LNs need to inhibit not just RNs, but also (to a lesser degree) LNs and PNs. The model weight strengths for LNRN, LN, and PN are in the ratio of 6:2:1. That LNs would inhibit LNs makes sense when the goal is maximizing PN output of the active glomerulus: By inhibiting the LNs of rival glomeruli, the active glomerulus reduces the amount of inhibition directed at itself. Similarly, that LNs would inhibit PNs makes sense if the goal is to reduce the PN output of rival glomeruli.

Octopamine’s effects on different neuron types

Octopamine increases the responsivity of a neuron to incoming signals. It is unclear how or whether octopamine affects various neuron types (ie RNs, PNs, LNs, KCs). Calibration of the Network Model’s AL behavior, and tuning of KC behavior to enable learning, indicate that octopamine needs to target RNs and LNs, but not PNs, KCs, or ENs. Logical arguments support these findings:

RNs: Because RNs initially receive the odor signal, these are logical neurons to stimulate with octopamine, because it sharpens their response to the exact signature being trained, which in turn sharpen the AL’s output code for that odor.

LNs: LNs have the dual roles of inhibiting rival glomeruli and limiting overall PN output in the AL. For the first role, increased LN response to RNs will tend to sharpen AL response to the trained odor, by accentuating inhibition of rival glomeruli PNs. For the second role, increased LN activity mitigates the risk that increased RN activity (due to octopamine) might blow up the overall PN output of the AL.

PNs: Network Model simulations suggest that PNs should receive little or no octopamine stimulation. While increasing PN responsivity would benefit RN-induced sharpening of the trained odor’s signature, there are three downsides. First, RN input to PNs is intrinsically noisy, so higher PN responsivity amplifies noise as well as signal. Second, since PNs respond to LNs, higher PN activity tends to reduce the impact of LN inhibition, and thus reduces the inhibition-induced sharpening of the AL odor response caused by octopamine. Third, increasing PN responsivity can have an outsize effect on overall PN firing rates, ie it is a “high-gain” knob and therefore risky.

KCs: Network Model simulations indicate that direct octopamine stimulation of KCs greatly reduces sparseness in the MB, which can be disastrous to learning. Thus we expect that octopamine has no, or only slight, direct stimulative effect on KCs.

3.2 Noise filtering role of the sparse, high-dimensional stage


Projection from a dense, low-dimensional coding space (eg the AL) to a sparse, high-dimensional coding space (eg KCs in the MB) is a widespread motif of biological neural systems, with size shifts are routinely on the order of 20x to 100x [31, 35, 32]. The reasons for this pattern are not fully understood. Some proposed reasons include information capacity, long-range brain communication, and reduced training data needs [31], as well as improved discrimination ability [11, 32].

Network Model experiments bring to light another, central, role of sparseness: It acts as a robust noise filter, to protect the Hebbian growth process from amplifying upstream noise to out-of-control levels. Though noise may be useful (or unavoidable) in upstream networks such as the AL, noise that reaches the neurons on both sides of a synaptic connection will be amplified by Hebbian growth during learning, swamping the system’s downstream neurons (eg ENs) with noise. However, the “fire together, wire together” principle of Hebbian learning is an AND gate. Thus it suffices to remove noise from just one of the two connected neurons to prevent synaptic growth. Sparsity does precisely this.

Network Model experiments show that sparseness in the MB ensures that noise does not get amplified during training, so that post-training EN spontaneous firing rates and EN control odor responses remain unchanged. Conversely, when KC response is not sufficiently sparse, any training leads rapidly to noisy EN spontaneous response levels and amplified EN responses to control odor. This implies that the noise filtering induced by MB sparseness is necessary for a workable Hebbian learning mechanism. This finding agrees with [11], where experiments with a computational MB model indicated that sparse MB response gave stronger built-in discriminating ability.

Setting aside the particular demands of Hebbian plasticity, robust noise filtering may be a core function of sparse, high-dimensional stages within any network cascade where noise accumulates due to (beneficial) use in upstream stages.

3.3 Roles of octopamine


The levels of octopamine stimulation in our Network Model were calibrated to in vivo data on PN responses to octopamine. Thus, the simulations give insights into downstream effects of octopamine on plasticity, KC responses, EN responses, and Hebbian learning.

Accelerant

Moths can learn to respond to new odors remarkably quickly, in just a few exposures. Simulations indicates that while Hebbian growth can occur without octopamine, it is so slow that actionable learning, ie in terms of amplified EN responses, does not occur.

This implies that octopamine, through its stimulative effect, acts as a powerful accelerant to learning. Perhaps it is a mechanism that allows the moth to work around intrinsic organic constraints on Hebbian growth of new synapses, constraints which would otherwise restrict the moth to an unacceptably slow learning rate. To the degree that octopamine enabled a moth to learn more quickly, with fewer training samples, it would clearly be highly adaptive.


Active learning

Simulations indicate that octopamine strongly stimulates the EN response to even an unfamiliar odor. Since octopamine is delivered as a reward, this has a beneficial effect in the context of reinforcement learning

[36]

, with the moth as the learning agent. An agent (the moth) can in some cases learn more quickly when it has choice as to the sequence of training samples (Active Learning

[37]).

In particular, when a certain class of training sample is relatively rare, it benefits the agent to actively seek out more samples of that class [38]. Octopamine enforces high EN response to a reinforced odor, ensuring that ENs will consistently exceed their “take action” threshold during training. If the action is to “approach”, the moth is more likely to again encounter the odor, thus reaping the benefits predicted by Active Learning theory. This advantage applies in the context of positively-reinforced odors.

In the case of aversive learning, the high EN responses to unfamiliar but objectionable odors, due to dopamine, would cause the moth to preferentially avoid further examples of the odor. This would slow learning of aversive responses (a drawback), but would also minimize the moth’s exposure to bad odors (danger avoidance, a benefit).

Exploration of optimization space

A limitation of Hebbian growth is that it can only reinforce what already exists. That is, it only strengthens channels that are transmitting signals deemed (by association) relevant to the stimulus being reinforced. Absent a mechanism like octopamine, this constrains growth to channels that are already active. Simulations indicate that octopamine induces much broader activity, both upstream from and within the plastic layer, thus activating new transmitting channels. This allows the system to strengthen, and bring permanently online, synaptic connections that were formerly silent. This expands the solution space the system can explore during learning. This function may be particularly important given the constraint of sparsity placed on odor codes in the MB.

Injury compensation

There is evidence that many forms of injury to neurons result in dropped spikes and thus lower firing rates in response to odors [39]. This injury-induced drop in the signals reaching the ENs could induce behavioral consequences, by lowering EN responses to below key behavioral action thresholds. In this case, the generalized stimulation induced by octopamine might compensate for the reduced signal strength sufficiently to lift the EN responses back above threshold during training. This in turn would allow Hebbian growth to strengthen the synaptic connections to those ENs, such that even the reduced signals from the still-injured upstream network would be sufficient to push EN response above the action threshold. This mechanism (octopamine stimulation plus Hebbian synaptic growth) would allow the moth to regain behavioral function lost due to damage to the upstream neural system.

3.4 The value of randomness


The principle of randomness permeates the moth olfactory network, for example in random neural connection maps [9] and in highly variable performance characteristics of chemical receptors in the antennae [33]. A key result from Network Model experiments is that a biologically-based neural net permeated with the random principle can robustly learn. This is in marked contrast to engineered computers, where poorly-spec’ed components are a liability. Particular benefits of randomness include:

Random KCEN connections

These guarantee that projections onto ENs are incoherent relative to whatever low-dimensional manifold in KC-space, which ensures (by the Johnson-Lindenstrauss lemma) that the EN readouts will preserve distinctions between elements in KC-space [31].

Variable gaba-sensitivity in glomeruli

This increases the range of sensitivity to odor concentration, because some glomeruli simply don’t turn off when a different, stronger odor tries to mask them through lateral inhibition (LNs) [40].

Variable sensitivity of antennae receptors

This gives a natural sensitivity to concentration, as progressively stronger odor will progressively activate the less-sensitive receptors, increasing the total RN input to glomeruli.

Resistance to injury

A randomly-connected network is more robust to damage. When exact connection maps and strengths are not required in the first place, damage has less impact on the fundamental nature of the system.

Most importantly, randomness (of connections and component properties) is evolutionarily cheap, easy, and available. So perhaps the core benefit of randomness to the moth olfactory network is that it works at all.

3.5 The value of noise


Noise in biological neural systems is believed to have value, for example by encoding probability distributions and enabling neural Baysian estimations of posteriors

[41]. In addition, injection of noise while training engineered NNs can improve trained accuracy [42]. Simulations indicate that in the AL-MB, noise has potential benefits, coupled with caveats. Figure 3 shows how noise in the AL adds an extra dimension to MB odor encoding, increasing the granularity of its odor responses.

The MB responds to odors in two ways: (i) by the number of KCs that are responsive, and (ii) by the reliability (eg from 10% to 100%) of their responses. This can be seen in the effect of octopamine on KC odor response, Fig 3 (B). Octopamine boosts MB odor response by increasing the number of active KCs (horizontal shift in response curves) and also by increasing the reliability of responsive KCs (vertical shift in responsivity curves). Both these shifts represent a stronger MB response and translate into stronger EN response.

Taken together, they provide a finer granularity of the response range, versus the binary response of a noise-free system. Looked at another way, the MB response to noisy inputs from the AL is a concrete example of a mechanism used by a neural system to translate the probability distributions encoded by noisy neurons into actionable signals with high dynamic range and granularity.

A caveat is that noise in the AL-MB must be confined to the AL, i.e. upstream from the encoding layer, in order to protect the readout neurons and Hebbian learning mechanism from noise. The system’s success depends on robust noise filtering at the MB layer, via global inhibition from the LH. So the three-stage architecture consisting of:

Noisy pre-amplifier layer Sparse noise-reduction layer Decision layer
is an interdependent system well-suited to nuanced decision-making.

Given a layer (the MB) that effectively protects the decision neurons from upstream noise, the system is also potentially robust to noisy stimuli. In the neural net context, input samples (i.e. inputs to the feature-reading layer) can be thought of as a de facto “first layer” of the neural net. A system that is robust to upstream noise may also be naturally robust to noisy inputs, a further potential advantage of judicially-placed sparse layers.

3.6 Applications to Machine Learning


The model and simulations in this paper characterize key features of the AL-MB system which might usefully be ported to machine learning algorithms. These features include: Generalized stimulation during training; Hebbian growth; sparse layers to control plastic connections and filter noise; and noisy initial layers. Advantages of this biological toolkit include:

Fast learning

Moths can reliably learn a new odor in less than 10 exposures. Biological brains in general can learn given few training samples. This contrasts by orders of magnitude with the voracious demands of DNNs, where assembling sufficient training data can be a serious chokepoint in deployment.

Robustness to noise

The sparse layer in the AL-MB acts as an effective noise filter, protecting the readout neurons from a noisy upstream layer (the AL). Since the system is designed to accommodate upstream noise, it is possible that it can also readily accommodate noisy input samples. NNs have a troublesome property, that input-output score functions are not locally continuous [43]. Biological neural nets seem to avoid this particular fault. The noisy layer sparse layer motif may be one reason for this. It may thus be a useful motif to apply in ML architectures.

Novel training mechanism

Hebbian growth, combined with generalized stimulation via octopamine, is (in the context of ML) a novel mechanism to explore a solution space and train a classifier. In particular, it works on a fundamentally different principle than backpropagation algorithms: It does not minimize a loss function via gradient descent; rather, it selectively strengthens only those connections that are transmitting meaningful signals. The addition of a sparse layer acts to control and focus learning, by leveraging the AND-function nature of Hebbian growth.


Biological plausibility

One characteristic (not criticism) of backprop optimization is that it is biologically implausible, since it requires a neuron to have more than local knowledge of the system. The search for neural network architectures (for example with recurrent connections to transport non-local information) and variants of backprop which are biologically plausible, and which thus might narrow the gap between biological and engineered NNs, is currently an area of interest, especially in the context of DNNs [44]. This paper demonstrates that the triad of octopamine stimulation + Hebbian growth + sparse layers can efficiently train a NN, and is thus a possible candidate to address the biological plausibility gap.

4 Materials and Methods


In this section, we describe the biological moth olfactory network, as well as our Network Model. We also provide a Glossary, and describe the in vivo data used for model calibration.

4.1 Moth olfactory system overview


The parts of the AL-MB implicated in learning are organized as a feed-forward cascade of five distinct networks, as well as a reward mechanism [21, 22]. Figure 1 gives a system schematic along with typical firing rate (FR) timecourses (from simulation) for neurons in each network.

  1. Antennae. Roughly 30,000 noisy chemical receptors detect odor and send signals to the Antenna Lobe [6].

  2. Antenna Lobe (AL). Contains roughly 60 units (glomeruli), each focused on a single odor feature [21]. The AL essentially acts as a pre-amp, boosting faint signals and denoising the antennae inputs [23]. AL neurons are noisy [10].

  3. Lateral Horn (LH). Though not fully understood, one key function is global inhibition of the Mushroom Body to enforce sparseness [25].

  4. Mushroom Body (MB), here synonymous with the Kenyon Cells (KCs). About 4000 KCs are located in the calyx of the Mushroom Body (MB). These fire sparsely and are believed to encode odor signatures [24, 4, 8].

  5. Extrinsic Neurons (ENs), numbering 10’s, located downstream from the KCs. These are believed to be “readout neurons” that interpret the KC codes and convey actionable messages (such as “fly upwind”) [26, 27].

  6. Reward Mechanism. A large neuron sprays octopamine globally over the AL and MB, in response to reward, such as sugar at the proboscis. Learning does not occur without this octopamine input [28, 1].

  7. Inter-network connections: In the AL-MB these are strictly feed-forward, either excitatory or inhibitory. In particular, AntennaeAL, ALLH, KCsENs are all excitatory. LHKCs is inhibitory. ALKCs have both excitatory and inhibitory channels.

  8. Plasticity: The connections into the KCs (ALKCs) and out of the KCs (KCsENs) are known to be plastic during learning [29, 6]. The AL is not plastic.

4.2 Glossary

Antenna lobe (AL)

A collection of neurons innervated by odor receptors in the antennae. It sends signals to the mushroom body via projection neurons. Connections in the AL are not plastic.

Mushroom body (MB)

A collection of neurons (Kenyon cells - KCs) downstream from the antenna lobe. The MB is believed to store odor codes that serve as a memory, allowing the moth to recognize odors. Connections in the MB are plastic.

Lateral horn (LH)

A collection of neurons which receives input from the AL and sends inhibitory output to the MB. One of its roles is to enforce sparse firing in MB neurons.

Receptor neuron (RN)

These neurons respond to odors (volatiles) at the antennae and stimulate the antenna lobe. RNs respond to different, distinct odors.

Glomerulus

The antenna lobe is divided into about 60 glomeruli, each of which is a self-contained collection of neurons (projection and lateral), innervated by RNs that respond to particular odors.

Projection neuron (PN)

Each glomerulus contains projection neurons, whose output innervates the KCs and also the lateral horn, but not other glomeruli in the AL, ie they are feed-forward only. Most PNs start in one glomerulus and are excitatory. A few PNs arborize in several glomeruli and are inhibitory (we refer to inhibitory PNs as “QNs”). Each glomerulus initiates about five PNs.

Lateral neuron (LN)

Each glomerulus contains lateral neurons, which innervate other glomeruli in the AL. LNs are inhibitory. One function is competitive inhibition among glomeruli. Another function is gain control, ie boosting low signals and damping high signals.

Kenyon cell (KC)

Neurons in the calyx of the MB. These have very low FRs, and tend to respond to particular combinations of PNs. KCs respond sparsely to a given odor. There are about 4000 KCs, ie a two-orders-of-magnitude increase over the number of glomeruli. Each KC synapses with about ten PNs. Connections into and out of KCs are plastic.

Extrinsic neuron (EN)

A small number of neurons downstream from the KCs. ENs are thought to be “readout” neurons. They interpret the odor codes of the KCs, deciding to eg “ignore”, “approach”, or “avoid”.

Firing rate (FR)

The number of spikes/second at which a neuron fires. Typically FRs are counted using a window (eg 500 ms). The moth’s response to odor stimulations is episodic, with FR spikes in FR and rapid return to spontaneous FRs. Neurons respond to relative changes in FR, rather than to raw magnitude changes. A neuron’s relative change in FR is scaled by its spontaneous FR (see section 4.5 below).

Octopamine

A neuromodulator which stimulates neural firing. The moth spritzes octopamine on both the AL and MB in response to sugar, as a feedback reward mechanism. Dopamine has a similar stimulating effect on both AL and MB, but it reinforces adverse rather than positive events.

4.3 Component networks and their Network Model representations


This subsection offers a more detailed discussion of the constituent networks in the biological AL-MB, and details about how they are modeled in our Network Model.

Antennae and receptor neurons

The Antennae receptors, activated by chemical molecules in the air, send excitatory signals to Receptor Neurons (RNs) in the AL. Several thousand antennae converge onto 60 units (glomeruli) in the AL [45]. All the receptors for a given atomic volatile converge onto the same glomerulus in the AL, so the glomeruli each have distinct odor response profiles [46]. Since natural odors are a blend of atomic volatiles, a natural odor stimulates several units within the AL [47].

Our model does not explicitly include antennae. Rather, the first layer of the model consists of the RNs entering the glomeruli. Though 500 RNs feed a given glomerulus, the model assumes one RN. The benefit of many RNs converging appears to be noise reduction through averaging [33]. This can be simulated by one RN with a smaller noise envelope. Each glomerulus’ RN has a spontaneous FR and is excited, according to random weights, by odor stimuli.

Antenna lobe and projection neurons

The AL is fairly well characterized in both structure and dynamics, with a few important gaps. Moths and flies are similar enough that findings in flies (Drosophila) can generally be transferred to the moth (in contrast, locusts and honeybees are more complex and findings in these insects do not safely transfer) [48].

The AL contains about 60 glomeruli, each a distinct unit which receives RN input and projects to the KCs via excitatory PNs. The same PN signal also projects to the LH [25]. The AL, unique among the networks, has inhibitory lateral neurons (LNs) [49], the only neurons that are not strictly feed-forward. (There is some evidence of excitatory LNs, eg [50]; the Network Model excludes this possibility.) The LNs act as a gain control on the AL, and also allow odors to mask each other by inhibiting other glomeruli’s RNs [51, 40]. It is not known whether LNs also inhibit PNs and LNs. Based on calibrations to in vivo data, in Network Model LNs inhibit all neuron types (cf section 3.1). Thus each glomerulus contains dendrites (ie outputs) for PNs and LNs, and axons (ie inputs) for RNs and LNs, as shown in Figure 6.

Each glomerulus does the following: Receives RN input from the antennae receptors upstream; inhibits other glomeruli within the AL via LNs; and sends excitatory signals downstream via Projection Neurons (PNs).

In general, each PN is innervated in a single glomerulus. In moths, there are 5 PNs rooted in each glomerulus (60 glomeruli, 300 PNs). The Network Model assumes all PNs from a given glomerulus carry the same signal (because they share the same glomerulus and therefore inputs, and perhaps also because of ephaptic binding) [52].

Glomeruli also initiate pooled Inhibitory Projection Neurons (QNs) that send inhibitory signals downstream to the KCs.

The AL contains a powerful macro-glomerulal complex (MGC), which processes pheromone. Because pheromone response has fundamentally different dynamics than food odor response [53], the model ignores it. Only the glomeruli associated with non-pheromone (food) odors are modeled.

Connections in the AL are not plastic with long-term persistence [5]. While some evidence of short-term plasticity exists, the Network Model ignores this option.

Figure 6: Schematic of AL glomeruli. Detail of neural connections within a glomerulus. Red = inhibitory, green = excitatory, blue = increases responsiveness. RNs enter from the antennae. LNs enter from other glomeruli; one full LN is shown. It is not known if octopamine modulates LNs and PNs (see section 3.1).

Lateral horn

The LH receives input from the PNs. It then sends an inhibitory signal to the KCs. This inhibition from the LH appears to ensure that the KCs fire very sparsely and thus act as coincidence detectors for signals from the AL [52, 54, 55].

The LH is also suspected of containing a parallel system for processing certain intrinsically-known odors in short-cut fashion (labeled lines) [56]. Since this parallel system is (by definition) not involved with learning, the Network Model ignores it. The LH is modeled solely as a simple sparsifying inhibition on the KCs.

(Note: The locust and honeybee, which have more complex olfactory systems and different use-cases in terms of odor processing, have a time-oscillating depolarization mechanism (local potential fields, LPF) [57] which serves a similar purpose to LH inhibition in the moth. LPF oscillations are absent in the moth [21].)

Mushroom body and Kenyon cells

The KCs (4000) in the MB are believed to encode odor memories in a high-dimensional, sparse space [30]. Odors with no meaning to the moth still have non-zero codes in the KCs.

KCs receive excitatory input from the PNs and inhibitory input from QNs, both of which vary greatly between KCs, since each KC is innervated by only 10 PNs [21]. The connection map appears to be random [58]. The KCs also receive generalized damping inhibition from the LH. (There is some evidence in drosophila of an MBMB global inhibitory neuron [54], with the same essential effect as LH inhibition; the Network Model excludes this possibility.) KCs fire very sparsely, generally respond to only a single odor, and are silent absent that odor [8]. KCs are treated as noise-free. Their output is an excitatory signal sent to the extrinsic neurons (ENs) [26].

In addition to olfactory input, the KCs receive input signals from other parts of the moth (eg hearing) [52]. Because the Network Model targets olfactory learning, it ignores these other inputs and uses a reduced number of KCs (2000 instead of 4000).

The synaptic connections in the MB (PNsKCs, QNsKCs, and KCsENs) are plastic, ie they can be modified during training [59]. The generalized inhibition from LHKCs is modeled as non-plastic (actual physiology is not known).

Extrinsic neurons

Though located in the lobes of the MB, here ENs are not considered part of the MB, which is taken to be synonymous with the KCs. ENs are few in number compared to the KCs (10s ) [26, 27]. They are believed to be “readout” neurons, that interpret the KC codes as actionable signals (eg “approach”, “avoid”) [6]. We assume that ENs trigger actions when their output FRs exceed some threshold.

We define Learning as: Permanently boosting EN responses beyond their naive (untrained) level, so that EN responses to reinforced stimuli can consistently exceed an action-triggering threshold. This is tantamount to modifying the moth’s behavior.

Octopamine (reward circuit)

A large neuron delivers octopamine to the entire AL and MB, in response to positive stimuli, eg sugar at the proboscis. It acts as a reward feedback to the system. A similar neuron delivers dopamine to the AL and MB in response to negative stimuli, and acts as an aversive feedback signal [5]. Learning does not occur without octopamine (or dopamine) [1].

Despite their opposite reward values, both octopamine and dopamine act in the same way when sprayed on a neuron: They increase the neuron’s general tendency to fire [60]. In the Network Model this effect is modeled as making a neuron more responsive to excitatory inputs (eg from odors and RNs) and less responsive to inhibitory inputs (eg from LNs). Details of octopamine’s effects if any on various neural types are not well-characterized. In the Network Model octopamine directly affects RNs and LNs but not PNs in the AL (cf section 3.1); has no direct effect on KCs or ENs (though there are strong indirect effects); and has no effect on the LH inhibitory signal.

It is unclear whether octopamine delivery to both the MB and AL is necessary and sufficient for learning [1, 5]. The Network Model assumes that octopamine controls an “on/off” switch for Hebbian growth, ie there is no plasticity in the MB (and therefore no learning) without octopamine.

4.4 Network Model description


This section describes our Network Model in detail. It covers the firing rate measure used to compare model output to in vivo data; model dynamics; plasticity and other details; model parameters; and moth generation. All coding was done in Matlab. Computer code for this paper will be found at:
https://github.com/charlesDelahunt/SmartAsABug

4.5 Firing rate measure


To compare PN firing rate statistics from in vivo experiments and from Network Model simulations (ie model calibration), we use a measure of firing rate (FR) based on Mahalanobis distance, similar to the measure common in the literature [26, 40, 30, 61]. The premise is that neurons downstream respond to a 1 std change in FRs equally (modulo different connection weights), independent of the sometimes large (up to 40x) magnitude differences in the raw spontaneous FRs of different neurons. The FR measure is defined as follows:

  1. Each PN has a spontaneous firing rate (FR) with a gaussian noise envelope.

  2. PNs with FR 1 spike/sec are ignored, on the assumption that such PNs represent artifacts of experiment (also, the gaussian noise assumption fails). About 10% of PNs in experimental data fall in this category.

  3. Output FR activity of PNs is measured as = distance from mean spontaneous FR, in units of time-varying std dev of spontaneous FR (ie Mahalanobis distance): Let
    = raw firing rate (spikes per second).
    = spontaneous firing rate (no odor).
    = moving average of S (no odor).
    = smoothed estimate of the moving average , eg a quadratic or spline fit.

    = standard deviation of

    , calculated using values within a moving window centered on .
    and are typically steady absent octopamine, but are often strongly modulated by octopamine.
    Then the measure of FR activity is:

    (3)
  4. is related to the measure :
    , ie is change in FR, normalized by spontaneous FR. The key difference between and is whether or how is estimated, due to varying exigencies of experiment. Our experimental data allow reasonable estimates of and . Network Model simulations produce very good estimates, since computers are more amenable to repeated trials than live moths.

4.6 Model dynamics


Our Network Model uses standard integrate-and-fire dynamics [16], evolved as stochastic differential equations [62].
Let = firing rate (FR) for a neuron. Then

(4)

w = connection weights;
u = upstream neuron FRs;

is a sigmoid function or similar.


PN dynamics are given here as an example. Full model dynamics are given in Section 5.2. PNs are excitatory, and project forward from ALMB:

(5)

= brownian motion process;
;
= octopaminePN weight matrix (diagonal );
= LNPN weight matrix ( with );
= RNPN weight matrix (diagonal );
indicates if octopamine is active ( = 1 during training, 0 otherwise).

= LN FRs, vector

;
= RN FRs ();
= scaling factor for effects on inhibition.

Discretization

The discretization uses Euler-Maruyama, a standard step-forward method for SDEs [62].
Euler (ie noise-free):
Euler-Maruyama: , where controls the noise intensity.

Convergence

Timestep was chosen such that noise-free E-M evolution gives the same timecourses as Runge-Kutta (4th order), via Matlab’s ode45 function. mSec suffices to match E-M evolution to R-K in noise-free moths. Values of mSec gives equivalent simulations in moths with AL noise calibrated to match in vivo data. Values of mSec show differences in evolution outcomes given AL noise.

Plasticity

The model assumes a Hebbian mechanism for growth in synaptic connection weights [63, 29]. That is, the synaptic weight between two neurons and increases proportionally to the product of their firing rates (“fire together, wire together”):
Thus, synaptic plasticity is defined by:

(6)

There are two layers of plastic synaptic weights, pre- and post-MB: ALMB (), and MBENs () . Learning rate parameters of the Network Model were calibrated to match experimental effects of octopamine on PN firing rates and known moth learning speed (eg 4 - 8 trials to induce behavior modification) [60]. The Network Model does not decay unused synaptic weights. Training does not alter octopamine delivery strength matrices (). That is, the neuromodulator channels are not plastic (unlike, for example, the case in [64]).

Odor and octopamine injections

Odors and octopamine are modeled as hamming windows. The smooth leading and trailing edges ensures low stiffness of the dynamic ODEs, and allows a 10 mSec timestep to give accurate evolution of the SDEs in simulations.

Training

Training on an odor consists of simultaneously applying stimulations of the odor, injecting octopamine, and “switching on” Hebbian growth. Training with 5 to 10 odor stimulations typically produces behavior change in live moths.

4.7 Model parameters


There is a risk, when modeling a system, of adding too many free parameters in an effort to fit the system. Fewer free parameters are better, for the sake of generality and to avoid overfitting. Conversely, we wish to reasonably match the physiological realities of the system. Because the key goal of this paper was to demonstrate that a simple model, in terms of parameters and structure, can reproduce the learning behavior of the AL-MB, we made efforts to minimize the number of free parameters. For example, neuron-to-neuron connections in the model are defined by their distributions, ie two parameters each. These are (usually) distinct for different source-to-target pairs (eg LNRN, LNLN, etc).
Some mean and std dev parameters for distributions are shared among different neuron types (eg LNs, PNs, and QNs all share the same variance scaling parameter).

Parameter list

There are in total 47 free parameters:
1. Structure: 5 (eg number of neurons in each network)
2. Dynamics: 12 (noise: 2. decay tau and sigmoid: 3. Hebbian growth: 6. misc: 1).
3. Spontaneous RN FRs: 3.
4. Connection matrices: 27 (to control non-zero connection ratios: 5; synaptic weights (eg ): means 12, std devs 4; octopamine weights (eg ): means 6, std devs 2).

Dynamics parameters

The differential equations of all neuron types share the same decay rate, set to allow return to equilibrium in 1 second, consistent with in vivo data. Neurons also share parameters of the sigmoid function within the differential equation. Noise added via the SDE model is controlled by a single parameter , the same for all neuron types. It is determined by empirical constraint on , as shown in column 2 of Figure 2.

Connection matrix generation

Connection weight matrices (eg

etc) are generated in a standard way, from Gaussian distributions with std dev

defined proportional to the mean , using a scaling factor :
where depends on the neuron types being connected, and . Many connection types typically share the same .

A special feature of the AL is that all the neurons in a given glomerulus share a common environment. For example, all the neurons, of whatever type, in glomerulus A will share the same strong (or weak) LN axon from glomerulus B. Thus, the RN, LN, and PNs in a given glomerulus are all correlated. In addition, neuron types are correlated. To model this dual set of correlations, connection matrices in the AL are generated as follows. As an example, consider LN connection matrices in the AL:

  1. A glomerulus-glomerulus connection matrix is created, which defines LN arborization at the glomerular level.

  2. This connection matrix is multiplied by a neural type-specific value to give ,, and connection matrices. This is particularly important when tuning the various inhibitory effects of LNs on RNs, PNs (QNs), and LNs.

  3. Sensitivity to GABA: A separate variance factor determines glomerular sensitivity to GABA (ie sensitivity to inhibition). This is tuned to match data in the literature [40], and applies to LN-to-PN(QN) (ie ) connections only.

The goal of this two-stage approach is to enforce two types of similarity found in the AL: (i) Connections to all neurons within a single glomerulus are correlated; and(ii) connections to all neurons of a certain type (LN, PN, RN) are correlated.

Due to constraints of the biological architecture there are many zero connections. For example, about 85% of entries in the ALMB weight matrix are zero because MB neurons connect to only 10 projection neurons [58]. All MB

EN weights are set equal at the start of training. Training leads rapidly to non-uniform distributions as inactive connections decay and active connections strengthen.

RN spontaneous firing rates

RNs in the glomeruli of the AL have noisy spontaneous firing rates [23]

. The Network Model simulates this by assigning spontaneous firing rates to RNs. These spontaneous firing rates are drawn from a gamma distribution plus a bias:


, where are shape and rate parameters, and is the Gamma function.
This can be thought of as a source of energy injected into the system, at the furthest upstream point (absent odor). Other energy sources are odor signals and octopamine. The spontaneous firing rates of all other neurons in the Network Model are the result of their integrate-and-fire dynamics responding as RN spontaneous FRs propagate through the system.

4.8 Discrepancies between biology and model


There are some known discrepancies between our Network Model and the moth AL-MB. These are listed below.

Connection weight distributions

This model version uses gaussian distributions to generate initial connection weights. However, moths used in live experiments are older and thus presumably have modified PNKC and KCEN connection weights. If this modification was strong, we might expect the connection weight distributions to tend towards a scale-free rather than gaussian distribution [65]. This represents an unknown discrepancy between structure parameters of the live moths used in experiments vs the model.

Hebbian pruning

The Network Model contains no pruning mechanism to offset, via decay, the Hebbian growth mechanism. Such pruning mechanisms are common in nature, so it is reasonable to suppose that one might exist in the AL-MB. The moth has inhibitory as well as excitatory feed-forward connections from AL to MB. In the Network Model, pruning is functionally replaced by Hebbian growth of QNKC inhibitory connections, which act to inhibit KCs and thus offset the growth of excitatory PNKC connections (this does not directly offset KCEN Hebbian growth). Thus omitting a separate Hebbian decay mechanism is a matter of convenience rather than a match to known biology.

Non-olfactory input to KCs

In addition to olfactory input, the KCs receive signals from other parts of the moth, eg hearing. Because this model targets only olfactory learning, it ignores these other inputs to the KCs, and reduces the total number of KCs (from 4000 to 2000).

Number of QNs

There are believed to be about 3-6 QNs projecting from the AL to the MB. This model sets their number at about 15. The reason is that, absent a Hebbian pruning system in the model, the QNs function as the brake on runaway increases in KC responses due to Hebbian growth. So the increased number of QNs is a compensation for the lack of a weight-decay system.

Number of ENs

This model version has only one EN, since its goal is to demonstrate simple learning. The moth itself possesses multiple ENs.

LH inhibition

The LHKC inhibitory mechanism used in this chapter is modeled as a time-invariant global signal, delivered equally to all KCs. This simplifies the model parameter space while retaining the essential functionality of the LH. A more refined version of LHKC inhibition might vary in strength according to PN output, since the same PN signals that excite the KCs also excite the LH. The actual dynamics of the ALLHKC linkage are not known, beyond the principle that inhibition from the LH sparsifies the KC codes and makes the individual KCs act as coincidence detectors.

4.9 in vivo neural recordings data


Model parameters were calibrated by matching Network Model performance to in vivo electrode readings from the ALs of live moths. The various performance metrics are described in Results.

Electrode data was collected by the lab of Prof Jeff Riffell (Dept of Biology, UW). It consists of timecourses of PN firing rates measured via electrode in the AL of live moths, during a variety of regimes including:

  1. Series of 0.2 sec odor stimulations delivered without octopamine. These experiments gave data re PN response to odor relative to PN spontaneous (baseline) FRs, absent octopamine.

  2. Series of 0.2 sec odor stimulations delivered coincident with sugar reward (which delivers octopamine). This gave data re how PN odor response is modulated by octopamine, relative to octopamine-free spontaneous FR. See Figure 7 panel A.

  3. Series of 0.2 sec odor stimulations, delivered first without and then coincident with an octopamine wash applied to the AL. This gave data re how PN spontaneous FR and PN odor response are modulated by octopamine. See Figure 7 panel B.

In most cases the applied odor consisted of a collection of 5 volatiles, which taken together stimulate many glomeruli in the AL. It was selected to ensure sufficient odor-responsive PNs, such that inserted electrodes would detect interesting (ie responsive) PNs. Further details re in vivo data collection can be found in [66] and in Sec. 5.1. Example timecourses are shown in Figure 7.

4.10 Simulation setup


For Network Model learning experiments, the time sequence of events for simulations, shown in Fig 1, is as follows:

  1. A period of no stimulus, to assess baseline spontaneous behavior.

  2. Four odor stimuli are delivered, 16 stimulations each (two odors were used in MB sparseness experiments).

  3. A period of control octopamine, ie without odor or Hebbian training.

  4. The system is trained (odor + octopamine + Hebbian mechanism) on one of the odors.

  5. A period of no stimulus, to assess post-training spontaneous behavior.

  6. The odors are re-applied (16 stimulations each), without octopamine, to assess effects of training on odor response.

Figure 7: Time series of PN firing rates from in vivo experiments. x-axis = time, y-axis = FR. Blue lines = mean spontaneous rate, shaded regions = 1 and 2 std. Red dots are odor responses. Green dots are response to control (mineral oil).
A: PN response, given odor plus coincident sugar reward, ie plus octopamine (time series for PNs with odor only are similar, but with less strong odor responses). Top row: unresponsive to odor. Middle row: excited response to odor. Bottom row: inhibited response to odor.
B: PNs with octopamine wash added in mid-experiment, then rinsed away (duration shown by black line). Octopamine can alter (up, down, or not at all) the spontaneous FR and/or the odor response, so there are 9 possible modulation regimes. This grid of timecourses shows a typical PN from each regime. Top row: spontaneous FR in unaffected. Middle row: spontaneous FR is boosted. Bottom row: spontaneous FR is inhibited. First column: odor response is unaffected. Second column: odor response is boosted. Third column: odor response is inhibited.

5 Supporting Information


5.1 Moth AL neural recording datasets


Multichannel Recording Methods

Adult male M. sexta were reared in the laboratory at the University of Washington on an artificial diet under a long-day (17/7 hr light/dark cycle) photoperiod and prepared for experiments 2–3 d after emergence. In preparation for recording, the moth was secured in a plastic tube with dental wax, and the caudal end of the head capsule was opened and the piece of cuticle that is the attachment site of the pharyngeal dilator muscles is positioned forward and readhered to the headcapsule with myristic acid. The proboscis was extended and adhered to a piece of teflon tubing, 7-cm in length, that allowed movement and extension of the proboscis along its length and at the tip. Behavioral response to odor stimuli was determined by the percentage of proboscis movement response between treatments, allowing for a conservative estimate of a learned response. This preparation allows full access to the exposed AL while having no effect on the moth’s ability to feed normally. During recordings, the brain was superfused slowly with physiological saline solution (in mM: 150 NaCl, 3 CaCl2, 3 KCl, 10 N-tris[hydroxymethyl] methyl-2 aminoethanesulfonic acid buffer, and 25 sucrose, pH 6.9) throughout the experiment.

Olfactory stimuli were delivered to the preparation by pulses of air from a constant air stream were diverted through a glass syringe containing a piece of filter paper bearing odor stimuli. The stimulus was pulsed by means of a solenoid-activated valve controlled by the SciWorks acquisition software (Data Wave Technologies, Longmont, CO, USA). The outlet of the stimulus syringe was positioned 2 cm from and orthogonal to the center of the antennal flagellum ipsilateral to the ALs.

We used a classic conditioning paradigm to examine the effects on AL neurons while the moth learned to associate an odor with a sugar reward. Odor stimuli were delivered in a three second pulses, and one second after odor onset, the unconditioned stimulus (US; 1 μL of 20% sucrose solution) was applied to the proboscis for ca. 2 s. A ten minute inter-trial interval separated each training trial, and moths were trained over eight trials. After the conditioning trials were completed, a test trial was performed during which only the trained odor was presented to assess the behavioral and odor-driven ensemble responses as a result of the conditioning treatment.

AL Recordings were made with 16-channel silicon multielectrode recording arrays (a4×4-3mm50-177; NeuroNexus Technologies, Ann Arbor, MI, USA). The spatial distribution design of the recording array suits the dimensions of the AL in M. sexta, with four shanks spaced 125 μm apart, and each with four recording sites 50 μm apart. The four shanks were oriented in a line parallel to the antennal nerve. The probe was advanced slowly through the AL using a micromanipulator (Leica Microsystems, Bannockburn, IL, USA) until the uppermost recording sites were just below the surface of the AL. Extracellular activity was acquired with a RZ2 base station (Tucker-Davis Technologies, Alachua, FL, USA) and a RP2.1 real time processor (Tucker-Davis Technologies) and extracellular activity in the form of action potentials, or spikes, were extracted from the recorded signals and digitized at 25 kHz using the Tucker-Davis Technologies data-acquisition software.Threshold and gain settings were adjusted independently for each channel, and spikes were captured in the 4-channel, or ‘tetrode’, recording configuration: any spike that passed threshold on one channel triggered the capture of spikes recorded on the other 3 channels on the same shank. Offline Sorter v.3 (Plexon Neurotechnology Research Systems, Dallas, TX, USA) was used to sort extracellular spikes based on their waveform shape and spikes were assigned timestamps to create raster plots and calculate peri-stimulus time histograms (PSTH). Only those clusters that were separated in three dimensional space (PC1–PC3) after statistical verification (multivariate ANOVA; P 0.05) were used for further analysis (typically 6–18 units were isolated per ensemble). Each spike in each cluster was time-stamped, and these data were used to create raster plots and to calculate peristimulus time histograms (PSTHs), interspike interval histograms, cross-correlograms, and rate histograms.

List of in vivo data sets

  1. AL, odor only: PNs, one odor, no octopamine. 7 preps with 8 - 16 PNs each.

  2. AL, odor + octopamine. PNs, one odor, sugar reward. 10 preps with 9 - 21 PNs each.

  3. AL + MB, odor + octopamine. PNs and KCs, one odor, sugar reward. 1 prep, with 7 PNs and 12 KCs.

  4. AL, odor + octo wash: PNs, one odor, octopamine directly applied to AL. 7 preps: 6 preps with 8 - 13 PNs each; 1 prep with one pheromone-responsive neuron

  5. AL, odors only (BEA): PNs, several odors and concentrations. 12 preps with 14 - 17 PNs each.

  6. AL, odors only (ESO): PNs, several odors and concentrations. 4 preps with 12 - 14 PNs each.

5.2 Full equations of model dynamics


(7)
(8)
(9)
(10)
(11)
(12)

where

Symbol Type Size/Value Description and Remarks
R superscript Refers to the receptor neurons subpopulation.
P superscript Refers to the excitatory projection neurons subpopulation.
Q superscript Refers to the inhibitory projection neurons subpopulation.
L superscript Refers to the lateral neurons subpopulation.
K superscript Refers to the kenyon cells subpopulation.
E superscript Refers to the readout extrinsic neurons subpopulation.
O superscript Refers to the octopamine neurotransmitter.
scalar 60 Number of glomeruli in the antenna lobe.
scalar 2-4 Number of different stimuli (odors).
scalar Number of inhibitory projection neurons.
scalar 2000 Number of kenyon cells.
scalar 1 Number of extrinsic neurons.
vector FRs of the receptor neurons subpopulation.
vector FRs of the exc. projection neurons subpopulation.
vector FRs of the inh. projection neurons subpopulation.
vector FRs of the lateral neurons subpopulation.
vector FRs of the kenyon cells subpopulation. Sparse.
vector FRs of the extrinsic neurons subpopulation.
vector
vector
matrix Stimulus connections.
matrix Octopamine connections. Diagonal matrix.
matrix Octopamine connections. Diagonal matrix.
matrix Connection weights   .
matrix Connection weights   . Diagonal matrix.
matrix Connection weights   .
matrix Connection weights   .
matrix Connection weights   .
matrix Connection weights   .
matrix Connection weights   .
matrix Connection weights   .
matrix Connection weights   .
matrix Connection weights   .
function 0 or 1 Flags when octopamine is active (typically during training).
scalar 0.5 Scaling factor for octopamine’s effects on inhibition.
scalar
scalar
scalar
scalar
scalar
scalar

Each glomerulus receives one RN and one octopamine input, and initiates one PN and one LN.
Octopamine decreases the response to inhibition less than it increases the response to excitation

Table 1: Variables and parameters for neuronal network model

5.3 ANOVA analysis of Network Model learning


The differential increase in EN response to trained vs control odors was almost always significant to . When odors’ naive EN response magnitudes differed by 3, sometimes differences in either raw increases or percentage increases (not both) did not attain this level of significance, while the other metric did. An example of learning results when odors’ naive EN response magnitudes differ widely is shown in Fig 8.

The p-values of 336 trained odor/control odor pairs are plotted in Fig 9, against the ratio of their mean naive odor responses , for 28 moths randomly generated from a template, with three control odors and one trained odor. Each p-value is for the trained odor vs one control odor (so there are 12 data points per moth). Column 1 shows p-values for the change in raw EN response, as in Fig 8 (C), trained vs control. Trained odors with very low-magnitude naive response often did not have raw increases larger than high-magnitude control odors. Column 2 shows p-values for the percentage change in EN response, as in Fig 8 (B), trained vs control.

Unless the naive EN responses for the two odors were highly disparate (eg by factor of 3x), the differential increase in EN response of the trained vs control odors is almost always significant, measured both as raw and as percentage. Fig 10 plots the percentage of 336 trained-control pairs that had p-values for both measures of EN response increase (ie as raw and as percentage) below the listed threshold (eg ), for 336 trained-control pairs whose ratio ( or ) is within the bound given on the x-axis. Fig 10 shows how many moths, generated from template with no constraint on unbalanced naive odor EN responses, had differential post-training EN responses with significance , for both measures (as raw and as a percentage).

Figure 8: Effect of training on EN FRs, given odors with unequal naive response magnitudes. When odors induced naive EN responses of very different magnitudes, then trained odor response increased much more than control odor responses either in raw magnitude, or as a percentage, or both.
A: Typical timecourse showing magnitudes EN responses before and after training the third (red) odor, indicated by red arrow, over 15 odor stimulations. This corresponds to the third column in panels B - D, at index 3 on the x-axis. Note that only the third (red) odor’s EN response changes magnitude.
Panels B - D: Changes to ENs during training. x-axis = number of training sessions. Each column shows results of training a given odor, color coded: blue, black, red. y-axis measures raw EN or percent change in EN. 21 trials per data point.
B: Percent change (from pre-training) in ENs, mean 2 stds.
C: Raw EN FRs, mean 2 stds.
D: Changes in raw EN FRs, normalized by trained odor (ie subtract the trained odor’s changes from all odors), mean 2 std devs. This shows how far each control odor lagged behind the trained odor.
Note that the trained odor dominates in either raw increase (panels C, D) if naive response to trained odor was large, or in percent increase (panel B) if naive response to trained odor was small.
Figure 9: p-values for trained-control odor pairs: A: p-values for change in raw EN responses. B: p-values for percentage change in EN responses. P-values are sometimes high (for one metric or the other) when trained and control odors have highly disparately-scaled naive responses (= mean raw T ) (= mean raw C). Plots show results given 20 training stimulations.
When is larger (right end of x-axis), the p-value for raw change (A) is consistently very low, but the p-value for percentage change (rB) can be high, since even a small incidental change to a low-intensity odor can be a large percentage change.
When is larger (left end of x-axis), the p-value for percentage change (B) is consistently very low, but the p-value for raw change (A) can be high, since even a small percentage change to a high-response odor corresponds to a large raw change.
When naive odor responses are roughly matched, eg within 3x (ie 0.33 to 3), p-values for both raw and percentage change are very low.
Figure 10: Fractions of p-values below 0.01 for trained-control odor pairs In most cases, the trained odor shows much larger increases in EN response magnitude. A: The percentage of trained-control odor pairs with EN response magnitudes within the ratios given on the x-axis. B: The percentage of trained-control pairs, with EN response magnitudes within the ratios given on the x-axis, whose training-induced changes in EN responses were distinct with p-value 0.01. Each curve is for a different number of training stimulations. More training increases distinctions, up to 15 stimulations. But additional training actually hinders distinctions, as control odor response reinforcement begins to overtake trained odor reinforcement.

References

  • [1] Martin Hammer and Randolf Menzel. Multiple sites of associative odor learning as revealed by local brain microinjections of octopamine in honeybees. Learn Mem, 5(1):146–156, May 1998. 10454379[pmid].
  • [2] Jeffrey A. Riffell, Leif Abrell, and John G. Hildebrand. Physical processes and real-time chemical measurement of the insect olfactory environment. Journal of Chemical Ecology, 34(7):837–853, Jul 2008.
  • [3] Rachel I Wilson. Neural and behavioral mechanisms of olfactory perception. Current Opinion in Neurobiology, 18(4):408 – 412, 2008. Sensory systems.
  • [4] Robert A.A. Campbell and Glenn C. Turner. The mushroom body. Current Biology, 20(1):R11 – R12, 2010.
  • [5] Andrew M. Dacks, Jeffrey A. Riffell, Joshua P. Martin, Stephanie L. Gage, and Alan J. Nighorn. Olfactory modulation by dopamine in the context of aversive learning. Journal of Neurophysiology, 108(2):539–550, 7 2012.
  • [6] Nicolas Y. Masse, Glenn C. Turner, and Gregory S.X.E. Jefferis. Olfactory information processing in drosophila. Current Biology, 19(16):R700 – R713, 2009.
  • [7] Gilles Laurent. Olfactory network dynamics and the coding of multidimensional signals. Nature Reviews Neuroscience, 3:884 EP –, Nov 2002. Review Article.
  • [8] Kyle S. Honegger, Robert A. A. Campbell, and Glenn C. Turner. Cellular-resolution population imaging reveals robust sparse coding in the drosophila mushroom body. Journal of Neuroscience, 31(33):11772–11785, 2011.
  • [9] Sophie J. C. Caron. Brains don’t play dice—or do they? Science, 342(6158):574–574, 2013.
  • [10] C. Giovanni Galizia. Olfactory coding in the insect brain: data and conjectures. European Journal of Neuroscience, 39(11):1784–1795, 2014.
  • [11] Fei Peng and Lars Chittka. A simple computational model of the bee mushroom body can explain seemingly complex forms of olfactory learning and memory. Current Biology, 27(2):224 – 230, 2017.
  • [12] Mark Roper, Chrisantha Fernando, and Lars Chittka. Insect bio-inspired neural network provides new evidence on how simple feature detectors can enable complex visual generalization and stimulus location invariance in the miniature brain of honeybees. PLOS Computational Biology, 13(2):1–23, 02 2017.
  • [13] Thiago S Mosqueiro and Ramón Huerta.

    Computational models to understand decision making and pattern recognition in the insect brain.

    Current Opinion in Insect Science, 6:80 – 85, 2014. Pests and resistance/Parasites/Parasitoids/Biological control/Neurosciences.
  • [14] Paolo Arena, Luca Patané, Vincenzo Stornanti, Pietro Savio Termini, Bianca Zäpf, and Roland Strauss. Modeling the insect mushroom bodies: Application to a delayed match-to-sample task. Neural Networks, 41:202 – 211, 2013. Special Issue on Autonomous Learning.
  • [15] Faramarz Faghihi, Ahmed A. Moustafa, Ralf Heinrich, and Florentin Wörgötter. A computational model of conditioning inspired by drosophila olfactory system. Neural Networks, 87:96 – 108, 2017.
  • [16] Peter Dayan and L. F. Abbott. Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. The MIT Press, 2005.
  • [17] D. Hubel and T. Wiesel. Receptive fields, binocular interaction, and functional architecture in the cat’s visual cortex. Journal of Physiology, 160:106–154, 1962.
  • [18] Kunihiko Fukushima. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36:193–202, 1980.
  • [19] Jurgen Schmidhuber. Deep learning in neural networks: An overview. Neural Networks, 61(Supplement C):85 – 117, 2015.
  • [20] Yann LeCun. Facebook ai director yann lecun on his quest to unleash deep learning and make machines smarter. IEEE Spectrum, 2015.
  • [21] Joshua P. Martin, Aaron Beyerlein, Andrew M. Dacks, Carolina E. Reisenman, Jeffrey A. Riffell, Hong Lei, and John G. Hildebrand. The neurobiology of insect olfaction: Sensory processing in a comparative context. Progress in Neurobiology, 95(3):427 – 447, 2011.
  • [22] Pal Kvello, Bjarte Lofaldli, Jurgen Rybak, Randolf Menzel, and Hanna Mustaparta. Digital, three-dimensional average shaped atlas of the heliothis virescens brain with integrated gustatory and olfactory neurons. Frontiers in Systems Neuroscience, 3:14, 2009.
  • [23] Vikas Bhandawat, Shawn R Olsen, Nathan W Gouwens, Michelle L Schlief, and Rachel I Wilson. Sensory processing in the drosophila antennal lobe increases reliability and separability of ensemble odor representations. Nature Neuroscience, 10:1474–1482, 2007.
  • [24] Emmanuel Perisse, Christopher Burke, Wolf Huetteroth, and Scott Waddell. Shocking revelations and saccharin sweetness in the study of drosophila olfactory memory. Curr Biol, 23(17):R752–R763, Sep 2013. S0960-9822(13)00921-4[PII], 24028959[pmid].
  • [25] Maxim Bazhenov and Mark Stopfer. Forward and back: Motifs of inhibition in olfactory processing. Neuron, 67(3):357 – 358, 2010.
  • [26] RAA Campbell, KS Honegger, H Qin, W Li, E Demir, and GC Turner. Imaging a population code for odor identity in the drosophila mushroom body. Journal of Neuroscience, 33(25):10568–81, 2013.
  • [27] Toshihide Hige, Yoshinori Aso, Gerald M. Rubin, and Glenn C. Turner. Plasticity-driven individualization of olfactory coding in mushroom body output neurons. Nature, 526:258 EP –, Sep 2015.
  • [28] M Hammer and R Menzel. Learning and memory in the honeybee. Journal of Neuroscience, 15(3):1617–1630, 1995.
  • [29] Stijn Cassenaer and Gilles Laurent. Hebbian stdp in mushroom bodies facilitates the synchronous flow of olfactory information in locusts. Nature, 448:709 EP –, Jun 2007.
  • [30] Glenn C. Turner, Maxim Bazhenov, and Gilles Laurent. Olfactory representations by drosophila mushroom body neurons. Journal of Neurophysiology, 99(2):734–746, 2008.
  • [31] Surya Ganguli and Haim Sompolinsky. Compressed sensing, sparsity, and dimensionality in neuronal information processing and data analysis. Annual Review of Neuroscience, 35(1):485–508, 2012. PMID: 22483042.
  • [32] Ashok Litwin-Kumar, Kameron Decker Harris, Richard Axel, Haim Sompolinsky, and L.F. Abbott. Optimal degrees of synaptic connectivity. Neuron, 93(5):1153 – 1164.e7, 2017.
  • [33] Shawn R. Olsen, Vikas Bhandawat, and Rachel Irene Wilson. Divisive normalization in olfactory population codes. Neuron, 66(2):287–299, Apr 2010. 20435004[pmid].
  • [34] Hong Lei, Thomas A. Christensen, and John G. Hildebrand. Local inhibition modulates odor-evoked synchronization of glomerulus-specific output neurons. Nature Neuroscience, 5:557 EP –, May 2002. Article.
  • [35] Baktash Babadi and Haim Sompolinsky. Sparseness and expansion in sensory representations. Neuron, 83(5):1213 – 1226, 2014.
  • [36] Richard S Sutton and Andrew G Barto. Reinforcement Learning. MIT Press, 1998.
  • [37] B. Settles. Active Learning.

    Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool, 2012.

  • [38] Josh Attenberg and Foster Provost. Why label when you can search?: Alternatives to active learning for applying human resources to build classification models under extreme class imbalance. In Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’10, pages 423–432, New York, NY, USA, 2010. ACM.
  • [39] Pedro D. Maia and J. Nathan Kutz. Reaction time impairments in decision-making networks as a diagnostic marker for traumatic brain injuries and neurological diseases. Journal of Computational Neuroscience, 42:323–347, 2017.
  • [40] Elizabeth J. Hong and Rachel I. Wilson. Simultaneous encoding of odors by channels with diverse sensitivity to inhibition. Neuron, 85(3):573 – 589, 2015.
  • [41] Wei Ji Ma, Jeffrey M. Beck, Peter E. Latham, and Alexandre Pouget. Bayesian inference with probabilistic population codes. Nature Neuroscience, 9:1432 EP –, Oct 2006. Article.
  • [42] Guozhong An. The effects of adding noise during backpropagation training on a generalization performance. Neural Comput., 8(3):643–674, April 1996.
  • [43] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. CoRR, abs/1312.6199, 2013.
  • [44] Yoshua Bengio and Asja Fischer. Early inference in energy-based models approximates back-propagation. arXiv e-prints, abs/1510.02777, October 2015.
  • [45] Katherine I. Nagel and Rachel I. Wilson. Biophysical mechanisms underlying olfactory receptor neuron dynamics. Nature Neuroscience, 14:208 EP –, Jan 2011. Article.
  • [46] Nina Deisig, Martin Giurfa, Harald Lachnit, and Jean-Christophe Sandoz. Neural representation of olfactory mixtures in the honeybee antennal lobe. European Journal of Neuroscience, 24(4):1161–1174, 2006.
  • [47] Jeffrey A. Riffell, H. Lei, and John G. Hildebrand. Neural correlates of behavior in the moth manduca sexta in response to complex odors. Proceedings of the National Academy of Sciences, 106(46):19219–19226, 2009.
  • [48] Jeffrey A. Riffell, Hong Lei, Thomas A. Christensen, and John G. Hildebrand. Characterization and coding of behaviorally significant odor mixtures. Current Biology, 19(4):335 – 340, 2009.
  • [49] Rachel I. Wilson and Gilles Laurent. Role of gabaergic inhibition in shaping odor-evoked spatiotemporal patterns in the drosophila antennal lobe. Journal of Neuroscience, 25(40):9069–9079, 2005.
  • [50] Shawn R. Olsen, Vikas Bhandawat, and Rachel I. Wilson. Excitatory interactions between olfactory processing channels in the ¡em¿drosophila¡/em¿ antennal lobe. Neuron, 54(4):667, 2008.
  • [51] Shawn R. Olsen and Rachel I. Wilson. Lateral presynaptic inhibition mediates gain control in an olfactory circuit. Nature, 452:956 EP –, Mar 2008. Article.
  • [52] Marcus Sjoholm. Structure and function of the moth mushroom body. Swedish Univ of Agricultural Sciences, Alrarp, 2006. PhD thesis.
  • [53] Gregory S.X.E. Jefferis, Christopher J. Potter, Alexander M. Chan, Elizabeth C. Marin, Torsten Rohlfing, Calvin R. Maurer Jr., and Liqun Luo. Comprehensive maps of drosophila higher olfactory centers: Spatially segregated fruit and pheromone representation. Cell, 128(6):1187 – 1203, 2007.
  • [54] Andrew C. Lin, Alexei M. Bygrave, Alix de Calignon, Tzumin Lee, and Gero Miesenböck. Sparse, decorrelated odor coding in the mushroom body enhances learned odor discrimination. Nature Neuroscience, 17:559 EP –, Feb 2014. Article.
  • [55] Eyal Gruntman and Glenn C. Turner. Integration of the olfactory code across dendritic claws of single mushroom body neurons. Nature Neuroscience, 16:1821 EP –, Oct 2013. Article.
  • [56] Sean X. Luo, Richard Axel, and L. F. Abbott. Generating sparse and selective third-order responses in the olfactory system of the fly. Proceedings of the National Academy of Sciences, 107(23):10713–10718, 2010.
  • [57] Javier Perez-Orive, Ofer Mazor, Glenn C. Turner, Stijn Cassenaer, Rachel I. Wilson, and Gilles Laurent. Oscillations and sparsening of odor representations in the mushroom body. Science, 297(5580):359–365, 2002.
  • [58] SJ Caron, V Ruta, LF Abbott, and R Axel. Random convergence of olfactory inputs in the drosophila mushroom body. Nature, 497(5):113–7, 2013.
  • [59] Randolf Menzel and Gisela Manz. Neural plasticity of mushroom body-extrinsic neurons in the honeybee brain. Journal of Experimental Biology, 208(22):4317–4332, 2005.
  • [60] Jeffrey A. Riffell, Hong Lei, Leif Abrell, and John G. Hildebrand. Neural basis of a pollinator’s buffet: Olfactory specialization and learning in manduca sexta. Science, 2012.
  • [61] Ana F. Silbering and C. Giovanni Galizia. Processing of odor mixtures in the drosophila antennal lobe reveals both global inhibition and glomerulus-specific interactions. Journal of Neuroscience, 27(44):11966–11977, 2007.
  • [62] Desmond J. Higham. An algorithmic introduction to numerical simulation of stochastic differential equations. SIAM Rev., 43(3):525–546, March 2001.
  • [63] D. O. Hebb. The organization of behavior : a neuropsychological theory. Wiley New York, 1949.
  • [64] W. Shane Grant, James Tanner, and Laurent Itti. Biologically plausible learning in neural networks with modulatory feedback. Neural Networks, 88(Supplement C):32 – 48, 2017.
  • [65] Albert-Laszlo Barabasi and Reka Albert. Emergence of scaling in random networks. Science, 286(5439):509–512, 1999.
  • [66] Eli Shlizerman, Jeffrey A. Riffell, and J. Nathan Kutz. Data-driven inference of network connectivity for modeling the dynamics of neural codes in the insect antennal lobe. Frontiers in Computational Neuroscience, 8:70, 2014.