The Gabor-Einstein Wavelet: A Model for the Receptive Fields of V1 to MT Neurons

Our visual system is astonishingly efficient at detecting moving objects. This process is mediated by the neurons which connect the primary visual cortex (V1) to the middle temporal (MT) area. Interestingly, since Kuffler's pioneering experiments on retinal ganglion cells, mathematical models have been vital for advancing our understanding of the receptive fields of visual neurons. However, existing models were not designed to describe the most salient attributes of the highly specialized neurons in the V1 to MT motion processing stream; and they have not been able to do so. Here, we introduce the Gabor-Einstein wavelet, a new family of functions for representing the receptive fields of V1 to MT neurons. We show that the way space and time are mixed in the visual cortex is analogous to the way they are mixed in the special theory of relativity (STR). Hence we constrained the Gabor-Einstein model by requiring: (i) relativistic-invariance of the wave carrier, and (ii) the minimum possible number of parameters. From these two constraints, the sinc function emerged as a natural descriptor of the wave carrier. The particular distribution of lowpass to bandpass temporal frequency filtering properties of V1 to MT neurons (Foster et al 1985; DeAngelis et al 1993b; Hawken et al 1996) is clearly explained by the Gabor-Einstein basis. Furthermore, it does so in a manner innately representative of the motion-processing stream's neuronal hierarchy. Our analysis and computer simulations show that the distribution of temporal frequency filtering properties along the motion processing stream is a direct effect of the way the brain jointly encodes space and time. We uncovered this fundamental link by demonstrating that analogous mathematical structures underlie STR and joint cortical spacetime encoding. This link will provide new physiological insights into how the brain represents visual information.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 13

page 14

page 15

page 20

page 22

page 23

page 25

07/31/2015

A Sinc Wavelet Describes the Receptive Fields of Neurons in the Motion Cortex

Visual perception results from a systematic transformation of the inform...
04/16/2019

Response of Selective Attention in Middle Temporal Area

The primary visual cortex processes a large amount of visual information...
01/10/2019

Early recurrence enables figure border ownership

The face-vase illusion introduced by Rubin demonstrates how one can swit...
12/28/2015

Approximate Hubel-Wiesel Modules and the Data Structures of Neural Computation

This paper describes a framework for modeling the interface between perc...
04/08/2014

Notes on Generalized Linear Models of Neurons

Experimental neuroscience increasingly requires tractable models for ana...
07/20/2016

The encoding of proprioceptive inputs in the brain: knowns and unknowns from a robotic perspective

Somatosensory inputs can be grossly divided into tactile (or cutaneous) ...
03/08/2022

Joint rotational invariance and adversarial training of a dual-stream Transformer yields state of the art Brain-Score for Area V4

Modern high-scoring models of vision in the brain score competition do n...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

1.1 Aim

To introduce a model which innately and efficiently represents the salient properties of receptive fields of neurons in the V1 to MT motion processing stream.

1.1.1 Specific Aim

As one traces the neural pathway connecting the input layers of V1 to area MT, neurons become increasingly specialized for motion detection. In ascending order, attributes such as orientation selectivity, direction selectivity, speed tuning, increasing preferred speeds, component spatial frequency selectivity, and pattern (plaid) spatial frequency selectivity are sequentially acquired. Furthermore, the temporal frequency filtering properties of the neuronal population changes in a particular way along this pathway. Specifically, the proportion of bandpass temporal frequency filtering neurons to lowpass temporal frequency filtering neurons increases. Existing receptive field models do not represent this fundamental emergent property. Our specific aim is to introduce and describe a model for the receptive fields of V1 to MT neurons which innately and efficiently represents the aforementioned properties.

1.2 Motivation and Background

The detection of moving objects in our environment is both critical to our experience of the world and to our very survival. Imagine a hunter in rapid pursuit of a small agile animal. Both predator and prey detect abrupt changes in velocity occurring over an interval of less than a tenth of a second. The precision of our visual motion detection system is undoubtedly striking.

The middle temporal (MT) area of the mammalian brain plays a key role in the visual processing of motion. The importance of area MT in motion processing is supported by a growing wealth of electrophysiological evidence which was initiated by Dubner and Zeki’s 1971 report (Dubner and Zeki, 1971; Zeki, 1974, 1980; Van Essen, Maunsell, and Bixby, 1981; Maunsell and Van Essen, 1983a, b; Felleman and Kaas, 1984; Ungerleider and Desimone, 1986; Maunsell and Newsome, 1987; Rodman and Albright, 1989; Movshon and Newsome, 1996; Smith, Majaj, and Movshon, 2005). For purposes of motion processing, the main input signals into MT originate in specialized cells of the primary visual cortex (V1).

Kuffler’s early studies of retinal ganglion cell response properties led to similar studies of simple cortical cells by Hubel and Wiesel. Together, their work generated great interest in neuronal receptive fields (Kuffler, 1953; Hubel, 1957, 1959; Hubel and Wiesel, 1959; Marčelja, 1980; Daugman, 1985). The functional forms of these receptive fields are at once beautifully simple yet enormously complex. Hence mathematical models have been used in step with electrophysiological studies to advance our understanding of their properties. Initially, focus was predominantly on their spatial structure (Marčelja, 1980; Gabor, 1946). Now, however, their temporal structure is increasingly studied in tandem. In particular, for most neurons in the V1 to MT processing stream, it is now appreciated that spatial and temporal features cannot be studied separately. They are spatiotemporally inseparable entities. In particular, motion is encoded by orientation in the spectral domain, and spatiotemporally-oriented filters are therefore motion detectors (Watson and Ahumada, 1983, 1985; Heeger, 1987). Therefore at first glance it may seem an easy matter to mathematically model motion detecting neurons. The challenge, however, is to develop physiologically sound receptive field models which reflect the hierarchical structure of the motion processing stream. Various models do exist which are spatiotemporally-oriented filters, and are therefore motion detectors from a mathematical standpoint (Adelson and Bergen, 1985; Qian, 1994; Qian, Andersen, and Adelson, 1994; Qian and Andersen, 1997; Qian and Freeman, 2009). However, they fail to represent one of the most salient characterizing attributes of the motion processing stream: the lowpass to bandpass distribution of temporal frequency filtering properties along the V1 to MT specialization hierarchy. This feature is discussed further in the next subsection.

In addition to the above temporal frequency filter property profile, the neurons of the V1 to MT motion processing stream have certain characteristics which set them apart and prescribe bounds on models of their spatiotemporal receptive field structure. Namely, they are tuned to specific orientations, directions, and speeds (Movshon and Newsome, 1996); and they are more likely than other V1 cells to exhibit “end-stopping” phenomena (Sceniak, Hawken, and Shapley, 2001). Of the V1 cells, the subclass with direct synaptic projections from V1 to MT are the most specialized towards motion and are thought to be the main channels of motion substrates incident into MT neurons. For instance, Churchland et al showed that V1 and MT neurons lose direction selectivity for similar values of spatial disparity, suggesting V1 is MT’s source of direction selectivity (Churchland, Priebe, and Lisberger, 2005). Additionally, these V1 to MT neurons cluster motion substrate properties such as strong direction selectivity and tuning to high speeds. For example, Foster et al found that highly direction selective neurons in the macaque V1 and V2 were more likely to be tuned to higher temporal frequencies and lower spatial frequencies, i.e. higher speeds (Foster, Gaska, Nagler, and Pollen, 1985). In other words, neurons that are highly direction selective are more likely to also be tuned to higher speeds. Such neurons are higher up in the V1 to MT hierarchy. Similarly, McLean and Palmer studied direction selectivity in cat striate cortex, and found that space-time inseparable cells were direction selective while space-time separable cells were not direction selective McLean and Palmer (1989). Again reflecting the hierarchy of the motion processing stream.

The distribution of some distinct anatomical and histological features have been shown to correlate with the motion-specialization hierarchy. For instance, Shipp and Zeki’s retrograde tracer studies showed that the majority of direct V1 to MT projecting neurons originate within layer 4B (Shipp and Zeki, 1989). Based on their results, they suggested that layer 4B contained a functionally subspecialized anatomically segregated group whose members each have direct synaptic connections with MT cells. The exact properties of the V1 to MT projectors are likely to be intermediate between those of cells in V1 layer 4C and MT neurons. This is because input into MT appears to be largely constituted of magnocellular predominant streams from 4C, while parvocellular predominant streams may play a much less role  (Livingstone and Hubel, 1988; Maunsell, Nealey, and DePriest, 1990; Yabuta, Sawatari, and Callaway, 2001). Progressive specialization along the V1 to MT motion processing stream is also observable in the morphological characteristics of the neurons. For instance, distinguishing attributes of the V1-MT or V2-MT projectors such as size, arborization patterns, terminal bouton morphology, and distribution have been observed  (Rockland, 1989, 1995; Anderson, Binzegger, Martin, and Rockland, 1998; Anderson and Martin, 2002). Sincich and Horton demonstrated that layer 4B neurons projecting to area MT were generally larger than those projecting to layer V2. Overall, neurons in the V1 to MT motion processing stream are highly and progressively specialized. Hence description of their receptive fields requires adequately specialized models which reflect not only their individual attributes, but also the emergent hierarchical properties of the network. Next we discuss one of the most fundamental of such emergent network properties: the particular distribution of temporal frequency filtering types along the stream (Foster et al., 1985; DeAngelis et al., 1993b; Hawken et al., 1996).

1.3 Temporal Frequency Filtering Property Distribution

Hawken et al found that direction-selective cells were mostly bandpass temporal frequency filters, while cells which were not direction-selective were equally distributed into bandpass and lowpass temporal frequency filtering types (Hawken, Shapley, and Grosof, 1996). Foster et al found a similar phenomenon in macaque V1 and V2 neurons. V1 neurons were more likely to be lowpass temporal frequency filters, while their more specialized downstream heirs, V2 neurons, were more likely to be bandpass temporal frequency filters (Foster, Gaska, Nagler, and Pollen, 1985). Less specialized neurons located anatomically upstream (lower down in the hierarchy) are more likely to have lowpass temporal frequency filter characteristics, while more specialized neurons located anatomically downstream (higher up in the hierarchy) are more likely to display bandpass temporal frequency filter characteristics. For example, LGN cells are at best only weakly tuned to direction and orientation ( Xu et al. (2001)Ferster and Miller (2000)) and are hence equally distributed into lowpass and bandpass categories. Layer 4B V1 cells and MT cells on the other hand, are almost all direction selective  (Maunsell and Van Essen, 1983b) and hence are mostly bandpass temporal frequency filters. We firmly believe this classification is not arbitrary, but instead is a direct manifestation of the particular spatiotemporal structure of the V1 to MT motion processing stream. Consequently, the receptive field model must reflect this increased tendency for bandpass-ness with ascension up the hierarchy. In other words, the representation scheme must be one that is inherently more likely to deliver bandpass-ness to a more specialized cell (such as in layer 4B or MT) and lowpass-ness to a less specialized cell (such as in layer 4A, 4C, or 4C). However, none of the existing receptive field models reflect this underlying spatiotemporal structure. In contrast, as will be seen in Section (2) below, the Gabor-Einstein wavelet’s wave carrier is a sinc function which directly confers the aforementioned salient property. Individual Gabor-Einstein basis elements are lowpass temporal frequency filters; and bandpass temporal frequency filters can only be obtained by combinations of basis elements. A study by DeAngelis and colleagues also corroborates the above. They found that temporally monophasic V1 cells in the cat were almost always low pass temporal frequency filters, while temporally biphasic or multiphasic V1 cells were almost always bandpass temporal frequency filters (DeAngelis, Ohzawa, and Freeman, 1993b). The explanation for this finding is inherent and explicit in the Gabor-Einstein wavelet basis, where bandpass temporal character necessarily results from biphasic or multiphasic combination. All monophasic elements on the other hand, are lowpass temporal frequency filters. However, we will see that according to the model, the converse is not true: i.e. not all lowpass temporal frequency filters are monophasic, and not all multiphasic combinations yield bandpass temporal frequency filters.

1.4 Analogy to Special Relativity

Perceptual and electrophysiological studies reveal intriguing similarities between how space and time are mixed in the visual cortex and how they are mixed in the special theory of relativity (STR). For instance, within certain limits, binocular neurons cannot distinguish a temporal delay from a spatial difference. As a result, inducing a monocular time delay by placing a neutral density filter over one eye but not the other, causes a pendulum swinging in a 2D planar space to be perceived as swinging in 3D depth space. This is known as the Pulfrich effect. Also, most neurons in the V1 to MT stream are maximally excited only by stimuli moving at that neuron’s preferred speed. They are speed tuned. In this subsection, we briefly review and summarize the special theory of relativity. We identify the stroboscopic pulfrich effect and speed tuning as cortical analogues of STR’s joint encoding of space and time. And finally, we explain how STR is used in (and motivates) the design of the Gabor-Einstein wavelet.

1.4.1 Review of Special Relativity

In 1905, Albert Einstein proposed the special theory of relativity (Einstein, 1905). He was motivated by one thing: a firm belief that Maxwell’s equations of electromagnetism are laws of nature. Maxwell’s equations are a classical description of light (Maxwell, 1861a, b, 1862a, 1862b). According to Maxwell, light is the propagation of electric and magnetic waves interweaving in a particular way perpendicular to each other. The speed of propagation in a vacuum is the constant m/s. The other governing constraint on STR is the relativity principle. It has been around and generally accepted since the time of Galileo, and it states that the laws of nature are true and the same for all non-accelerating observers. Einstein believed Maxwell’s equations to be laws of nature, and therefore to satisfy the relativity principle. In other words, Einstein believed the speed of light must be the same to all non-accelerating observers regardless of their velocity relative to each other. For the speed of light to be fixed, something(s) had to yield and be unfixed. What had to yield were the components of the definition of speed, i.e. space and time. Space and time had to become functions of relative velocity and of each other, to ensure the speed of light remains constant to all non-accelerating observers. The transformation rule —of space and time coordinates between two observers moving with constant velocity relative to each other— is called the Lorentz equations. It specifies how space and time are mixed in STR. More specifically, the Lorentz equations are the transformation rule between the (spacetime) coordinates of two inertial reference frames. For two inertial reference frames, and , moving exclusively in the direction with constant velocity relative to each other, the Lorentz transformation is given by,

(1)

where () are the coordinates of , () are the coordinates of , is the speed of light, and is the Lorentz factor and is given by,

(2)

We demonstrate below that the stroboscopic pulfrich effect is analogous to STR in one respect, and speed tuning of neurons is analogous to STR in another respect.

1.4.2 The Stroboscopic Pulfrich Effect

In the classical pulfrich effect, a monocular temporal delay results in a perception of depth in pendular swing (Pulfrich, 1922). It has a simple geometric explanation. By the time the temporally delayed eye “sees” the pendulum at a given retinal position, the pendulum has advanced to a further position in its trajectory, hence the visual cortex always receives images that are spatially offset between the eyes. The spatial disparity results in stereoscopic depth perception. This form of pulfrich phenomena is termed classical to distinguish it from the stroboscopic pulfrich effect. In the stroboscopic pulfrich effect, the pendulum is not a continuously moving bulb, but instead is a strobe light which samples the trajectory of the classical pendulum in time and space (Lee, 1970; Morgan, 1976; Burr and Ross, 1979; Morgan, 1979). The stroboscopic pulfrich effect cannot be explained by the simple geometric illustration that explains the classical pulfrich effect. This is because depth is still perceived in sequences involving a temporal delay in flash between eyes, but particularly lacking an interocular spatial disparity in the flash (Burr and Ross, 1979).

(a)
(b)
Figure 1: The pulfrich effect: Top figure, (a), shows planar 2D pendular swing. Bottom figure, (b), shows perception of 3D pendular swing (depth) resulting from interocular temporal delay.

In stereoscopic depth perception, a planar (x,y) difference in retinal image position () transforms into (is perceived as) a displacement along the direction. In the stroboscopic pulfrich effect, a temporal difference () between eyes in retino-cortical transmission of image signal, results in a planar pendulum trajectory being perceived as (transforming into) depth. In STR, the transformation is between two inertial reference frames moving relative to each other. By analogy, we assert that for the stroboscopic pulfrich effect, there is a transformation between the interocular retinal space () and the perceptual depth space (). We identify the variables () of the interocular space as the difference between eyes of the corresponding variables in the retino-cortical space i.e. is the difference between the left and right eye retinal positions of an image; and is the difference between the right and left eyes in retino-cortical image transmission times. The transformation may take the form , were is proportional to pendulum velocity and is a function of . Note the similarity between and the equation for in the Lorentz transformation, . This analogy suggests that similar mathematical structures underlie STR and the joint encoding of space and time in the visual cortex. The stroboscopic pulfrich effect is not by itself direct evidence of joint encoding of spatial and temporal disparities  (Read and Cumming, 2005a, b, c), however direct electrophysiological evidence does exist for such linkage in cortical encoding (Carney, Paradiso, and Freeman, 1989; Anzai, Ohzawa, and Freeman, 2001; Pack, Born, and Livingstone, 2003). This mathematical similarity influences our design of the Gabor-Einstein wavelet in a particular way which we describe in Section (1.5) below. But first, we discuss another neuronal phenomenon, speed tuning, which is analogous to STR in a certain physical sense.

1.4.3 Speed Tuning

Most motion processing stream neurons in V1 or area MT are speed tuned. They have a preferred speed, which is the speed of moving stimuli to which they maximally respond. Other speeds also elicit a response, but with a lower rate of action potential. When presented moving sine wave grating stimuli, truly speed tuned neurons respond maximally to their preferred speed independent of the spatial or temporal frequency of the stimuli. The velocity, , of a moving wave is,

(3)

where is temporal frequency and is spatial frequency. In higher dimensions,

(4)

where is the spatial frequency in the direction, is the spatial frequency in the direction, is the component of velocity, and is the component of velocity. From both equations above, it is clear that for a speed tuned neuron, a change in the spatial frequency of the stimuli would necessary require a complementary change in the temporal frequency. Hence a broad range of stimuli with widely varying spatial and/or temporal frequencies can maximally excite the neuron, so long as the above equations are satisfied for the neuron’s preferred speed . This is analogous to how the special theory of relativity is motivated by fixing the speed of light. In the case of STR the variables which necessarily change in complementary fashion are the space and time coordinates as described in the Lorentz equations.

1.5 Design of the Gabor-Einstein Wavelet

We constrained the Gabor-Einstein model by requiring the following properties:

  1. The minimum possible number of parameters

  2. Relativistic-invariance of the wave carrier

1.5.1 Minimal Number of Parameters

In addition to orientation in the spatiotemporal frequency domain, localization in the space, time, spatial frequency, and temporal frequency domains are the essential features of the V1 to MT neuron receptive field. These are the most basic attributes of the receptive field. As such, they correspond with the minimum number of model parameters needed. Specifically, speed tuning, the spatial and the temporal localization envelopes, the amplitude modulation parameter, and the spatial and temporal frequencies must all be represented. If one adopts the notion of a gaussian spatial localizer (Marčelja, 1980; Daugman, 1985), the minimum number of parameters add up to eight: four for the spatial envelope and four for the wave carrier. The four necessary spatial envelope parameters are: amplitude factor,

; variance in the

direction, ; variance in the direction, ; and spatial orientation of envelope, . The four necessary wave carrier parameters are: temporal frequency, ; spatial frequency in the x direction, ; spatial frequency in the direction , and the phase, . The orientation of the wave carrier is not an independent parameter. We use it in computer simulations below only for convenience. The essential feature that is not yet explicitly accounted for in the above parameter tally is the temporal envelope of the receptive field. This can be implemented either through the gaussian envelope or through the wave carrier, and in either case may conceivably involve an additional parameter. The next design criteria, relativistic-invariance of the wave carrier, resolves this ambiguity without adding another parameter.

1.5.2 Relativistic-invariance of the Wave Carrier

Given the analogies of spatiotemporal mixing in the visual cortex to spatiotemporal mixing in the special theory of relativity, it is reasonable to conjecture that their endpoints are within proximity of each other. Einstein eventually sought a lexicon in which physical laws are expressed the same way in any two inertial reference frames. That lexicon is called relativistic-invariance (or Lorentz-invariance). By analogy, we ultimately seek a lexicon in which physical laws can be expressed the same way in both the interocular and perceptual spaces. Lorentz-invariance of physical law is the specification (or endpoint) arising out of STR. Hence, requiring Lorentz-invariance of the receptive field’s wave carrier is desirable. Moreover, this immediately resolves the above ambiguity regarding implementation of the temporal envelope profile. The temporal envelope must be implemented via the wave carrier for relativistic invariance to hold. And since the receptive field amplitude eventually falls, the sinc function emerges as a natural descriptor. Furthermore, the sinc function has the distinct advantage of not introducing any additional parameters.

1.6 Summary

The way space and time are mixed in the visual cortex bears resemblance to the way they are mixed in the special theory of relativity. We demonstrated that the stroboscopic pulfrich effect and speed tuning are cortical analogues of the spacetime mixing mechanics of STR. In STR, the mixture of space and time is described by the Lorentz transform which relates the spacetime coordinates of inertial reference frames moving with constant velocity relative to each other. In the striate and extrastriate motion processing stream, the analogous transformation is between the 3D interocular space (x,y planar inter-retinal space + inter-retinocortical time) and the 3D perceptual space (x,y,z 3D space). Though on the surface, STR and cortical spacetime encoding appear to be unrelated processes, we conjecture and have partly shown a shared underlying mathematical structure. This structure can be exploited to deepen our understanding of cortical encoding of motion, depth, and more fundamentally, the joint encoding of space and time in the visual cortex. Here, we proceeded by requiring relativistic-invariance of the receptive field’s wave carrier. We simultaneously required that the model be constrained to the minimum possible number of parameters. Under these constraints, the sinc function with energy-momentum relation as argument emerged as a natural descriptor of the receptive field’s wave carrier. Furthermore, the Gabor-Einstein wavelet explains a number of salient physiological attributes of the V1 to MT spectrum. Chief amongst these being the distribution of bandpass to lowpass temporal frequency filter distribution profile; which we postulate is a fundamental manifestation of the way space and time are mixed in the visual cortex.

The remainder of this paper is organized as follows: Section (2) introduces the Gabor-Einstein wavelet. Section (3) presents computer simulations. Section (4) is the discussion. It includes the following subsections: Subsection (4.1

) introduces a simple framework for classifying and naming the components and hierarchical levels of receptive field models. Subsection (

4.1) also identifies the Gabor-Einstein’s place within this larger framework of receptive field modeling. Subsection (4.2) briefly discusses related work. Subsection (4.3) discusses some phenomena which cannot be explained by models such as the native Gabor-Einstein wavelet in isolation, i.e. models of single neuron receptive fields early in the visual pathway. Specifically, it discusses some higher order phenomena and non-linearities which necessarily arise from neuronal population and network interactions. Section (5) concludes the paper.

2 The Gabor-Einstein Wavelet

In this section, we present the Gabor-Einstein wavelet. It has the following properties:

  • Like the Gabor function, it is a product of a gaussian envelope and a sinusoidal wave carrier.

  • It differs from the Gabor function in that its wave carrier is a relativistically-invariant sinc function whose argument is the energy-momentum relation.

  • Its gaussian envelope contains only spatial arguments. i.e. its spatial envelope is not time-dependent.

  • It has the minimum possible number of parameters.

  • Its fourier transform is the product of a mixed frequency gaussian and a temporal frequency step function.

  • Like the Gabor function, it generates a quasi-orthogonal basis.

We define the Gabor-Einstein wavelet as follows,

(5)

where the constant multiplier is amplitude; and are the gaussian variances in the x and y directions; and are the respective and coordinates of the gaussian center; , , and are the frequencies of the wave carrier in the , , and directions respectively; is the sinusoid phase; and the sinc function is defined as,

(6)

We have rotated our coordinates by an angle from a reference state to a state which we denote for notational simplicity. The transformation,

(7)

is given by,

(8)

We proceed here with the following instance of the Gabor-Einstein wavelet,

(9)

where we have set equal to one. We can always do so for one reference neuron by simply defining the unit of time as , the duration the neuron’s frame cycle. As we will see, this value, , is the shortest frame duration to which the neuron can respond. In the above equation, we have also set , and .

The fourier transform is as follows,

(10)

where is given by,

(11)

and the sign function is defined as,

(12)

The above fourier transform was obtained using Mathematica symbolic software. The fourier transform of the general case will likely be challenging to obtain analytically either by hand or symbolic software. Hence, we anticipate numerical methods may have an important role to play.

The maximum magnitude of the above response function, Equation (10), is attained where the argument of the exponent is zero, i.e. where,

(13)

The neuron’s preferred spatial frequency, , is dependent on the temporal frequency, , of the stimulus, and is given by the above bivariate quadratic equation’s solution,

(14)

The location where the response maximum is attained is a parametrized curve in 3D frequency space. It is given by,

(15)

Although the location in space where the neuron’s maximum response is attained depends on , the magnitude of the maximum response is itself a constant. It is given by,

(16)

The half magnitude response is attained at values of satisfying,

(17)

Taking the natural logarithm of the above equation yields,

(18)

Defining and , Equation (18) becomes,

(19)

In the above form, one readily recognizes this as the ellipse centered at , whose principal axes radii are and in the and directions respectively. The long axis in the frequency domain is the short axis in the spatial domain and vice versa. Without loss of generality, we can assume that prior to rotation of the spatial axes by angle , the long axis of the receptive field envelope is parallel to the x axis. Then the orientation of the on-off bands, i.e. planes of the spatial wave, are aligned parallel to the long axis of the envelope when the rotation angle, , is related to the polar angle by the relationship,

(20)

On the other hand, the on-off bands are perpendicular to the long axis of the envelope when,

(21)

In the case of Equation (20), the half magnitude frequency bandwidth is readily seen to equal , the length of the short axis, while in the case of Equation (21), the half magnitude frequency bandwidth equals

, the length of the short axis. The cases of skewed alignment take on values between

and and are also computable from the geometry. Unlike the preferred spatial frequency, the half-magnitude frequency bandwidth is not dependent on the temporal frequency of the stimulus.

In summary, the Gabor-Einstein wavelet spatiotemporal receptive field model predicts that the magnitude of a V1 to MT neuron’s preferred spatial frequency is linearly dependent on the temporal frequency of the stimulus as shown in Equation (14). In the next section, we present some receptive field simulations using the Gabor-Einstein wavelet.

(a)
(b)
(c)
(d)
Figure 2: Receptive field spatial envelopes. The following parameters were used in the above gaussians. In Fig(2a), , , and ; In Fig(2b) , , and ; In Fig(2c) , , and ; In Fig(2d) , , and .

3 Computer Simulations

In this section, we present computer simulations demonstrating the Gabor-Einstein wavelet’s properties. We start with simulations which show the model’s adherence to basic physiological form. We then present simulations which demonstrate how the Gabor-Einstein model innately represents the temporal frequency filtering property distribution along the V1 to MT neuronal hierarchy.

The following notations are used in the figure captions: is the angle of rotation of the axes of the gaussian envelope as described in Equations (7) and (8).

is the angle of rotation of the axes of the wave carrier. ctr is the 2-component vector consisting of the

and coordinates of the gaussian envelope center respectively. is the 2-component vector consisting of the gaussian variances of the envelope in the and directions respectively. is phase of the wave carrier. is time. And is the 3-component vector consisting of the temporal, x-spatial, and y-spatial frequencies of the wave carrier respectively. For succinctness, , for instance, is taken to be equivalent to . The same short-hand notation applies to the other multi-component vectors.

3.1 Basic Physiological Form

Figure (2) shows receptive field envelopes with varying aspect ratios and centers; Figure (3) shows the sinc function wave carriers at varying times and at one different orientation; Figures (4) and (5) show Gabor-Einstein wavelets with varying envelope centers; Figure (7) shows Gabor-Einstein wavelets with varying spatial frequencies; and Figure (6) shows Gabor-Einstein wavelets of varying phase.

(a) ,
(b) ,
(c) ,
(d) ,
Figure 3: Receptive field wave carrier. 3D sinc waves model the wave carrier. The following parameters were common to each simulation in the figure: , , .
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4: The Gabor-Einstein wavelet. The following parameters were common to each simulation in the figure: , , , , , .

In Figure (4), one sees the anisometry that is conferred by the sinc function wave carrier. The anisometry increases from Fig (4a) through (4d), due to the rapid fall off in the sinc function amplitude outside of the central peak over the two half cycles on each side of zero. In contrast, Fig (4e) and Fig (4f) appear similar to each other in form, and each look more isometric than Fig (4d). This illustrates the presence of both isometric and anisometric basis elements which differ only in a single parameter (phase, or time, or envelope center). This expressiveness in the basis should allow for economical representation of a visual scene. Of note, the amplitudes of the more isometric elements Figs (4e) and (4f) are much smaller than those of the more anisometric preceding elements. If desired, this can be readily modulated by the constant amplitude factor parameter.

(a)
(b)
(c)
(d)
(e)
(f)
Figure 5: Gabor-Einstein space-modulation. The following parameters were common to each simulation in the figure: , , , , , .

Figure (5) shows 3-dimensional plots illustrating the above discussed isometry modulation feature of the Gabor-Einstein basis. Figure (7) shows receptive fields with various spatial frequency tuning. Figure (6) shows various receptive fields which are phase-shifted relative to each other. For any given spatial location in the receptive field, the peak amplitude of a cycle is phase dependent. This is a feature distinct to the Gabor-Einstein model. And like the anisometry-modulation feature, it confers expressiveness to the basis family. This should allow for efficient representation of natural visual scenes. In the model, time has a similar effect on peak amplitude. And this behavior is in agreement with space-time inseparable receptive field profiles in cat striate cortex, where a time dependent phase drift has been shown (DeAngelis, Ohzawa, and Freeman, 1993a).

(a)
(b)
(c)
(d)
Figure 6: Gabor-Einstein phase-modulation. The following parameters were common to each simulation in the figure: , , , , .
(a)
(b)
(c)
(d)
Figure 7: Gabor-Einstein frequency-modulation. The following parameters were common to each simulation in the figure: , , , , , , , .
(a) Sinc,
(b) Fourier of (a)
(c) Sinc,
(d) Fourier of (c)
(e) (c) minus (a)
(f) (d) minus (b)
Figure 8: Sinc function and its Fourier transform
(a)
(b)
(c)
Figure 9: Fourier representation of the 2D (x,t) Gabor-Einstein wavelet. . (a) is the surface plot, while (b) is the p colormap. The 3D analogue is more readily imagined as an extension of Figure (b). In particular, Figure (b) is a slice perpendicular to the y-spatial frequency (or “”) direction of the 3D extension. (c) is the (x,t) domain representation.
(a)
(b)
(c)
(d)
(e) (c) minus (a)
(f) (d) minus (b)
Figure 10: Construction of temporal frequency bandpass Gabor-Einstein wavelet. Each surface plot in the right column is of the same Gabor-Einstein element as its colormap neighbor on the left. Parameters used: .

3.2 Temporal Frequency Filter Property Distribution

Figure (8) shows the essential property which the Gabor-Einstein wavelet inherits from the sinc function. The sinc function’s fourier transform is a lowpass filter. Higher frequency sinc functions have wider bandwidth. Bandpass filters are formed by taking the difference between sinc functions of different frequency as illustrated in Figure (8). The Gabor-Einstein wavelet’s fourier transform has a step function factor inherited from the sinc function. This allows it explain the particular distribution of lowpass to bandpass temporal frequency filter properties of V1 to MT neurons (Foster et al 1985; DeAngelis et al 1993b; Hawken et al 1996) in a manner innately representative of the motion-processing stream’s neuronal hierarchy. For simplicity of illustration, Figure (9) shows a 2D (x,t) Gabor-Einstein element. Its fourier transform is shown in Figs (9a) and (9b), while its (x,t) domain representation is shown in Fig (9c). The lowpass temporal frequency nature of the fourier representation is apparent, as the support straddles the zero. Figure (10) shows the construction of temporal frequency bandpass Gabor-Einstein wavelets. As illustrated, the temporal frequency bandpass Gabor-Einstein element results from the difference of two temporal frequency lowpass Gabor-Einstein elements. Fig (10a) and Fig (10b) are plots of Equation (10), i.e. they are the fourier transform of the Gabor-Einstein wavelet described by Equation (9). We label this basis element “”, meaning it is the zero-centered lowpass temporal frequency filter whose temporal frequency bandwidth is one. Accordingly, Fig (10c) and Fig (10d) are plots of the “” Gabor-Einstein basis element, i.e. they are plots of the zero-centered lowpass temporal frequency filter whose temporal frequency bandwidth is three. Fig (10e) and Fig (10f) plot the bandpass temporal frequency filter basis element obtained by taking the difference of the “” and “” basis elements. The left-hand column of Figure (11) plots the Gabor-Einstein basis elements of Fig (10), while the right hand column plots the corresponding temporal response probed at the spatial origin of the receptive field. There is an increasing complexity of the temporal waveform structure as one progresses from lowpass to bandpass basis element.

All pure Gabor-Einstein basis elements are lowpass temporal frequency filters. On the other hand, the bandpass temporal frequency filter property necessarily results from summation of Gabor-Einstein basis elements. Hence bandpass temporal elements are necessarily complex (not pure). However, not all lowpass temporal frequency filters are pure; and not all basis summations yield bandpass temporal frequency filters. For instance, Figure (12) shows the sum of two pure Gabor-Einstein basis elements which yield another lowpass temporal frequency filter. The temporal waveform of the composite element is indeed more complex than that of its two pure constituents, however, it appears less complex than the composite temporal waveform of Fig (10).

(a)
(b)
(c)
(d)
(e) (c) minus (a)
(f) (d) minus (b)
Figure 11: Progressive complexity: The bandpass temporal frequency filters such as (e) are necessarily superpositions of two or more Gabor-Einstein wavelets. The profiles on the right are the temporal response weights probed at the spatial origin. Note that the pattern is more complex in the bandpass temporal frequency filter. Parameters used: .
(a)
(b)
(c)
Figure 12: A superposition of two lowpass filters which yields a hierarchically more complex neuron which is nonetheless low pass. Specifically, the above is a plot of “” + “”. Parameters used: .

4 Discussion

The neurons in the V1 to MT dorsal stream are highly specialized towards motion detection. Furthermore, they are embedded in a hierarchical order of increasing motion specialization. The earliest (least specialized) members of the chain are the non-speed tuned V1 cells, while the most specialized members are the MT cells. Naturally, LGN cells precede input layer V1 cells both anatomically as well as in level of motion specialization. Similarly, cells located in the medial superior temporal (MST) area and the parietal lobe are further along than MT cells both anatomically and in motion specialization. We have focused our current study on the V1 to MT spectrum.

We set out to obtain a good mathematical representation for the receptive fields of these neurons. The term good here implies that the representation should faithfully capture the salient features of the motion specialization spectrum and serve as an effective label or address schema for neurons in the motion processing stream. For instance, it is known that MT neurons receive input from V1 neurons and most directly from V1-to-MT projectors, therefore in a good basis, the representation for an MT neuron should result from some combination of representations of some V1 neuron(s). Furthermore, the resulting MT representation should naturally manifest known attributes of MT neurons.

Appropriate comparison and discussion of the current work and related work requires a more precise language than currently available. Hence, we begin this discussion by introducing a simple framework for classifying and naming the components and hierarchical levels of receptive field models.

4.1 Receptive Field Model Classification

In the literature, there is currently no clear categorization of the various mathematical descriptions of receptive fields. Ironically, the laudable increase in discovery of receptive field attributes and modulation mechanisms may have led to further conceptual entanglement of the mathematical forms of receptive fields. Here, we propose a simple framework to organize and classify receptive field descriptions and briefly review some basic concepts. This should help place the current work, the Gabor-Einstein model, in the appropriate context; and furthermore, should help organize and direct receptive field research. We will identify a receptive field by its constituents and method of construction. A receptive field can be constructed using two essential things:

  1. Primary input receptive fields

  2. Input combination scheme

The primary input receptive field is the response function of the neuron at the lowest level on the hierarchy of study. For instance, when studying the motion-processing stream between V1 and MST, the primary input receptive field is the response function of a V1 input layer neuron. An input combination scheme is a model for the network of neurons projecting onto the study neuron. For instance, a common scheme is linear summation of weighted V1 responses to yield an MT neuron. The input combination scheme can be classified according to the stage at which input processing, if any, occurs. Three classes arise:

  1. Pre-processing

  2. Intra-processing

  3. Post-processing

Pre-processing is modification of the input responses prior to combining them. For instance, V1 elements are squared in the motion-energy model, after which they are summed to yield the MT response (Adelson and Bergen, 1985, 1986). Intra-processing is modification of the input responses during the combination process. For example, the Soft MAX scheme (Lampl, Ferster, Poggio, and Riesenhuber, 2004; Riesenhuber and Poggio, 1999; Nowlan and Sejnowski, 1995). Post-processing is modification of the response after input combination. For instance, divisive normalization schemes (Heeger, 1992, 1993; Simoncelli and Heeger, 1998; Tsui, Hunter, Born, and Pack, 2010). The divisive normalization scheme, proposed by Heeger in 1992, is a neuron population based model of V1 to MT function. In that model, each neuron’s behavior is modulated by that of a population module of which it was a member. Specifically, its manifest response is its native response divided by the net sum of the responses of the other neurons in its population module plus a regularization factor. The model bears resemblance to receptor models in pharmacokinetics, where the regularization factor plays the role of a half-saturation constant. The third categorization attribute is the hierarchical level from which input originates relative to the study neuron. The three following classes arise:

  1. Feed-forward

  2. Isostratal

  3. Feedback

Feed-forward implies the input comes in from a lower hierarchical level. For instance V1 projecting to MT. Isostratal implies that the input comes in from the same level. For instance, normalization of MT response by neighborhood neuron pool (Heeger, 1992, 1993; Simoncelli and Heeger, 1998; Tsui, Hunter, Born, and Pack, 2010). Feedback implies the input comes in from a higher level in the hierarchy. Of note, most receptive fields likely receive contributions from all three types of inputs. In light of the above framework, the Gabor-Einstein model is a primary input receptive field for the V1 to MT motion processing stream. It is compatible with any of the aforementioned input combination schemes, hierarchical input routing schemes, and input processing schemes. The work presented in this paper focused mainly on the primary input receptive field, and for simplicity of illustration, assumed a basic (i.e. linear) input combination scheme. Certain important phenomena cannot be explained at the level of early (e.g. input layer V1) primary input receptive fields alone. Next, we discuss some aspects of such phenomena including end-stopping, a short selection of non-linearity mechanisms, and contrast-modulated speed tuning.

4.2 Related Work

A key distinguishing feature of the Gabor-Einstein model is how it naturally represents the distribution of lowpass to bandpass temporal frequency filtering properties along the V1 to MT motion specialization hierarchy. Ratio models are a class of models with a different objective. The ratio models aim to mechanistically model the way speed tuned units arise from combinations of non-speed tuned bandpass and low pass temporal frequency filters. These models are based on the idea that stimulus temporal frequency is proportional to the ratio of outputs of spatiotemporally separable bandpass to lowpass neuronal outputs (Tolhurst, Sharpe, and Hart, 1973; Adelson and Bergen, 1986; Perrone and Thiele, 2002; Perrone, 2004, 2005).

Various summation-based input combination schemes along the V1 to MT stream have been used. Heeger and colleagues proposed a two stage weighted linear summation model in which MT fields resulted from weighted sums of V1 fields (Heeger, 1992, 1993; Simoncelli and Heeger, 1998)

. Sereno as well as Nowlan & Sejnowski proposed multilevel neural network reinforcement learning models in which MT neuron response was a function of a linear summation of V1 responses 

(Sereno, 1993; Nowlan and Sejnowski, 1995). Tsui et al proposed a model in which MT response was obtained by a soft-max weighted summation of V1 inputs (Tsui, Hunter, Born, and Pack, 2010). Each of these schemes differs fundamentally in approach from the Gabor-Einstein model. Namely, we focus on the receptive field of a single neuron along the V1 to MT pathway, implicitly representing the network spatiotemporal structure in the inherent attributes of the basis itself. Specifically, the natural representation of temporal frequency filter distribution along the V1 to MT spectrum. In the Gabor-Einstein basis, the basis itself mandates summation as a means of ascending up the specialization hierarchy. This is a gratifying result, and supports the notion that the sinc function is a natural way to describe the V1 to MT spectrum.

4.3 Higher-order Phenomena

Here, we discuss certain higher order phenomena whose description exceeds the scope of single early neuron receptive fields.

4.3.1 End-stopping

End-stopping is a center-surround type phenomenon in which —for instance— a full length bar stimulus results in a submaximal response, while some shorter bar stimulus generates maximal response. In other words, the excitatory portion of the receptive field is shorter than the full length of the receptive field. Typically, the periphery of the receptive field, i.e. its end, elicits suppression (or “stopping”). Hence neurons exhibiting such behavior are said to be “end-stopped”.

Jones and colleagues found that majority (94%) of V1 cells in the macaque were end-stopped to various degrees (Jones et al., 2001). Moreover, the most prominently end-stopped V1 cells have been found in layer 4B which is known to be the dominant projection layer to MT (Sceniak et al., 2001). Furthermore, Tsui et al have argued a role for V1 end-stopping in MT motion integration and solution to the aperture problem (Tsui, Hunter, Born, and Pack, 2010). It seems therefore that a complete model of V1 to MT neurons must account for end-stopping. End-stopping phenomena would be difficult to explain at the single input layer V1 neuron stage, since it likely involves isostratal interactions. Tsui and colleagues modeled end-stopping by specifying the normalization pool as surround cells arranged along the orientation axis of a center cell. With this spatially-specific implementation of normalization, surround cell activity yielded end-stopping (Tsui et al., 2010). The response was then fit to a saturation kinetics model equivalent to Heeger’s divisive normalization model (Heeger, 1992). Regarding center-surround phenomena such as end-stopping, the Gabor-Einstein wavelet can be considered a classical receptive field. A scheme similar to that of Tsui et al can be used to implement end stopping using the Gabor-Einstein wavelet as both center and surround, or in hybrid with other wavelet transform as either center or surround. Of note, other schemes have also been used to implement end-stopping. For instance, Skottun proposed an orientation-modulated cancellation strategy (Skottun, 1998). The Gabor-Einstein wavelet is compatible with this scheme as well.

4.3.2 Nonlinearity Mechanisms

The perception of motion is likely a higher order phenomenon that emerges out of the elaborate nonlinear connection network of which MT is only a part. Some of these non-V1 areas from which MT receives input include LGN  (Sincich, Park, Wohlgemuth, and Horton, 2004; Nassi, Lyon, and Callaway, 2006), superior colliculus (Rodman, Gross, and Albright, 1990; Berman and Wurtz, 2010, 2011), and extrastriate regions (Maunsell and Van Essen, 1983a; Boussaoud, Ungerleider, and Desimone, 1990; Ungerleider and Desimone, 1986). Some combination of these non-V1 inputs may explain the persistence of visual responsiveness and direction selectivity after V1 lesions or cooling  (Girard, Salin, and Bullier, 1992; Azzopardi, Fallah, Gross, and Rodman, 2003; Rodman and Albright, 1989; Rodman, Gross, and Albright, 1990). Indeed several researchers have drawn specific attention to the necessity of non-linear models. Moreover, these nonlinearities are not just a feature of late (specialized) visual neurons, but have been demonstrated very early in the visual pathway. For example, Schwartz et al pointed out the need for nonlinear retinal ganglion cell receptive field models (Schwartz et al., 2012). And Rosenberg et al showed that LGN Y cells utilize a nonlinear mechanism to represent interference patterns (Rosenberg, Husson, and Issa, 2010). Fine mechanistic modeling of the single V1 to MT neuron’s receptive field’s most salient features is certain to yield valuable physiological insight into the myriad non-V1 connections to MT.

4.3.3 Contrast-Modulated Effects

Contrast-modulated speed tuning is likely unexplainable at the level of early primary input receptive fields of single neurons. Existing explanations of such phenomena appear to necessarily involve nonlinear isostratal interactions. For instance, divisive normalization (Heeger, 1992, 1993; Simoncelli and Heeger, 1998).

In general, the effects of contrast on the receptive field of V1 to MT neurons can safely be assumed to be highly modulated by neighboring neurons. Significant data exists to support such population encoding. For instance, DeAngelis et al showed that end and side stopping were strongest along the preferred excitatory orientation, yet superimposed suppressive stimuli was much more broadly tuned than the classical receptive field, suggesting a modulatory pool of neurons (DeAngelis et al., 1994). Such neuron pools also likely mediate the effects of contrast on speed tuning. Krekelberg et al found that most MT cells in alert macaques preferred lower speeds for lower stimulus contrast (Krekelberg, van Wezel, and Albright, 2006). Priebe et al found that a significant population of V1 cells have contrast-modulated speed tuning curves (Priebe, Lisberger, and Movshon, 2006). Livingstone and Conway found that the speed tuning curves of most V1 neurons in alert macaques were dependent on contrast stimuli. Specifically, for lower contrast stimuli, V1 cells shifted their tuning curves to lower speeds; while for higher contrast stimuli, the opposite was observed (Livingstone and Conway, 2007). Psychophysical studies also show that both high contrast stimuli and high spatial frequency stimuli are associated with higher perceived speeds (Brooks, Morris, and Thompson, 2011). The increase in preferred speed with contrast is a phenomenon that likely involves nonlinear isostratal interactions. As such, early single neuron receptive field models in isolation are unable to explain such phenomena. Nonetheless the primary input receptive field is the theoretical and physiological foundation of higher order phenomena. Hence it is essential to have a physiologically sound model such as the Gabor-Einstein wavelet which naturally generates the hierarchical specialization structure of the motion stream. The Gabor-Einstein wavelet can be readily plugged-in to divisive normalization schemes which may be necessary to explain contrast-modulated phenomena.

Figure 13: A schematic illustration of the V1 to MT motion specialization spectrum. The arrow gradient represents increasing motion specialization along the spectrum. In the second column, the neurons depict progressively complex specializations in size and arborization patterns along the spectrum. The third column depicts the temporal profile probed at the spatial origin of the receptive field. The waveform progresses from a pure sinc function in the upstream layers to composite forms downstream. The rightmost column depicts the increased proportion of bandpass to lowpass temporal frequency filters as one ascends the hierarchy.

5 Conclusion

In this paper, we introduced the Gabor-Einstein Wavelet, a new family of functions for modeling the receptive fields of neurons in the V1 to MT motion processing stream. We showed that the way space and time are mixed in the visual cortex has analogies to the way they are mixed in the special theory of relativity. We therefore constrained the Gabor-Einstein model to a relativistically invariant wave carrier and to the minimum possible number of parameters. These constraints yielded a sinc function wave carrier with energy-momentum relation as argument. The model innately and efficiently represents the temporal frequency filtering property distribution along the motion processing stream. Specifically, on the V1 end of the stream, the neuron population has an equal proportion of lowpass to bandpass temporal frequency filters; whereas on the MT end, they have mostly bandpass temporal frequency filters. From our analysis and simulations, we showed that the distribution of temporal frequency filtering properties along the motion processing stream is a direct effect of the way the brain jointly encodes space and time. We uncovered this fundamental link by demonstrating an analogous mathematical structure between the special theory of relativity and the joint encoding of space and time in the visual cortex. The Gabor-Einstein model and the experiments it motivates will provide new physiological insights into how the brain represents visual information.

Acknowledgements

The author thanks Rudi Weikard, Marius Nkashama, John Mayer, Ian Knowles, Xiaobai Sun, and Peter Blair for helpful suggestions on how to approach the fourier transform of the sinc function of multidimensional argument. He thanks Greg Schwartz and Ari Rosenberg for helpful discussion on nonlinearities in visual receptive fields. He thanks his mentors and role models in Ophthalmology at Howard University and the Washington area for their dedication to training residents: Robert A. Copeland Jr., Leslie S. Jones, Bilal Khan, Earl Kidwell, Janine Smith-Marshall, David Katz, William Deegan III, Melissa Kern, Ali Ramadan, Frank Spellman, Reggie Sanders, and Michael Rivers, Emily Chew, Brian Brooks, and Wai Wong. He thanks Claude L. Cowan Jr whose excellence inspired him to pursue a career in Medical Retina. He thanks his co-residents at Howard Ophthalmology for the wonderful experience which resulted from their enthusiasm for patient care and collegial learning: Sir Gawain Dyer, Salman J. Yousuf, Animesh Petkar, Ninita Brown, Mona Kaleem, Mikelson MomPremier, Saima Qureshi, Nikisha Richards, Neal Desai, Chris Burris, Natasha Pinto, Usiwoma Abugo, Chinwe Okeagu, and Katrina del Fierro. He thanks Richard Mooney, Michael Platt, Pate Skene, Fan Wang, Vic Nadler, and Kafui Dzirasa for supporting his membership to the Society for Neuroscience. He thanks Susan Elner, Mark W. Johnson, John R. Heckenlively, Paul P. Lee and all the amazing faculty at the University of Michigan–Ann Arbor (W.K. Kellogg Eye Center) for awarding me the clinical fellowship training position in Medical Retina. He thanks Stuart Fine and the board of directors of the Heed ophthalmic foundation (Nicholas J. Volpe, Stephen McLeod, Joan Miller, Eduardo Alfonso, David Wilson, Julia Heller, and Froncie Gutman) for inviting him to participate in the 8th Annual Heed resident’s retreat in Chicago Illinois. It was a delight to be at that retreat. He thanks Eydie Miller-Ellis, Mildred Olivier, and the the Rabb Venable Excellence in Research Program Board for selecting him as a participant and for encouraging a career in academia.

References

  • Adelson and Bergen (1985) E. H. Adelson and J. R. Bergen. Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am. A, 2(2):284–299, 1985.
  • Adelson and Bergen (1986) E. H. Adelson and J. R. Bergen. The extraction of spatiotemporal energy in human and machine vision. In Proc. IEEE Workshop on Visual Motion, Charleston, pages 151–156, 1986.
  • Anderson and Martin (2002) J. C. Anderson and K. A. C. Martin. Connection from cortical area V2 to MT in macaque monkey. Journal of Comparative Neurology, 443(1):56–70, 2002.
  • Anderson et al. (1998) J. C. Anderson, T. Binzegger, K. A. C. Martin, and K. S. Rockland. The connection from cortical area V1 to V5: a light and electron microscopic study. The Journal of Neuroscience, 18(24):10525–10540, 1998.
  • Anzai et al. (2001) A. Anzai, I. Ohzawa, and R. D. Freeman. Joint-encoding of motion and depth by visual cortical neurons: neural basis of the Pulfrich effect. Nature Neuroscience, 4(5):513–518, 2001.
  • Azzopardi et al. (2003) P. Azzopardi, M. Fallah, C. G. Gross, and H. R. Rodman. Response latencies of neurons in visual areas MT and MST of monkeys with striate cortex lesions. Neuropsychologia, 41(13):1738–1756, 2003.
  • Berman and Wurtz (2010) R. A. Berman and R. H. Wurtz. Functional identification of a pulvinar path from superior colliculus to cortical area MT. The Journal of Neuroscience, 30(18):6342–6354, 2010.
  • Berman and Wurtz (2011) R. A. Berman and R. H. Wurtz. Signals conveyed in the pulvinar pathway from superior colliculus to cortical area MT. The Journal of Neuroscience, 31(2):373–384, 2011.
  • Boussaoud et al. (1990) D. Boussaoud, L. G. Ungerleider, and R. Desimone. Pathways for motion analysis: cortical connections of the medial superior temporal and fundus of the superior temporal visual areas in the macaque. Journal of Comparative Neurology, 296(3):462–495, 1990.
  • Brooks et al. (2011) K. R. Brooks, T. Morris, and P. Thompson. Contrast and stimulus complexity moderate the relationship between spatial frequency and perceived speed: Implications for MT models of speed perception. Journal of Vision, 11(14), 2011.
  • Burr and Ross (1979) D. C. Burr and J. Ross. How does binocular delay give information about depth? Vision Research, 19(5):523–532, 1979.
  • Carney et al. (1989) T. Carney, M. A. Paradiso, and R. D. Freeman. A physiological correlate of the Pulfrich effect in cortical neurons of the cat. Vision Research, 29(2):155–165, 1989.
  • Churchland et al. (2005) M. M. Churchland, N. J. Priebe, and S. G. Lisberger. Comparison of the spatial limits on direction selectivity in visual areas MT and V1. Journal of Neurophysiology, 93(3):1235–1245, 2005.
  • Daugman (1985) J. G. Daugman. Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. Optical Society of America, Journal, A: Optics and Image Science, 2(7):1160–1169, 1985.
  • DeAngelis et al. (1993a) G. C. DeAngelis, I. Ohzawa, and R. D. Freeman. Spatiotemporal organization of simple-cell receptive fields in the cat’s striate cortex. I. General characteristics and postnatal development. Journal of Neurophysiology, 69(4):1091–1117, 1993a.
  • DeAngelis et al. (1993b) G. C. DeAngelis, I. Ohzawa, and R. D. Freeman. Spatiotemporal organization of simple-cell receptive fields in the cat’s striate cortex. II. Linearity of temporal and spatial summation. Journal of Neurophysiology, 69(4):1118–1135, 1993b.
  • DeAngelis et al. (1994) G. C. DeAngelis, R. D. Freeman, and I. Ohzawa. Length and width tuning of neurons in the cat’s primary visual cortex. Journal of Neurophysiology, 71(1):347–374, 1994.
  • Dubner and Zeki (1971) R. Dubner and S. M. Zeki. Response properites and receptive fields of cells in an anatomically defined region of the superior temporal sulcus in the monkey. Brain Research, 1971.
  • Einstein (1905) A. Einstein. On the electrodynamics of moving bodies. Annalen der Physik, 17(891):50, 1905.
  • Felleman and Kaas (1984) D. J. Felleman and J. H. Kaas. Receptive-field properties of neurons in middle temporal visual area (MT) of owl monkeys. Journal of Neurophysiology, 52(3):488–513, 1984.
  • Ferster and Miller (2000) D. Ferster and K. D. Miller. Neural mechanisms of orientation selectivity in the visual cortex. Annual Review of Neuroscience, 23(1):441–471, 2000.
  • Foster et al. (1985) K. H. Foster, J. P. Gaska, M. Nagler, and D. A. Pollen. Spatial and temporal frequency selectivity of neurones in visual cortical areas V1 and V2 of the macaque monkey. The Journal of Physiology, 365(1):331–363, 1985.
  • Gabor (1946) D. Gabor. Theory of communication. Part 1: The analysis of information. Electrical Engineers-Part III: Radio and Communication Engineering, Journal of the Institution of, 93(26):429–441, 1946.
  • Girard et al. (1992) P. Girard, P. A. Salin, and J. Bullier. Response selectivity of neurons in area MT of the macaque monkey during reversible inactivation of area V1. Journal of Neurophysiology, 67(6):1437–1446, 1992.
  • Hawken et al. (1996) M. J. Hawken, R. M. Shapley, and D. H. Grosof. Temporal-frequency selectivity in monkey visual cortex. Visual Neuroscience, 13(03):477–492, 1996.
  • Heeger (1987) D. J. Heeger. Model for the extraction of image flow. JOSA A, 4(8):1455–1471, 1987.
  • Heeger (1992) D. J. Heeger. Normalization of cell responses in cat striate cortex. Visual Neuroscience, 9(02):181–197, 1992.
  • Heeger (1993) D. J. Heeger. Modeling simple-cell direction selectivity with normalized, half-squared, linear operators. Journal of Neurophysiology, 70(5):1885–1898, 1993.
  • Hubel (1957) D. H. Hubel. Tungsten microelectrode for recording from single units. Science, 125(3247):549–550, 1957.
  • Hubel (1959) D. H. Hubel. Single unit activity in striate cortex of unrestrained cats. The Journal of Physiology, 147(2):226–238, 1959.
  • Hubel and Wiesel (1959) D. H. Hubel and T. N. Wiesel. Receptive fields of single neurones in the cat’s striate cortex. The Journal of Physiology, 148(3):574–591, 1959.
  • Jones et al. (2001) H. E. Jones, K. L. Grieve, W. Wang, and A. M. Sillito. Surround suppression in primate V1. Journal of Neurophysiology, 86(4):2011–2028, 2001.
  • Krekelberg et al. (2006) B. Krekelberg, R. J. A. van Wezel, and T. D. Albright. Interactions between speed and contrast tuning in the middle temporal area: implications for the neural code for speed. The Journal of Neuroscience, 26(35):8988–8998, 2006.
  • Kuffler (1953) S. W. Kuffler. Discharge patterns and functional organization of mammalian retina. J. Neurophysiol., 16(1):37–68, 1953.
  • Lampl et al. (2004) I. Lampl, D. Ferster, T. Poggio, and M. Riesenhuber. Intracellular measurements of spatial integration and the MAX operation in complex cells of the cat primary visual cortex. Journal of Neurophysiology, 92(5):2704–2713, 2004.
  • Lee (1970) D. N. Lee. A stroboscopic stereophenomenon. Vision Research, 10(7):587–593, 1970.
  • Livingstone and Hubel (1988) M. Livingstone and D. Hubel. Segregation of form, color, movement, and depth: anatomy, physiology, and perception. Science, 240(4853):740–749, 1988.
  • Livingstone and Conway (2007) M. S. Livingstone and B. R. Conway. Contrast affects speed tuning, space-time slant, and receptive-field organization of simple cells in macaque V1. Journal of Neurophysiology, 97(1):849–857, 2007.
  • Marčelja (1980) S. Marčelja. Mathematical description of the responses of simple cortical cells. JOSA, 70(11):1297–1300, 1980.
  • Maunsell and Van Essen (1983a) J. H. Maunsell and D. C. Van Essen. The connections of the middle temporal visual area (MT) and their relationship to a cortical hierarchy in the macaque monkey. The Journal of Neuroscience, 3(12):2563–2586, 1983a.
  • Maunsell and Van Essen (1983b) J. H. Maunsell and D. C. Van Essen. Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. Journal of Neurophysiology, 49(5):1127–1147, 1983b.
  • Maunsell et al. (1990) J. H. Maunsell, T. A. Nealey, and D. D. DePriest. Magnocellular and parvocellular contributions to responses in the middle temporal visual area (MT) of the macaque monkey. The Journal of Neuroscience, 10(10):3323–3334, 1990.
  • Maunsell and Newsome (1987) J. H. R. Maunsell and W. T. Newsome. Visual processing in monkey extrastriate cortex. Annual Review of Neuroscience, 10(1):363–401, 1987.
  • Maxwell (1861a) J. C. Maxwell. LI. On physical lines of force. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 21(141):338–348, 1861a.
  • Maxwell (1861b) J. C. Maxwell. XLIV. On physical lines of force. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 21(140):281–291, 1861b.
  • Maxwell (1862a) J. C. Maxwell. XIV. On physical lines of force. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 23(152):85–95, 1862a.
  • Maxwell (1862b) J. C. Maxwell. III. On physical lines of force. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 23(151):12–24, 1862b.
  • McLean and Palmer (1989) J. McLean and L. A. Palmer. Contribution of linear spatiotemporal receptive field structure to velocity selectivity of simple cells in area 17 of cat. Vision Research, 29(6):675–679, 1989.
  • Morgan (1976) M. J. Morgan. Pulfrich effect and the filling in of apparent motion. Perception, 5(2):187–195, 1976.
  • Morgan (1979) M. J. Morgan. Perception of continuity in stroboscopic motion: a temporal frequency analysis. Vision Research, 19(5):491–500, 1979.
  • Movshon and Newsome (1996) J. A. Movshon and W. T. Newsome. Visual response properties of striate cortical neurons projecting to area MT in macaque monkeys. The Journal of Neuroscience, 16(23):7733–7741, 1996.
  • Nassi et al. (2006) J. J. Nassi, D. C. Lyon, and E. M. Callaway. The parvocellular LGN provides a robust disynaptic input to the visual motion area MT. Neuron, 50(2):319–327, 2006.
  • Nowlan and Sejnowski (1995) S. J. Nowlan and T. J. Sejnowski. A selection model for motion processing in area MT of primates. The Journal of Neuroscience, 15(2):1195–1214, 1995.
  • Pack et al. (2003) C. C. Pack, R. T. Born, and M. S. Livingstone. Two-dimensional substructure of stereo and motion interactions in macaque visual cortex. Neuron, 37(3):525–535, 2003.
  • Perrone (2004) J. A. Perrone. A visual motion sensor based on the properties of V1 and MT neurons. Vision Research, 44(15):1733–1755, 2004.
  • Perrone (2005) J. A. Perrone. Economy of scale: A motion sensor with variable speed tuning. Journal of Vision, 5(1), 2005.
  • Perrone and Thiele (2002) J. A. Perrone and A. Thiele. A model of speed tuning in mt neurons. Vision Research, 42(8):1035–1051, 2002.
  • Priebe et al. (2006) N. J. Priebe, S. G. Lisberger, and J. A. Movshon. Tuning for spatiotemporal frequency and speed in directionally selective neurons of macaque striate cortex. The Journal of Neuroscience, 26(11):2941–2950, 2006.
  • Pulfrich (1922) Carl Pulfrich. Die stereoskopie im dienste der isochromen und heterochromen photometrie. Naturwissenschaften, 10(35):751–761, 1922.
  • Qian (1994) N. Qian. Computing stereo disparity and motion with known binocular cell properties. Neural Computation, 6(3):390–404, 1994.
  • Qian and Andersen (1997) N. Qian and R. A. Andersen. A physiological model for motion-stereo integration and a unified explanation of Pulfrich-like phenomena. Vision Research, 37(12):1683–1698, 1997.
  • Qian and Freeman (2009) N. Qian and R. D. Freeman. Pulfrich phenomena are coded effectively by a joint motion-disparity process. Journal of Vision, 9(5), 2009.
  • Qian et al. (1994) N. Qian, R. A. Andersen, and E. H. Adelson. Transparent motion perception as detection of unbalanced motion signals. III. Modeling. The Journal of Neuroscience, 14(12):7381–7392, 1994.
  • Read and Cumming (2005a) J. C. A. Read and B. G. Cumming. Effect of interocular delay on disparity-selective V1 neurons: relationship to stereoacuity and the Pulfrich effect. Journal of Neurophysiology, 94(2):1541–1553, 2005a.
  • Read and Cumming (2005b) J. C. A. Read and B. G. Cumming. The stroboscopic Pulfrich effect is not evidence for the joint encoding of motion and depth. Journal of Vision, 5(5), 2005b.
  • Read and Cumming (2005c) J. C. A. Read and B. G. Cumming. All Pulfrich-like illusions can be explained without joint encoding of motion and disparity. Journal of Vision, 5(11), 2005c.
  • Riesenhuber and Poggio (1999) M. Riesenhuber and T. Poggio. Hierarchical models of object recognition in cortex. Nature Neuroscience, 2(11):1019–1025, 1999.
  • Rockland (1989) K. S. Rockland. Bistratified distribution of terminal arbors of individual axons projecting from area V1 to middle temporal area (MT) in the macaque monkey. Vis. Neurosci., 3(2):155–70, 1989.
  • Rockland (1995) K. S. Rockland. Morphology of individual axons projecting from area V2 to MT in the macaque. Journal of Comparative Neurology, 355(1):15–26, 1995.
  • Rodman and Albright (1989) H. R. Rodman and T. D. Albright. Single-unit analysis of pattern-motion selective properties in the middle temporal visual area (MT). Experimental Brain Research, 75(1):53–64, 1989.
  • Rodman et al. (1990) H. R. Rodman, C. G. Gross, and T. D. Albright. Afferent basis of visual response properties in area MT of the macaque. II. Effects of superior colliculus removal. The Journal of Neuroscience, 10(4):1154–1164, 1990.
  • Rosenberg et al. (2010) A. Rosenberg, T. R. Husson, and N. P. Issa. Subcortical representation of non-fourier image features. The Journal of Neuroscience, 30(6):1985–1993, 2010.
  • Sceniak et al. (2001) M. P. Sceniak, M. J. Hawken, and R. Shapley. Visual spatial characterization of macaque V1 neurons. Journal of Neurophysiology, 85(5):1873–1887, 2001.
  • Schwartz et al. (2012) G. W. Schwartz, H. Okawa, F. A. Dunn, J. L. Morgan, D. Kerschensteiner, R. O. Wong, and F. Rieke. The spatial structure of a nonlinear receptive field. Nature Neuroscience, 15(11):1572–1580, 2012.
  • Sereno (1993) M. E. Sereno. Neural Computation of Pattern Motion: Modeling Stages of Mation Analysis in the Primate Visual Cortex. The MIT Press, 1993.
  • Shipp and Zeki (1989) S. Shipp and S. Zeki. The organization of connections between areas V5 and V1 in macaque monkey visual cortex. European Journal of Neuroscience, 1(4):309–332, 1989.
  • Simoncelli and Heeger (1998) E. P. Simoncelli and D. J. Heeger. A model of neuronal responses in visual area MT. Vision Research, 38(5):743–761, 1998.
  • Sincich et al. (2004) L. C. Sincich, K. F. Park, M. J. Wohlgemuth, and J. C. Horton. Bypassing V1: a direct geniculate input to area MT. Nature Neuroscience, 7(10):1123–1128, 2004.
  • Skottun (1998) B. C. Skottun. A model for end-stopping in the visual cortex. Vision Research, 38(13):2023–2035, 1998.
  • Smith et al. (2005) M. A. Smith, N. J. Majaj, and J. A. Movshon. Dynamics of motion signaling by neurons in macaque area MT. Nature Neuroscience, 8(2):220–228, 2005.
  • Tolhurst et al. (1973) D. J. Tolhurst, C. R. Sharpe, and G. Hart. The analysis of the drift rate of moving sinusoidal gratings. Vision Research, 13(12):2545–2555, 1973.
  • Tsui et al. (2010) J. M. G. Tsui, J. N. Hunter, R. T. Born, and C. C. Pack. The role of V1 surround suppression in MT motion integration. Journal of Neurophysiology, 103(6):3123–3138, 2010.
  • Ungerleider and Desimone (1986) L. G. Ungerleider and R. Desimone. Cortical connections of visual area MT in the macaque. Journal of Comparative Neurology, 248(2):190–222, 1986.
  • Van Essen et al. (1981) D. C. Van Essen, J. H. R. Maunsell, and J. L. Bixby. The middle temporal visual area in the macaque: myeloarchitecture, connections, functional properties and topographic organization. Journal of Comparative Neurology, 199(3):293–326, 1981.
  • Watson and Ahumada (1983) A. B. Watson and A. Ahumada. A look at motion in the frequency domain, volume 84352. National Aeronautics and Space Administration, Ames Research Center, 1983.
  • Watson and Ahumada (1985) A. B. Watson and A. J. Ahumada, Jr. Model of human visual-motion sensing. Journal of the Optical Society of America. A, Optics and Image Science, 2(2):322–341, 1985.
  • Xu et al. (2001) X. Xu, J. Ichida, Y. Shostak, A. B. Bonds, and V. A. Casagrande. Are primate lateral geniculate nucleus (LGN) cells really sensitive to orientation or direction? Visual Neuroscience, 19(1):97–108, 2001.
  • Yabuta et al. (2001) N. H. Yabuta, A. Sawatari, and E. M. Callaway. Two functional channels from primary visual cortex to dorsal visual cortical areas. Science, 292(5515):297–300, 2001.
  • Zeki (1974) S. M. Zeki. Cells responding to changing image size and disparity in the cortex of the rhesus monkey. The Journal of Physiology, 242(3):827–841, 1974.
  • Zeki (1980) S. M. Zeki. The response properties of cells in the middle temporal area (area MT) of owl monkey visual cortex. Proceedings of the Royal Society of London. Series B. Biological Sciences, 207(1167):239–248, 1980.

Author Biography

Dr. Stephen G. Odaibo is Chief Scientist and Founder of Quantum Lucid Research Laboratories. He received the 2005 Barrie Hurwitz Award for Excellence in Clinical Neurology from Duke University School of Medicine where he topped the class in Neurology. Dr. Odaibo is a Mathematician, Computer Scientist, Physicist, and Physician. He obtained a B.S. in Mathematics (UAB, 2001), M.S. in Mathematics (UAB, 2002), M.S. in Computer Science (Duke, 2009), and Doctor of Medicine (Duke, 2010). Dr. Odaibo completed his internship in Internal Medicine at Duke University Hospital and is currently an Ophthalmology resident at Howard University Hospital in Washington DC. He is author of the book, “Quantum Mechanics and the MRI Machine” (Symmetry Seed Books, Oct 2012). He invented the Trajectron method with which he provided the first quantitative demonstration of non-paraxial light bending within the human cornea. His other awards and recognitions include: He won the 2013 Best Resident Research Presentation Award at the 23rd Annual Washington Retina Symposium; in 2012 he was selected as a Featured Alumnus of the Mathematics Department at UAB; and his cornea paper was selected by MIT Technology Review as one of the best papers from Physics or Computer science submitted to the arXiv the first week of Oct 2011. Dr. Odaibo’s research interests are at the fusion of Mathematics, Computer Science, Physics, and the Biomedical Sciences, with a special focus on the representation of motion in the mammalian visual cortex. His clinical interests are in the diagnoses and medical management of retinal disease. In the 2014-2015 academic year, Dr. Odaibo will be a Medical Retina fellow at the University of Michigan–Ann Arbor. In his spare time he enjoys time with his wife, family and friends, and studying the bible.