- About this Journal ·
- Abstracting and Indexing ·
- Advance Access ·
- Aims and Scope ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents

Neural Plasticity

Volume 2008 (2008), Article ID 658323, 12 pages

http://dx.doi.org/10.1155/2008/658323

## Linking Cellular Mechanisms to Behavior: Entorhinal Persistent Spiking and Membrane Potential Oscillations May Underlie Path Integration, Grid Cell Firing, and Episodic Memory

Center for Memory and Brain, Department of Psychology and Program in Neuroscience, Boston University, 2 Cummington Sreet, Boston, MA 02215, USA

Received 11 January 2008; Accepted 14 May 2008

Academic Editor: Roland S.G. Jones

Copyright © 2008 Michael E. Hasselmo and Mark P. Brandon. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

The entorhinal cortex plays an important role in spatial memory and episodic memory functions. These functions may result from cellular mechanisms for integration of the afferent input to entorhinal cortex. This article reviews physiological data on persistent spiking and membrane potential oscillations in entorhinal cortex then presents models showing how both these cellular mechanisms could contribute to properties observed during unit recording, including grid cell firing, and how they could underlie behavioural functions including path integration. The interaction of oscillations and persistent firing could contribute to encoding and retrieval of trajectories through space and time as a mechanism relevant to episodic memory.

#### 1. Introduction

The entorhinal cortex plays an important role in memory function. In the rat, entorhinal cortex lesions impair performance in both spatial memory tasks [1] as well as in odor memory tasks [2]. In monkeys, memory function in delayed match to sample tasks is impaired by lesions of entorhinal cortex [3] and adjacent parahippocampal structures [4]. This article describes cellular and circuit mechanisms in the entorhinal cortex that could underlie its role in spatial and episodic memory functions. Computational modeling links data across multiple levels of function, including: (a) properties of single cell physiology, including persistent spiking and membrane potential oscillations, (b) properties of unit recording, including grid cells, place cells, and head direction cells, and (c) the role of entorhinal cortex in spatial path integration and episodic memory function. This article will review physiological data and modeling across these different levels.

#### 2. Physiological Data

Recordings from neurons in slice preparations of entorhinal cortex demonstrate important cellular properties including (i) persistent spiking and (ii) membrane potential oscillations. These cellular properties could contribute to properties described in unit recordings from entorhinal cortex in awake, behaving rats.

##### 2.1. Persistent Spiking

In slices, pyramidal neurons in different layers of entorhinal cortex demonstrate the capacity to display persistent spiking activity after a depolarizing current injection or a period of repetitive synaptic input [5–8]. Pyramidal neurons in layer II of medial entorhinal cortex show persistent spiking that tends to turn on and off over periods of many seconds [5]. This cyclical persistent spiking is shown in Figure 1(a). As described below, this could underlie the spatial periodicity of grid cells. Pyramidal neurons in deep layers of entorhinal cortex can maintain spiking at different graded frequencies for many minutes [8] as shown in Figure 2(a). The persistent spiking appears to due to muscarinic or metabotropic glutamate activation of a calcium-sensitive nonspecific cation current [7, 9, 10]. This graded persistent firing could allow these neurons to integrate synaptic input over extended periods. Persistent firing has also been shown in layer III of lateral entorhinal cortex [6].

##### 2.2. Membrane Potential Oscillations

Entorhinal layer II stellate cells show prominent subthreshold membrane potential oscillations when depolarized near firing threshold [11, 12]. These are small oscillations of a few millivolts in amplitude that can influence the timing of action potentials [13] and can contribute to network oscillations [14, 15]. The frequency of membrane potential oscillations differs systematically along the dorsal to ventral axis of the medial entorhinal cortex [16]. A model presented below discusses how the membrane potential oscillation properties could underlie differences in grid cell firing properties along the dorsal to ventral axis [16–19]. The oscillations appear to be due to a hyperpolarization activated cation current or h-current [20], that differs in time constant along the dorsal to ventral axis [21]. Depolarizing input increases the frequency of these oscillations such that the phase of the oscillation integrates the depolarizing input over time. Membrane potential oscillations do not usually appear in layer II or layer III pyramidal cells [12], but are observed in layer V pyramidal cells, where they may be caused by M-current [22]. Membrane potential oscillations do not appear in neurons of the lateral entorhinal cortex [23].

##### 2.3. Unit Recording Data

Recordings of neural activity in awake behaving rats provide important clues to the functional role of entorhinal cortex. In particular, many cells in medial entorhinal cortex fire as “grid cells.” A single grid cell responds as a rat forages in a hexagonal array of different locations in an open-field environment [24, 25]. Examples of the pattern of firing in modeled grid cells are shown in Figures 1 and 2. Grid cells differ in spatial periodicity along the dorsal to ventral axis of medial entorhinal cortex, with larger spacing between larger fields in more ventral regions [24, 25].

Grid cells appear in all layers of entorhinal cortex, but in layers V and VI of entorhinal cortex the grid cells often only respond when the rat is facing in a particular direction [25]. This resembles head direction cells in areas such as the postsubiculum (dorsal presubiculum), which respond at all locations in the environment but only when the rat faces a particular direction [26–30]. The conjunctive grid-by-direction cells resemble the theta-modulated place-by-direction cells observed in the post- and parasubiculum, which respond only when the rat faces a preferred direction while occupying a single location [31].

#### 3. Path Integration

The cellular mechanisms described above may contribute to the function of path integration. Path integration involves an animal using its self-motion cues to maintain an accurate representation of the angle and distance from its start position, even during performance of a complex trajectory through the environment [32–35]. Many species demonstrate the behavioral capacity to remember the distance as well as the angle of return to the starting location (here represented in Cartesian coordinates by a two component return vector). As shown here, persistent firing provides a single neuron mechanism to integrate the distance and angle of trajectory segments to compute the overall distance and angle from start location to an end or goal location. Here, the vector from start to goal is called the goal vector (and the negative of the goal vector is called the return vector). An example of a return vector is shown as a dashed line in Figure 3(a).

In general, the location of an agent can be
determined by computing the integral of the velocity vector. In Cartesian coordinates, the velocity vector
is . Integration of the velocity vector over a
period of time *T* after starting at
location yields
the location vector at time *T*:

For example, if the velocity of an animal is 10 cm/sec in the direction and 5 cm/sec in the direction, integration over 5 seconds yields a final location of [,] = [50 cm, 25 cm] relative to the start location. Note that this integral corresponds to the goal vector , which is the negative of the return vector.

#### 4. Integration by Persistent Firing Can Code Location

Integration of the goal vector or return
vector could be provided by the mechanism of graded persistent firing in deep
layer entorhinal neurons [7, 8]. These neurons could
integrate a velocity vector coded by neurons responsive to the head direction
of the rat and to the speed of the rat.
Head direction cells have been shown in deep entorhinal cortex [25] and in the postsubiculum, which provides direct input to the
entorhinal cortex [26, 27]. Head direction cells respond selectively when
the rat is heading in a specific allocentric direction. Some neurons show sensitivity to speed
(translational motion) in the postsubiculum [27] as well as in the hippocampus [36] and in the medial mammillary nucleus which receives output from the
postsubiculum and medial entorhinal cortex [37]. Here, the activity of a
population of head direction cells modulated by speed is represented by
multiplying the velocity vector of the rat with a head direction matrix . The head direction matrix consists of rows
with unit vectors representing the preference angles of individual speed
modulated head direction cells, that transforms the velocity vector into a head
direction vector . For example, a matrix representing two head
direction cells with preference angles *θ*_{1} and *θ*_{2} has two rows: . For a cell with a preference angle of 0, the
activity of the head direction cell would be . This framework has the advantage
that it allows for a simple inverse transform turning head direction space back
into Cartesian coordinates. The inverse [38] of the matrix *H* is

Note that this framework assumes that the response of head direction cells is like a cosine function, whereas head direction cells usually only show positive activity and have narrower, triangular response functions with no activity outside this range. Head direction input corresponding to a cosine function of actual input could be provided by summed input converging from a population of head direction cells with different magnitudes of tuning for values at different angles from the preferred direction. Note also that this representation combines the properties of different neurons responding to translational velocity or to head direction in the postsubiculum [27] and other regions [36, 37]. If head direction is not computed based on velocity, then it could be integrated from angular velocity as a distinct element of the state vector or could be computed based on angle to a different reference point.

Using
this mathematical representation of head direction input, the firing rate of a
set of graded persistent firing cells could integrate the input from a set of
head direction cells to yield a firing rate as follows:
where
vector represents the firing rate of a population of
graded persistent firing cells at time *T*. For example, imagine two cells and with capacity for graded persistent spiking that receive
input from two head direction cells with preference angles of 0 degrees and 60
degrees. Imagine that the rat moves at
10 cm/sec in the *x* direction for 4 seconds, and the scaling factor * β* is 0.25 Hz/cm. Moving in the

*x*direction is equivalent to moving at 0 degrees, which would result in the head direction cells having the activity and . The computation in (3) would then result in the frequency of the graded persistent firing cells as follows: . Thus, the graded persistent firing cells would increase their activity to = 10 Hz and = 5 Hz. Mathematically, the inverse transform of this firing rate vector computes the location vector in Cartesian coordinates (see Figure 3(a)). However, the difference in neural activity could guide behavior without use of the inverse transform. This could involve forming associations between the start location and this pattern of graded firing, and then forming associations between this pattern of graded firing and the associated head direction signal. At the start location, the pattern of graded firing could be activated, and this could retrieve the associated head direction signal. The animal could change directions until its actual head direction matched this retrieved head direction. This could give the animal the correct angle to the goal.

The same mechanism computes both the goal vector and its negative, the return vector. The return vector allows a rat to return to the starting location from any arbitrary location in the environment. In contrast, the goal vector can be used to store the distance and angle to important locations in the environment. For example, if a rat is started in one location in an open field, and wanders until it finds food in another location, the integrated activity vector at that point is the goal vector—it provides a simple description of the angle and distance from the start to the goal. This goal vector could be associated with all elements of the preceding path by backward replay of place cells coding the full pathway [39, 40]. This could allow storage of an association between place cells active at the start location and the subsequent goal vector. Retrieval of the goal vector at the start location could then allow the rat to go directly to the location of food reward. If the spatial locations leading to a goal are associated with the goal vector at each goal location, and then integration is reset, a sequential series of trajectories to goals could be stored separately. The rat could then use this activity to sequentially retrieve pathways to different rewarded locations in the environment, as in some behavioral tasks [41]. The resetting of integration activity could underlie the different pattern of place cell firing shown with this type of directed task compared to open field activity. Thus, the resetting of integration could explain the shift in firing location for place cells between scavenging in an open field and following sequential trajectories between reward locations [41] as well as the shift in firing location for grid cells between open field scavenging and running on a long hairpin track [42] in which the view of each new segment could cause phase reset.

Graded persistent spiking could also hold initial head direction and update this by integrating input from neurons coding angular head velocity in areas such as the postsubiculum [27].

#### 5. Persistent Spiking Could Underlie Grid Cell Firing

Because the neurons that show persistent firing can integrate the synaptic input from speed modulated head direction cells, and thereby can code spatial location, these persistent firing neurons could potentially be the grid cells recorded in awake behaving animals [24, 43, 44]. This section describes two potential mechanisms for persistent firing neurons to contribute to the activity of grid cells. The first mechanism involves the cyclical persistent firing shown in layer II (see Figure 1(a)). The second mechanism involves the graded persistent firing shown in layer V (see Figure 2(a)).

In a general manner, the experimental data
on firing of single grid cells can be described by
where is the probability of
firing of the grid cell over time. The
product sign *Π* represents
multiplication of the output from each row (each head direction * θ*) of the head
direction transform matrix

*H*described above. The description of the experimental data here directly uses the vector representation of location over time . Orientation of the grid is determined by the head directions

*in*

*θ**H*, the spatial phase is determined by , and the spacing between fields is determined by the angular frequency

*ω*. This equation resembles other representations of grid cells [18, 45] but simplifies the representation by using the head direction transform matrix.

##### 5.1. Cyclical Persistent Firing

The pattern of
periodic spatial firing of grid cells could arise from the pattern of cyclical
persistent firing as shown in Figure 1(a).
The tendency for persistent firing to turn on and off could contribute
to grid cell firing if the oscillation could be gated by integration of input
from different populations of head direction cells with different preferred
angles. Simulations shown in Figures 1(b) and 1(c) show that the following equations can generate grid cell firing properties:
where represents changes in current due to positive
components of the head direction input (note that this uses cosine modulated
head direction input), and represents current due to negative
components of the cosine modulated head direction input. Note that the equations separately use
positive and negative elements of the speed modulated head direction matrix *H* transforming the rat movement velocity . The parameter *ω* determines the time scaling of input effects
on activity levels. In the equations, represents the voltage change in individual compartments each of which receive
input from the positive and negative components of one head direction
input. As noted above, the cosine
modulated head direction input could be provided by summing over head direction
cells with different angles of preference.
The negative influence of head direction in the equation could be due to
feedforward inhibition or inhibitory GABAergic projections from the
postsubiculum to the medial entorhinal cortex.
The function represents the firing of grid cells over
time. The square brackets indicate
that spiking occurs whenever crosses a threshold.

This pattern of activity could be obtained if neurons respond to different head direction inputs with cyclical persistent firing (Klink and Alonso, 1997), as shown in Figure 1(a). In this case, when going one direction, head direction input will cause phasic changes in firing in that direction, possibly due to build up first of calcium and then of calcium inactivation. When going the exact opposite direction, head direction input would have to activate the reverse processes, possibly reducing calcium inactivation and then reducing calcium.

Examples of grid field plots obtained with this model are shown in Figures 1(b) and 1(c). The grid fields are more stable in the trajectory data from the Moser laboratory than in randomly created trajectories (see Figure 1(b)) or in a trajectory obtained in our own laboratory (see Figure 1(c)). This indicates that the statistics of rat movement can determine appearance of the grid in this new model, and this could underlie variability in detection of grid cell firing properties depending on the trajectory of rat movement in the behavioral foraging task.

##### 5.2. Graded Persistent Firing

As an alternative to cyclical persistent firing, graded persistent firing as shown in Figure 2(a) could provide the basis for grid cell firing. In this framework, different graded persistent firing cells start out with the same baseline frequency of spiking and provide convergent input to a grid cell that fires whenever the inputs are in synchrony. Speed modulated head direction input to different graded persistent firing cells will transiently alter the frequency and thereby the phase of firing. Therefore, if the rat moves, it shifts the frequency of a graded cell out of phase with the other cells and thereby reduces or stops the grid cell firing until the phase is shifted enough to come into phase with the other neurons. A grid cell simulated with this model is shown in Figure 2(b). This mechanism uses graded persistent firing in a manner similar to the mechanism of membrane potential oscillations described in the following section.

Both of these models will yield a pattern of firing similar to grid cells as long as the head direction cells providing input have preference angles at multiples of 60 degrees. For path integration, the head direction angles used for integration might be determined at the start location. For example, a single pyramidal cell showing persistent firing might receive input from three head direction cells that code the heading angle at the start, as well as the angle of eye direction. Rats have binocular overlap of about 60 degrees [46]. If the total visual field of one eye is 180 degrees and the center of view is at 90 degrees, then the center of view for each eye should be offset about 60 degrees from head direction. Therefore, a rat may choose angles of 0, −60, and 60 degrees for path integration. These angles have the 60-degree difference necessary for the head direction input to cause hexagonal arrays in the grid cell model. The rat can use these initial angles of view as reference angles, but if it turns far enough from the initial heading (e.g., 180 degrees from the initial head direction), then it may need to select additional reference angles at 60-degree intervals from the previous reference angles.

Some grid cells respond selectively only for certain head direction [25]. These head direction sensitive grid cells might result from the input only being suprathreshold for a population of head direction cells responding near one preferred angle, with input being subthreshold from other populations of head direction cells coding other preferred angles (e.g., at 60 or 120 degrees differences).

##### 6. Integration by Membrane Potential Oscillation Phase Can Code Location

As an alternative mechanism for path integration, the phase of membrane potential oscillations in medial entorhinal stellate cells can also be used to integrate speed modulated head direction input. This mechanism was proposed in a model of grid cells developed by Burgess et al. [18] and O’Keefe and Burgess [47]. This mechanism uses the physiological fact that depolarizing inputs to stellate cells cause a change in frequency of membrane potential oscillations [16]. This could change oscillation phase based on an integral of the depolarizing input.

##### 6.1. Model of Grid Cells Using Membrane Potential Oscillations

This
computational model shows how activity of a single grid cell could arise from
membrane potential oscillations within that cell modulated by depolarizing
input from head direction cells:
where is the firing in time and space of a single modeled grid cell. *ω* represents the baseline angular frequency of membrane
potential oscillations (2**π**) in different
portions of the neuron. * β_{H}* is the experimentally determined scaling factor relating membrane
potential oscillations to grid cell spacing.
The input from head direction cells is determined by the matrix

*H*and the velocity vector . The inner product of each row of

*H*with the velocity vector represents input to one dendrite from one head direction cell modulated by the speed of the rat. This input alters the frequency of dendritic membrane potential oscillations and thereby shifts the phase of the dendritic oscillations in proportion to the integral of the velocity vector over time. Both the starting location of the rat and the spatial phase of the grid cell are combined in the initial location vector

*x*. This initial location vector is also transformed by the matrix

_{0}*H*. The square brackets represent a Heaviside step function generating a spiking output for each time point when the product crosses a threshold (set at 1.8).

This
model generates grid cell firing fields with spacing between fields dependent
upon the frequency of membrane potential oscillations [16–18]. A grid cell created with
this model is shown in Figure 2(c). The
model generated the prediction that the systematic change in spacing of grid
cell firing fields along the dorsal to ventral axis of entorhinal cortex would
depend upon a systematic difference in frequency of membrane potential
oscillations in entorhinal neurons. This
prediction was tested and supported by whole cell patch recordings from
entorhinal layer II stellate cells [16]. Based on experimental data
alone, it appears that membrane potential oscillation frequency *f* is
scaled to grid cell spacing *G* by a constant factor [16, 17].

As noted above, membrane potential oscillations appear in specific medial entorhinal populations such as layer II stellate cells and layer V pyramidal cells, but not in other cells such as layer II pyramidal cells, or neurons in medial entorhinal layer III, or in the lateral entorhinal cortex. Based on these data, the generation of grid cells responses based on membrane potential oscillations would only occur in layer II stellate cells and layer V pyramidal cells, and would appear in other neurons due to network interactions or due to the persistent spiking mechanisms described above.

##### 6.2. Grid Cell Activity Codes Location

In the model, the depolarizing input from head direction cells increases the frequency of membrane potential oscillations in proportion to the velocity vector. This shift in frequency alters the phase of oscillations in proportion to the integral of the velocity vector transformed by head direction: . In addition, the interference pattern increases and decreases in proportion to the difference in oscillation frequency of the soma and dendrite, so that the vector of angular phases of interference is (see Figures 3(b) and 3(c)) and the equation for grid cell activity can be written for location as

The location can be extracted from grid cell phase by using the inverse of the head direction transform matrix as follows: . Figure 3(d) shows the trajectory obtained from this inverse transform of phase.

##### 6.3. Theta Phase Precession

In addition to replicating the spatial periodicity of grid cell firing fields, the model based on interference of membrane potential oscillations also replicates experimental data showing systematic changes in phase of grid cell firing relative to network theta rhythm oscillations [48], a phenomenon known as theta phase precession. The phenomenon of theta phase precession was initially shown for place cell firing in the hippocampal formation [49, 50] and was proposed to arise from the interaction of network theta rhythm oscillations with cellular theta rhythm oscillations [49, 51, 52]. As an alternative model, precession was proposed to arise from the readout of sequences of place cell activity [53–55].

The oscillatory interference model of grid
cells [18, 47] extended the models of
hippocampal phase precession and can account for grid cell phase precession [48]. In the oscillatory
interference model, the interference has two components that appear
physiologically: (1) as described above, the size of grid fields is determined
by the envelope that has a frequency depending on the difference of the two
angular frequencies , and (2) the pattern of phase precession depends on a higher
frequency component that corresponds to the mean of the two frequencies . The phase of this
second-high frequency component of the summed oscillation is . The spiking will occur near the peak phase of
this summed oscillation which is equal to some multiple *n* of the full
cycle: .

The spiking caused by the phase of the summed oscillations can then be plotted relative to the network theta rhythm by plotting the phase of the soma at the time of each spike. This can be obtained analytically from the above equation if we consider the case of the movement at a constant speed continuously in the preferred direction of one head direction cell. In this case, the integral of head direction for that cell is simply the integral of speed, which is equal to the location . Note also that the phase of the soma oscillations is the product of soma frequency and time: . Therefore, the equation for the phase of the summed oscillation can be reduced to .

Plotting of theta phase precession
essentially involves plotting the timing of spikes (which occur when ) relative to the phase of the network
oscillations (which here correspond to the phase of the soma because the soma
is being driven by network oscillations with fixed frequency * ω*). Thus, the vertical axis of a plot of theta
phase precession shows the phase of the soma oscillation at the time of each
spike: plotted relative to location

*x*on the horizontal axis. Figure 4 shows the plotting of spikes in the simulation during runs on a linear track through the firing field of the neuron. Note that the phase precession in this model resembles the phase precession found in experimental data [50] but only covers about 180 degrees of the network theta oscillation cycle.

Note that the phase precession for a single direction gives a partial readout of the phase code of location, but when considering the phase in two-dimensional space, it confounds the phase of the two or more dendrites, so it is radially symmetric and dependent upon the direction of trajectory through the field (see Figure 5(a)). Thus, the phase precession code is less accurate for use in path integration, in contrast to the overall mean firing rate that would be observed in a grid cell due to persistent firing with a very large firing field, which could code location for distances smaller than one half the spacing between two grid fields (e.g., for 80 cm spacing, distances less than 40 cm could be coded).

##### 6.4. Mechanism for Path Integration

As
noted above, path integration involves an animal using its self-motion cues to
maintain an accurate representation of angle and distance from start
position. In the grid cell model, the
update of oscillation phase by speed-modulated head direction integrates the
velocity vector, thereby integrating the distance from the starting position in
specific directions determined by *H*.
If oscillation phases are reset to zero at the starting location, the
return vector giving distance and direction back to the starting location
can be obtained by applying the inverse of the head direction matrix to the
dendritic phase vector at any position . Figure 3(a) shows that after resetting phase at the
starting location, applying the inverse head direction transform to the dendritic phase vector, and taking the negative of this vector gives the direction and angle directly back
to the starting location (dotted line).

##### 6.5. Interaction of Path Integration and Visual Stimuli

Path integration based on idiothetic cues alone builds up substantial error [34] that is usually corrected by comparison with sensory cues such as visual stimuli [35]. Grid cells show dependence upon visual stimuli in the environment. They maintain the same properties when returned to a familiar room [24], and they change their scale during a period of time after manipulations of environment size [44]. This influence of visual stimuli could result from the fact that the angle to visual stimuli has the same properties as phase of grid cell oscillations. Figure 5 shows how the angle of a single distal visual stimulus changes as a rat moves in a manner that is consistent with the change in phase of individual dendrites of a single modeled grid cell.

Alternately, the grid cell could be influenced by the angle and distance to visual stimuli, and neural mechanisms could update the expected angle and distance to visual stimuli in a manner similar to the mechanism for updating the angle and distance from start location (return vector). This requires updating the angles of the initially selected stimuli (that might be determined by eye direction) according to the direction and velocity of movement. This will update the expected absolute angle of visual stimuli. The further computation of expected relative angle (the actual visual input) requires combining absolute angle with current head direction. The process of updating head direction could depend upon input from cells coding angular velocity of movement. These have been shown in the postsubiculum [27, 56] as well as in structures including the anterior dorsal thalamic nucleus [29]. Grid cells appear to be more consistent when there are clear barriers on the edge of the open field, suggesting that rats might use the vertical angle of a boundary to judge distance.

The basic grid cell model assumes speed modulation of head direction cells, but most head direction cells show stable persistent firing even when the rat is motionless. In contrast, place cells show more speed modulation. In keeping with the physiological data that shows stable persistent firing for head direction cells and speed dependent firing for place cells, it might be appropriate to represent state as the static head direction cell activity combined with visual stimulus angle, and to use the speed-dependent activity of place cells as the action of the rat.

##### 6.6. Grid Cell phase Represents Continuum of Locations for Reinforcement Learning

The grid cells can be used as a representation of state for goal directed behavior. Many reinforcement learning theory models have used discrete representations of state for goal directed behavior. However, this causes difficulties for representing movement in continuous space. The phase of oscillations in the grid cell models is a continuous representation of space that can be updated in a continuous manner by actions held by persistent spiking. As noted previously [16, 57], this could allow grid cells to provide an effective mechanism for representations of state and action in continuous space and time.

#### 7. Model of Episodic Memory

The interaction of head direction cells and grid cells described here provides a potential mechanism for episodic memory involving the storage of trajectories through space and time [57]. As shown in Figure 6, this model uses a functional loop that encodes and retrieves trajectories via three stages: (1) head direction cells update grid cells, (2) grid cells update place cells, and (3) place cells activate associated head direction activity [57]. This model is consistent with the anatomical connectivity (see Figure 6). The head direction cells could update grid cells via projections from the postsubiculum (dorsal presubiculum) to the medial entorhinal cortex [58–60], causing updating of persistent firing as described above, or influencing the phase of membrane potential oscillations [16–18]. Grid cells can update place cells via the extensive projections from entorhinal cortex layer II to dentate gyrus and CA3 and from layer III to region CA1 [45, 61]. The connectivity from grid cells to place cells could be formed by different computational mechanisms [45, 62, 63]. Place cells can become associated with head direction activity via direct projections from region CA1 to the postsubiculum [58], or via indirect projections from region CA1 to the subiculum [61, 64], and projections from the dorsal and distal regions of the subiculum to the postsubiculum and medial entorhinal cortex [65], both of which contain head direction cells.

During initial encoding of a trajectory in the model, the head direction cell activity vector would be set by the actual head direction of the rat during exploration, and associations would be encoded between place cell activity and head direction activity. These associations would be stored in the form of a synaptic connectivity matrix with strengthened connections between active place cells and the head direction cell activity vector as follows:

In this equation, head direction vectors
associated with individual place cell locations are identified with the place
cell index *p*. During retrieval,
the head direction activity depends upon synaptic input from place cell
representations as follows: .

This model has the capacity for performing
episodic encoding and retrieval of trajectories in simulations [57], including trajectories based on experimental data or trajectories
created by an algorithm replicating foraging movements of a rat in an open
field [17]. During encoding, a series
of place cells is
created associated with particular locations . Each place cell is also associated with input
from the grid cell population activity and with the head direction vector that occurred during the initial movement from
that location. For retrieval, the
simulation is cued with the grid cell phase vector and head direction vector present at the start location. The head
direction vector updates the grid cell phase vector ,
which alters the activity of grid cells.
The grid cell firing drives place cells *p* associated with
subsequent locations on the trajectory.
These mechanisms are summarized in Figure 6.

The activation of each new place cell activates a new head direction vector associated with that place cell. This new head direction vector then drives the further update of dendritic phases of grid cells. This maintenance of the head direction vector might require graded persistent spiking [8] of head direction cells in deep layers of entorhinal cortex. Essentially, the retrieval of the place cell activity representing the state drives the retrieval of the new head direction vector representing the action from that state. This action is then used for a period of time to update the grid cell state representation until a new place cell representation is activated.

Because retrieval of the trajectory depends on updating of phase by head direction cells, this allows retrieval of a trajectory at a time course similar to the initial encoding. This can allow effective simulation of the slow time course of place cell replay observed in previous experimental data collected during REM sleep [66]. The spread of activity from place cells to cells coding head direction could contribute to patterns of firing in the postsubiculum that appear as cells responding dependent on both place and head direction [31]. These cells might code the action value for retrieval of a trajectory from a particular location, firing only when actual head direction matches the head direction previously associated with specific place cell activity. The strong theta phase specificity of these cells could be due to separate dynamics for encoding and retrieval within each cycle of theta rhythm [67]. These cells might selectively fire during the retrieval phase.

##### 7.1. Enhancement by Arc Length Cells

The retrieval mechanism mediated by place cells can be enhanced by inclusion of cells that fire dependent upon the arc length of the trajectory [57], or by the time interval alone [17]. These types of responses help prevent a breakdown in trajectory retrieval caused by overlaps in the trajectory. The associations between place cells and head direction cells cannot disambiguate between two segments of a trajectory passing through the same location with different head directions. However, coding of arc length or time since the start of the trajectory can disambiguate these overlapping locations. Oscillatory interference between neurons directly modulated by speed but not head direction can activate arc length cells coding arc length from the start of a trajectory, or from the last time that oscillations are reset along the trajectory. Simulations based on this coding of arc length can account for many features of unit recording data in behavioral tasks. Persistent spiking in layer III of entorhinal cortex could provide a means for driving the coding of arc length (or time) along a trajectory. Persistent spiking in layer III with specific frequencies [6] could activate neurons in region CA1 with different phases relative to arc length (or time) on a trajectory. During retrieval, arc length cells from one portion of a trajectory can activate associated speed modulated cells to trigger the next arc length cell along a trajectory. This retrieval process can be accelerated or decelerated via modulation of the frequency of entorhinal oscillations during persistent firing.

##### 7.2. Predictions of Arc Length Model

Since the output of arc length cells is essentially dependent upon distance, a simple manipulation of running distance should directly influence spatial specificity of arc length cells. For example, in a rat running continuously clockwise around a circular track with a circumference of two meters, an arc length cell may match the periodicity of the track and display a stable place field somewhere on the track. Here, the cell is firing at an arc length of two meters. If we expand the track by a small amount to say, a circumference of 2.1 meters, the arc length cell would be expected to continue to fire at two-meter intervals, thus the field will translocate in the counterclockwise direction (or backwards in relation to the rat) by 10 centimeters for each lap.

The reset version of the arc length coding model assumes that oscillations reset at a specified location or during a key event such as food reward. This is supported experimentally given that the theta oscillation does reduce when an animal stops or consumes food. By using the same manipulation on the circular track as before, a similar but quite different prediction surfaces. Here, the field will shift counterclockwise abruptly, but will remain at that location for subsequent laps. This location stability is a direct consequence of stability of the food reward location since now the oscillatory interference is anchored to the food reward, and not from the previous location that the cell had discharged. Interestingly, the distance a field moves will be linearly proportional to the distance the original field was from the food reward location. Thus, a field will move 10 centimeters only if the field was originally located at the end of a lap (at the feeder), and a field will shift 5 centimeters if the original field was located at the midpoint of the lap.

The reset model prediction of the expanded circular track leads us to a further prediction. Since the discharge of an arc length cell in the reset model is dependent on and anchored to the reward location, a manipulation of the location of food reward will cause a relative movement of the location of the arc length’s discharge. For example, the movement of the food reward by 10 centimeters in the clockwise directly will cause an arc length cell to correspondingly shift its field 10 centimeters in the clockwise direction.

#### 8. Conclusions

The cellular physiological phenomena described in this paper provide mechanisms important for behavioral functions including path integration and the episodic encoding and retrieval of trajectories. Detailed computational models demonstrate the potential behavioral role of cellular mechanisms of persistent spiking and membrane potential oscillation, demonstrate how these could underlie unit recording data such as grid cell firing, and generate predictions for future experimental studies.

#### Acknowledgments

Research supported by Silvio O. Conte Center grants NIMH MH71702 and MH60450, by NIMH R01 grants MH60013 and MH61492, by NSF SLC SBE 0354378, and by NIDA grant DA16454 (part of the CRCNS program).

#### References

- H.-A. Steffenach, M. Witter, M.-B. Moser, and E. I. Moser, “Spatial memory in the rat requires the dorsolateral band of the entorhinal cortex,”
*Neuron*, vol. 45, no. 2, pp. 301–313, 2005. View at Publisher · View at Google Scholar - T. Otto and H. Eichenbaum, “Complementary roles of the orbital prefrontal cortex and the perirhinal-entorhinal cortices in an odor-guided delayed-nonmatching-to-sample task,”
*Behavioral Neuroscience*, vol. 106, no. 5, pp. 762–775, 1992. View at Publisher · View at Google Scholar - B. W. Leonard, D. G. Amaral, L. R. Squire, and S. Zola-Morgan, “Transient memory impairment in monkeys with bilateral lesions of the entorhinal cortex,”
*The Journal of Neuroscience*, vol. 15, no. 8, pp. 5637–5659, 1995. - S. Zola-Morgan, L. R. Squire, and S. J. Ramus, “Severity of memory impairment in monkeys as a function of locus and extent of damage within the medial temporal lobe memory system,”
*Hippocampus*, vol. 4, no. 4, pp. 483–495, 1994. View at Publisher · View at Google Scholar - R. Klink and A. Alonso, “Muscarinic modulation of the oscillatory and repetitive firing properties of entorhinal cortex layer II neurons,”
*Journal of Neurophysiology*, vol. 77, no. 4, pp. 1813–1828, 1997. - B. Tahvildari, E. Fransén, A. A. Alonso, and M. E. Hasselmo, “Switching between “On” and “Off” states of persistent activity in lateral entorhinal layer III neurons,”
*Hippocampus*, vol. 17, no. 4, pp. 257–263, 2007. View at Publisher · View at Google Scholar - E. Fransén, B. Tahvildari, A. V. Egorov, M. E. Hasselmo, and A. A. Alonso, “Mechanism of graded persistent cellular activity of entorhinal cortex layer V neurons,”
*Neuron*, vol. 49, no. 5, pp. 735–746, 2006. View at Publisher · View at Google Scholar - A. V. Egorov, B. N. Hamam, E. Fransén, M. E. Hasselmo, and A. A. Alonso, “Graded persistent activity in entorhinal cortex neurons,”
*Nature*, vol. 420, no. 6912, pp. 173–178, 2002. View at Publisher · View at Google Scholar - M. H. Shalinsky, J. Magistretti, L. Ma, and A. A. Alonso, “Muscarinic activation of a cation current and associated current noise in entorhinal-cortex layer-II neurons,”
*Journal of Neurophysiology*, vol. 88, no. 3, pp. 1197–1211, 2002. - M. Yoshida, E. Fransén, and M. E. Hasselmo, “Cholinergic-independent persistent firing in entorhinal layers III and V neurons,”
*Society for Neuroscience Abstract*, vol. 33, p. 935.9, 2007. - A. Alonso and R. R. Llinás, “Subthreshold ${\text{Na}}^{+}$-dependent theta-like rhythmicity in stellate cells of entorhinal cortex layer II,”
*Nature*, vol. 342, no. 6246, pp. 175–177, 1989. View at Publisher · View at Google Scholar - A. Alonso and R. Klink, “Differential electroresponsiveness of stellate and pyramidal-like cells of medial entorhinal cortex layer II,”
*Journal of Neurophysiology*, vol. 70, no. 1, pp. 128–143, 1993. - E. Fransén, A. A. Alonso, C. T. Dickson, J. Magistretti, and M. E. Hasselmo, “Ionic mechanisms in the generation of subthreshold oscillations and action potential clustering in entorhinal layer II stellate neurons,”
*Hippocampus*, vol. 14, no. 3, pp. 368–384, 2004. View at Publisher · View at Google Scholar - C. D. Acker, N. Kopell, and J. A. White, “Synchronization of strongly coupled excitatory neurons: relating network behavior to biophysics,”
*Journal of Computational Neuroscience*, vol. 15, no. 1, pp. 71–90, 2003. View at Publisher · View at Google Scholar - A. Alonso and E. García-Austt, “Neuronal sources of theta rhythm in the entorhinal cortex of the rat. I. Laminar distribution of theta field potentials,”
*Experimental Brain Research*, vol. 67, no. 3, pp. 493–501, 1987. View at Publisher · View at Google Scholar - L. M. Giocomo, E. A. Zilli, E. Fransén, and M. E. Hasselmo, “Temporal frequency of subthreshold oscillations scales with entorhinal grid cell field spacing,”
*Science*, vol. 315, no. 5819, pp. 1719–1722, 2007. View at Publisher · View at Google Scholar - M. E. Hasselmo, L. M. Giocomo, and E. A. Zilli, “Grid cell firing may arise from interference of theta frequency membrane potential oscillations in single neurons,”
*Hippocampus*, vol. 17, no. 12, pp. 1252–1271, 2007. View at Publisher · View at Google Scholar - N. Burgess, C. Barry, and J. O'Keefe, “An oscillatory interference model of grid cell firing,”
*Hippocampus*, vol. 17, no. 9, pp. 801–812, 2007. View at Publisher · View at Google Scholar - N. Burgess, C. Barry, K. J. Jeffery, and J. O'Keefe, “A grid and place cell model of path integration utilizing phase precession versus theta,” in
*Proceedings of the 1st Annual Computational Cognitive Neuroscience Conference*, Washington, DC, USA, April 2005. - C. T. Dickson, J. Magistretti, M. H. Shalinsky, E. Fransén, M. E. Hasselmo, and A. Alonso, “Properties and role of I(h) in the pacing of subthreshold oscillations in entorhinal cortex layer II neurons,”
*Journal of Neurophysiology*, vol. 83, no. 5, pp. 2562–2579, 2000. - L. M. Giocomo and M. E. Hasselmo, “Time constant of I(h) differs along dorsal to ventral axis of medial entorhinal cortex,” submitted.
- M. Yoshida and A. Alonso, “Cell-type-specific modulation of intrinsic firing properties and subthreshold membrane oscillations by the M(Kv7)-current in neurons of the entorhinal cortex,”
*Journal of Neurophysiology*, vol. 98, no. 5, pp. 2779–2794, 2007. View at Publisher · View at Google Scholar - B. Tahvildari and A. Alonso, “Morphological and electrophysiological properties of lateral entorhinal cortex layers II and III principal neurons,”
*The Journal of Comparative Neurology*, vol. 491, no. 2, pp. 123–140, 2005. View at Publisher · View at Google Scholar - T. Hafting, M. Fyhn, S. Molden, M.-B. Moser, and E. I. Moser, “Microstructure of a spatial map in the entorhinal cortex,”
*Nature*, vol. 436, no. 7052, pp. 801–806, 2005. View at Publisher · View at Google Scholar - F. Sargolini, M. Fyhn, T. Hafting, et al., “Conjunctive representation of position, direction, and velocity in entorhinal cortex,”
*Science*, vol. 312, no. 5774, pp. 758–762, 2006. View at Publisher · View at Google Scholar - J. S. Taube, R. U. Muller, and J. B. Ranck Jr., “Head-direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis,”
*The Journal of Neuroscience*, vol. 10, no. 2, pp. 420–435, 1990. - P. E. Sharp, “Multiple spatial/behavioral correlates for cells in the rat postsubiculumml: multiple regression analysis and comparison to other hippocampal areas,”
*Cerebral Cortex*, vol. 6, no. 2, pp. 238–259, 1996. View at Publisher · View at Google Scholar - H. T. Blair and P. E. Sharp, “Anticipatory head direction signals in anterior thalamus: evidence for a thalamocortical circuit that integrates angular head motion to compute head direction,”
*The Journal of Neuroscience*, vol. 15, no. 9, pp. 6260–6270, 1995. - J. S. Taube, “Head direction cells and the neurophysiological basis for a sense of direction,”
*Progress in Neurobiology*, vol. 55, no. 3, pp. 225–256, 1998. View at Publisher · View at Google Scholar - R. U. Muller, J. B. Ranck Jr., and J. S. Taube, “Head direction cells: properties and functional significance,”
*Current Opinion in Neurobiology*, vol. 6, no. 2, pp. 196–206, 1996. View at Publisher · View at Google Scholar - F. Cacucci, C. Lever, T. J. Wills, N. Burgess, and J. O'Keefe, “Theta-modulated place-by-direction cells in the hippocampal formation in the rat,”
*The Journal of Neuroscience*, vol. 24, no. 38, pp. 8265–8277, 2004. View at Publisher · View at Google Scholar - M. Muller and R. Wehner, “Path integration in desert ants, cataglyphis fortis,”
*Proceedings of the National Academy of Sciences of the United States of America*, vol. 85, no. 14, pp. 5287–5290, 1988. View at Publisher · View at Google Scholar - R. Wehner and M. Muller, “The significance of direct sunlight and polarized skylight in the ant's celestial system of navigation,”
*Proceedings of the National Academy of Sciences of the United States of America*, vol. 103, no. 33, pp. 12575–12579, 2006. View at Publisher · View at Google Scholar - A. Cheung, S. Zhang, C. Stricker, and M. V. Srinivasan, “Animal navigation: the difficulty of moving in a straight line,”
*Biological Cybernetics*, vol. 97, no. 1, pp. 47–61, 2007. View at Publisher · View at Google Scholar - R. Wehner, B. Michel, and P. Antonsen, “Visual navigation in insects: coupling of egocentric and geocentric information,”
*Journal of Experimental Biology*, vol. 199, part 1, pp. 129–140, 1996. - J. O'Keefe, N. Burgess, J. G. Donnett, K. J. Jeffery, and E. A. Maguire, “Place cells, navigational accuracy, and the human hippocampus,”
*Philosophical Transactions of the Royal Society B*, vol. 353, no. 1373, pp. 1333–1340, 1998. View at Publisher · View at Google Scholar - P. E. Sharp and S. Turner-Williams, “Movement-related correlates of single-cell activity in the medial mammillary nucleus of the rat during a pellet-chasing task,”
*Journal of Neurophysiology*, vol. 94, no. 3, pp. 1920–1927, 2005. View at Publisher · View at Google Scholar - G. Strang,
*Linear Algebra and Its Applications*, Harcourt Brace Jovanovich, San Diego, Calif, USA, 1988. - D. J. Foster and M. A. Wilson, “Reverse replay of behavioural sequences in hippocampal place cells during the awake state,”
*Nature*, vol. 440, no. 7084, pp. 680–683, 2006. View at Publisher · View at Google Scholar - R. A. Koene and M. E. Hasselmo, “Reversed and forward buffering of behavioral spike sequences enables retrospective and prospective retrieval in hippocampal regions CA3 and CA1,”
*Neural Networks*, vol. 21, no. 2-3, pp. 276–288, 2008. View at Publisher · View at Google Scholar - E. J. Markus, Y.-L. Qin, B. Leonard, W. E. Skaggs, B. L. McNaughton, and C. A. Barnes, “Interactions between location and task affect the spatial and directional firing of hippocampal neurons,”
*The Journal of Neuroscience*, vol. 15, no. 11, pp. 7079–7094, 1995. - D. Derdikman, M. Fyhn, T. Hafting, M. B. Moser, and E. I. Moser, “Breaking up the entorhinal grid in a hairpin maze,”
*Society for Neuroscience Abstract*, vol. 33, p. 68.10, 2006. - M. Fyhn, T. Hafting, A. Treves, M.-B. Moser, and E. I. Moser, “Hippocampal remapping and grid realignment in entorhinal cortex,”
*Nature*, vol. 446, no. 7132, pp. 190–194, 2007. View at Publisher · View at Google Scholar - C. Barry, R. Hayman, N. Burgess, and K. J. Jeffery, “Experience-dependent rescaling of entorhinal grids,”
*Nature Neuroscience*, vol. 10, no. 6, pp. 682–684, 2007. View at Publisher · View at Google Scholar - T. Solstad, E. I. Moser, and G. T. Einevoll, “From grid cells to place cells: a mathematical model,”
*Hippocampus*, vol. 16, no. 12, pp. 1026–1031, 2006. View at Publisher · View at Google Scholar - C. P. Heesy, “On the relationship between orbit orientation and binocular visual field overlap in mammals,”
*The Anatomical Record. Part A*, vol. 281, no. 1, pp. 1104–1110, 2004. View at Publisher · View at Google Scholar - J. O'Keefe and N. Burgess, “Dual phase and rate coding in hippocampal place cells: theoretical significance and relationship to entorhinal grid cells,”
*Hippocampus*, vol. 15, no. 7, pp. 853–866, 2005. View at Publisher · View at Google Scholar - T. Hafting, M. Fyhn, M.-B. Moser, and E. I. Moser, “Phase precession and phase locking in entorhinal grid cells,”
*Society for Neuroscience Abstract*, vol. 32, p. 68.8, 2006. - J. O'Keefe and M. L. Recce, “Phase relationship between hippocampal place units and the EEG theta rhythm,”
*Hippocampus*, vol. 3, no. 3, pp. 317–330, 1993. View at Publisher · View at Google Scholar - W. E. Skaggs, B. L. McNaughton, M. A. Wilson, and C. A. Barnes, “Theta phase precession in hippocampal neuronal populations and the compression of temporal sequences,”
*Hippocampus*, vol. 6, no. 2, pp. 149–172, 1996. View at Publisher · View at Google Scholar - A. P. Maurer, S. R. VanRhoads, G. R. Sutherland, P. Lipa, and B. L. McNaughton, “Self-motion and the origin of differential spatial scaling along the septo-temporal axis of the hippocampus,”
*Hippocampus*, vol. 15, no. 7, pp. 841–852, 2005. View at Publisher · View at Google Scholar - M. Lengyel, Z. Szatmáry, and P. Érdi, “Dynamically detuned oscillations account for the coupled rate and temporal code of place cell firing,”
*Hippocampus*, vol. 13, no. 6, pp. 700–714, 2003. View at Publisher · View at Google Scholar - M. V. Tsodyks, W. E. Skaggs, T. J. Sejnowski, and B. L. McNaughton, “Population dynamics and theta rhythm phase precession of hippocampal place cell firing: a spiking neuron model,”
*Hippocampus*, vol. 6, no. 3, pp. 271–280, 1996. View at Publisher · View at Google Scholar - G. V. Wallenstein and M. E. Hasselmo, “GABAergic modulation of hippocampal population activity: sequence learning, place field development, and the phase precession effect,”
*Journal of Neurophysiology*, vol. 78, no. 1, pp. 393–408, 1997. - O. Jensen and J. E. Lisman, “Hippocampal CA3 region predicts memory sequences: accounting for the phase precession of place cells,”
*Learning & Memory*, vol. 3, no. 2-3, pp. 279–287, 1996. View at Publisher · View at Google Scholar - P. E. Sharp and K. Koester, “Lesions of the mammillary body region severely disrupt the cortical head direction, but not place cell signal,”
*Hippocampus*. In press. View at Publisher · View at Google Scholar - M. E. Hasselmo, “Arc length coding by interference of theta frequency oscillations may underlie context-dependent hippocampal unit data and episodic memory function,”
*Learning & Memory*, vol. 14, no. 11, pp. 782–794, 2007. View at Publisher · View at Google Scholar - T. van Groen and J. M. Wyss, “The postsubicular cortex in the rat: characterization of the fourth region of the subicular cortex and its connections,”
*Brain Research*, vol. 529, no. 1-2, pp. 165–177, 1990. View at Publisher · View at Google Scholar - M. Caballero-Bleda and M. P. Witter, “Regional and laminar organization of projections from the presubiculum and parasubiculum to the entorhinal cortex: an anterograde tracing study in the rat,”
*The Journal of Comparative Neurology*, vol. 328, no. 1, pp. 115–129, 1993. View at Publisher · View at Google Scholar - C. Köhler, “Intrinsic projections of the retrohippocampal region in the rat brain. I. The subicular complex,”
*The Journal of Comparative Neurology*, vol. 236, no. 4, pp. 504–522, 1985. View at Publisher · View at Google Scholar - D. G. Amaral and M. P. Witter, “The 3-dimensional organization of the hippocampal formation: a review of anatomical data,”
*Neuroscience*, vol. 31, no. 3, pp. 571–591, 1989. View at Publisher · View at Google Scholar - M. C. Fuhs and D. S. Touretzky, “A spin glass model of path integration in rat medial entorhinal cortex,”
*The Journal of Neuroscience*, vol. 26, no. 16, pp. 4266–4276, 2006. View at Publisher · View at Google Scholar - E. T. Rolls, S. M. Stringer, and T. Elliot, “Entorhinal cortex grid cells can map to hippocampal place cells by competitive learning,”
*Network*, vol. 17, no. 4, pp. 447–465, 2006. View at Publisher · View at Google Scholar - L. W. Swanson, J. M. Wyss, and W. M. Cowan, “An autoradiographic study of the organization of intrahippocampal association pathways in the rat,”
*The Journal of Comparative Neurology*, vol. 181, no. 4, pp. 681–715, 1978. View at Publisher · View at Google Scholar - P. A. Naber and M. P. Witter, “Subicular efferents are organized mostly as parallel projections: a double-labeling, retrograde-tracing study in the rat,”
*The Journal of Comparative Neurology*, vol. 393, no. 3, pp. 284–297, 1998. View at Publisher · View at Google Scholar - K. Louie and M. A. Wilson, “Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep,”
*Neuron*, vol. 29, no. 1, pp. 145–156, 2001. View at Publisher · View at Google Scholar - M. E. Hasselmo, C. Bodelón, and B. P. Wyble, “A proposed function for hippocampal theta rhythmml: separate phases of encoding and retrieval enhance reversal of prior learning,”
*Neural Computation*, vol. 14, no. 4, pp. 793–817, 2002. View at Publisher · View at Google Scholar