Episodic memory retrieval of events at a specific place and time is effective for future planning. Sequential reactivation of the hippocampal place cells along familiar paths while the animal pauses is well suited to such a memory retrieval process. It is, however, unknown whether this awake replay represents events occurring along the path. Using a subtask switching protocol in which the animal experienced three subtasks as ‘what’ information in a maze, I here show that the replay represents a trial type, consisting of path and subtask, in terms of neuronal firing timings and rates. The actual trial type to be rewarded could only be reliably predicted from replays that occurred at the decision point. This trial-type representation implies that not only ‘where and when’ but also ‘what’ information is contained in the replay. This result supports the view that awake replay is an episodic-like memory retrieval process.https://doi.org/10.7554/eLife.08105.001
Place cells are neurons that respond to a particular location in the physical world. For example, as a rat runs around a maze, some place cells will become active when the rat reaches one corner. When the rat moves on towards a different corner, other place cells activate instead. The real-time activity of these place cells helps the rat to work out where it is in the maze. This activity also contains information about what the rat is doing.
In addition to their real-time activity, place cells also help previous events to be ‘replayed’ mentally, which is important for making decisions. Previous studies have shown that when a rat pauses during a task, place cell replays allow it to mentally map out the route it needs to take. However, it is less clear whether these replays also provide information about what the rat needs to do.
Takahashi gave rats a number of tasks to perform inside a figure of eight maze. In one of these tasks, the rat had to assess which one of two lights was lit up, and run towards it. In the other two tasks, the rat had to remember the direction it took on the previous occasion, and go in the opposite direction. During these tasks, the rat would occasionally pause to replay information about the task.
Takahashi recorded what the rats' place cells were doing during these pauses, and found that the place cell replays contained information about both the path the rat needed to take (‘where and when’ information), and which task it needed to carry out (‘what’ information). This suggests that replays are important for the ability to recall information about specific events, which is known as episodic memory. Takahashi's results may therefore also help us to learn more about this “mental time travel” and human conditions that damage episodic memory, such as Alzheimer's disease.https://doi.org/10.7554/eLife.08105.002
One of the most important aspects of episodic memory is the mental retrieval of personal recollections of what happened “where” and “when”, referred to as “mental time travel” (Tulving, 1983). Episodic memory retrieval is important not only for remembering recent events but also for imagining future ones (Hassabis et al., 2007). The hippocampus is critically involved in episodic memory retrieval, and within this structure reside place cells—principal cells exhibiting place-specific firing patterns in particular locations. The presence of such cells has led to the hypothesis that the hippocampus is the locus of the “where” element of episodic memory (O'Keefe and Dostrovsky, 1971). Following on from this seminal finding, more recent studies have demonstrated that hippocampal place cell activity simultaneously represents a broad range of aspects of memory, including odors and their match/non-match status in a particular location, called the “place field”, as well as elapsed time (Otto and Eichenbaum, 1992; Sakurai, 1996; Wood et al., 1999; Pastalkova et al., 2008; MacDonald et al., 2011). In the place field, both firing rate and location can change independently and are affected by external factors, such as spatial environment and task demand, and internal factors such as prospective and retrospective memory (Markus et al., 1995; Frank et al., 2000; Wood et al., 2000; Anderson and Jeffery, 2003; Ferbinteanu and Shapiro, 2003; Smith and Mizumori, 2006). During maze running, place cell activity sequences in rodents convey information on both familiar paths and accompanying subtasks (Allen et al., 2012; Takahashi, 2013).
Within sharp wave/ripples (SWRs) in local field potentials (LFPs) during slow-wave sleep or periods of awake immobility, internally generated place cell activity sequences are often reactivated in a temporally compressed manner (Wilson and McNaughton, 1994; Foster and Wilson, 2006; O'Neill et al., 2006; Diba and Buzsáki, 2007; Karlsson and Frank, 2009; Gupta et al., 2010). During brief periods of immobility, pivotal paths to a remembered goal can be predicted, regardless of the location in which the SWRs have previously occurred (Pfeiffer and Foster, 2013). Such “replay” is thus considered a neuronal substrate for retrieving memories on familiar paths (Carr et al., 2011). However, previous studies investigating the content of replays have primarily focused on the geography and timeline of the animals' running path as experienced in the environment. It is therefore unclear whether the replay conveys information on events (i.e., “what” information) occurring along the reactivated path.
I recently recorded the ensemble activity of place cells in the hippocampal CA1 of rats made to continuously navigate four journeys in a figure-eight maze under three intermittently switching subtasks within a single session: visually guided discrimination (VD), non-delayed spatial alternation (NA), and delayed spatial alternation (DA) (Takahashi, 2013). This task design allowed for comparison of hippocampal neuronal activity between and within different journeys and subtasks. I found that while the animal is running in the maze, differences between individual journeys and between subtasks in the maze are independently encoded in the firing locations and rates of place cells, respectively. Furthermore, all trial types (i.e., journey type and subtask) experienced in the maze could be decoded from the ensemble activity of place cells. Since the rats ran along similar spatial paths while performing different subtasks throughout this task, differences between subtasks can be interpreted as non-spatial “what” information. Provided that the mechanisms are preserved in temporally compressed replays that occur during brief periods of immobility, the subtask as “what” information occurring along the path in the maze might be examined in the replay. Building on this previous study (Takahashi, 2013), I accordingly investigated whether “what” information is contained in the replay of place cell activity sequences during the brief immobility periods that occur while the animal is engaging in the task.
According to the subtask switching protocol, four rats were trained for about one hour in a specific sequence—VD (20 laps), NA (20 laps), VD (10 laps), DA (20 laps), VD (20 laps), NA (20 laps), VD (10 laps), and DA (20 laps)—and were rewarded each time they arrived at the reward zone at the correct return rail (see ‘Materials and methods’). In the VD subtask, a cue light on the left or right side was illuminated to indicate which direction to turn at a decision point (Figure 1A) to receive a reward. In both the NA and DA subtasks, cue lights on both sides were illuminated and the rats had to alternate between left and right at the decision point (Figure 1B). The difference between the NA and DA subtasks was that in the DA subtask, a barrier appeared in the middle of the central stem for five seconds to impose a delay period. The subtask to be rewarded (i.e., VD, NA, or DA) could therefore only be distinguished at the decision point, based on visual cues (Takahashi, 2013).
I tested whether the methods used for the subsequent replay analyses could predict running path and trial type from the place cell activity sequences recorded during running. A total of 1084 principal cells in the dorsal hippocampal CA1 were recorded using an array of 10 extracellular dodecatrodes (Takahashi and Sakurai, 2005) in four well-trained rats (individuals that had achieved an overall task performance of >90% for a week), while they were performing the task (Table 1). Because running speed, head direction, and physical position can influence place cell firing (McNaughton et al., 1983; Wiener et al., 1989), only 556 place cells were examined in the following analyses. These cells, which had spatial information >0.3 bits/spike, fired at significantly different rates during different trial types (i.e., journey type and subtask), even when running speed, head direction, and x and y coordinates were taken into account (Analysis of covariance (ANCOVA) with these covariates, p < 0.05) (Wood et al., 2000). The rats were exposed to each of the subtasks intermittently at least twice throughout the entire session, but my previous study suggested that place fields are not remapped between trials of the same type, regardless of differences in levels of exposure to the trials (Takahashi, 2013). I therefore combined place cell activity from all trials of the same type, irrespective of the level of exposure.
Differences between individual journeys are encoded in the timing of firing across place cells while the animal is running (Figure 1—figure supplement 1A,B) (Takahashi, 2013). For each trial type, the rats' paths could be accurately decoded from the place cell activity sequences (Figure 1C, top) using a memoryless Bayesian decoder (Figure 1C, middle; median error: 6.0–13.5 cm). Since similar paths could be decoded from the place cell activity sequences during trial types with the same journey but different subtasks (e.g., VDLR, NALR, and DALR), the Bayesian decoder cannot per se identify trial types. As my previous study reported (Takahashi, 2013), differences between subtasks are encoded in firing rates across place cells. Therefore, using a prediction method based on these firing rates (Allen et al., 2012), I predicted trial type from the firing rate pattern across place cells in conjunction with the decoded path. The bottom portion of Figure 1C shows trial types predicted from rat I in a single trial. In the visually guided VDRR and VDLL trials, some mismatches occurred because the rats could not accurately predict the trial type until they reached the junction. The predictions were most accurate for the spatial alternation trials. Confusion matrices between predicted and actual trial types for each rat during the entire session (Figure 1D) show that a similar pattern was observed for all rats (overall mean, 74.3–84.5% (leave-one-out estimate), chance = 12.5%; Table 1). The Bayesian decoder and the prediction method together are sufficient for interpreting the representation of path and trial type in place cell activity sequences.
Before examining the replays, I investigated the rats’ behavior during the brief periods of immobility, when the replays occurred and the animals' behavior suggested that they were actively gathering information (Benjamini et al., 2011). Specifically, I assessed the directions in which the rats' heads were pointing at these times. In correct trials, the rats' heads pointed towards a goal in the opposite direction from the respective start zone (Watson–Williams test, P: 1.1 × 10−16, n = 588 (left), 1305 (right); Rayleigh test; P: left, 1.1 × 10−56, right, 4.9 × 10−324) and stem (Watson–Williams test, P: 1.1 × 10−16, n = 257 (left), 395 (right); Rayleigh test; P: left, 1.7 × 10−235, right, 5.2 × 10−125) (Figure 2). This suggests that the rats sampled a memory-guided goal in preparation for switching to either the NA or the DA subtask. In this experiment, head direction is therefore considered a behavioral sign indicating expected future choice. In erroneous trials, however, head direction was ambiguous only in the maze stem (Figure 2C,D; Watson–Williams test, (2C) P: 4.4 × 10−16, n = 29 (left), 30 (right), (2D) P: 0.29, n = 26 (left), 10 (right); Rayleigh test, P: (2C) left, 3.1 × 10−5, right, 8.2 × 10−18, (2D) left, 2.0 × 10−14, right, 5.1 × 10−7). Considering that the barrier in the maze stem would have been an indication to the rats of the ongoing DA subtask, this suggests that the spatial memory demand in preparation for NA/DA subtasks decreased within the maze stem. In addition, prior to making erroneous choices, the rats did not make a decision at the behavioral level within the maze stem.
I examined path representation in the temporally compressed replay of place cell activity sequences during the brief periods of immobility in the awake behavioral state. Some reactivations of place cell activity sequences during SWRs were related to the rats' upcoming path on the task. For instance, consider the spike raster plots of nine replays that occurred in the start zone, central stem, or junction (Figure 3). To arrange the sequences in terms of the cells' place fields along the journey, I ordered the cells by the latency of their peak firing rates during the corresponding trial type. The cell number decreases from the start zone to the goal. Sequential reactivation of the place cell activity during the replays shown in the raster plots began where the rat paused (blue circle in the trial-type illustration) and moved forward toward a goal.
To test whether the replays consistently related to upcoming behavior, I used the Bayesian decoder at a 10-fold compressed scale (Davidson et al., 2009; Pfeiffer and Foster, 2013) in the replay analyses. To avoid confusing replay and sequential activation during movement-related phase precession, population bursts of spiking activity during periods of awake immobility, when rats were moving less than 2 cm/s, were identified as candidate replays (Figure 3—figure supplement 1A). Of a total of 2820 candidates, 552 were identified as statistically significant replays with a continuous path, whose length and duration satisfied several criteria (Davidson et al., 2009; Pfeiffer and Foster, 2013) (see ‘Materials and methods’; Figure 3—figure supplement 1; Table 2). LFPs during the path replays were largely coincident with the SWRs (99.8%; Table 2), and their theta power (4–12 Hz) was significantly smaller than during running (Figure 3—figure supplement 2). This confirms that the path replays identified here were similar to those reported previously (Foster and Wilson, 2006; Diba and Buzsáki, 2007; Davidson et al., 2009; Pfeiffer and Foster, 2013) and did not include low-speed replay events associated with theta oscillation (Johnson and Redish, 2007; Jezek et al., 2011).
To examine the spatial specificity of path representation in the replays, I analyzed the posterior probability of locations decoded from path replays by the Bayesian decoder. The middle portions of Figure 3 show nine representative decoded paths from the spiking activity shown in the raster plots during candidate replays. As the spike raster plots suggest, the decoded paths also began where the rats paused and moved forward toward a goal in the corresponding trial type. In addition, despite the fact that the actual goal in the VDRR trial was on the right side, the decoded path ended on the left side (green arrow), suggesting that the replay represents an upcoming path to a memory-guided goal (i.e., opposite to the previous goal). Across all replays, the posterior probability of decoded locations was spatially concentrated around the vicinity of the goal (Figure 4A). To statistically compare the strength of path representation in the replays between selected regions, path representation strength was defined as the sum total of the posterior probability. Path representation in the vicinity of both goals was significantly greater than in the remainder of the maze area (Figure 4B; Wilcoxon rank-sum test, p = 3.3 × 10−106, n = 552). As awake replay often begins at the animal's current location (Foster and Wilson, 2006; Diba and Buzsáki, 2007) and is enhanced by rewarding outcomes (Singer and Frank, 2009), the replay may be initiated at the goal. Although reward expectation increases occupancy time and place-field representation around the goals, such initiation bias was not correlated with either occupancy time (Figure 4—figure supplement 1A,C) or spatial distribution of place-field representation (Figure 4—figure supplement 1B,D). This suggests that the path replays were not a simple consequence of either reward expectation or place-specific firings in the location where they occurred (Gupta et al., 2010; Carr et al., 2011). Rather, they may represent paths on a memory-guided journey towards an upcoming goal (Pfeiffer and Foster, 2013), or on the immediately preceding journey (O'Neill et al., 2006; Karlsson and Frank, 2009).
To test this hypothesis, I examined the representation of future and past journeys in the replays. The path replays predominantly occurred in the start zone and in the maze stem (Figure 2E,F). Unlike the start zones, the maze stem was common to all journeys. There were therefore two types of path replays: path-start and path-stem. I centered the posterior probability of decoded locations on the start position and rotated it according to the direction of the memory-guided goal. As expected from the initiation bias, the rats' current location was strongly represented in the posterior probability of decoded locations in path-start replays (Figure 5A). To statistically test the difference between future and past journeys, I defined future and past regions as those from the start position to memory-guided and previous goals, respectively. It is worth noting that the memory-guided goal was always opposite to the previous goal even in visually guided trials (VDLL, VDRR), because the task design did not allow the rats to identify the subtask until they reached the junction (see ‘Materials and methods’). Path representation (i.e., the sum total of the posterior probabilities of decoded locations) of future regions was significantly greater than that of past regions in the correct spatial alternation trials (Figure 5B; Wilcoxon signed-rank test, p = 0.0037, n = 345). Similarly, future regions were strongly and statistically significantly represented in the correct visually guided trials (Figure 5C,D; p = 0.0078, n = 52). In erroneous trials, path representation of future regions was slightly greater than that of past regions (Figure 5E), but the difference was not significant (Figure 5F; p = 0.73, n = 9). However, due to the physically different start zones of different journeys, these characteristics may be affected by the place-specificity of the locations in which the replays occurred.
To compare the representation of past and future journeys in the path-stem replays, I centered the posterior probability of decoded locations on the rat's physical location at the time of occurrence, rotated it according to the direction of the memory-guided goal, and scaled it according to the distance to the memory-guided goal. Path representation was similar to that in the replays that occurred in the start zones (Figure 5G,I; Figure 5H: p = 9.8 × 10−8, n = 190; Figure 5J: p = 0.034, n = 38). Surprisingly, however, path representation in path-stem replays on journeys toward an erroneous goal (Figure 5K) was significantly different from that on journeys toward a correct goal (Figure 5L; p = 0.0078, n = 8), despite the fact that the rats sampled both goal sides evenly. Similar to the findings of a previous study (Pfeiffer and Foster, 2013), this suggests that in the maze stem, where spatial memory retrieval is a prerequisite for predicting a future goal, path representation in the replay is directly linked to the animal's future path.
The exact path encoded in the replay may be decoded from the locations showing the highest a posteriori probability. I therefore reanalyzed the path replays using point estimates of locations based on a posteriori probability. Similar to the posterior probability analyses, the decoded locations in the replays tended to represent an upcoming journey to a memory-guided goal (Figure 5—figure supplement 1). To corroborate the results from the Bayesian decoder, I furthermore applied a simple spatial reconstruction algorithm, based on place fields during running, to the path replays. To estimate path representation in the replay, the place maps of the maze in neurons participating in the replay were simply summed up in response to the number of spikes (see ‘Materials and methods’). Similarly to the results obtained using the Bayesian decoder, the path representation estimated using the simple spatial reconstruction algorithm also largely predicted an upcoming path to a memory-guided goal (Figure 6). A previous study (Pfeiffer and Foster, 2013) reported that the awake replays represent an upcoming path to a memory-guided goal, but did not examine erroneous behaviors. My findings strongly support this view, with the additional finding that the upcoming path encoded in the awake replay can be linked to the animal's future actions.
Although hippocampal place cell activity per se represents non-spatial information as well as spatial information (Wood et al., 1999), the question of whether replays include non-spatial information remains to be addressed. I next examined whether place cell activity during replays can convey information not only on the path but also on the subtask occurring along it, similarly to place cell activity occurring during running.
I observed that the firing rate of place cells participating in replays during different subtasks was dramatically different between subtasks even if the replay depicted a similar decoded path. For instance, consider the firing rate histograms for participating neurons in replays occurring in the start zone, central stem, or junction (Figure 3). As expected from the timing of temporally compressed firings across place cells, the maximum firing rates of place cells during replays (∼80 Hz) were about 10 times faster than typical maximum firing rates during running (∼8 Hz). To estimate the exact temporal compression rate of place cell firing during replays, I divided the average interspike interval during candidate replays by that during the entire rest of the session. The median of the estimated compression rate was 9.0 (Figure 3—figure supplement 3), suggesting that similarly to the timing of firing, place cell firing rates were also temporally compressed during replays. During replays occurring at the junction, the raster plots of spiking activity in DARL and VDRL trials recorded from rat II showed a similar activity sequence (Figure 3C, rightmost, arrows). The replays also depicted a similar decoded path. However, the firing rates during DARL trials were substantially greater than during VDRL trials, suggesting that the firing rates of neurons participating in the replay may contain information on the differences between subtasks.
To test whether subtask information is encoded in the firing rates in the replay, I predicted trial types based on the firing rates of a group of cells recorded during path replays, using the prediction method in conjunction with the decoded locations (see ‘Materials and methods’; Figure 3—figure supplement 1). It was necessary to ensure that the replays used predicted a trial type with a probability greater than that dictated by chance. Accordingly, of the total 552 path replays, 108 were classified as episodic replays that satisfied criteria concerning the most frequently predicted trial type and its predicted probability sum (see ‘Materials and methods’; Table 2). As expected, only the replays that occurred in the junction (Figure 3C) appeared to predict the actual trial type correctly.
To examine this further, I investigated preference for subtask (i.e., VD, NA, DA subtasks). I examined whether there was any preference in the replays for particular rewarded subtasks. As expected, based on a previous study suggesting that awake SWRs support spatial working memory and not reference memory (Jadhav et al., 2012), the occurrence rate of overall episodic replays across all actual subtasks to be rewarded was significantly biased toward the DA subtask, which required spatial working memory (Figure 7A; Chi-square goodness-of-fit test, post hoc multiple binomial tests corrected by the Benjamini and Hochberg procedure, VD vs DA: p = 8.6 × 10−5, NA vs DA: p = 9.9 × 10−4). The null hypothesis tested was that the occurrence rates of predicted subtasks were equal to the rates expected based on the ratio of subtasks experienced throughout the entire session (60 laps VD: 40 laps NA: 40 laps DA).
Does this bias depend on where the replay occurred? To address this question, episodic replays were divided into two types according to the occurrence location of the replay: episodic-start and episodic-stem replays. This was done because the head direction analyses suggested that spatial memory demand at the behavioral level was lower within the maze stem. The partitioning showed that the subtask bias occurred in episodic-start replays (Figure 7B; VD vs DA: p = 3.7 × 10−9; NA vs DA: p = 5.1 × 10−7), but not in episodic-stem replays (Figure 7C). Similarly, in the path-start replays, all candidates and SWR events showed such a bias (Figure 7—figure supplement 1A–C). This bias may be accounted for by occupancy time while the rats paused. However, since the rats specifically spent more time pausing in the DA subtask in both the start zones and the maze stem (Figure 7—figure supplement 1D,E), the bias cannot be explained by the time spent pausing. Combined with behavioral signs of decreased spatial memory demand within the maze stem, these results suggest that the occurrence rate of replays is enhanced by upcoming spatial working memory demand. In contrast, the subtasks predicted from the episodic replays did not show such a bias, and instead the prediction rates reflected the ratio of subtasks experienced throughout the entire session (Figure 7D). This suggests that the enhancement is not specifically due to the reactivated DA subtask, but instead due to the reactivation of all previously experienced subtasks.
Examination of trial type, composed of journey type and subtask, should provide further understanding of subtask preference. Even if the replay can represent an upcoming path, disambiguation of the subtasks is still required for identifying the actual trial type. I therefore investigated the relationship between the actual trial type to be rewarded and trial-type representation in the episodic replays. Since the path representation suggested that the replay encodes an upcoming path to a memory-guided goal, I only examined the memory-guided trial types (i.e., VDLR, VDRL, NALR, NARL, DALR, and DARL). Trial-type representation in the episodic replays did not match the actual trial type in the start zones (Figure 7E; Choen's κ = 0.017; z test, p = 0.83, n = 71), but did match it in the stem (Figure 7F; Choen's κ = 0.31; p = 8.4 × 10−3, n = 34). This discrepancy may be the result of the extent of spatial working memory demand at the location where the replay occurs.
To further examine trial-type preference, I divided the episodic-stem replays into two types: episodic-center and episodic-junction replays. Trial-type representation matched actual trial type moderately well in replays at the junction (i.e., the decision point) where the rats could evidently identify the actual trial type based on the visual cue (Figure 7H; Choen's κ = 0.58; p = 3.7 × 10−4, n = 17), but not at the center of the stem (Figure 7G; Choen's κ = 0.022; p = 0.93, n = 17). To corroborate these results, the Bayesian decoder in the trial-type prediction method was replaced by the simple spatial reconstruction algorithm. Similarly, trial types predicted using this method matched actual trial types moderately well only in replays at the junction (Figure 7—figure supplement 2). To further check whether the predicted and actual trial types were coincidently matched, the probability of trial-type prediction across all replays occurring at the junction was analyzed on a time-window basis. There was a significant effect of the probability of trial-type prediction for five trial types (NALR, DALR, VDRL, NARL, and DARL) in episodic replays (ANOVA, p < 0.01; Figure 7—figure supplement 3). The predicted trial types showing the highest probability matched actual trial types for NALR, VDRL, NARL, and DARL trials. Post hoc comparisons using the Tukey honest significant difference (HSD) test indicated that the predicted trial types with the highest probability were significantly different from most of the others, suggesting that the matches were not coincidental.
At the junction, only a single visual cue was present in the visually guided trials. To test whether the trial type encoded in these replays was therefore just a representation of the available sensory cues, I examined whether the difference in the number of visual cues showing (one or both) could be used to distinguish VD from the other subtasks encoded in replays at the junction. The number of visual cues showing did not account for VD representation, even at the junction (Figure 8A; χ2 test, p = 0.20). The barrier that appeared at the center of the stem only in DA trials may also represent a sensory cue, which would distinguish NA from DA trials. However, the subtask predicted in the center of the stem did not match actual NA and DA trials (Figure 8B; χ2 test, p = 0.26). This suggests that trial-type representation in the replays is more than just a representation of external cues, and that it is internally generated. I therefore conclude that the internally generated replay represents the actual trial type—not only the upcoming path but also the actual subtask to be rewarded—when the trial type can be identified, but otherwise it evenly represents all trial types thus far experienced on the reactivated path.
My investigations into whether the subtask and path experienced during task performance are represented in replays of place cell activity sequences associated with SWRs during brief immobility periods, produced the following findings. Firstly, the replay can represent non-spatial information on subtask as well as spatial information concerning the path. Secondly, whereas the path in the replay is encoded in temporally compressed firing timings across place cells, the accompanying subtask is encoded in their temporally compressed firing rates. Thirdly, the replay is enhanced by spatial working memory demand. Finally, the replay can represent the actual trial type accurately when the trial type can be identified, but otherwise it evenly represents all trial types previously experienced along the reactivated path.
Place cell activity can encode not only spatial information about past, current, and future places (Frank et al., 2000; Wood et al., 2000; Ferbinteanu and Shapiro, 2003) but also non-spatial information including task demand (Markus et al., 1995; Anderson and Jeffery, 2003; Smith and Mizumori, 2006), odors and their match/non-match status (Otto and Eichenbaum, 1992; Sakurai, 1996; Wood et al., 1999), and elapsed time (Pastalkova et al., 2008; MacDonald et al., 2011). The sequential activation of place cells reflects distinct episodic-like memories (Pastalkova et al., 2008); in fact, reactivation of the firing rates of single neurons in the human hippocampus represents specific episodes during free recall (Gelbard-Sagiv et al., 2008). However, such episodic-like memory traces are present across time-scales similar to those during the actual experience, and are thus distinct from the rapid, temporally compressed replays reported here. Although numerous previous reports have concluded that place cell activity sequences are reactivated in a temporally compressed manner in the awake state (Foster and Wilson, 2006; O'Neill et al., 2006; Diba and Buzsáki, 2007; Singer and Frank, 2009; Gupta et al., 2010) as well as during slow-wave sleep (Wilson and McNaughton, 1994), they have shown only that replays encode familiar paths. Until now, the possibility that replays simultaneously represent both spatial and non-spatial information has been purely conjectural. It is well known that spatial information is encoded in temporally compressed firing timings across place cells in replays. I have now shown that replays also encode non-spatial information in temporally compressed firing rates, revealing that global and rate remapping mechanisms during running (Leutgeb et al., 2005; Takahashi, 2013) are preserved in the temporally compressed replays that occur during brief periods of immobility.
Since awake replay is more common in a novel environment (Cheng and Frank, 2008), a few previous studies (O'Neill et al., 2006) have speculated that the content of replays reflects the total of previous experience. Contradicting with this hypothesis, growing evidence suggests that replays are not a simple function of experience (Gupta et al., 2010), but rather that reward outcomes enhance the reactivation of experiences (Singer and Frank, 2009) and that replays can depict future paths to remembered goals (Pfeiffer and Foster, 2013). In addition, awake replays often begin at the animal's current location (Diba and Buzsáki, 2007). The relationship between replay content and experience therefore does not seem to be straightforward. Some previous reports suggest that awake replay is similar to vicarious trial-and-error (VTE) events (Hu and Amsel, 1995; Johnson and Redish, 2007), because it can represent possible future paths even in completely unfamiliar places (Gupta et al., 2010). Although VTE events are reported to occur outside of SWRs, a subsequent study suggested that awake replay associated with SWRs is seen frequently at the choice point in a similar task, where animals must make a memory-guided decision between two journeys (Karlsson and Frank, 2009). Consistent with activity observed in VTE events, these awake replays extend out to the left or right arms of the track, as would be expected if the animal were playing out possible upcoming choices. Frank et al. therefore proposed a unified interpretation in which replays provide information on possible upcoming paths to downstream brain structures such as the prefrontal cortex and nucleus accumbens, which can assess the value of different paths and make a decision about future actions (Carr et al., 2011).
In this study, I trained rats to perform a similar task, but path representation in these replays was directly linked to the animal's future path. This apparent disparity may reflect the proficiency of the task in previous studies. In contrast to previous VTE studies (Johnson and Redish, 2007; Karlsson and Frank, 2009; Gupta et al., 2010), well-trained rats in the present study could accurately predict a goal-directed path at the behavioral level even at the decision point. However, the task design did not allow prediction of the subtask until the animal had entered the decision point. This suggests that not only the path but also the subtask representation in the replay is similar to VTE events. By extending the unified interpretation to include non-spatial memory in the retrieved memory, the fact that replays in the present study reflected all previously experienced trial types (except at the decision point) suggests that replays prepare the brain for unforeseen changes, which would allow continuous switching between subtasks.
I also found that replays were enhanced by spatial working memory demand. Since the initiation of an awake replay is often related to external spatial input at the animal's location (Karlsson and Frank, 2009), this enhancement of replays suggests that the internal demand of the hippocampal-dependent spatial working memory is another factor. Considering that this is similar to a cued memory retrieval process, my findings strongly support the hypothesis that awake replay plays a key role in episodic-like memory retrieval, which contributes to memory-based navigational planning and decision making (Carr et al., 2011).
The sequential activation of place cell activity associated with SWRs can also represent paths where the animal has never experienced (Dragoi and Tonegawa, 2011; Dragoi and Tonegawa 2013a; 2013b; Ólafsdóttir et al., 2015). As a result, a fairly large proportion of events representing different and multiple experiences can spontaneously occur, even in naive animals. I speculate that the reason why paths and subtasks were encoded in temporally compressed firing timings and rates, respectively, in the replays may be to arrange such multiple preplayed episodes into a single replay event to easily recall and imagine an upcoming episode based on prior experience. In the present study, the rats were sequentially trained in the VD, NA, and DA subtasks. Together with the preplay mechanism, this training protocol might have helped to recruit a consistent path representation across the three subtasks. Until now, previous studies investigating the content of preplays have primarily focused on the geography and timeline of the animals' running. Non-spatial information in the preplay may provide further insight into memory encoding and retrieval.
This study has shown that non-spatial subtask information occurring along temporally ordered places, as experienced by the animal, could be predicted by the sequential reactivation of place cells occurring while the animal paused during engagement in the task. I therefore speculate that awake replay is linked to the typical abilities of episodic memory: mental time travel and foreseeing future situations. This provides novel insight into debates on animals' capacities to use the faculties provided by episodic memory (Allen and Fortin, 2013).
Four male Wistar rats implanted with a custom-made microdrive were housed individually in cages where the light was maintained in a 12-hr light–dark cycle. The tests were performed during the light phase. The weight of all rats was kept at 80% of free-feeding body weight. All procedures were approved by the Doshisha University and Kyoto Sangyo University Institutional Animal Care and Use Committees.
The rats were initially trained to run along the left/right O-shaped track unidirectionally for reward signals (medial forebrain bundle (MFB) stimulation) in the start zone. After the rats were running smoothly along both left and right O-shaped tracks, they were trained in the figure-eight maze (overall: 100 × 140 cm, 20-cm height; path: 20-cm width; Figure 1A) to perform the VD subtask for a reward. In the VD subtask, one of the two visual cues (LED-lights at the right and left corners) was illuminated randomly; the visual cue at the decision point of the maze (Figure 1A, decision point) indicated which direction to turn to receive the reward. The rats were trained until they achieved at least 80% correct decisions for >20 laps. Next, they were trained to perform the NA subtask for >20 laps. In this case, both LEDs were illuminated, so the rats could not rely on the visual cue. Instead, they had to choose the direction opposite to the previous goal. Once they had achieved at least 80% correct decisions for >20 laps, they were trained to perform the DA subtask for 20 laps. This was almost identical to the NA subtask except that it included a delay period. This was enforced by a barrier appeared for 5 s, 20 cm ahead of the entrance to the maze stem (Figure 1A, red dotted line). The rats paused reliably, facing the forward direction, in front of the barrier, and achieved a 90% correct decision rate from the beginning of this task. The detailed training protocols are described in a previous study (Takahashi, 2013).
To prevent the rats from receiving any unintended distal room cues, the task was performed under dim light. They did not receive the visual cue until they reached the decision point because the height of the walls of the maze (20 cm) was much higher than the level of the rat's eyes (∼5 cm) and they did not stand up during the experiment. The sole signal for subtask switching was therefore LEDs that were visible only at the decision point.
Under isoflurane anesthesia, stimulation electrodes were inserted into the MFB in the right lateral hypothalamus (AP 2.5, ML 1.0, DV 9.5). A custom-made microdrive with 10 independently movable dodecatrodes was then fixed to the skull above the left hippocampus (AP 3.8, ML 3.0, DV 0.5). A week after surgery, the electrodes were individually lowered into the pyramidal cell layer of the hippocampal CA1. The extracellular signals were unity-gain buffered, filtered (600 Hz–6 kHz), amplified (gain = 5000), and continuously sampled at 25 kHz. One channel of each electrode was duplicated for detecting LFPs and was filtered (0.1 Hz–6 kHz) and amplified (gain = 500). The LFP recordings were further band-pass filtered (150–250 Hz) using the Hilbert transform. Data recorded during the MFB stimulation were removed prior to analysis. Local peaks in the power of this filtered signal of magnitude >mean + 3SD were identified and extracted as periods. The boundaries for each period were defined as the point at which the amplitude crossed the mean. Periods during which running speed was <2 cm/s were identified as SWR events. For theta-wave detection, the z-score power of a band-pass filtered (4–12 Hz) signal was calculated. After spike sorting as previously described (Takahashi et al., 2003; Takahashi and Sakurai, 2009; Takahashi, 2013), putative principal cells were distinguished from putative fast-spiking cells based on spike width (0.4 ms) and average firing rate (5 Hz). Cells whose firing rate was <0.1 Hz were excluded. Only the putative principal cells were used in the analyses described below. Spikes during delay periods in the DA subtask, when hippocampal activity was not place-specific (Pastalkova et al., 2008), were excluded.
The trajectory of the rats was linearized for each trial by projecting the actual trajectory onto a predefined idealized journey using nearest-neighbor Delaunay triangulation. Spatial bins had a resolution of approximately 2 cm. A one-dimensional map of the place field for each place cell was then constructed for each trial type in the standard manner (Takahashi, 2013) (smoothed with a Gaussian kernel of 3-cm width). The place map was constructed from spikes only when running speed was >5 cm/s so that spikes generated while the rats paused in front of the barrier wall in the DA subtask were excluded.
A memoryless Bayesian decoder (Zhang et al., 1998) was used to decode the rats' locations on the basis of place cell activity. Firstly, the probability of a rat's location given place cell firings within a time window was estimated as follows:
where fi and ni represent the place map and the number of spikes of the i-th place cell within the time window, respectively, N indicates the total number of place cells, and τ represents the duration of the time window.
The probability within each time window was normalized for every location as follows (Pfeiffer and Foster, 2013):
where Prob(Posk|spikes) represents the probability at the k-th location bin within the time window, and M represents the total number of location bins.
Time windows for task performance and replays were set at 250 ms and 20 ms, respectively. Provided that nProb(Posk|spikes) showed unimodality (Hartigan's dip test, p < 0.05), a point estimation of the location on the journeys was made based on this value, using maximum likelihood estimation.
A simple spatial reconstruction algorithm was used to virtually reconstruct the place preference of replays from place cell activity while the animal briefly paused. The place preference of replays, Rp, was estimated as follows:
where fi and ni represent the place map and the number of spikes of the i-th place cell firing in the replay, respectively, and N indicates the total number of place cells. For trial-type prediction, the location encoded in the replay was estimated based on the location where Rp was maximized.
Firstly, candidate replays were selected as follows (as described in Figure 3—figure supplement 1A). A candidate replay was defined as a period where the smoothed (Gaussian kernel; SD: 10 ms) population activity was greater than the mean and the peak was above the defined threshold (mean + 3SD). To minimize ambiguous detections, periods were decreased based on the following criteria. Time windows were reduced as much as possible such that at least 10% of all detected cells were detected within the time window, each cell included at least two spikes, and the duration was at least 30 ms. Secondly, for each period, the rat's location was decoded using the Bayesian decoder, for 20-ms time windows advanced in 5-ms increments. Periods were concatenated when the distance between neighboring decoded locations was less than 25 cm. Thirdly, concatenated periods whose sequence of decoded locations covered a total distance greater than that covered during four time windows were classified as candidate replays. Fourthly, two significance tests were conducted for every candidate, using a Monte Carlo method with two different random shuffling modes: cell identity and cell place field (5000 times each). Finally, candidates with p < 0.05 for both shuffle modes were defined as path replays.
Trial type was predicted for every time window, using mean firing rate as a point of reference (Allen et al., 2012). Firstly, a binary population vector, C, was constructed. This vector could exist in one of two states, high or low. The state was determined on the basis of 200-ms time windows throughout the entire session, based on whether the cell's firing rate in each time window was higher or lower than its mean firing rate for the whole session. Eight population reference vectors, R, were constructed to store the probability of the place cells being in the high or low state in each of the eight trial types, for every possible linearized location. The probability of obtaining the vector C within each time window was computed as follows:
where represents the probability, taken from vector R, that the i-th place cell is in state c during the t-th trial type, at location p, decoded from place cell activity within the corresponding time window using the Bayesian decoder or the simple spatial reconstruction algorithm. For each time window, the trial type with the highest value for was recorded as predicted for that time window.
For each path replay, trial type was predicted for each time window. The representative trial type for each path replay was defined as the most frequently predicted trial type across all time windows in that replay. Significance tests were conducted using a Monte Carlo method with two random shuffling modes: cell identity and decoded location (5000 times each). The path replays with p < 0.05 for both shuffle modes were defined as episodic replays.
All analyses were performed using custom-made programs based on Matlab (v8.3; MathWorks) and R (R Development Core Team, 2013) functions.
After the rats were sacrificed by pentobarbital sodium overdose and perfused with formalin, their brains were cut coronally at 30 μm and stained with cresyl violet. The location of the tip of each electrode was estimated.
Hippocampal place cells can encode multiple trial-dependent features through rate remappingJournal of Neuroscience 32:14752–14766.https://doi.org/10.1523/JNEUROSCI.6175-11.2012
Heterogeneous modulation of place cell firing by changes in contextThe Journal of Neuroscience 23:8827–8835.
Colloquium paper: Quantifying the buildup in extent and complexity of free exploration in miceProceedings of the National Academy of Sciences of USA 108:15580–15587.https://doi.org/10.1073/pnas.1014837108
Forward and reverse hippocampal place-cell sequences during ripplesNature Neuroscience 10:1241–1242.https://doi.org/10.1038/nn1961
Distinct preplay of multiple novel spatial experiences in the ratProceedings of the National Academy of Sciences of USA 110:9100–9105.https://doi.org/10.1073/pnas.1306031110
Patients with hippocampal amnesia cannot imagine new experiencesProceedings of the National Academy of Sciences of USA 104:1726–1731.https://doi.org/10.1073/pnas.0610561104
A simple test of the vicarious trial-and-error hypothesis of hippocampal functionProceedings of the National Academy of Sciences of USA 92:5506–5509.https://doi.org/10.1073/pnas.92.12.5506
Neural ensembles in CA3 transiently encode paths forward of the animal at a decision pointThe Journal of neuroscience 27:12176–12189.https://doi.org/10.1523/JNEUROSCI.3761-07.2007
Awake replay of remote experiences in the hippocampusNature Neuroscience 12:913–918.https://doi.org/10.1038/nn.2344
Interactions between location and task affect the spatial and directional firing of hippocampal neuronsThe Journal of Neuroscience 15:7079–7094.
R: A language and environment for statistical computingR: A language and environment for statistical computing, Available at: http://www.R-project.org/.
Hippocampal and neocortical cell assemblies encode memory processes for different types of stimuli in the ratThe Journal of Neuroscience 16:2809–2819.
Automatic sorting for multi-neuronal activity recorded with tetrodes in the presence of overlapping spikesJournal of Neurophysiology 89:2245–2258.https://doi.org/10.1152/jn.00827.2002
Elements of episodic memoryNew York: Oxford University Press.
Spatial and behavioral correlates of hippocampal neuronal activityThe Journal of Neuroscience 9:2737–2763.
Interpreting neuronal population activity by reconstruction: unified framework with application to hippocampal place cellsJournal of Neurophysiology 79:1017–1044.
Upinder S BhallaReviewing Editor; National Centre for Biological Sciences, India
eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see review process). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.
Thank you for submitting your work entitled “Episodic-like memory trace in awake replay of hippocampal place-cell activity sequences” for peer review at eLife. Your submission has been evaluated as potentially interesting by Eve Marder (Senior editor) and three reviewers, one of whom is a member of our Board of Reviewing Editors.
The reviewers have discussed the reviews with one another and the Reviewing editor has drafted this decision to help you prepare a revised submission. For your interest, at the very bottom of this letter we have included the initial reviews, so that you can see where the summary conclusions come from.
This paper reports that one can predict which actions a rat will take, from replay sequences and firing rates at a decision point. This extends previous work that shows that it is possible to predict goals from sequences of brain activity. The key differences are that the nature of the task is a non-spatial datum, and that this is encoded not in sequences but in firing rates of replays.
The reviewers all felt that the paper was potentially interesting. They also agreed that the paper was quite confusing and needs considerable rewriting and clarification of methodology. It especially needs much improved reporting of data and a more thorough statistical analysis. This must include:
1) Direct representation of neuronal recordings.
2) Other forms of analysis of the p-values of the replay sequences.
3) A distinct analysis of encoding beyond the Bayesian decoder.
4) The author must also provide evidence to support the conclusion that “replay in the junction encodes trial-type” is more than just representation of available sensory cues.
The reviewers felt they need to see these clarifications in order to make a final decision on the paper.
This paper reports that one can predict which actions a rat will take, from replay sequences and firing rates at a decision point. There is some similarity in the overall result with David Foster's work on predicting goals. The key differences stated by the author is that task demand information is non-spatial, and that this is encoded not in sequences but in firing rates of replays.
In general I found the findings of the paper potentially quite interesting, but it turned out to be very difficult to track down the source of each of the presented conclusions. The key measurements were under-emphasized. The author seems to have put a layer of analysis all over the direct data, and everything is viewed through the filter of a Bayesian decoder. The paper is confusing in many places.
1) I would like to see direct data showing the results. The figures don't show rates or rasters, instead using a Bayesian decoder for almost all figures. I really think the author needs to use an alternative and more direct method to support each of the conclusions, since much is hidden by the inner workings of the decoder. For example, there is a small sample of a replay in Figure 3-figure supplement 1. I would like to see each of the points about replay events illustrated by a raster, a firing rate histogram, or other forms.
2) The paper lacks clarity in explaining where each of the conclusions comes from. For example, the author states in the Discussion that the task demand is encoded in “temporally compressed firing rates of place cells”. I struggled to find where this result was obtained in the paper, not least because firing rates were hidden from view as mentioned above. This turns out to be mentioned almost in passing in subsection “Replay is enhanced by spatial working memory demand”, and is easy to miss because it refers to the supplemental figure in Materials and Methods, rather than to presented data.
3) The paper builds a case for task demand representation in replay, and asserts that the task demand information is non-spatial. At face value the task demand can also be interpreted as a spatial datum: where to go next? I would ask that the author clarifies why this should be treated as non-spatial.
The author of this study investigates the phenomenon of place cell sequence replay during awake states under several experimental conditions. There are several findings, most of which are confirmatory of previous studies. The main, new finding of the study is that replay is specific to different experimental conditions. While this finding is interesting, there are a number of important aspects that need to be first clarified in a revised version.
First, the author claims that different types of trials result in differential neuronal activity in the hippocampus. However, what is important and should be shown is how different was the neuronal activity across these types of trials and how different was the decoding of this activity. These should be shown at least with average (and standard deviation/error) probability values across conditions in addition to the assessment of overall significance, considering and comparing all experimental possibilities like it was done in Figure 1D.
Second, the reported decoded probabilities are quite small in their average value, which brings into question how did they pass the significance test from control shuffles. Will the main effect still hold if a lower p-value (e.g., 0.01) is chosen as significance for each individual decoded event?
Third, what are some of the mechanisms by which the recruitment of context-specific replay is achieved in this study? Is the medial forebrain bundle stimulation administered as reward playing any role in the context-specific replay?
Finally, what could be the meaning of the replay events presented here, given that a quite large proportion of events significantly representing different and multiple experiences can occur spontaneously even in naive animals, such as in the case of pre-play?
Previous studies have suggested the replay of hippocampal firing sequences that occur during SWR may encode past or future task related behaviors. Here the author uses a nice design to further address this important question. Rats are trained on a task-switching paradigm in which whether they turn left or right at the junction of a Figure 8 maze is determined by visual cues (visual discrimination) or from memory (alternation tasks). Importantly the rats do not know until the juncture which type of task they are performing (but see below). One of the main findings is that by analyzing the spike patterns during replay with a Bayesian decoder it was possible to predict the trial type/condition when the replay occurred at the juncture, but not in the start point or stem.
While the experiments seem well executed, my concern is that it is not seem that the conclusions are supported by the results.
The main finding that replay in the juncture encodes trial-type is difficult to interpret, because at the juncture point trial type is also present in the available sensory cues (that is, the rat can see what the trial type is based on the lights). While the author attempts to address this issue I did not find this conclusion compelling.
I don't think the statements that trial type is predicted during replay in the stem junction are backed by any statistics. While the red bars are above the black in Figure 3F only two trial types are shown (why?). Additionally, I think Figure 3 D-F is mentioned out of order in the text.
It is stated that the rat only knows the correct response at the decision point (see Results section). But in the DA case doesn't it know it will have to alternate (as stated in subsection “Rats' behavior during brief periods of immobility”). Is DA versus NA encoded during the replay in the maze stem?
While trial prediction was well above chance, it was not clear how trial type was encoded. E.g. in Figure 1C conditions VDlr, NAlr, and DAlr look very similar, nevertheless the decoder extracts the trial type. How is this information encoded. Are some place cells only place cells during specific trial types? The relationship between Figure 1D and 1E was not clear. Figure 1D on the VDrr trials most of the predictions were wrong (sum of the black was more than the red). Why does this not show up in the confusion matrix for Rat 1?https://doi.org/10.7554/eLife.08105.022
- Susumu Takahashi
- Susumu Takahashi
- Susumu Takahashi
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
I would like to thank Y Sakurai and F Fujiyama for discussion and comments on the manuscript, H Shimazaki for helpful suggestions on the prediction method. This study was supported by the SCOPE from Ministry of Internal Affairs and Communications (MIC) (152107008), by the JST PRESTO, and by the JSPS KAKENHI (24300148 and 25560435).
Animal experimentation: All procedures were approved by the Doshisha University (approved number: 1229) and Kyoto Sangyo University (approved numbers: 2011-04 and 2011-05) Institutional Animal Care and Use Committees.
- Upinder S Bhalla, Reviewing Editor, National Centre for Biological Sciences, India
- Received: June 2, 2015
- Accepted: September 18, 2015
- Version of Record published: October 20, 2015 (version 1)
© 2015, Takahashi
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.