Abstract
Theoretical assessment of the function of the hippocampus has suggested that given certain physiological constraints at both the neuronal and cortical level, the hippocampus is best suited to associate discontiguous items that occur in different temporal or spatial positions. Conceptually, “discontiguous” refers to events that are to be associated with one another but do not temporally or spatially overlap. However, given that humans can actively maintain information “online” by rehearsing it, even when the information is no longer being presented to the sensory system, the right way to experimentally define “discontiguity” is still a question. Does it refer to a “gap” in the presentation of information (temporal discontiguity) or to an “interruption” of the active maintenance of working memory (WM) information (functional discontiguity)? To assess this, participants were imaged by functional magnetic resonance imaging (fMRI) when making judgments on whether two words were semantically related or not. In contrast with recognition memory that can be carried out through perceptual familiarity heuristics, judgments on semantic relatedness can only be accomplished through associative processing. To assess this experimentally, two words are either (1) presented at the same time (Event AB) or (2) one after the other with an unfilled, cross-viewing delay (Event A_B) (the uninterrupted discontiguity) or (3) presented one after the other, between which participants are required to perform a calculation task (Event A#B) (the interrupted discontiguity). Results of event-related fMRI analysis revealed that relative to Event AB, Event A_B was not associated with more hippocampal activity, whereas Event A#B was. The direct contrast of Event A#B relative to Event A_B also revealed significant hippocampal and parahippocampal activity. This result implied that functional discontiguity (the interruption of online maintenance of the inputted information) could be more apt at engaging the function of the hippocampus.
1. Introduction
The hippocampus has been recognized as the important cortical region for episodic memory. Its key feature has been proposed to be the representation of event sequences [1–3]. Theories have pointed to the role of the hippocampus in associating events across time [4]. Correspondingly, computational models also demonstrated how “local context” neurons (i.e., hippocampal cells that represent events in relation to the preceding and following events) could link discontiguous events to create a network of related episodic memories [2]. In addition, study also found the “time cells” in the hippocampus that mediated the bridging of temporal gap in memory for discontiguous events [5–7]. Conceptually, this “discontiguity” refers to the events that are to be associated with one another but do not temporally or spatially overlap [2]. However, the working memory (WM) system, especially that of humans, can actively maintain information “online” even when the stimulus is no longer being presented to the sensory system. For example, one can hold a piece of verbal information online by rehearsing it, meaning that the information that is not perceptually available may not necessarily be “discontiguous.” This raises the question of how to experimentally define this concept of “discontiguity.” Does it simply refer to a “gap” in the presentation of information (temporal discontiguity) or an “interruption” of the active maintenance of information stored in WM (functional discontiguity)?
To assess this, we asked participants to perform a semantic relatedness judgment task during neuroimaging scanning. Participants were required to judge whether two words were semantically related or not. The two words were presented either (1) at the same time (the AB trials) or (2) one after the other separated by a cross-viewing delay (the A_B trials) or (3) one after the other, between which participants performed a calculation task (the A#B trials) (Figure 1). AB trials served as a baseline, whereas the other two conditions contained the association of the two items that did not temporally overlap with one another. In the A_B trials, the first word could be actively maintained in WM by rehearsing it during the cross-viewing delay between the display of the two words. For this reason, the first word could also be maintained “online” at the moment of associative processing elicited by the presentation of the second word. However, in the A#B trials, participants were asked to engage in a distracting calculation task between the presentation of the two words, thereby interrupting the active maintenance of the first word. In this case, the first word could be “offline” at the moment of associative processing.
So far, the role of the hippocampus in bridging temporally discontiguous events has largely been investigated and proven in animal studies [5, 6, 8]. A few neuroimaging studies in human subjects have addressed the activation of the medial temporal lobe (MTL), including the hippocampus and parahippocampus, during the association of discontiguous events [9–12]. Our previous studies investigated how pairs of temporally discontiguous events (e.g., name-face pairs or object and face pairs) were encoded into long-term memory by MTL using a subsequent memory test paradigm [11, 12]. Additionally, there were also studies that examined the involvement of the MTL in the associative processing between the continuous or discontinuous memory retrieval. For example, in Sakai et al.’s [9] study, participants memorized a sequence of five consonant letters and were asked to perform a recognition judgment after a brief delay period with or without the interruption of an arithmetic calculation. However, their results only revealed MTL activation (but only in parahippocampus) during the retrieval of letter sequence in the presence of an interruption. This was not completely consistent with animal studies that found the function of the entorhinal cortex-hippocampal network in associating events separated by time or temporal gaps [7]. One possible reason for this lack of hippocampal activation was the recognition memory task in Sakai et al.’s [9] study which could be accomplished based on the perceptual familiarity of the stimulus and did not require hippocampal engagement [13]. In contrast, our previous study did observe left hippocampal activation in an associative processing task requiring participants to judge whether two words, presented simultaneously or one after the other separated by a brief delay, were semantically related or not [10]. However, in that study, there was no distraction task inserted in the delay period between the presentation of the two words. Therefore, we still do not know whether it was the “temporal discontiguity” or the “functional discontiguity” that was responsible for activating the hippocampus. This study’s design will enable us to precisely discriminate between these two factors.
It should be noted that with regard to the function of the MTL in discontiguity binding, two issues had been investigated. One relates to whether the MTL participates in the temporary or transient binding of two or more discontiguous events. The other relates to whether the MTL contributes to consolidating these (temporarily formed) discontiguity bindings into one’s long-term memory. The first issue could be investigated by assessing the involvement of MTL areas during the moment of making discontiguity bindings (relative to the nondiscontiguous ones) [10, 14, 15]. In contrast, the second issue could be investigated by assessing the involvement of the MTL areas in the moment of the discontiguity binding which could be successfully remembered in the later long-term memory task (relative to the forgotten ones) [11, 12, 16]. In the present study, we only focused on the first issue and did not assign the subsequent memory test to further check if the given “discontiguity binding” was successfully maintained in long-term memory or not.
2. Materials and Methods
2.1. Participants
Twelve healthy, right-handed volunteers (6 females and 6 males, aged 20 to 31 years old) recruited from the undergraduate and graduate students of University of Tsukuba participated in the experiment as paid volunteers. They were interviewed one or two days before beginning the fMRI experiment and had provided informed consent in accordance with the Advanced Industrial Science and Technology (AIST) MRI ethics committee.
2.2. Cognitive Tasks
This experiment consisted of two types of cognitive tasks. One consisted of a semantic-relatedness judgment task in which participants were asked to judge whether two words were semantically related. The other consisted of a calculation task in which triples of numbers were presented, with one on the top and two on the bottom. Subjects were asked to judge which one of the two bottom numbers (the bottom left or the bottom right number) was “closest” to the top number (Figure 1). For example, the number “3” on the bottom left is closer to the top number “4” than the number “8” on the bottom right (i.e., ). For both tasks, participants were asked to indicate their selection by pressing the left or right key on the button box using their right index or middle finger. The left key-pressing connoted “yes/related” in the semantic relatedness judgment or “bottom left” in the calculation task; right key-pressing connoted “no/unrelated” in the semantic relatedness judgment or “bottom right” in the calculation task. To minimize judgment bias and maintain participants’ arousal, in the semantic relatedness judgment, half of the word pairs in each condition contained were obviously semantically related, while half of them were not [10]. In addition, in the calculation task, there was an equal chance for the target number to appear in the bottom left as in the bottom right.
The presentation times of each stimulus are shown in Figure 1. For a trial in AB condition, the two words were presented at the same time and lasted for 2.5 sec. Participants should judge if these two words were semantically related or not during this period. Then, after a 2.3 sec cross-viewing delay, a calculation task was presented for 2 sec and followed by a 2.3 sec cross-viewing delay. For a trial in A_B condition, the first word was presented for 1 sec followed by a 6.6 sec cross-viewing delay. Then, the second word was presented for 2.5 sec and participants should make a semantic relatedness judgment during this moment. Then, after a 2.3 sec cross-viewing delay, a mental a calculation task was presented for 2 sec and followed by a 2.3 sec cross-viewing delay. For a trial in A#B condition, the first word was presented for 1 sec and followed by a 2.3 sec cross-viewing delay. Then, a mental a calculation task was presented for 2 sec and followed by a 2.3 sec cross-viewing delay. Finally, the second word was presented for 2.5 sec (participants should make a judgment) and followed by a 2.3 sec delay. In sum, in the A#B trials, the two words were presented one after the other separated by a 6.6 sec delay during which participants performed the calculation task. In the AB trials, however, the two words were presented at the same time. In the A_B trials, the two words were presented one after the other separated by a 6.6 sec unfilled, cross-viewing interval. Subjects were required to keep the first word in their mind and judge whether the first word was semantically related to the second word on seeing the second one. Calculation tasks were also assigned (but in separate ways from the session of semantic relatedness judgment task) in the AB and A_B trials to create a situation comparable to that in the A#B trials.
Ninety pairs of two-character Japanese Kanji words were used in a total of 30 trials for each of the three conditions. Word pairs were assigned counterbalanced across subjects. The 30 trials in each condition were assigned to four blocks with 7 or 8 trials in each block (in each block, the ratios of related versus unrelated word pairs were 4 : 4 in the 8-item block and 3 : 4 or 4 : 3 in the 7-item block). The whole session, therefore, consisted of a total of 12 blocks. These 12 blocks were presented in semirandom order with the constraint being that two same condition blocks (e.g., two blocks belonged to AB condition) were not to be presented continuously. These 12 task blocks were separated by three long resting (cross-viewing) blocks () and 8 short ones (). To familiarize subjects with the procedure and pace of the task, prior to the formal experiment, they were trained with another set of similar materials performed at the same pace.
2.3. Data Acquisition
All imaging was performed using a 3.0-T MRI scanner (GE 3T Signa) equipped with EPI capability. Eighteen axial slices (5.3 mm thick, interleaved) were prescribed to cover the whole brain. A T2 weighted gradient echo EPI was used. The imaging parameters were , , degrees, ( mesh). To reduce susceptibility noise artifacts (especially the EPI distortion) in the lower parts of the brain, including the anteromedial temporal lobes, we used a wider bandwidth (130 kHz) and moved participants’ chins such as to face down. To reduce head movement, participants were asked to put on a neck brace and were requested not to talk or move during scanning. Motion correction was also performed by a standard realignment process in SPM software (Wellcome Department of Cognitive Neurology, Institute of Neurology, London, UK).
2.4. Data Analysis
The imaging data were analyzed using SPM software. Each participant’s data was first preprocessed (timeslice adjusted, realigned, normalized, and smoothed) on an individual basis. Next, these preprocessed data were individually examined to establish first-level models (using the event-related analysis module) and then subjected to second-level, random effect models that incorporated the contrasts of all 12 participants. The aim of this study was to investigate transient brain activation in the moment of associative processing or semantic-relatedness judgment tasks. In the AB trials, the associative processing occurred when the two words were simultaneously presented, whereas in the A_B and A#B trials, the associative processing occurred when the second word was presented. Therefore, the association of discontiguous events was time-locked to the presentation of the second word in the A_B or the A#B trials (which we hereinafter call Event A_B or A#B). The association of contiguous events was time-locked to the presentation of the two words in the AB trials (named hereafter Event AB) (Figure 1). Besides these three major types of associative processing events, the presentation of the first word in the A_B and A#B trials, together with the calculation task, was also defined. In sum, we defined seven types of events in the estimation. Events 1-3 represented, respectively, associative processing in the AB trials (Event AB), A_B trials (Event A_B), and A#B trials (Event A#B). Events 4–6 represented, respectively, the calculation task in the AB trials (Event AB), A_B trials (Event A_B), and A#B trials (Event A#B). Event 7 represents the presentation of the first word in the A_B and A#B trials. All events were time-locked to the beginning of the stimulus presentation and modeled with a canonical hemodynamic response function (HRF) and as impulse (zero duration). The threshold for significant was set at (uncorrected for multiple comparisons), and the threshold for cluster size was set at 100 or more continuous voxels.
3. Results
Mean reaction times (RTs) in each type of event are given in Table 1. There was no significant difference between the RTs of Events A#B and A_B (, ). However, these were both significantly shorter than the RT of Event AB (Event A#B versus AB: , ; Event A_B versus AB: , ). The accuracy of semantic relatedness judgment is given in Table 2. In contrast with the old/new recognition task that has standard criteria, the semantic-relatedness judgment tasks were based on one’s subjective “feeling” of the word pairs. Therefore, any judgment was reasonable if it was made based on the individual’s careful consideration. The above-mentioned accuracy scores were just provided for reference. Subjects’ posttask reports demonstrated that they had neither any difficulty nor inability to retrieve the first word in A_B and A#B trials.
We consider the effects of the delayed presentation and disturbed delayed presentation in three pairs of comparisons, i.e., “Event A#B vs. AB,” “Event A#B vs. A_B,” and “Event A_B vs. AB,” therefore consisting of a total of six contrasts. Lists of activation shown in each comparison are given in Tables 3–5. Relative to Event AB, Event A#B was associated with activations in the left side middle and superior temporal gyrus, medial and middle frontal gyrus, posterior cingulate, fusiform gyrus, lentiform nucleus, insula, amygdala, claustrum, putamen, caudate, hippocampus, and parahippocampal gyrus, as well as activations in the right side superior temporal gyrus, fusiform gyrus, middle frontal gyrus, hippocampus, and parahippocampal gyrus. Relative to Event AB, Event A_B was associated with activation in the left side paracentral lobule, postcentral gyrus, and posterior cingulate gyrus. Relative to Event A_B, Event A#B was associated with activation in left middle and superior temporal gyrus, anterior cingulate gyrus, amygdala, hippocampus, and parahippocampal gyrus and activations in right middle, superior and inferior temporal gyrus, fusiform gyrus, insula, caudate, amygdala, and hippocampus. In this experiment, we only concentrated on activation of the hippocampus and other MTL areas. Two contrasts, “Event A#B minus AB” and “Event A#B minus A_B,” revealed significant hippocampal and parahippocampal activity. These two comparisons revealed activation of the amygdala and anterior hippocampus extending into posterior hippocampal and parahippocampal regions (Figure 2). The comparison of each of the three associative processing conditions (i.e., Events AB, A_B, and A#B) and their corresponding calculation tasks yielded consistent results (Figure 3).
4. Discussion
In this study, we observed that the hippocampus and parahippocampus were more activated in Event A#B than in Events A_B and AB. This MTL activation could not be attributed to the semantic associative judgment itself because this process is equally involved in all three conditions. In addition, this MTL activation could not be interpreted as the bridging of temporal discontiguity because Event A_B also contained a 6 sec delay period but was not associated with more MTL activation relative to Event AB. Instead, the hippocampal activation in Event A#B implied that the hippocampus was more involved in the bridging of the “functional discontiguity” produced by the interruption task between the two to-be-combined events. Specifically, we observed hippocampal activation in two contrasts: “Event A#B minus AB” and “Event A#B minus Event A_B.” In the former contrast, besides the difference in the simultaneous and delayed presentation of the two words, Event A#B also differed from Event AB in the number of words (one or two) that participants saw in the moment of associative processing. In the latter contrast, however, the difference in the number of words no longer existed. However, Event A#B was still found to be associated more with hippocampal activation, relative to Event A_B, thus excluding that possible confounding.
This result is consistent with early studies on hippocampal amnestic patients. These patients could hold normal conversations but could not remember facts across conversations [17–19]. Or they could remember a three-figure number but would forget it the instant their attention was diverted [20]. One such particular example was the patient H.M., who could remember a number for 15 minutes by constantly rehearsing it, forgetting that he had even been given a number to remember the moment the examiner switched subjects of talk [21]. Our results are also consistent with neuroimaging studies reporting hippocampal or MTL activity in WM tasks [9, 22–25] and in delayed-match-to-sample tasks [26, 27]. For example, Elliott and Dolan [23] reported that a long delay was associated with more hippocampal activity than a short delay in a delayed-match-to-sample task and that this result could be explained as the longer delay increasing the chances of functional discontiguity occurring. In Luo and Niki’s [10] experimental design that only included two conditions (AB and A_B trials), hippocampal activation was also observed in the comparison of “Event A_B minus AB.” However, in the present research that included three conditions (AB, A_B, and A#B trials), we failed to observe any significant hippocampal activation in the comparison of “Event A_B minus AB.” This implied the insertion of A#B trials across the experimental session might have altered the general processing strategy that participants took in accomplishing all the cognitive tasks including the A_B ones. They might have been more involved in the active maintenance of the first words during the delay period in the A_B condition, and this might eliminate the MTL difference in the associative processing of Event A_B relative to AB. Another difference between Luo and Niki’s [10] study and the present one was in that previous study the contrast of “Event A_B minus AB” found left hippocampal activation, whereas in the present study we observed bilateral hippocampus activations in the contrasts of “Event A#B minus AB” or “Event A#B minus A_B.” The left hippocampal activation in Luo and Niki’s [10] study, together with the activation of left middle temporal gyrus in that study, could be related to the more intensive processing of semantic retrieval and matching in Event A_B. In contrast, the involvement of bilateral hippocampus in Event A#B of the present study, especially the additional participation of the right hippocampus, might embody the extra processing demands on memory retrieval. In support of this possibility, the WM performance has been found to be more fragile to the damage in right lateral hippocampus relative to the left ones [28].
In the present study, we found that the bilateral hippocampus was more activated when the associative processing was carried out between the two items, which were separated by the interruptive calculation task. However, the exact cognitive components that caused this effect remain to be specified. Given that the only difference between Events A#B and A_B was the interruption of the online maintenance of the first words, we guessed it was the extra memory retrieval of the first word in Event A#B that evoked hippocampal activation. The nature of the memory retrieval involved in Event A#B, however, could be different from the ones in a typical WM task, although the delay period between the presentation of the first and second word was very brief (6 sec). First, in the typical WM retrieval, the solicited information is constantly kept online in the visual-spatial sketch pad and/or articulatory loop of the WM buffer. However, in Event A#B, the WM buffer that maintained the perceptual and semantic processing results of the first words was refreshed or updated by the interruptive calculation task [9]. Second, the memory retrieval of the first word was not triggered by a typical recognition memory or recall memory requirement but triggered by the presentation and processing of the second word. This task not only required the memory retrieval of the first word but also and even more dominantly required the extensive semantic retrieval and comparison of the first and second words.
These features inherent to our cognitive tasks made it difficult for us to further identify the neural correlates corresponding to the WM components involved in conditions A#B and A_B. This is due to the fact that both the memory retrieval of the first word and the perceptual encoding of the second word, as well as the semantic comparisons between the first and second words, cooccurred during the moment of associative processing. We have tried to identify common activation patterns involved in Events A#B and A_B by contrasting them with the AB condition but found few common activation patterns in Events A#B and A_B. This suggested that the possible common cognitive components of memory retrieval (of the first word) contained in Events A#B and A_B were rare. This could be due to the fact that individuals in the A_B condition might have tried to retrieve the first word during the delay period and at the presentation of the second word, rendering the memory retrieval component less significant. However, technically, we had no way to know the exact time at which participants started to rehearse the first word during the relay period. In addition, we did not know whether they did so in explicit or implicit ways. Therefore, it was difficult for us to further identify the neural signals that were specifically related to the process of memory retrieval during the maintenance period of the first word in conditions A#B and A_B. The only time window to check the WM components is the onset time of the second word in the A#B or A_B trials. However, at that time, the processes of semantic retrieval, selection, and comparison between the two words were also involved and mixed with the processes of memory retrieval, thus rendering us unable to ensure a reasonable disassociation.
Although computational models essentially pointed to the function of the hippocampus in discontiguity binding [2, 4], the present study found both hippocampal and parahippocampal activation. This is different from the previous study that only detected hippocampal activation [10]. A reason for the coactivation of the hippocampus and parahippocampus in functional discontiguity binding may be the extensive short-term memory and semantic memory retrieval involved in Event A#B. This conjecture was consistent with Wais et al.’s [29] study that suggested the processes of recollection and familiarity in memory retrieval that could be jointly supported by the hippocampus and parahippocampus.
While the present study used a calculation task in which the presented numbers were compared, could this comparison task leave enough room to still rehearse the first word in the A#B trials? In order to estimate whether the calculation task could prohibit the rehearsal of the first word in the A#B trials, we compared the calculation in A#B trials with that in A_B trials. Results showed that, relative to the calculation in A_B trials, the calculation in A#B trials was associated with activation in (a) left precentral gyrus (; ; BA4), (b) right cuneus (; ; BA7); (c) left thalamus (; ), and (d) medial frontal gyrus (; ; BA6). These activation patterns, especially in the medial frontal gyrus, revealed that an extra process of mental shifting was evoked in the calculation task of A_B trials. However, brain areas linked to rehearsal, including the dorsolateral prefrontal cortex, left superior temporal region, and Broca’s area [9], were not detected as being activated in the comparison. This implied there was no obvious rehearsal component inherent to the calculation task in A#B trials. In addition, in the comparison of “the calculation in A#B trials minus the calculation in A_B trials,” no activation difference in the MTL region was detected. This implied that the MTL activation associated with Event A#B was transient in the moment of associative processing and did not appear during the interruption interval.
This study sought to address how to experimentally define “discontiguity” associated with hippocampal activity. The study’s results showed that “discontiguity” could be more directly related to the “interruption” of the online maintenance of information held in WM than to the simple time or space “gap” (interval) between two items. Not only is this conclusion consistent with the general point of view that the function of the hippocampus is to recall a context or bridge a contextual gap, but it also provides an alternative explanation to the experimental observations that seem to be inconsistent with the discontiguity-binding hypothesis. For example, a previous study found that a discontiguous presentation of events and difficulty in performing the task inherent to the long delay interval between the conditioned stimulus and the unconditioned stimulus could impair the performance of animals with hippocampal lesions [30]. In addition, according to the implication of the present study, this phenomenon could be interpreted in light of the fact that the long interval allowed for a higher chance for functional discontiguity to occur.
This study had a few limitations. First, the sample size of the study was relatively small, although a random effect analysis was conducted and each condition contained enough trials. A larger sample size could be more reliable for drawing further conclusions. Second, as we said in the Introduction, although our study assessed the involvement of the hippocampus in discontiguity binding, no subsequent memory test was used to further examine if these bindings were successfully encoded in long-term memory or not. As such, we did not have any information on hippocampal involvement in the successful (versus unsuccessful) long-term memory storage of these discontiguity bindings. This lack of information may somewhat limit the deductive power of this study in its conclusions on the function of the hippocampus in binding the two discontiguous words and may lead to a “memory retrieval” interpretation. It implies that the memory retrieval of the first word, rather than the binding of the first and second words, may be what really contributed to the hippocampal activation in Event A#B. These components of memory retrieval, though they also occurred in the task context of associative processing, could somewhat deviate from the major concerns of the two-word binding processes. More objective measurement of the internal processing of discontiguity binding should also be considered and tested to clarify this issue.
5. Conclusions
In sum, in this study, we demonstrated that the hippocampus is more activated in the bridging of “functional discontiguity” than in the bridging of “temporal discontiguity.” This observation showed that the hippocampus could be mostly suited to associating items that are separated by distractors. This study lays forth additional information relevant to the theories that propose a hippocampal role in associating discontiguous items that occur in different temporal or spatial positions [2].
Data Availability
The datasets are available from the corresponding author Jing Luo ([email protected]).
Conflicts of Interest
The authors declare that this study was conducted in the absence of any commercial or financial relationships that could be construed as potential conflicts of interest.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (31871093 and 31671124), JSPS KAKENHI Grant (JP18H05318), Capacity Building for Sci-Tech Innovation-Fundamental Scientific Research Funds (047-19530050133 and 025-185305000), and the Beijing Brain Initiative of Beijing Municipal Science & Technology Commission.