Cognitive Neurorehabilitation in Acquired Neurological Brain InjuryView this Special Issue
Research Article | Open Access
Effects of Systematic Categorization Training on Cognitive Performance in Healthy Older Adults and in Adults with Traumatic Brain Injury
This study investigated the effects of hierarchical cognitive training using the categorization program (CP), designed initially for adults with cognitive deficits associated with traumatic brain injury (TBI). Fifty-eight participants were included: a group of fifteen young adults with TBI (ages 18-48), another group of fifteen noninjured young adults (ages 18-50), and two groups of adults over 60 randomly assigned into the experimental group () or the control group (). Following neuropsychological testing, the two young adult groups and the experimental older adult group received the CP training for 10-12 weeks. The CP training consisted of 8 levels targeting concept formation, object categorization, and decision-making abilities. Two CP tests (administered before and after the training) and three probe tasks (administered at specified intervals during the training) assessed skills relating to categorization. All treated groups showed significant improvement in their categorization performance, although younger participants (with or without TBI) demonstrated greater gains. Gains on the categorization measures were maintained by a subgroup of older adults up to four months posttraining. Implications of these findings in terms of adult cognitive learning and directions for future research on adult cognitive rehabilitation and cognitive stimulation programs are discussed.
Several studies demonstrated that older adults benefit from cognitive training efforts that target specific processes. These findings create a paradigm shift because they suggest that the aging brain continues to be somewhat plastic and adaptable in old age. Improvement associated with training could be attributed to the development of a new skill or the facilitation of existing cognitive mechanisms and capacities [1, 2] that may be affected due to the aging process. Ultimately, cognitive training in aging could contribute to the increase of cognitive reserve, the brain’s ability to withstand pathology .
Age-related cognitive changes are generally highly heterogeneous and are typically noted in tasks with high demands on speed of information processing and executive control (such as complex working memory tasks requiring manipulation of information), as compared to simple or automatic tasks (such as digit recall tasks) [3–6]. These changes in episodic memory can be measurable starting at age 30, and the rate of age-related decline in verbal episodic memory is normally mediated by working memory capacity . The goal of cognitive training is to reduce the impact of the aging process on the cognitive system,  and in order to be successful, training programs should be based on sound theoretical models of cognition.
Contemporary cognitive theory organizes human cognition into a hierarchy of basic and complex processes or systems. Basic processes such as sensory perception, attention, and memory underlie more complex systems such as language, categorization, and executive functioning (for an extensive review, refer to Constantinidou and Thomas ). Deficits in categorization could interfere with the successful execution of daily activities because categorization skills are integral to memory and learning of new information and are essential processes for decision-making and successful problem-solving. Given the fundamental importance of categorization to all of intelligent behavior, it is surprising to observe the scarcity of investigation specific to the effects of aging on classification behavior. This is in contrast to other domains such as speed of processing and memory for which a substantial body of work can be found [10, 11]. The current study proposes to fill some of this gap in knowledge and is part of a systematic research program on the effects of categorization training on cognitive performance. Our previous research suggests that the CP is effective in young adults with TBI [12, 13]. The next paragraphs provide the theoretical framework for categorization-classification behavior relevant to this study.
The literature on categorization, especially that which is focused on visual object classification, divides into two largely nonoverlapping areas: those studies concerned with how we go about recognizing and categorizing ordinary objects in the world [14, 15] and those studies whose purpose is primarily to explain how novel situations or categories are acquired and later used to classify novel instances [16–18].
Regarding object perception, recognition, and categorization, semantic knowledge models support a bilateral temporal lobe hub which links object properties and conceptual understanding . Furthermore, evidence in vision indicates the presence of a hierarchical recognition process that begins with early feature processing (such as orientation, motion, and color) and leads to the processing and representation of objects and object classes in the inferotemporal cortex [15, 20, 21]. Existing research suggests that both healthy aging and TBI result in changes in object classification [22, 23], indicating that treatment of object classification behavior should begin with feature identification in order to retrain the passive object recognition system .
For new category learning and classification of novel instance, empirical evidence suggests that people recruit one of two explicit systems (i.e., their processes and outputs are consciously available to the individual). The rule-based or rule-governed category system is the most important of the two. It involves the use of explicit verbalizable rules and hypothesis testing to determine category membership. This system relies heavily on executive functioning for its operation and engages frontal-subcortical networks. The other explicit system that requires significant episodic memory in that categorization is accomplished by the recall of previously experienced category members, or exemplars, that are similar to the present novel object [9, 17]. This latter system works by evaluating similarity, in terms of object or situation features, of to-be-categorized instances to those encountered previously , and its operation is sometimes labeled as procedural or nonexplicit, in the sense that one cannot easily describe the strategy used. Our knowledge of natural categories and common objects is likely to have been acquired through the use of the similarity-based system [20, 21, 25]. A long list of neuropsychological and experimental studies supports the above distinction [26–30]. The categorization program implemented in this study is organized hierarchically in order to address these two distinct areas of classification.
While the process of healthy aging is different from the pathology of TBI, healthy aging results in changes in categorization abilities, similar to TBI [22, 23]. Even though higher levels of cognitive reserve result in improved neurocognitive performance and moderate the effects of aging, age continues to be a robust predictor of neurocognitive functioning [3, 31].
Both TBI and age have been established as significant risk factors for the development of pathological aging resulting in dementia. TBI is now viewed as a chronic condition, and there is accumulating evidence suggesting that TBI and dementia of the Alzheimer’s type share a common neuropathologic sequelae, such as chronic neuroinflammation (see review by Breunig et al. ), tauopathy , and the accumulation of beta amyloid (Aβ) , leading to significant neurocognitive impairment. Moderate-severe TBI sets off a neurodegenerative cascade as manifested by significant reductions in brain volume and lingering neurocognitive deficits  associated with longer time since injury placing the survivor at risk for dementia in middle/later life .
The current project responds to the challenge set by the NIH consensus report on preventing AD and cognitive decline . The primary objective of this project was to investigate the utility of intense neurocognitive training in healthy older adults who experience neurocognitive changes associated with the normal aging process, using the categorization program (CP). A secondary objective was to compare the performance of older adults to younger healthy adults and to adults with cognitive deficits secondary to brain injury. Results from this study could guide future research investigating the effects of intense neurocognitive training on older adults with TBI and adults with mild cognitive impairment, the prodromal phase to dementia.
The CP is a rigorous systematic, hierarchical, eight-level program initially designed as a restorative cognitive rehabilitation program in adults with acquired brain injury. It addresses the aforementioned two distinct areas of human categorization, i.e., passive object recognition and new category learning. Initial research findings  and a subsequent randomized controlled trial  indicate that the CP is an effective therapy tool for adults with brain injury who exhibit categorization deficits. Constantinidou et al.  offered the following explanations regarding the active ingredients of the CP: (i)The CP addresses both aspects of categorization, novel category learning, and categorization of established concepts or categories. It incorporates concrete visual stimuli and gradually progresses into abstract concepts through the use of repetition, cueing, and strategy building(ii)The CP was built using a very systematic hierarchical structure that corresponds to the neurodevelopmental order of categorization and classification process hierarchy(iii)Tasks gradually increase in difficulty and cognitive abstraction. CP tasks begin with basic feature identification and feature extraction (such as color, shape, and size) and progress to higher levels of concept formation and abstraction (such as rule-based decision-making) [38, 39](iv)The program integrates cognitive processes such as executive skills, attention, organization, conceptual reasoning, linguistic flexibility, and explicit memory for the completion of the categorization tasks(v)The redundancy and the repetition integrated in each level, along with the extensive cueing systems and errorless learning principles, provide support and organization for participants with more passive learning styles. The program provides a standardized approach to categorization training; yet, it incorporates mastery criteria for each level in order to account for individual differences. In conclusion, the CP targets cognitive domains such as complex working memory, information processing, and fluid intelligence that traditionally have been associated with age-related cognitive decline
The present study investigated categorization abilities and the effects of training on healthy older adults over 60 and in a group of young adults with moderate to severe TBI. In order to account for previous methodological flaws in adult training studies and determine the true effects of training, this study incorporated an older adult control group (who did not receive the training) and a young adult group who received training. The primary hypotheses were the following: (1)All participants who receive the CP training would demonstrate improvements in their categorization performance as measured by gains on the CP-dependent measures(2)Participants who receive the CP training would be able to generalize their knowledge to other categorization tasks not directly related to the CP, to a greater degree than participants who do not receive the CP training(3)Performance of older adults on formal neuropsychological measures measuring memory, executive function, perception, conceptual reasoning, and attention would correlate significantly with performance on categorization tests(4)Older adults who receive the CP training would be able to maintain their gains on categorization performance at one and four months posttraining
2. Materials and Methods
Fifty-eight participants were included in the study. One group consisted of young adults with TBI who received the CP training (), a second group of young healthy adults who also received the CP training (), a third group of healthy older adults who received the training (), and a fourth group of healthy older adults who did not receive the training (). Participants with TBI were recruited from brain injury rehabilitation centers collaborating in the project. The rest of the participants (groups 2-4) were volunteers from southwest Ohio areas. Recruitment of the TBI and noninjured groups was done in parallel and in a rolling admission process. All work was conducted in accordance with the Declaration of Helsinki (1964), and the project was approved by the Miami University Institutional Review Board for Human Subjects in Research. All study participants provided written informed consent prior to participation. Study participants were English-speaking adults and met the study’s inclusion/exclusion criteria as follows.
2.2. Young Adults with TBI
The following are the inclusion/exclusion criteria for participants with TBI, which are consistent with the Constantinidou et al.  criteria.
2.2.1. Inclusion Criteria
(1)Adult males and females between 18 and 55 years of age(2)Primary diagnosis of moderate to severe CHI. The indication of an initial moderate to severe head injury was determined by the presence of three or more of the following severity indices: (a) initial Glasgow Coma Scale score less than 12, (b) abnormal initial computed tomography (CT) findings indicating acute central nervous system pathology, (c) length of impaired consciousness greater than 20 minutes as specified by the emergency records, (d) length of posttraumatic amnesia greater than 24 hours as specified in the acute hospital/emergency records, (e) length of acute hospital stay greater than 3 days, (f) positive neurological examination on hospital admission and discharge indicating focal sensory and motor neurological deficits or changes in the mental status attributed to brain injury, (g) medical complications secondary to the injury, and (h) head injury severity classification according to hospital records [40, 41](3)Rancho Los Amigos Scale  Level VI or higher (which indicates appropriate, goal-oriented behavior and posttraumatic amnesia (PTA) resolution)(4)No aphasia present with the exception of mild to moderate word-finding problems due to cognitive deficits(5)Resolution of PTA as evidenced by a score of 76 or higher on the Galveston Orientation and Amnesia Test (6)Enrollment in a residential comprehensive postacute rehabilitation program at the onset of the study(7)Participants were within 4 years of their injury
2.2.2. Exclusion Criteria
(1)Penetrating head injuries(2)Diagnosis of stroke at the time of injury(3)Premorbid central nervous system disorder or learning disability(4)Documentation of premorbid major depression or other significant psychiatric disorders as defined by the Diagnostic and Statistical Manual of Mental Disorders [DSM-IV]  that resulted in hospitalization and/or incapacity to work or perform activities of daily living(5)Current Beck Depression II  score of 25 or higher indicating the presence of depression that could interfere with performance on the protocol. This score is higher than the typical cut-off score of 15 because as previously described , TBI results increased symptomatology for reasons relevant to TBI and not because of clinical depression. A score of 25 or higher indicates clinical depression in this population(6)Active or current alcohol, drug, or other controlled substance abuse that interferes with participation(7)Deficits in auditory comprehension and moderate to severe word-finding problems, two standard deviations below the mean on the Boston Naming Test , which could interfere with the subject’s ability to follow test or task instructions(8)English as a second language(9)Color blindness as measured by the Ishihara test for color blindness 
Seventy percent of the participants were injured in motor vehicle accidents, and another 30% were injured as a result of falls. All participants received comprehensive rehabilitation at the time of participation in the project.
2.3. Noninjured Adult Groups
2.3.1. Inclusion Criteria
(1)Adult males and females between 18 and 55 years of age for the young adult group(2)Adults over 60 for the older adult groups
2.3.2. Exclusion Criteria
(1)A medical history of a central nervous system trauma, disorder, or organic brain disease, learning disability, or language learning disability(2)Documentation of psychological or psychiatric disorder as defined by the DSM-IV  that resulted in hospitalization for major depression and/or incapacity to work(3)Current Beck Depression II  score of 15 or higher indicating the presence of depression that could interfere with performance on the protocol(4)Mini-Mental State Examination score of 25 or lower (5)Active or current alcohol, drug, or other controlled substance abuse(6)Uncorrected visual or hearing deficits(7)Color blindness as measured by the Ishihara test for color blindness 
2.4. Group 1: Experimental Young Adult Group with TBI ()
Participants were residents of postacute rehabilitation centers and were enrolled in the project through a rolling admission process. They remained in the study for an average of 10-12 weeks which coincided with their length of stay at the rehabilitation centers. Participants in this group ranged in age from 18 to 48 years with a mean age of 28.13 (). Education ranged from 12 to 17 years, with a mean of 13.67 ().
2.5. Group 2: Young Adult Group ()
The ages of the participants ranged from 19 to 50 years with a mean age of 29.73 years (). Education ranged from 12 to 18 years, with a mean of 14.16 ().
2.6. Group 3: Experimental Older Adult Group ()
Participants ranged in age from 60 to 82 years with a mean age of 67.28 years (). Education ranged from 12 to 19 years, with a mean of 13.9 (). Their average MMSE score was 28.5 ().
2.7. Group 4: Older Adult Control Group ()
Their ages ranged from 60 to 88 years with a mean age of 68.64 years (). Education ranged from 10 to 18 years, with a mean of 13.92 (). Their average MMSE score was 29.42 ().
Healthy older adults who met the inclusion/exclusion criteria for the project were randomly assigned to either the experimental or the control group. The two older adult groups did not differ significantly in age or education level, , and , . There was no difference in their gross cognitive ability as measured by the MMSE , . Finally, there was no significant difference between the four groups on the years of education, , .
Participants were administered a neuropsychological assessment at the beginning of their participation in the project. Following the neuropsychological testing, participants were administered two categorization tests designed for this project and the first probe task (see sections 2.11 and 2.12 for a description of the tests and probe tasks). All participants remained in the study for 10-12 weeks. Participants who received the CP training (groups 1, 2, and 3) participated in individualized hourly sessions for a total of 2-4 hours of training per week until they completed the CP protocol. On average, participants required 27 hours to complete the CP training. Table 1 displays the experimental design.
Note: pre/posttests include the neuropsychological assessment and the 2 tests of the categorization program. Probe 2 was inserted at about 5 weeks after the onset of treatment and probe 3 at 8 weeks for the control group.
2.9. Experimental Items
2.10. The Categorization Program (CP)
The CP was based on theories of implicit and explicit categorization systems. Therefore, tasks were grouped into two major parts: (1) recognition and categorization of everyday objects and (2) new category learning . Principles of learning, concept formation, and rehabilitation were incorporated in order to develop the hierarchical tasks [49–52]. Table 2 presents the 8 levels of the CP.
2.10.1. Part A: Object Categorization Tasks
This part consists of 5 different levels. The tasks begin with teaching perceptual features in order to describe objects or living things and move to higher levels of cognition including analyses, synthesis, linguistic flexibility, and abstract reasoning.
2.10.2. Part B: New Category Learning Tasks
The new category learning tasks consist of three levels. Under each level, there are 5 steps that increasingly demand a higher level of rule-governed responses. Errorless learning principles and cueing hierarchies are applied under each step.
The CP-dependent measures were developed to measure the effectiveness of the CP program. These were the CP Test 1, CP Test 2, and probe tasks 1, 2, and 3.
2.11. CP Tests 1 and 2
CP Test 1 relates to the categorization of common objects (Part A of the CP). Participants were required to describe pictures of objects and identify core attributes such as their primary function and alternate uses of the object. These objects were not part of the CP training. There were a total of 10 objects; five have high frequency and five have low frequency in occurrence . The number correctly obtained from 120 possible was recorded for each subject. Cronbach’s alpha for the 10 items is .90.
The second test relates to the new category learning portion (Part B of the CP). Participants were required to follow a logical rule in categorizing objects. These objects were not part of the CP. There were a total of 5 rules with increased complexity. For instance, the first rule asks the subject to “put all red items in the basket.” The last rule is more complex and requires that participants to “put all things that are blue but not used for coffee in the basket.” Both informal tests were administered at the beginning and at the end of the study. The number correctly obtained from 36 possible was recorded for each subject. Cronbach’s alpha for CP Test 2 was .69.
2.12. CP Probe Tasks
The probe tasks were designed to assess how participants generalize information learned on the CP to other tasks not directly related to the CP training tasks. Participants were presented with an array of 10 objects and were required to categorize objects based on a self-generated rule. Following that, they were asked to categorize the same objects twice, each time using a different self-generated rule.
The probe tasks were administered at 3 different times during the study. The first probe task was administered prior to the CP training, the second after Level 2 (Part A), and the third after Level 5 (Part A). For the treated control group participants, the first probe was administered prior to the onset of therapy, the second after 5 weeks of treatment, and the third after 8 weeks of treatment. Each of the probe task uses a set of 10 different objects, equal in familiarity and frequency of occurrence. The three different sets of probe tasks are of similar difficulty, and analyses of variance yielded no statistical difference in performance () between the three different sets of items. The order of administration was counterbalanced to avoid order effects. Participants obtained one point for each object they sorted correctly by their self-generated rule. The subject was asked to sort the items three times under each probe task, each time using a different rule. Hence, each probe trial was worth 10 points with a total of 30 points for each probe task.
2.13. CP Stimuli and Scoring System
The CP protocol was provided in a manualized format in order to ensure consistency. The CP packet included the administration manual with cueing instructions, the CP stimuli (objects, photos, written words, etc.), and score sheets.
2.14. Neuropsychological Tests
The following tests were administered at pre- and posttesting. (i)Wechsler Abbreviated Scale of Intelligence (WASI) (ii)Mini-Mental State Examination (iii)Rey Complex Figure Test (iv)Trail Making Tests A and B (v)Digit Span Forward and Backwards and Visual Span Forward and Backwards—Wechsler Memory Scale-III (WMS-III) (vi)California Verbal Learning Test-II (vii)Wisconsin Card Sorting Test (viii)The Booklet Category Test (ix)Symbol Digit Modalities Test (x)Control Oral Word Association (xi)The Picture Recognition, Spatial Relations, Analysis and Synthesis, Concept Formation, Decision Speed, and Verbal Comprehension Subtests from the Woodcock-Johnson III (WJIII, Tests of Cognitive Abilities) 
2.15. Data Scoring and Analyses
Data were included in the analyses to the fullest extent possible. The primary statistical design was a multivariate mixed model design followed by preplanned univariate comparisons. The alpha level was set at .05.
Participants who received the CP protocol required about 27 hours of treatment spread over 10-12 weeks. The participants in the control group did not receive any cognitive treatment other than information regarding memory strategies and a list of suggested activities for cognitive stimulation.
3.1. Performance on CP Measures
CP measures consisted of the CP Test 1 (pre and post), CP Test 2 (pre and post), and the three probe tasks (Probe 1, Probe 2, and Probe 3).
3.1.1. CP Test 1
The first CP test assessed the ability to categorize common objects. In order to determine the effects of the CP training on categorizing familiar objects, a mixed model analyses of variance () compared the pre- and postperformance of the four groups on the CP Test 1 with time (pre/post) as the within-subjects factor and group as the between-subjects factor. The analyses revealed a significant time effect (, , , ), group by time interaction (, , , ), and an overall group effect (, , , ).
Planned lower-order ANOVAs indicated that there was an overall group difference on the CP Test 1 baseline performance (, ). However, there was no difference at the baseline among the three healthy groups on CP Test 1 accuracy (, ). Post hoc pairwise Bonferroni comparisons indicated that the baseline difference at CP Test 1 was due to the lower performance of the young group with TBI. Participants with TBI performed significantly lower on CP Test 1 at the baseline as compared to the healthy young adults and the two older adult groups.
At Time 2, there was a significant group main effect (, ). Post hoc pairwise Bonferroni comparisons revealed no difference between the TBI and the healthy young adult or the TBI and the older experimental group at Time 2. However, healthy young adults performed significantly better than their older counterparts who received the CP training. The older untreated group had the lowest performance among all groups at Time 2, and their performance was significantly lower than that of participants in the other groups (e.g., TBI, young adult, and older experimental groups). While all treated groups demonstrated significant growth from Time 1 to Time 2, the young adults (TBI and healthy young adults) demonstrated the most significant growth as compared to the older treated group, , and , . The degree of improvement on CP Test 1 (i.e., difference score between pre-post performance) between the two young adult groups was similar (, ). Figure 1 displays time by group interaction depicting the change in performance from Time 1 to Time 2 on the CP Test 1 tasks. Table 3 presents the performance of each group on the CP-dependent measures.
Note: the maximum possible score on CP Test 1 is 120, and on CP Test 2 is 36. The total number of possible points for each probe task is 30.
3.1.2. CP Test 2
The second CP test assessed the ability to implement logical rules to categorize objects consistent with theories of category learning. In order to determine the effects of the CP training on implementing logical rules, a mixed model analyses of variance () compared the pre- and postperformance of the four groups on the CP Test 2 with time (pre/post) as the within-subjects factor and group as the between-subjects factor. There was a time main effect (, , , ) and a group main effect (, , , ). The group by time interaction was not significant (, , , ). Results indicate that the patterns of performance were similar among the groups. Planned lower-order ANOVAs indicated that there was a difference at the baseline between the four groups (, ). The only significant difference was between the noninjured young adults and the two older adult groups (, ). There were no other significant group differences at the baseline.
At Time 2, there was a significant group simple main effect (, ). A priori pairwise comparisons () revealed significant differences between the young TBI group and the treated older adult group (, ) and between the healthy young adult group and the treated older adult group and untreated older adult groups (, and , , respectively). Figure 2 displays group performance on the CP Test 2.
3.2. Probe Tasks
The probe tasks were designed to assess the participants’ ability to implement skills learned during the CP training and categorize new objects. The three probe tasks were administered at three different intervals: before the CP training, after Level 2 (at 5 weeks after the onset of treatment), and after Level 5 (at 8 weeks after the onset of treatment). Participants in the control group received the probes before the onset of the study (i.e., baseline), at 5 weeks, and at 8 weeks.
A mixed model analysis of variance was performed with probe as the within-subjects factor and groups as the between-subjects factor. There was a significant probe effect (, , , ), group by probe interaction (, , , ), and a significant group main effect (, , , ).
Pairwise analyses (mean difference estimations, ) indicate that there was a significant improvement between the first and second probe administrations (). There was also a significant difference between the first and third probes (). However, there was no significant difference between the second and third probes ().
The groups demonstrated different patterns on performance on the probe tasks. The greatest gain in performance was obtained by the young adult group with TBI (, , , ). The performance of young adults with TBI and older adults during Probe 1 was more variable (compared to younger adults). However, their performance during Probes 2 and 3 became more uniformed. The performance of the young adult group was fairly stable across the three probes, probably due to a ceiling effect. Figure 3 displays group performance on the probe task across time.
3.3. Follow-Up Effects of CP Training
Out of the 14 older participants who received the CP training, 9 participants were able to complete follow-up assessments at one month and at 4 months after study completion in order to determine possible long-term effects of the CP training. The remaining five dropped out because they missed one of the two follow-up sessions due to illness, planned surgeries, death in the family, and travel. A repeated measures ANOVA () on the CP Test 1 scores obtained during the four different administration times (pretest, posttest, follow-up Time 1, and follow-up Time 2) revealed significant differences in performance (, , , ). Preplanned Helmert contrasts indicated a significant difference in performance between the pretest score (, ) and the rest of the posttest scores (, , , ) (, ; , ; and , ). There was no significant difference between the three posttest scores on CP 1 (, , , ).
Similar to the CP Test 1, a repeated measures ANOVA () on the CP Test 2 scores obtained over four different administration times (pretest, posttest, follow-up Time 1, and follow-up Time 2) revealed a significant overall effect (, , , ). There was no significant difference between the three posttest scores on CP Test 2 (, , , ) (, ; , ; and , ). In summary, the small group of participants who received the follow-up testing maintained their gains on CP Tests 1 and 2 after the training up to 4 months after the study completion.
3.4. Pretest Neuropsychological Measures and Performance on the CP Tests
One objective of the present study was to determine the relationship between neuropsychological performance and the CP-dependent measures for the older adults. Scores from the full set of baseline neuropsychological assessments were combined into a set of five composite scores representing the conceptually motivated constructs [13, 65] of Memory Processing, Executive Functioning, Perceptual/Visual Processing, Conceptual Reasoning, and Organization/Attention using a method advocated by Cahn and colleagues  and Cohen and colleagues . Each measure was converted into a -score. The resulting -scores for the measures within each construct were then averaged to derive a score for the constructed measure. The two Executive Function measures (Wisconsin Card Sorting Test: Trials to First Category, Total Number of Categories, and Failure to Maintain Set), which were not significantly correlated with the Executive Function composite variable, were removed from the composite, and the Executive Function composite was recomputed from the remaining measures in that set. Table 4 presents the group means on the pre- and postneuropsychological measures. Values under means are standard deviations; Table 5 displays the correlations between individual measures and their composite scores.
Note: ; .
The correlation between the pretest scores of CP Test 1 and CP Test 2 was weak and nonsignificant (, ) because the two dependent measures assessed different categorization constructs. The left columns of Table 6 contain the Pearson product-moment correlations between the composite scores and the categorization pretest measures (CP Test 1, CP Test 2, and Probe 1) for both groups of older adults. At Time 1 (pretest), CP Test 1 performance correlated significantly with all five composite indices; CP Test 2 correlated significantly with all indices except the Organization/Attention Composite. Probe 1 correlated with all the composite scores except the Perception/Visual Processing composite.
Note: Executive Functioning composite does not include the following Wisconsin Card Sort Measures: Total Number of Categories, Trials to 1st Category, and Failure to Maintain Set. Posttests and pre-post difference scores include the experimental older participants only. ; . + indicates partial correlations holding the corresponding pretest performance as a covariate.
Finally, in order to examine whether neuropsychological measures could predict who benefited most from the categorization program, the -score composite measures were correlated with the three difference scores (CP Test 1 posttest minus pretest, CP Test 2 posttest minus pretest, and Probe 3 minus Probe 1) using the data from the older adults who received the CP training. Additionally, partial correlations (using the pretest score as a covariate) between the composite scores and the categorization posttest measures (CP Test 1, CP Test 2, Probe 2, and Probe 3) were conducted. As can be seen in Table 6, the Perceptual/Visual, Conceptual/Reasoning, and Global Cognitive were the most useful composites in predicting improvement in performance on the CP-dependent measures.
The current study is part of a systematic research program exploring the effects of a hierarchical cognitive training program in adult rehabilitation. The primary objective of the present study was to determine the utility of such training in healthy older adults and compare their performance to healthy young adults and to young adults with known neurocognitive deficits resulting from TBI. The results support the notion that older adults similar to adults with TBI can benefit from cognitive activities that enhance organization and conceptual knowledge. Overall, the three groups of participants who received the CP training demonstrated improvement in their categorization performance. Categorization in the project was measured directly by the two CP tests developed specifically for this training program. Participants who received the CP training showed improvement in their ability to categorize common objects as measured by their performance on CP Test 1. During CP Test 1, participants were required to describe objects effectively and to generate creative uses that could improve functional problem-solving abilities. Participants in the experimental groups demonstrated significant gains in describing common objects and in creative uses of objects. In comparison, participants in the control group did not demonstrate significant gains in this area. Greater gains however were noted in the young adults with TBI and young healthy adults, in comparison to the older experiment adult group.
CP Test 2 measures the ability to categorize based on predetermined rules. During this test, participants were provided the rule and asked to classify objects based on the rule. This type of task is considered passive in nature because participants were not required to delineate the rule themselves. In contrast, during the actual training tasks of the CP, participants were required to delineate the rule based on feedback they received from the clinician. While young participants (both with TBI and noninjured young adults) demonstrated significant improvement on the CP Test 2, older adults who received the training did not demonstrate significant gains. Therefore, it appears that older adults demonstrate greater gains in the classification of objects as compared to rule-based learning.
Older adults were followed up to four months upon completion of their training. This was not possible for the TBI group since completion of the CP training also coincided with discharge from the rehabilitation facility. Findings indicated that older adults who received the training were able to maintain their gains at one and four months posttraining. Future studies need to explore the posttraining effects systematically. This information can guide future research on the CP and on potential benefits of “tune-up” sessions at specified intervals.
One of the primary challenges faced by cognitive rehabilitation programs centers on the realization that knowledge acquired during training may not transfer into other novel (untrained) tasks. Transfer success may be influenced by a number of factors stemming from the nature of the tasks, types of stimuli, and cognitive distance between the trained task and the untrained task as well as unfamiliarity with the untrained task itself. This challenge in transferring skills from domain-specific to more general can limit the practical impact of cognitive training programs as well as the motivation of participants to continue their training. An additional variable that contributes to the aforementioned challenge might be the advanced age of some older participants (i.e., over 75), which in turn could create difficulties in learning, speed of information processing, mental flexibility, and reduction in strategy use .
Hence, another important objective of the present study was to determine whether the CP training facilitates the ability to generalize categorization skills to novel tasks as measured by the probes. The current results indicated that our older adult group, similarly to adults with TBI, demonstrated gains in the probe performance. However, the gains of the young TBI group were greater than that of the older adult group. This might be due to the fact that the injured TBI group had a greater room for improvement on this task. Future studies may want to increase the complexity of the probe tasks in order to ensure that they provide opportunity for improvement in performance as a result of training.
4.1. Relationship between the CP Measures and Formal Neuropsychological Tests
Given the focus of CP Test 1 on object recognition and categorization, it is understandable that the skills this test measures overlap significantly with those assessed by the neuropsychological tests comprising the Perception Composite score and the Conceptual/Reasoning composite. On the other hand, rule-governed categorization has been consistently related to executive functioning, attention, and reasoning processes using a variety of methodologies [27, 29]. The present findings are largely consistent with this view as can be seen by the fact that composite measures having to do with executive function, attention, and reasoning correlate with performance on CP Test 2 during the pretest and posttest administrations. Probe task improvement is largely related to conceptual/reasoning and perceptual/visual measures, although this task may be affected by a ceiling effect. Finally, the Memory Composite and the Global Cognitive composite scores relate to all CP-dependent measures. The present findings with older adults are consistent with our previous work with the CP and TBI .
Future research should continue to explore the interaction between task properties and learner capacity. Evidence suggests  that individuals with high working memory capacities (as measured, for instance, by the digit span task) will learn rule-based problems faster than they can learn problems requiring a nonexplicit, similarity reasoning strategy. Alternatively, individuals with lower working memory capacity learn the nonrule-governed category structures faster than they learn those defined by rules. In the present study, baseline performance on the CP-dependent measures was highly correlated with the memory composite scores. This suggests that it may be profitable, in the designing of neurocognitive programs like the CP, to understand the relationship between individual neuropsychological characteristics and performance in categorization retraining tasks. The present study implemented composite scores in order to create theoretically meaningful constructs and also as a strategy to reduce the number of comparisons. Due to the preliminary nature of the findings and the small sample size, further corrections for multiple comparisons were not implemented. This limitation should be taken into consideration when interpreting the above findings.
4.2. Conclusions, Clinical Implications, Limitations, and Future Research
The present results contribute to the growing body of literature supporting cognitive training in healthy older adults. Improvements noted by cognitively healthy older adults and by younger participants with TBI, are consistent with our previous work implementing the CP with younger adults who sustained neuropsychological deficits secondary to closed head injury [12, 13]. However, gains demonstrated by older adults are not as dramatic as those demonstrated by their younger counterparts. Future research needs to determine the utility of the CP in treating older patients with TBI during rehabilitation and in the chronic phase post injury. This is in light of recent evidence on the neurodegenerative effects of chronic TBI on the brain structure and function [35, 69].
The Task Force on Promotion and Dissemination of Psychological Procedures and the World Health Organization define efficacy research as the examination of an intervention’s effect under highly controlled experimental conditions [50, 70]. The current results suggest that the CP training is efficacious in enhancing certain aspects of categorization performance. The CP training implemented a standardized, manualized protocol and followed the necessary procedures in order to adhere to the standards and rigor of experimental research. However, given the small sample sizes in the present study, the results should be interpreted with caution and could not be generalized to the larger population of TBI and older adults. Future research should incorporate larger sample sizes to reduplicate these findings and also incorporate quality of life outcome measures in order to determine the potential generalizability of cognitive training in other aspects of daily activities.
One of the strengths of the current project was the careful selection of participants in order to create a homogeneous sample of subjects, improve internal validity, and reduce variability in performance. In addition to determining the utility of the CP training in older adults with TBI, future research should include adults with mild cognitive decline in order to determine the potential utility of this type of training in adults who may be at risk for developing dementia.
Present findings with a small subgroup of older adults demonstrate that the positive effects of CP training last for up to 4 months post treatment. This finding can guide future research on determining the timing and potential benefit of periodic booster sessions. Additionally, the link between CP training and improving cognitive reserve in older adults may be a fruitful line of research. Furthermore, future studies exploring the long-term effects of treatment should incorporate older adults who do not receive the training in addition to both younger and older adults with TBI.
Biological aging does not seem to be a uniformed process; therefore, larger studies (with neurologically healthy and neurologically compromised groups of participants) will afford the necessary statistical power to identify subgroups of older adults and delineate who benefits the most from this type of training. The identification of individuals with specific neuropsychological and genetic (i.e., ApoE allele) profiles, determination of mitigating factors, and the development of predictive models indicating who would demonstrate the greatest gains would be an important contribution of larger scale studies.
In closing, the present results provide additional evidence supporting the continued investigation of the CP training in adult neurorehabilitation. Future larger scale clinical trials would provide stronger evidence for the CP utility. The long-term effects of CP training on categorization performance and the use of CP training in older participants who are at risk for dementia and in older adults with TBI would be a fruitful line of investigation.
The data used to support the findings of this study are available from the corresponding author upon request. Availability is dependent upon compliance with personal data protection laws and ethics board laws and regulations.
Conflicts of Interest
There are no conflicts of interest to report, financial or otherwise.
Special thanks are due to the study volunteers, to the clinical site collaborators at the Center for Neuro Skills (CNS), Bakersfield, CA; Center for Neuro Skills, Dallas, Texas; and Center for Comprehensive Services (CCS, NeuroRestorative), Carbondale, IL, and to the members of the NeuroCognitive Disorders Laboratory at Miami University for their contributions, especially Abigail Migut, Lacy Robinson, and Jessica Schneider. This research was supported in part by a grant from the Committee on Faculty Research at Miami University, Oxford, Ohio, where Constantinidou was Professor of Speech Pathology and Audiology. The statistical analyses, preparation, writing, and editing of this manuscript was supported by the author’s current employer, the University of Cyprus, Nicosia, Cyprus.
- G. W. Rebok, K. Ball, L. T. Guey et al., “Ten-year effects of the advanced cognitive training for independent and vital elderly cognitive training trial on cognition and everyday functioning in older adults,” Journal of the American Geriatrics Society, vol. 62, no. 1, pp. 16–24, 2014.
- S.-C. Li, F. Schmiedek, O. Huxhold, C. Röcke, J. Smith, and U. Lindenberger, “Working memory plasticity in old age: Practice gain, transfer, and maintenance,” Psychology and Aging, vol. 23, no. 4, pp. 731–742, 2008.
- E. Giogkaraki, M. P. Michaelides, and F. Constantinidou, “The role of cognitive reserve in cognitive aging: results from the neurocognitive study on aging,” Journal of Clinical and Experimental Neuropsychology, vol. 35, no. 10, pp. 1024–1035, 2013.
- F. Constantinidou and S. Baker, “Stimulus modality and verbal learning performance in normal aging,” Brain and Language, vol. 82, no. 3, pp. 296–311, 2002.
- A. W. Kaszniak, L. W. Poon, and W. H. Riege, “Assessing memory deficits: an information-processing approach,” in Handbook for Clinical Memory Assessment of Older Adults, L. W. Poon, T. Crook, K. L. Davis et al., Eds., pp. 168–188, American Psychological Association, Washington D.C., 1986.
- F. Constantinidou and M. Ioannou, “The effects of age and language on paragraph recall performance: findings from a preliminary cross-sectional study,” Psychologia, vol. 15, no. 4, pp. 342–361, 2008.
- F. Constantinidou, I. Zaganas, E. Papastefanakis, D. Kasselimis, A. Nidos, and P. G. Simos, “Age-related decline in verbal learning is moderated by demographic factors, working memory capacity, and presence of amnestic mild cognitive impairment,” Journal of the International Neuropsychological Society, vol. 20, no. 8, pp. 822–835, 2014.
- P. L. Ackerman, R. Kanfer, and C. Calderwood, “Use it or lose it? Wii brain exercise practice and reading for domain knowledge,” Psychology and Aging, vol. 25, no. 4, pp. 753–766, 2010.
- F. Constantinidou and R. D. Thomas, “Principles of cognitive rehabilitation in TBI: an integrative neuroscience approach,” in Traumatic Brain Injury: Rehabilitation, Treatment, and Case Management, Fourth Edition, M. J. Ashley and D. A. Hovda, Eds., pp. 513–540, Taylor & Francis: CRC Press, 2017.
- J. Karbach and P. Verhaeghen, “Making working memory work: a meta-analysis of executive-control and working memory training in older adults,” Psychological Science, vol. 25, no. 11, pp. 2027–2037, 2014.
- Y. Brehmer, H. Westerberg, and L. Bäckman, “Working-memory training in younger and older adults: training gains, transfer, and maintenance,” Frontiers in Human Neuroscience, vol. 6, p. 63, 2012.
- F. Constantinidou, R. D. Thomas, V. L. Scharp, K. M. Laske, M. D. Hammerly, and S. Guitonde, “Effects of categorization training in patients with TBI during post-acute rehabilitation: preliminary findings,” The Journal of Head Trauma Rehabilitation, vol. 20, no. 2, pp. 143–157, 2005.
- F. Constantinidou, R. D. Thomas, and L. Robinson, “Benefits of categorization training in patients with Traumatic Brain Injury during post–acute rehabilitation,” Journal of Head Trauma Rehabilitation, vol. 23, no. 5, pp. 312–328, 2008.
- A. Caramazza and J. R. Shelton, “Domain-specific knowledge systems in the brain: the animate-inanimate distinction,” Journal of Cognitive Neuroscience, vol. 10, no. 1, pp. 1–34, 1998.
- G. W. Humphreys, C. J. Price, and M. J. Riddoch, “From objects to names: a cognitive neuroscience approach,” Psychological Research, vol. 62, no. 2-3, pp. 118–130, 1999.
- F. G. Ashby, L. A. Alfonso-Reese, A. U. Turken, and E. M. Waldron, “A neuropsychological theory of multiple systems in category learning,” Psychological Review, vol. 105, no. 3, pp. 442–481, 1998.
- R. M. Nosofsky, “Exemplar-based approach to relating categorization, identification, ad recognition,” in Multidimensional Models of Perception and Cognition, F. G. Ashby, Ed., pp. 363–393, Earlbaum, Hillsdale, NJ, 1992.
- M. A. Erickson, “Executive attention and task switching in category learning: evidence for stimulus-dependent representation,” Memory & Cognition, vol. 36, no. 4, pp. 749–761, 2008.
- K. Patterson, P. J. Nestor, and T. T. Rogers, “Where do you know what you know? The representation of semantic knowledge in the human brain,” Nature Reviews Neuroscience, vol. 8, no. 12, pp. 976–987, 2007.
- G. S. Cree and K. McRae, “Analyzing the factors underlying the structure and computation of the meaning of chipmunk, cherry, chisel, cheese, and cello (and many other such concrete nouns),” Journal of Experimental Psychology: General, vol. 132, no. 2, pp. 163–201, 2003.
- G. W. Humphreys and E. M. Forde, “Hierarchies, similarity, and interactivity in object recognition:“category-specific” neuropsychological deficits,” Behavioral and Brain Sciences, vol. 24, no. 3, pp. 453–476, 2001.
- F. Constantinidou and A. Migut, “Common object categorization in TBI and in normal aging executive network functioning and implications for rehabilitation,” Journal of the International Neuropsychological Society, vol. 14, Supplement 1, p. 76, 2008.
- F. Constantinidou and L. T. Kreimer, “Feature description and categorization of common objects after traumatic brain injury: the effects of a multi-trial paradigm,” Brain and Language, vol. 89, no. 1, pp. 216–225, 2004.
- F. G. Ashby and E. M. Waldron, “The neuropsychological bases of category learning,” Current Directions in Psychological Science, vol. 9, no. 1, pp. 10–14, 2000.
- F. Constantinidou and R. D. Thomas, “Principles of cognitive rehabilitation in TBI: an integrative neuroscience approach,” in Traumatic Brain Injury: Rehabilitation, Treatment, and Case Management, M. Ashley, Ed., pp. 549–582, Taylor & Francis: CRC Press, 3rd edition edition, 2010.
- F. G. Ashby, S. Noble, J. V. Filoteo, E. M. Waldron, and S. W. Ell, “Category learning deficits in Parkinson’s disease,” Neuropsychology, vol. 17, no. 1, pp. 115–124, 2003.
- F. G. Ashby and B. J. Spiering, “The neurobiology of category learning,” Behavioral and Cognitive Neuroscience Reviews, vol. 3, no. 2, pp. 101–113, 2004.
- W. T. Maddox, B. C. Love, B. D. Glass, and J. V. Filoteo, “When more is less: feedback effects in perceptual category learning,” Cognition, vol. 108, no. 2, pp. 578–589, 2008.
- W. T. Maddox and F. G. Ashby, “Dissociating explicit and procedural-learning based systems of perceptual category learning,” Behavioural Processes, vol. 66, no. 3, pp. 309–332, 2004.
- W. T. Maddox and J. V. Filoteo, “Striatal contributions to category learning: quantitative modeling of simple linear and complex nonlinear rule learning in patients with Parkinson’s disease,” Journal of the International Neuropsychological Society, vol. 7, no. 6, pp. 710–727, 2001.
- A. M. Brickman, K. L. Siedlecki, J. Muraskin et al., “White matter hyperintensities and cognition: testing the reserve hypothesis,” Neurobiology of Aging, vol. 32, no. 9, pp. 1588–1598, 2011.
- J. J. Breunig, M. V. Guillot-Sestier, and T. Town, “Brain injury, neuroinflammation and Alzheimer’s disease,” Frontiers in Aging Neuroscience, vol. 5, p. 26, 2013.
- J. J. Iliff, M. J. Chen, B. A. Plog et al., “Impairment of glymphatic pathway function promotes tau pathology after traumatic brain injury,” Journal of Neuroscience, vol. 34, no. 49, pp. 16180–16193, 2014.
- V. E. Johnson, W. Stewart, and D. H. Smith, “Widespread tau and amyloid-beta pathology many years after a single traumatic brain injury in humans,” Brain Pathology, vol. 22, no. 2, pp. 142–149, 2012.
- N. Konstantinou, E. Pettemeridou, I. Seimenis et al., “Assessing the relationship between neurocognitive performance and brain volume in chronic moderate-severe traumatic brain injury,” Frontiers in Neurology, vol. 7, p. 29, 2016.
- E. Pettemeridou and F. Constantinidou, “Brain reserve as a predictor of functional outcome in chronic moderate-to-severe TBI,” Archives of Physical Medicine and Rehabilitation, vol. 99, no. 10, article e39, 2018.
- M. L. Daviglus, C. C. Bell, W. Berrettini et al., “National Institutes of Health state-of-the-science conference statement: preventing Alzheimer disease and cognitive decline,” Annals of Internal Medicine, vol. 153, no. 3, pp. 176–181, 2010.
- F. Constantinidou, R. D. Thomas, and P. Best, “Principles of cognitive rehabilitation after traumatic brain injury: a systematic hierarchical approach,” in Traumatic Brain Injury Rehabilitation, M. Ashley and F. T. Constantinidou, Eds., pp. 337–365, CRC Publishers, Boca Raton, FL, 2nd edition, 2004.
- F. Constantinidou, “The categorization program,” in This is a copyright protected therapy program and manual designed to treat adults with cognitive-communication disorders, Miami University. United States Library of Congress, copyright office (TXU 001345058), Oxford, OH, 2006.
- M. Gentilini, P. Nichelli, and R. Schoenhuber, “Assessment of attention in mild head injury,” in Mild Head Injury, H. S. Levin, H. M. Eisenberg, and A. L. Benton, Eds., pp. 163–175, Oxford University Press, New York, 1989.
- T. Kay, D. E. Harrington, R. Adams et al., “Definition of mild traumatic brain injury,” Journal of Head Trauma Rehabilitation, vol. 8, no. 3, pp. 86-87, 1993.
- C. Hagen, D. Malkmus, and P. Durham, “Levels of cognitive functioning,” in Rehabilitation of the Head-Injured Adult: Comprehensive Physical Management, C. A. Downey, Ed., Professional Staff Association of Rancho Los Amigos Hospital, 1972.
- H. S. Levin, M. A. O’Donnell, and R. G. Grossman, Galveston Orientation and Amnesia Test (GOAT): A Practical Scale to Assess Cognition after Head Injury, Williams & Wilkins, Baltimore, 1979.
- American Psychiatric Association, DSM-IV: The Diagnostic and Statistical Manual of Mental Disorders, The Psychological Corporation, San Antonio, 1994.
- A. T. Beck, R. A. Steer, and G. K. Brown, The Beck Depression Inventory-II, The Psychological Corporation, San Antonio, 1996.
- H. Goodglass, E. Kaplan, and B. Barresi, Boston Diagnostic Aphasia Examination, The Psychological Corporation, San Antonio, TX, 3rd edition, 2000.
- F. Dutton, “Ishihara test for color blindness,” 1999, March 2000, http://www.toledo-bend.com/colorblind/Ishihara.html.
- M. F. Folstein, S. E. Folstein, and P. R. McHugh, “"Mini-mental state": a practical method for grading the cognitive state of patients for the clinician,” Journal of Psychiatric Research, vol. 12, no. 3, pp. 189–198, 1975.
- B. B. Adamovich, J. A. Henderson, and S. Auerbach, Cognitive Rehabilitation of Closed Head Injured Patients: A Dynamic Approach, College Hill Press, San Diego, Ca, 1985.
- C. A. Coelho, F. DeRuyter, and M. Stein, “Treatment efficacy: cognitive-communicative disorders resulting from traumatic brain injury in adults,” Journal of Speech and Hearing Research, vol. 39, no. 5, pp. S5–S17, 1996.
- M. D. Lezak, Neuropsychological Assessment, Oxford University Press, New York, 3rd edition, 1995.
- A. R. Luria, Restoration of Function after Brain Injury, Pergamon Press, Oxford, 1963.
- S. M. Zeno, S. H. Ivens, R. T. Millard, and R. Duvvuri, The Educator’s Word Frequency Guide, Touchstone Applied Science Associates, New York, 1995.
- D. Wechsler, Wechsler Abbreviated Scale of Intelligence, The Psychological Corporation, San Antonio, 1999.
- M. F. Folstein, S. E. Folstein, and P. R. McHugh, Mini Mental State Examination, Psychological Assessment Resources, Inc., Lutz, FL, 2001.
- J. E. Myers and K. R. Meyers, Rey Complex Figure Test and Recognition Trial, Psychological Assessment Resources, Inc., Odessa, FL, 1995.
- M. D. Lezak, Neuropsychological Assessment, Oxford University Press, New York, 2nd edition, 1983.
- D. Wechsler, Wechsler Memory Scale, The Psychological Corporation, San Antonio, TX, 3rd edition, 1999.
- D. C. Delis, J. H. Kramer, E. Kaplan, and B. A. Ober, California Verbal Learning Test, The Psychological Corporation, San Antonio, TX, 2nd edition, 2000.
- D. A. Grant and E. A. Berg, Wisconsin Card Sorting Test, Los Angeles, Western Psychological Services, 1981.
- N. A. De Filippis and E. McCambell, Booklet Category Test, Psychological Assessment Resources, Inc., Odessa, FL, 2nd edition, 1998.
- A. Smith, Symbol Digit Modalities Test, Western Psychological Services, Los Angeles, 1973.
- A. Benton and K. Hamsher, Multilingual Aphasia Examination, University of Iowa Press, Iowa City, 1978.
- R. W. Woodcock, K. S. McGrew, and N. Mather, Woodcock Johnson Tests of Cognitive Abilities, Riverside Publishing, Itasca, IL, 3rd edition, 2001.
- M. D. Lezak, D. B. Howieson, and D. W. Loring, Neuropsychological Assessment, Oxford University Press, Oxford, 4th edition, 2004.
- D. A. Cahn, E. V. Sullivan, P. K. Shear, A. Pfefferbaum, G. Heit, and G. Silverberg, “Differential contributions of cognitive and motor component processes to physical and instrumental activities of daily living in Parkinson’s disease,” Archives of Clinical Neuropsychology, vol. 13, no. 7, pp. 575–583, 1998.
- J. Cohen, P. Cohen, S. G. West, and L. S. Aiken, Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, Erlbaum, Hillsdale, NJ, 3rd edition, 2003.
- M. S. DeCaro, R. D. Thomas, and S. L. Beilock, “Individual differences in category learning: sometimes less working memory capacity is better than more,” Cognition, vol. 107, no. 1, pp. 284–294, 2008.
- N. Konstantinou, E. Pettemeridou, E. A. Stamatakis, I. Seimenis, and F. Constantinidou, “Altered resting functional connectivity is related to cognitive outcome in males with moderate-severe Traumatic Brain Injury,” Frontiers in Neurology, vol. 9, article 1163, 2019.
- Task Force on Promotion and Dissemination of Psychological Procedures, “Training in and dissemination of empirically validated psychological treatments,” The Clinical Psychologist, vol. 48, pp. 13–23, 1995.
Copyright © 2019 Fofi Constantinidou. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.