Abstract

Mild cognitive impairment (MCI) is frequent in Parkinson’s disease (PD). Recently proposed criteria for MCI in PD (PD-MCI) indicate level I diagnosis based on abbreviated assessment and level II based on comprehensive neuropsychological evaluation. The study explored the sensitivity and specificity of the Italian versions of three neuropsychological tests for level I diagnosis of PD-MCI. We recruited 100 consecutive PD patients. After screening for inclusion criteria, 43 patients were included. The sensitivity and specificity of the Mini Mental State Examination (MMSE), the Montreal Cognitive Assessment (MoCA), and the Addenbrooke’s Cognitive Examination Revised (ACE-R) in comparison to level II diagnosis of PD-MCI were examined. PD-MCI was diagnosed (level II) in 51% of patients. Disease duration was significantly longer and PD motor scales were more severely impaired in MCI group. The receiver-operator characteristics curve documented nonsignificant difference in the performance of the three tests, with slight advantage of MMSE (corrected data). The time of administration favored MMSE. In Italian-speaking PD patients, MMSE might represent a good screening tool for PD-MCI, because of the shorter time of administration and the performance comparable to those of MoCA and ACE-R. Further studies are needed to validate the new PD-MCI criteria across different languages and cultures.

1. Introduction

Cognitive impairment is frequent in Parkinson’s disease (PD) [1], and the spectrum of cognitive dysfunction ranges from mild cognitive impairment (MCI) to PD dementia (PD-D) [2, 3]. The diagnosis of PD-D may to some extent be straightforward [4], but recognizing MCI in PD (PD-MCI) is more difficult. Cognitive deficits may occur early in PD course, and they can be documented in up to a quarter of newly diagnosed PD patients [5]. The biological validity of PD-MCI as a clinical entity is supported by converging morphological, functional neuroimaging, neurophysiological, genetic, and cerebrospinal fluid and histological data showing an association between a number of neuropathophysiological variables and cognitive impairment or cognitive decline in nondemented PD patients [2].

Identifying PD-MCI is clinically important, as these patients appear to be at increased risk for developing PD-D [6], and they often present functional impairment and have worse quality of life [2]. In the rehabilitation setting, recognizing PD-MCI is very important, in that it may negatively influence the outcome in patients undergoing motor rehabilitation. Moreover, PD-MCI may itself represent a target for cognitive training [7, 8], pharmacological treatment [9], or their combination.

A task force of the Movement Disorder Society (MDS) has recently delineated diagnostic criteria for PD-MCI [10]. These criteria indicate a two-step process with level I (possible PD-MCI) based on abbreviated assessment and level II diagnosis based on comprehensive neuropsychological evaluation permitting MCI subtyping [10], but they need to be validated, as well as the proposed neuropsychological scales and tests. A very recent study explored these criteria in a group of PD patients and the accuracy of three neuropsychological screening tests and found that none of them provided good combined sensitivity and specificity for PD-MCI [11]. For most of the neuropsychological tests, translation and validation across different languages and cultures are lacking, and this may represent a problem when assessing PD-MCI with level I criteria and a possible source of error when transferring data from a given population/language to other ones.

The present study was aimed to explore the sensitivity and specificity of the Italian versions of three neuropsychological tests for level I diagnosis of PD-MCI, namely, the Mini Mental State Examination (MMSE) [12], the Montreal Cognitive Assessment (MoCA) [13], and the Addenbrooke’s Cognitive Examination Revised (ACE-R) [14], for all of which an Italian translation and validation exist [1518]. Data from the three screening neuropsychological tests were compared to those from full neuropsychological testing (level II) [10], which represent the gold standard for MCI diagnosis.

2. Materials and Methods

2.1. Subjects

Our population sample was a group of 100 consecutive Italian PD patients. The study was carried out in accordance with the principles of the Declaration of Helsinki as revised in 2001 and approved by local ethics committee. All patients gave signed informed consent prior to inclusion in the study. Inclusion criteria were diagnosis of PD based on the UK PD Brain Bank Criteria [19]; (2) absence of PD-D [4]; (3) no other possible causes for cognitive impairment (e.g., delirium, stroke or cerebrovascular disease, head trauma, metabolic abnormalities, and adverse effects of medication); (4) no other PD-associated comorbid conditions (e.g., marked motor impairment, severe or unpredictable motor fluctuations and/or dyskinesia, severe anxiety, excessive daytime sleepiness, or psychosis) that may have significantly influenced cognitive testing [10].

Depression was assessed with the Beck Depression Inventory II (BDI-II) [20] with a cutoff of 14 for the presence of mild depression and a cutoff of 28 for severe depression [21]. Depression was not considered an exclusion criterion, except if severe (i.e., patients with a BDI-II score >28 were excluded), because it may be found in around 35% of PD patients [22] and including PD patients with mild to moderate depression would have resulted in a more real-life scenario. The severity of PD motor symptoms and related impairment and disability was measured with the Modified Hoehn and Yahr Staging Scale [23] and the Unified Parkinson’s disease rating scale [24]. Total daily levodopa equivalent dose was calculated for each patient [25].

After screening for inclusion criteria (Figure 1), 43 patients (27 males, 16 females, mean age , range 44–88; mean education years, range 4–13) were included in the study. Demographic and clinical characteristics of patients are reported in Table 1.

2.2. Neuropsychological Assessment

All patients underwent the Italian versions of MMSE, MoCA, and ACE-R and a full neuropsychological testing, which were performed by different expert neuropsychologists, who were blinded to each other’s results, on separate days at a similar time of the day, and with the patient in the ON state. Given overlapping items, the order of administration of the three screening tests was pseudorandom to avoid bias in performance related to fatigue, learning, or other effects secondary to order [11]. Since the ACE-R contains all the items of the MMSE, the common items were not administered twice. The time taken for administering each screening test and full neuropsychological testing was measured in each patient.

Full neuropsychological testing included at least two types of neuropsychological testing for each of the five following cognitive domains [10]. Attention and working memory were examined with four tests, namely, digit span, a subtest of the Wechsler memory scale [26], interference memory task (10 sec and 30 sec) based on the Brown-Peterson paradigm [27, 28], and trail making test (TMT) part A [29]. Executive function was explored with four tests, namely, TMT part B [29], frontal assessment battery [30], phonemic fluency test, and clock drawing test, the latter two being subtests of the short neuropsychological examination version 2 (ENB-2) [31]. Language was examined with four tests, namely, the short form of the Boston naming test [32] and three specific subtests of the neuropsychological examination of aphasia [33]. Memory was explored with four tests, namely, Rey’s auditory verbal learning test (immediate recall, delayed recall) [34], and two prose recall subtest (immediate recall, delayed recall) derived from ENB-2 [31]. Visuospatial function was examined with two tests, namely, Benton’s judgment of line orientation [35] and the geometrical figures copying test, a subtest of the mental deterioration battery [36].

The impairment on basic activities of everyday life (BADL) and instrumental activities of everyday life (IADL) was explored with specific questionnaires [37, 38].

2.3. PD-MCI Diagnosis

The diagnosis of PD-MCI was made according to the MDS Task Force level II criteria [10]. They included gradual decline, in the context of established PD, in cognitive ability reported by either the patient or informant or observed by the clinician, consisting of at least 1 item of the IADL scale; cognitive deficits that are not sufficient to interfere significantly with functional independence, although subtle difficulties on complex functional tasks may be present, as documented by normal BADL scale; impairment in at least two neuropsychological tests, represented by either two impaired tests in one cognitive domain (single-domain PD-MCI) or one impaired test in two different cognitive domains (multiple-domain PD-MCI). Impaired performance on a neuropsychological test was defined as a score that was at least 1.5 standard deviations (SDs) below the age-adjusted mean from normative data [11]. According to the MDS Task Force criteria, significant decline on serial cognitive testing or from estimated premorbid level may be used instead of normative data [10], but we did not use these alternative criteria, because the former would have required repeated full neuropsychological testing with the risk of learning bias and because of the difficulties found in applying the latter (see Section 4) [11].

2.4. Statistical Analysis

All tests were carried out with the IBM SPSS version 20.0 and the Stata 11.0 statistical packages. The normality of variable distribution was analyzed with the Skewness-Kurtosis test. Continuous variables were explored with ANOVA and post hoc -test with Bonferroni’s correction. Homogeneity of variance was analyzed with Levene’s test. The data were transformed (logarithmic transformation) before submitting them to ANOVA in case of an inequality in the variances. The nonparametrical Mann-Whitney test was applied in case the distribution was not normal. Pearson’s test with Yates’ correction for continuity was applied to dichotomous variables. Sensitivity and specificity of the MMSE (raw score and score corrected for age, sex, and education), MoCA (raw and corrected score), and ACE-R were calculated across all possible cutoff scores below which an individual would be classified as having PD-MCI. The area under the receiver-operator characteristics (ROC) curve (AUC) was calculated and compared across the three tests and the AUC 95% confidence intervals (CIs) were generated. (two-tailed) was taken as the significance threshold for all the tests.

3. Results

According to the MDS Task Force level II criteria [10], PD-MCI was diagnosed in 22 patients (51%). Eight out of the 22 (36%) PD-MCI patients were classified as single-domain MCI, with five of them showing impairment in executive function and three with impaired memory. The other 14 patients (64%) were classified as multiple-domain MCI. Among multiple-domain MCI cases, attention and working memory was impaired in 9 patients, executive function in 14, memory in 8, language in 2, and visuospatial function in 1. Demographic and clinical variables according to the presence or absence of MCI and the MCI subtype (i.e., single-domain versus multiple-domain) are reported in Table 2. Disease duration was significantly longer in patients with MCI ( years) than in those without MCI ( years, ; Table 2). PD motor and impairment scales were more severely impaired in MCI group (H-Y: ; UPDRS-III: ) than in patients without MCI (H-Y: , ; UPDRS-III: , ; Table 2). The other variables did not differ between the two groups. None of the demographic and clinical variables significantly differed according to the MCI subtype (Table 2).

3.1. Comparison between the Screening Tests

The ROC curves for the three screening tests (raw and corrected data) are illustrated in Figure 2. The AUC was 0.84 (95% CI: 0.72–0.97) for the MMSE (raw data), 0.88 (95% CI: 0.78–0.98) for the MMSE (corrected data), 0.80 (95% CI: 0.66–0.93) for the MoCA (raw data), 0.79 (95% CI: 0.66–0.93) for the MoCA (corrected data), and 0.81 (95% CI: 0.68–0.94) for the ACE-R. None of the pair-wise comparisons between AUC estimates were statistically significant.

The sensitivity and specificity of the three tests for detecting PD-MCI across different cutoff scores are reported in Tables 3 and 4.

3.2. Screening Cutoff Values

For raw MMSE data, the lowest cutoff value with sensitivity >0.80 was 29.5 (sensitivity = 0.96, specificity = 0.62). When using corrected MMSE data, the lowest cutoff value with sensitivity >0.80 was 28.6 (sensitivity = 0.86, specificity = 0.71). For raw MoCA data, the lowest cutoff value with sensitivity >0.80 was 24.5 (sensitivity = 0.82, specificity = 0.67). When analyzing corrected MoCA data, the lowest cutoff value with sensitivity >0.80 was 25.5 (sensitivity = 0.82, specificity = 0.67). For ACE-R, the lowest cutoff value with sensitivity >0.80 was 86.0 (sensitivity = 0.82, specificity = 0.67).

3.3. Diagnostic Cutoff Values

For raw MMSE data, the highest cutoff value with specificity >0.80 was 28.5 (sensitivity = 0.73, specificity = 0.81). When examining corrected MMSE data, the highest cutoff value with specificity >0.80 was 28.0 (sensitivity = 0.73, specificity = 0.81). For raw MoCA data, the highest cutoff value with specificity >0.80 was 21.5 (sensitivity = 0.55, specificity = 0.90). When using corrected MoCA data, the highest cutoff value with specificity >0.80 was 22.5 (sensitivity = 0.55, specificity = 0.86). When analyzing ACE-R findings, the highest cutoff value with specificity >0.80 was 77.5 (sensitivity = 0.59, specificity = 0.81).

3.4. Timing for Administering Screening Tests and Full Neuropsychological Testing

The average time for the administration of the screening tests was  min for MMSE,  min for MoCA, and  min for ACE-R. Full neuropsychological testing required  min.

4. Discussion

We have explored the sensitivity and specificity of the Italian versions of three screening tests for recognizing PD-MCI in comparison to full neuropsychological testing. Our data documented that the performances of the three tests were similar and that they could achieve a limited trade-off between sensitivity and specificity, with a slight advantage of MMSE and the use of corrected data.

The screening tests we examined were chosen because, to the best of our knowledge, they were the only ones with the availability of a validated Italian version at the time when the study was designed. None of them could reach combined sensitivity and specificity >0.80 at any cutoff value. The analysis of ROC curves for the screening scales showed a larger AUC and the best sensitivity-specificity profile for the corrected MMSE score. In particular, a cutoff of 28.6 resulted in sensitivity = 0.86 and specificity = 0.71, while a cutoff of 28.0 was associated in sensitivity = 0.73, and specificity = 0.81. The other scales performed slightly worse, but the difference between the ROC curves was not significant.

A number of previous studies compared different screening tests for assessing cognitive functions and/or early cognitive deficit in PD patients [5, 39], with conflicting results in terms of the best profile of sensitivity and specificity between them. The use of MMSE as a screening instrument in PD has been challenged because it does not specifically test subcortical executive function, which is impaired early in PD patients [40]. Some studies documented that MMSE has low sensitivity in detecting MCI and cognitive impairment in PD [41, 42], in particular when compared to MoCA [39, 4345]. At variance, other authors reported that MMSE might be useful in detecting cognitive deterioration in early PD [46]. Data on the use of ACE-R as a screening tool for PD-MCI are controversial [47], but a previous version was found to be a good test for evaluating MCI [48] and dementia [49, 50] in PD patients. A reason for these discrepancies might be that ACE-R includes an assessment by domains and its abilities may not be completely comparable to that of MMSE and MoCA, which represent true screening scales. Moreover, MMSE and ACE-R share some common items, and the total points of ACE-R (100 points) differ from that of MMSE and MoCA (30 points). However, the comparison of AUCs instead of cutoffs should have avoided the difference in total points among screening tests to represent a bias.

Comparison between the present results and those from most of previous studies is however difficult, because only a few of them used a comprehensive neuropsychological evaluation and standard criteria to detect and diagnose MCI. A couple of recent studies applied the new MDS criteria for MCI and yielded contrasting results, in that one documented limited sensitivity-specificity profile for both MMSE and MoCA [11], while the other reported high sensitivity of MoCA for predicting PD-MCI [51].

Other neuropsychological scales, such as the Mattis dementia rating scale [48, 52, 53], the Cambridge cognitive assessment [54], the cognitive linguistic quick test [55], the PD cognitive rating scale [56], and the SCOPA cognition [57], have been demonstrated to be helpful in exploring early cognitive decline in PD [10], but the absence of an Italian version impeded their exploration as a screening tool for PD-MCI in our PD patients sample. What is more, the long administration time of these scales (i.e., up to 25–45′) is not suitable for a screening procedure in the clinical setting.

Our data favor the correction for age, sex, and education when scoring MMSE, in that corrected MMSE data yielded a larger AUC and slightly better sensitivity-specificity profile than raw ones. At variance, correcting MoCA did not change the performance of the test. However, pair-wise statistical comparisons between ROC curves did not show any significant difference between them. In the clinical setting, MMSE correction is reasonable especially for older and less educated patients.

We recorded the time taken for administering the three screening scale, and this variable favored the MMSE ( min) compared to the MoCA ( min) and the ACE-R ( min). According to these combined figures (i.e., similar sensitivity-specificity profile, shorter time of administration), it is reasonable to prefer the use of MMSE in the setting of a busy clinic.

A number of factors may contribute to cognitive dysfunction in PD patients and lead to a false positive diagnosis of PD-MCI. All the possible contributing factors were considered and our strict inclusion criteria, which resulted in the exclusion of approximately half of the patients, should have reduced this bias. Drugs with possible effect on cognition represented an exclusion criterion, and the total LED was similar between patients with and without MCI. As a consequence, pharmacological effects should not have influenced our findings. Depression has been documented to be more frequent in PD-MCI patients in comparison to those without MCI [58], but this was not the case in our sample. We excluded only patients with severe depression according to the BDI-II, because mild to moderate depression is a common feature of PD and exclusion of all depressed patients might have resulted in a non-real-life scenario. We may argue that depression should not have been a bias factor in the present study.

PD patients with MCI had significantly longer disease duration and more severe motor impairment and disability, according to the H-Y and UPDRS-III scales. This finding is in accordance with some previous reports [58] but in contrast with other ones [11]. Differences in the sampling of PD patients across different studies, depending on different settings (e.g., primary care versus referral centre) or different populations, are the most likely reasons for this discrepancy.

The analysis of MCI subtypes indicated a prevalence of multidomain PD-MCI in comparison to single-domain. This finding is in accordance with previous reports using new MDS criteria [11, 59]. We could not document any difference in clinical variables between single- and multidomain PD-MCI patients, but the small samples might have impeded the recognition of small differences between the two groups. In accordance with previous studies [5, 7], we documented a high prevalence of executive alterations in our PD sample. This finding may stem from the use of four tests for this cognitive domain, which may have resulted in a higher likelihood of falling in two of them [60]. However, this potential bias effect seems not to be major, because the upper limit (maximum probability) for detecting impairment on a test was found to stabilize at two tests in the executive functions domain and did not increase with three or four tests [60].

When applying the MDS level II diagnostic criteria for PD-MCI [10], impairment on a neuropsychological test was defined as a score that was at least 1.5 SD below the age-adjusted normative data [11]. We avoided the use of the alternative criterion of a significant decline on serial cognitive testing [10], because of the lack of previous neuropsychological testing in the majority of our patients. For what concerns the other alternative criteria of a decline from estimated premorbid level [10], this was also not used for a number of reasons. They include the lack of any indication on how to use tests of premorbid intellectual functioning [10], the absence of a validated Italian version of the Wechsler test of adult reading [10], and the previous evidence of the ineffectiveness of the Italian version of the alternative national adult reading test for the estimation of premorbid reading ability [61]. In a previous study, the number of patients diagnosed as PD-MCI with level II criteria varied consistently (i.e., from 33% to 79%) by applying different criteria for impairment on a neuropsychological test [11], and this might represent an important source of uncertainty when applying level II criteria. Similarly, varying cutoff values for single tests had a large influence on the percentage of PD-MCI patients in the same population [62].

Limitations of the present study include the small sample and the high prevalence of PD-MCI. MCI was found in 51% patients in our PD sample, while cross-sectional studies documented that the prevalence of MCI ranges from 20 to 30% in PD [42, 58]. However, our sample is too small to provide a good approximation of the prevalence of the condition in the general population, and there may have been some bias due to the strict selection criteria. The present data should be confirmed in a larger PD patients group before generalizing our conclusions.

Another limitation is the absence of follow-up data. Serial testing of PD-MCI patients documented that a similar proportion of them might either progress to PD-D or revert to normal cognition (i.e., approximately 20%) after one year [63]. Reasons for this apparently paradoxical finding might include comorbidities, measurement errors, learning effects due to repeated neuropsychological testing, and improved cognition after initiation of symptomatic treatment [63], in addition to suboptimal treatment of motor symptoms at the time of first testing, motor fluctuations, or drug side effects.

BADL and IADL were evaluated with questionnaires [37, 38] that are not PD-specific, because, to the best of our knowledge, there is no Italian version of any disease-specific scale, such as the Parkinson’s disease cognitive functional rating scale [64]. We think that this point does not represent a major bias, because the questionnaires were used to group patients as having PD-MCI or not and not to quantitatively measure impairment on BADL and IADL.

5. Conclusions

Our data might be helpful in the clinical and the neurorehabilitation setting, because cognitive impairment is common in PD, PD-MCI may progress to PD-D, and both these conditions may have a negative impact on function, quality of life, and caregiver burden [43]. Identification and intervention at the earliest stage of PD-MCI is a crucial unmet need for the overall care of PD patients [10]. MMSE might represent a good tool for screening cognition throughout all stages of PD, because of the short time of administration and the sensitivity-specificity profile comparable to those of MoCA and ACE-R. Follow-up serial testing might be necessary in case of confounding factors. Complete neuropsychological testing, however, still represents the gold standard for a diagnosis of PD-MCI.

Future studies should better explore the reliability of level I and level II MDS criteria for MCI and incorporate biomarkers of cognitive dysfunction [2, 10].

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.