Estimating Everyday Neuropsychological Functioning in Multiple Sclerosis: Reliability and Validity of the Greek Multiple Sclerosis Neuropsychological Questionnaire

Konstantinopoulou, Eleni; Ioannidis, Panagiotis; Bakirtzis, Christos; Giantzi, Virginia; Afrantou, Theodora; Parissis, Dimitrios; Messinis, Lambros; Nasios, Grigorios; Grigoriadis, Nikolaos

doi:https://doi.org/10.1155/2018/6301535

Multiple Sclerosis International

On this page

Abstract Introduction Results and Discussion Discussion Conclusions Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2018 | Article ID 6301535 | https://doi.org/10.1155/2018/6301535

Estimating Everyday Neuropsychological Functioning in Multiple Sclerosis: Reliability and Validity of the Greek Multiple Sclerosis Neuropsychological Questionnaire

Eleni Konstantinopoulou,¹Panagiotis Ioannidis,¹Christos Bakirtzis,¹Virginia Giantzi,²Theodora Afrantou,²Dimitrios Parissis,¹Lambros Messinis,³Grigorios Nasios,⁴and Nikolaos Grigoriadis¹

Academic Editor: Peter Arnett

Received23 May 2018

Revised04 Aug 2018

Accepted04 Sept 2018

Published25 Sept 2018

Abstract

The Multiple Sclerosis Neuropsychological Questionnaire is a brief screening questionnaire for the assessment of everyday neuropsychological competence of patients with Multiple Sclerosis. The aim of the present study was to examine psychometric properties of the Greek version of the instrument. One hundred and three MS patients and 60 informants participated in the present study and completed the questionnaire. From the initial patient sample, 51 participants completed broadly used neuropsychological tests and measures estimating cognitive failures and depression. Moreover, after a six-month interval the MSNQ was administered to 58 patients from the initial sample in order to explore test-retest reliability. Cronbach's α was 0.92 and 0.93 for patient and informant forms, respectively. The patient form was correlated significantly with measures of cognitive failures and depression. Low correlations were found between the informant form and performance on cognitive tests. In regard to the patient form, significant correlation was observed between repeated administrations and, psychometrically, the three-factor structure was preferable than the one-factor structure. The present study confirms the already established pattern of correlations among the two MSNQ forms, neuropsychological test performance and depression measurements. Additional research is needed in order to define a cut-off score for the MSNQ-I providing further information about the diagnostic interpretability of the instrument.

1. Introduction

Cognitive impairment is frequent in MS occurring in about 45% of patients, although according to various studies there is a range in prevalence rates (40-65%) [1]. The heterogeneity of neuropsychological dysfunction among MS patients is highlighted and impairment can be observed in any disease stage affecting a range of cognitive functions such as memory, executive functions, attention, and processing speed [2].

Brief batteries of neuropsychological tests have been proposed for the assessment of cognitive functioning in MS [2]. The Multiple Sclerosis Neuropsychological Questionnaire (MSNQ) is a 15-item questionnaire for the identification of patients with possible neuropsychological impairment [3]. Apart from the self-report form (MSNQ-P), an informant form is also available (MSNQ-I). The patient-informant rating discrepancy offers information about possible impairment in self-awareness and is also associated with neuropsychiatric features [4]. Moreover, since lack of awareness of cognitive deficits is an important concern in MS, informant ratings regarding patients’ neuropsychological status are valuable and should be included in standard assessment procedures [5, 6]. Moreover, previous research supports the relationship between the MSNQ-P score and depressive symptoms and suggests the implementation of the MSNQ-I for the evaluation of cognitive deficits [3, 7].

Prevalence and incidence rates of MS have been increased in Greece [8] but neuropsychological assessment is not included in routine clinical evaluations. Furthermore, screening instruments for the detection of Greek MS patients with cognitive impairment are lacking. Consequently, the aim of the present study was to examine basic psychometric properties of the Greek version of the MSNQ.

2. Method

2.1. Participants

One hundred and three patients (65 women, 63%) attending the MS center at 2nd Neurology Department, AHEPA University Hospital, Aristotle University of Thessaloniki, participated in the present study and completed the Multiple Sclerosis Neuropsychological Questionnaire (MSNQ-P). Patients were included in the study regardless of any Disease Modifying Treatment and were diagnosed by standard criteria [9]. The majority of patients (n= 84, 81%) were diagnosed with Relapsing Remitting MS (RRMS), 14 with Secondary Progressive MS (SPMS), and five with Primary Progressive MS (PPMS). Exclusion criteria were (1) current or past neurological disorder other than MS; (2) relapse within eight weeks of assessment; (3) history of, or current, psychiatric disorder; (4) history of, or current, substance abuse. Mean Expanded Disability Status Scale (EDSS) score was obtained retrospectively (within six months) from medical records and was unavailable for one patient. More than half of MS participants (n=58, 56%) completed the MSNQ-P for a second time after a six-month interval (Subgroup 1). Similarly, the majority of patients (n= 48, 82%) in this subgroup were diagnosed with RRMS, eight with SPMS and two with PPMS.

Based on both criteria of being a spouse or a parent and having daily contact with the patient, 60 informants completed the MSNQ-I (Subgroup 2). The majority of patients whose informants completed the MSNQ-I (n= 47.78%) were diagnosed with RRMS, nine with SPMS and four with PPMS.

Moreover, from the initial MS sample 51 patients (49%) completed broadly used neuropsychological tests estimating verbal memory, processing speed, and executive functions (Subgroup 3), as well as the Cognitive Failures Questionnaire (CFQ) and the Beck Depression Inventory (BDI). Again, the majority of patients (n= 44, 86%) were diagnosed with RRMS, five with SPMS and two with PPMS. Table 1 presents demographic and clinical information for each MS subgroup, as well as for the total group of patients. Mean ratings on the MSNQ-P, the CFQ, the BDI, and mean performance on neuropsychological tests are presented in Table 2.

2.2. Measures

2.2.1. The Multiple Sclerosis Neuropsychological Questionnaire (MSNQ)

The MSNQ is a disease specific questionnaire estimating patient’s and informant’s perspective of patient’s everyday neuropsychological competence [2]. Both forms include 15 questions, which came up from a larger pool of items addressing the spectrum of neuropsychological symptoms occurring in MS [7]. Scoring is based on a 5-point Likert scale, ranging from 0 (does not occur) to 4 (very often, very disruptive).

2.2.2. The Cognitive Failures Questionnaire (CFQ) and The Beck Depression Inventory (BDI-I)

The CFQ was developed to assess cognitive failures and errors in everyday tasks. It is a self-report questionnaire including 25 items and its scoring is based on 5-point Likert scale [10]. The BDI-I contains 21 items which estimate depressive symptoms [2].

2.2.3. Neuropsychological Tests

In order to estimate objective neuropsychological performance, a brief battery of neuropsychological tests was used. Verbal memory was estimated with the Greek Verbal Learning Test (GVLT) [11] and delayed free recall score was utilized for analyses. Processing speed was assessed with the Stroop Color-Word Test (SCWT) [12–14] as well as with the Experimental Processing Speed Test (EPST), an experimental measure which was developed for the present study according to the oral version of Symbol Digit Modalities Test [15]. The total score on the first (number of correct words read in 45 seconds) and second (number of correct colors named in 45 seconds) conditions of the SCWT and the total score on EPST (number of correct answers in 90 seconds) were used for analyses. Executive functions were estimated via the interference condition of SCWT (number of correct answers in 45 seconds) and the Verbal Fluency Test (VFT, subscores on semantic and phonemic conditions) [16]. Moreover, patients completed the Digit Span test [17] and total score, as well as subscores on forward and backward conditions, was used for analyses.

2.3. Procedures

The Greek version of the MSNQ-P was provided by its developer (R. H. Benedict). Minor grammatical adjustments were made, in order to develop the informant form (MSNQ-I).

Patients were asked to complete the MSNQ-P and available informants completed the MSNQ-I. Both forms were administered to participants by a staff neurologist or a psychologist. Next, all patients were informed for their further potential participation requesting the administration of neuropsychological tests. At this phase of the study the presence of an appropriate informant was an inclusion criterion for participation.

Cognitive impairment was evaluated according to published Greek normative data. In order to define impairment the criterion of two standard deviations below mean was used and three cognitive measures were assumed: memory (i.e., abnormal score on CVLT), processing speed (i.e., abnormal score on color and/or word conditions of the SCWT), and executive functioning (i.e., abnormal score on the interference condition of the SCWT, on phonemic and/or semantic conditions of the VFT).

All participants gave their informed consent to participate and the study was conducted according to the ethical standards set forth in the Declaration of Helsinki.

2.4. Data Analysis

Spearman rank correlation coefficients were calculated in order to explore possible correlations between variables, to estimate test-retest reliability and criterion-related validity. Mann–Whitney U Test for independent samples and Wilcoxon signed-rank test for paired data were conducted in order to examine group differences and difference between MSNQ-P test and retest scores, respectively. Internal Consistency was estimated for both MSNQ forms via Cronbach’s alpha coefficient.

Exploratory factor analysis (EFA) with direct oblimin rotation was used to investigate the factor structure of MSNQ-P. Because of the small sample of available informants, factor structure of the MSNQ-I was not explored in the present study. Eigenvalue (number of factors with eigenvalue >1) and scree plot criteria (number of factors before the break) were used to determine the number of factors. Subsequently, factor solution extracted by the EFA was further analyzed using Confirmatory Factor Analysis (CFA). The following goodness-of-fit indices were used for model selection: Chi-square (χ²), comparative fit index (CFI), Tucker-Lewis index (TLI), root-mean-square error of approximation (RMSEA), standardized root-mean-square residual (SRMR), and Akaike’s information criterion (AIC). Parameters were obtained by maximum likelihood estimation (MLE). The recommended criteria for good fit are CFI and TLI close to or higher than 0.95, SRMR lower than 0.08, RMSEA lower than 0.06 (lower and upper limits of the confidence interval lower than .05 and .10, respectively), and minimum AIC [18]. According to Hu and Bentler [2], for model fit evaluation in small sample sizes (<250), the combination of CFI and SRMR fit indices is preferable to the combination of TLI and RMSEA.

Statistical analyses were performed using the Statistical Package for Social Sciences (SPSS, version 22). CFAs were conducted using the R software (version 3.2.3 for Windows). Level of significance was set at 0.05.

3. Results and Discussion

3.1. Descriptive Information and Group Differences

Clinical variables, namely, duration of disease and EDSS score, were not correlated with any MSNQ form. The two MSNQ forms were significantly correlated with each other (r_s=.530, p=.000). MSNQ-P score was higher than MSNQ-I score but this difference was not significant.

Thirteen patients (25%) were impaired in one cognitive domain and two patients (3%) in two cognitive domains. Impaired performance on three cognitive domains was not observed in any patient.

3.2. Reliability

Cronbach’s α was 0.92 and 0.93 for the patient and the informant MSNQ form, respectively, suggesting excellent internal consistency.

Also, test-retest reliability was high. In specific, significant correlation was observed between MSNQ-P repeated administrations (r_s=781, p=.000). MSNQ-P scores were lower at retesting (after the six-month interval), but this decrease was not significant.

3.3. Criterion-Related Validity

The MSNQ-P was correlated with the CFQ (r_s=.762, p=.000) and the BDI (r_s=.315, p=.025). However, nonsignificant correlations were observed between the MSNQ-P and neuropsychological test performance.

On the contrary, the MSNQ-I was correlated significantly with performance on the Verbal Fluency, the Greek Verbal Learning, and the Digit Span tests. Nevertheless, the observed correlations were low (Table 3). The correlation between MSNQ-I and BDI scores did not reach significance (p>.05) and a low, yet significant, correlation was observed between the MSNQ-I and the CFQ (r_s=. 355, p=.020).

3.4. Factor Structure

Exploratory Factor Analysis (EFA) with direct oblimin rotation was conducted in order to explore the factor structure of the MSNQ-P. Three factors were observed explaining the 65% of the total variance. According to the unrotated factor solution all items loaded onto the first factor explaining the 49% of the total variance. The total amount of variance and the amount of variance explained by each factor were not increased after rotation. However, more interpretable factor solution was observed. Items corresponding to the ability to remember of performing planned actions in the future loaded onto the first latent factor. Items related to memory and attention, as well as to uncaused emotional reactions, loaded onto the second factor. Finally, executive and problem solving items loaded onto the third factor. Four items loaded also onto a second latent factor, but observed loadings were low (<.40). Factor loadings after rotation, eigenvalues, and the amount of the variance explained by each factor are shown in Table 4.

The three-factor structure identified by the EFA was further explored using Confirmatory Factor Analysis (CFA). Moreover, an alternative model was tested, namely, the theoretically driven one-factor model [19]. The value of χ² in both models was not significant (p>.05). Apart from the RMSEA fit index, the remaining fit indices were adequate in the three-factor model only (Table 5).

4. Discussion

According to the results of the present study both forms of the Greek MSNQ have excellent internal consistency. Moreover the MSNQ-P has high test-retest reliability. However, a nonsignificant decline was observed in MSNQ-P scores after the six-month interval. In a previous study researchers noticed a significant decline in repeated administrations which was attributed to a number of factors, such as response shift and increased awareness [20]. It should be emphasized that, within this six-month interval, which is longer than the conventional retesting interval, disease course and disease characteristics were stable (i.e., no relapses, no medication/treatment modifications).

Also inconsistent with previous findings [19], the results of EFAs and CFAs support the multidimensionality of the MSNQ-P. Nevertheless, the MSNQ-P can also be assumed as a one-dimensional questionnaire, since according to the unrotated factor solution of the EFA all items loaded onto the first factor explaining a respectable amount of the total variance. The results from the CFAs though suggest that the three-factor model fitted the data more satisfactorily. This is a reasonable evidence since the MSNQ includes items that are related to different conceptual dimensions (attention/processing speed, memory, other cognitive ability, personality, and behavior) [7].

In agreement with previous findings the MSNQ-P was highly correlated with the CFQ. Moreover, the MSNQ-P was correlated positively with the BDI, a finding which can be added on evidence underlying the susceptibility of the patient form to symptoms associated with depression [2]. It is important to highlight though that patients with a diagnosis of depressive disorder and any other psychiatric comorbidity were not participated in the present study. In regard to the MSNQ-I, low correlations were observed with neuropsychological measures of executive functioning and memory that are usually impaired in MS patients. Despite the frequency of information processing speed impairment in MS [21], in the present study “pure” measures of information processing speed were not correlated with the MSNQ-I. Nevertheless, correlations were observed between MSNQ-I score and performance on the Verbal Fluency Test which is also a time-dependent task. The lack of significant correlations between the MSNQ-I and performance on tests of processing speed could be attributed to the administration of an experimental test instead of the original SDMT. However, it should be highlighted that the MSNQ includes only one item directly focusing on the domain of processing speed.

Decreased awareness of cognitive deficits is an important concern in MS [5]. Moreover, literature suggests that self-reports of brain-damaged patients tend to underestimate cognitive impairment [22]. Thus, informant ratings regarding patient’s neuropsychological status are helpful and should be included in standard assessment procedures in MS, similarly to other groups of brain-damaged patients [6]. In addition, in MS as well as in other clinical groups subjective cognitive complaints are linked to depression [3, 23, 24]. Considering the frequency of depressive symptoms in MS, the significant impact of depression on self-reports, and the susceptibility of the MSNQ-P to depression, increased scores should be treated with caution and additional information from an informant is valuable.

In spite of psychometric adequacy regarding internal consistency, test-retest reliability, construct, and concurrent validity of the Greek version of the MSNQ cut-off scores were not defined in the present study. This inadequacy is associated with the major limitation of the study concerning the small number of MS participants with severe cognitive impairment and/or the lack of a control group. Therefore, additional research is needed in order to examine the diagnostic interpretability of the instrument. However, it should be mentioned that cut-off scores for the MSNQ-P have not been defined in previous studies where low sensitivity has been repeatedly observed.

5. Conclusions

Cognitive impairment not only affects everyday functioning in MS [25] but also has important implications in clinical management of patients. Reliable neuropsychological screening offers essential information about neurobehavioral symptoms of the disease. However, the Greek MSNQ is not appropriate to serve as a screening tool per se, since correlations with neuropsychological test performance are low. The MSNQ scales can be used as an index of agreement between patient’s and informant’s perspective in order to provide important information related to awareness and to facilitate treatment and rehabilitation. Furthermore, concurrent implementation of the MSNQ with a brief and well-established measure of processing speed (i.e., the SDMT) [26] would be more appropriate for efficient cognitive screening in clinical and research settings.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors have no conflicts of interest to report.

Acknowledgments

This work was supported by the Research Committee of Aristotle University of Thessaloniki (grant number: 89397). Τhe authors wish to acknowledge Eleni Polychroniadou, M.D., for her contribution in data collection.

References

P. A. Arnett and A. R. Rabinowitz, “Neuropsychological presentation and treatment of demyelinating disorders,” in Handbook of Clinical Neuropsychology, J. Gurd, U. Kischka, and J. Marshall, Eds., pp. 528–543, Oxford University Press, Oxford, UK, 2010.
View at: Google Scholar
A. Achiron, G. M. Doniger, Y. Harel, N. Appleboim-Gavish, M. Lavie, and E. S. Simon, “Prolonged response times characterize cognitive performance in multiple sclerosis,” European Journal of Neurology, vol. 14, no. 10, pp. 1102–1108, 2007.
View at: Publisher Site | Google Scholar
R. H. Benedict, D. Cox, L. L. Thompson, F. Foley, B. Weinstock-Guttman, and F. Munschauer, “Reliable screening for neuropsychological impairment in multiple sclerosis,” Multiple Sclerosis Journal, vol. 10, no. 6, pp. 675–678, 2004.
View at: Publisher Site | Google Scholar
D. Carone, R. Benedict, F. E. Munschauer III, I. Fishman, and B. Weinstock-Guttman, “Interpreting patient/informant discrepancies of reported cognitive symptoms in MS,” Journal of the International Neuropsychological Society, vol. 11, no. 5, pp. 574–583, 2005.
View at: Publisher Site | Google Scholar
F. Patti, “Cognitive impairment in multiple sclerosis,” Multiple Sclerosis Journal, vol. 15, no. 1, pp. 2–8, 2009.
View at: Publisher Site | Google Scholar
R. S. M. Tsang, K. Diamond, L. Mowszowski, S. J. G. Lewis, and S. L. Naismith, “Using informant reports to detect cognitive decline in mild cognitive impairment,” International Psychogeriatrics, vol. 24, no. 6, pp. 967–973, 2012.
View at: Publisher Site | Google Scholar
R. H. Benedict, F. Munschauer, R. Linn et al., “Screening for multiple sclerosis cognitive impairment using a self-administered 15-item questionnaire,” Multiple Sclerosis Journal, vol. 9, no. 1, pp. 95–101, 2016.
View at: Publisher Site | Google Scholar
P. Papathanasopoulos, E. Gourzoulidou, L. Messinis, V. Georgiou, and M. Leotsinidis, “Prevalence and incidence of multiple sclerosis in Western Greece: A 23-year survey,” Neuroepidemiology, vol. 30, no. 3, pp. 167–173, 2008.
View at: Publisher Site | Google Scholar
C. H. Polman, S. C. Reingold, B. Banwell et al., “Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria,” Annals of Neurology, vol. 69, no. 2, pp. 292–302, 2011.
View at: Publisher Site | Google Scholar
D. E. Broadbent, P. F. Cooper, P. FitzGerald, and K. R. Parkes, “The cognitive failures questionnaire (CFQ) and its correlates,” The British Journal of Clinical Psychology, vol. 21, no. 1, pp. 1–16, 1982.
View at: Publisher Site | Google Scholar
C. H. Vlahou, M. H. Kosmidis, A. Dardagani et al., “Development of the Greek verbal learning test: Reliability, construct validity, and normative standards,” Archives of Clinical Neuropsychology, vol. 28, no. 1, pp. 52–64, 2013.
View at: Publisher Site | Google Scholar
M. Zafiri and M. H. Kosmidis, “Effects of demographic characteristics on the ‘Stroop Conflict’,” The Journal of the Hellenic Psychological Society, vol. 15, no. 4, pp. 319–141, 2008.
View at: Google Scholar
T. J. Arentsen, K. B. Boone, T. T. Y. Lo et al., “Effectiveness of the comalli stroop test as a measure of negative response bias,” The Clinical Neuropsychologist, vol. 27, no. 6, pp. 1060–1076, 2013.
View at: Publisher Site | Google Scholar
D. R. Denney and S. G. Lynch, “The impact of multiple sclerosis on patients' performance on the Stroop Test: Processing speed versus interference,” Journal of the International Neuropsychological Society, vol. 15, no. 3, pp. 451–458, 2009.
View at: Publisher Site | Google Scholar
A. Smith, Symbol Digit Modalities Test, Western Psychological Services, Los Angeles, Calif, USA, 1982.
M. H. Kosmidis, C. H. Vlahou, P. Panagiotaki, and G. Kiosseoglou, “The verbal fluency task in the Greek population: Normative data, and clustering and switching strategies,” Journal of the International Neuropsychological Society, vol. 10, no. 2, pp. 164–172, 2004.
View at: Publisher Site | Google Scholar
D. Wechsler, Wechsler Memory Scale, The Psychological Corporation, San Antonio, Tex, USA, 3rd edition, 1997.
D. Hooper, J. Coughlan, and M. R. Mullen, “Structural equation modelling: guidelines for determining model fit,” The Electronic Journal of Business Research Methods, vol. 6, no. 1, pp. 53–60, 2008.
View at: Google Scholar
J. M. Sonder, L. B. Mokkink, F. A. H. Van Der Linden, C. H. Polman, and B. M. J. Uitdehaag, “Validation and interpretation of the Dutch version of the Multiple Sclerosis Neuropsychological Screening Questionnaire,” Journal of the Neurological Sciences, vol. 320, no. 1-2, pp. 91–96, 2012.
View at: Publisher Site | Google Scholar
S. Morrow, P. O’Connor, C. Polman et al., “Evaluation of the symbol digit modalities test (SDMT) and MS neuropsychological screening questionnaire (MSNQ) in natalizumab-treated MS patients over 48 weeks,” Multiple Sclerosis Journal, vol. 16, no. 11, pp. 1385–1392, 2010.
View at: Publisher Site | Google Scholar
S. L. Costa, H. M. Genova, J. Deluca, and N. D. Chiaravalloti, “Information processing speed in multiple sclerosis: Past, present, and future,” Multiple Sclerosis Journal, vol. 23, no. 6, pp. 772–789, 2017.
View at: Publisher Site | Google Scholar
J. M. Bruce and P. A. Arnett, “Self-reported everyday memory and depression in patients with multiple sclerosis,” Journal of Clinical and Experimental Neuropsychology, vol. 26, no. 2, pp. 200–214, 2004.
View at: Publisher Site | Google Scholar
L. Chamelian and A. Feinstein, “The effect of major depression on subjective and objective cognitive deficits in mild to moderate traumatic brain injury,” The Journal of Neuropsychiatry and Clinical Neurosciences, vol. 18, no. 1, pp. 33–38, 2006.
View at: Publisher Site | Google Scholar
Y. Maor, L. Olmer, and B. Mozes, “The relation between objective and subjective impairment in cognitive function among multiple sclerosis patients—the role of depression,” Multiple Sclerosis Journal, vol. 7, no. 2, pp. 131–135, 2001.
View at: Publisher Site | Google Scholar
J. H. Kalmar, E. A. Gaudino, N. B. Moore, J. Halper, and J. DeLuca, “The Relationship Between Cognitive Deficits and Everyday Functional Activities in Multiple Sclerosis,” Neuropsychology, vol. 22, no. 4, pp. 442–449, 2008.
View at: Publisher Site | Google Scholar
S. M. i. c. w. t. C. F. Rao, A Manual for The Brief Repeatable Battery of Neuropsychological Tests in Multiple Sclerosis, Medical College of Wisconsin, Milwaukee, Wis, USA, 1990.

Copyright

Copyright © 2018 Eleni Konstantinopoulou et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

2516

Downloads

1069

Citations