Research Article | Open Access
Utility of the Mini-Cog for Detection of Cognitive Impairment in Primary Care: Data from Two Spanish Studies
Objectives. To study the utility of the Mini-Cog test for detection of patients with cognitive impairment (CI) in primary care (PC). Methods. We pooled data from two phase III studies conducted in Spain. Patients with complaints or suspicion of CI were consecutively recruited by PC physicians. The cognitive diagnosis was performed by an expert neurologist, after formal neuropsychological evaluation. The Mini-Cog score was calculated post hoc, and its diagnostic utility was evaluated and compared with the utility of the Mini-Mental State (MMS), the Clock Drawing Test (CDT), and the sum of the MMS and the CDT () using the area under the receiver operating characteristic curve (AUC). The best cut points were obtained on the basis of diagnostic accuracy (DA) and kappa index. Results. A total sample of 307 subjects (176 CI) was analyzed. The Mini-Cog displayed an AUC (±SE) of , which was significantly inferior to the AUC of the CDT (), the MMS (), and the (). The best cut point of the Mini-Cog was 1/2 (sensitivity 0.60, specificity 0.90, DA 0.73, and kappa index ). Conclusions. The utility of the Mini-Cog for detection of CI in PC was very modest, clearly inferior to the MMS or the CDT. These results do not permit recommendation of the Mini-Cog in PC.
The aging of the population has come along with an increase in the incidence of cognitive impairment (CI) , a clinical syndrome that, in about one-third of the patients, precedes dementia . An early detection of CI could produce benefits at different levels, including early dementia diagnosis, access to treatments, and delay or even reversion of cognitive deterioration [3–5].
Primary care (PC) presents optimal characteristics of accessibility and continuity of care, which are essential for early detection and management of CI . In this vein, the focus of the PC physicians should be the detection of CI, rather than dementia. A separation of mild cognitive impairment (MCI) and dementia would not only be difficult or arbitrary in many instances but would also lead to missing opportunities for treatment and research .
The detection of CI requires a proactive attitude and the use of cognitive tests. In PC, cognitive tests need to be brief and easy to administer and interpret. In addition, these tests should have been specifically validated in the PC setting, with an adequate control of the potential influence of age, education, and other social variables . Albeit not simple, rather long, and very influenced by education, the Mini-Mental State (MMS)  is still the most used cognitive test. As additional limitations, the MMS displayed modest diagnostic utility for detection of CI in PC [10, 11] and was recently protected by copyright .
The Mini-Cog is a very simple and brief cognitive test that comprises a three-item verbal memory task and a simplified evaluation of the Clock Drawing Test (CDT). According to their authors, the Mini-Cog is not influenced by education . Indeed, in a population-based study, this test was as effective as a formal neuropsychological battery for detecting dementia . The Mini-Cog was also well accepted by the patients , displayed good correlation with the MMS , and was recently recommended in some consensus guidelines and reviews [17–20]. However, there are very few studies addressing the yield of the Mini-Cog for detection of CI (i.e., both MCI and dementia), which displayed conflicting results [21, 22].
The objective of the present study was to evaluate the diagnostic utility of the Mini-Cog for detection of CI in patients who presented with complaints or suspicion of cognitive deterioration in PC.
We analyzed data from two phase III studies of evaluation of diagnostic instruments  that were conducted in two different cities of Spain (i.e., Madrid and Granada). The study of Madrid was conducted in the Hermanos Sangro health center (Health District 1, Autonomous Community of Madrid, urban area), and the study of Granada was conducted in the Almanjáyar/Cartuja, the Casería de Montijo, and the Salvador Caballero health centers (Health District Granada Norte, Autonomous Community of Andalucía, urban area). The methods of these studies were described in detail elsewhere [24, 25]. In the two studies, patients and caregivers were informed of the procedures and purpose of the studies and gave their consent to participate. Other essential common characteristics of the two studies were as follows.
The recruitment of patients and the administration of the screening instruments were performed in PC, whereas the formal neuropsychological evaluation and the final diagnosis (gold standard) were performed in specialized care (SC), which replicates the standards of clinical practice.
The patients were prospectively and consecutively recruited by the PC physicians. All the subjects who presented with complaints or suspicious (either by informant or by family physician) regarding cognitive dysfunction or cognitive deterioration were invited to participate in the study. Patients with a former diagnosis of CI were not included.
2.3. Screening Instruments
A validated Spanish version of the MMS was utilized, with the modification of ignoring the possibility of spelling world backwards . The CDT was scored from 0 to 7 ; in one of the sites (Granada), the requested time was “ten past eleven,” whereas in the other site (Madrid) the requested time was “twenty to four.” Otherwise, the instructions for the clock drawing test in the two sites were alike (i.e., “please, draw a big round clock, put all the numbers on it, and display the hands on (requested time)”). The score of the Mini-Cog was obtained post-hoc, for the purpose of the present investigation. One point was given for every recalled word from the MMS delayed recall task, and two additional points were given if the clock fulfilled the following conditions: (a) all the numbers from 1 to 12 were included and there was no repeated number, (b) all the numbers were in the right order and direction, and (c) the hands unequivocally displayed the requested time. Hence, the score of the Mini-Cog varied from 0 to 5 .
2.4. Cognitive Diagnosis (Gold Standard)
Mild cognitive impairment (MCI) was diagnosed on the basis of the following criteria: (a) an abnormal performance was documented in at least one neuropsychological test; (b) that abnormal performance was deemed to be of clinical relevance (i.e., not due to low premorbid level, lack of motivation, etc.), and (c) dementia criteria were not fulfilled. The diagnosis of dementia was made according to the 4th edition, text revised, of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR) . The cognitive diagnosis was performed by two senior neurologists with expertise in cognitive and behavioural neurology (CC, JO) after medical visit, neurological exam, mental status exam, and formal neuropsychological evaluation, which included tests of memory, attention/executive functions, language, and visual-spatial abilities [24, 25]. Throughout the study, laboratory determinations and other ancillary tests (e.g., cranial computed tomography scan) were performed as the clinical situation indicated.
2.5. Independence of the Cognitive Diagnosis
The results of the screening instruments were not utilized for the cognitive diagnosis in the original studies. However, for the present investigation, the CDT, which was a screening instrument in the study of Granada but was included in the formal neuropsychological evaluation in the study of Madrid, was analyzed as a screening instrument.
2.6. Double Blinding
The screening instruments were administered before the neuropsychological and neurological exams were conducted, in different settings (PC versus SC) and by different professionals. The neurologists, who performed the cognitive diagnosis, were blind to the results of the screening instruments with the exception of the CDT in the study of Madrid, as mentioned in the previous paragraph.
2.7. Reference Bias
Complete verification was conducted; that is, all the recruited patients received the same neuropsychological and neurological assessment and all of them received a final cognitive diagnosis, independently of the results in the screening instruments .
There were some differences between the two studies regarding age and period of inclusion. In the study of Madrid, only ≥50 year old patients were recruited, and the period of inclusion ran from April 1, 2000 to October 31, 2002 (i.e., 31 months). There was no age limit in the study of Granada and the period of inclusion of that study ran from February 1, 2008 to January 31, 2009 (i.e., 1 year). The study of Granada was approved by the Hospital Virgen de las Nieves Ethics Committee, and participants signed informed consent. In the study of Madrid, only verbal consent was required from patients and caregivers/informants.
Statistical Analysis. Bivariate comparisons between quantitative variables were performed using Student’s -test, whereas qualitative variables were compared using chi-square and Fisher exact tests. A potential influence of site was specifically addressed by means of two-factor (site [Granada versus Madrid] × cognitive diagnosis [CI versus no CI]) analysis of covariance (ANCOVA) adjusted for age (years), sex (man versus woman), and education (no or incomplete primary education versus primary or higher education).
The diagnostic utility of the Mini-Cog, the CDT, the MMS, and the sum of the MMS and the CDT scores (MMS + CDT) was evaluated and compared for detection of CI (MCI or dementia) using receiver operating characteristic (ROC) curves. Differences between the areas under the curve (AUC) were analyzed using the Hanley and McNeil correction to compare diagnostic tests in the same sample . In addition, the sensitivity (Sn), the specificity (Sp), the positive likelihood ratio (LR+), the diagnostic accuracy (DA) (proportion of patients correctly classified), and the kappa index of diagnostic concordance were obtained for all the cut points in all the tests. The best cut point was chosen for each test on the basis of maximization of DA and kappa index.
The interrater reliability of the Mini-Cog was evaluated in a subsample of 50 subjects (first 25 subjects included in each site). Four evaluators that were blind to one another and to all other study variables scored the clocks and the three-item recall tasks of those subjects, and the intra-class correlation coefficient (ICC) was calculated.
All the statistical tests were two tailed, and a level of was chosen for statistical significance. Statistical analyses were performed using the SPSS 15.01 and MedCalc 12.3 software.
The demographic and clinical characteristics of the total sample and of the two original samples are presented in Table 1. Overall, there was predominance of female sex (72.0%), and the educational achievement was low (50.8% of the participants had not complete primary education). The frequency of illiteracy was particularly high in the participants from Granada (14.1% versus 4.8%, ). Patients from Granada also presented with a higher frequency of dementia (34.5% versus 9.1, ) and obtained lower scores in all the cognitive tests (Table 1).
|Data represent number of individuals (%) or mean ± SD. MMS: Mini-Mental State; CDT: Clock Drawing Test; DEM: dementia; NC: normal cognition; MCI: mild cognitive impairment. |
As it was expected, the subjects with MCI or dementia were older than the subjects with normal cognition (NC). The subjects with CI had also less educational achievement and scored lower in all the cognitive tests (Table 2). There were no sex differences across the different cognitive diagnoses. Figure 1 represents the distribution of the Mini-Cog score in the two study groups.
|Data represent number of individuals (%) or mean ± SD. MMS: Mini-Mental State; CDT: Clock Drawing Test; NC: normal cognition; CI: cognitive impairment (MCI or dementia); DEM: dementia; MCI: mild cognitive impairment.|
*For the comparison between NC versus CI (MCI or dementia).
The score of the Mini-Cog was significantly influenced by age and education but not by sex or study site . A total of 16 participants belonging to the sample of Granada (11.3% of patients from Granada, 5.2% of the total sample, and 11 of them illiterates) did not even try to draw the clock, in which case a zero score was given.
The diagnostic yield of the Mini-Cog was inferior (AUC ± SE 0.78 ± 0.02), compared to that of the MMS (0.84 ± 0.02, ), the CDT (0.84 ± 0.02, ), and the MMS + CDT (0.86 ± 0.02, ) (Figure 2). The best cut point of the Mini-Cog was 1/2, with Sn 0.60 (confidence interval (CI) 95% 0.53–0.67), Sp 0.90 (CI 95% 0.84–0.95), LR+ 6.1 (CI 95% 3.6–0.3), DA 0.73, and kappa index 0.48 ± 0.05). The usually recommended cut point of the Mini-Cog of 2/3 displayed an inferior utility, with Sn 0.78 (CI 95% 0.71–0.84), Sp 0.59 (CI 95% 0.51–0.68), DA 0.70, and kappa index 0.38 ± 0.04 (Table 3). Using a cut point of 22/23, the MMS had similar Sn (0.76, CI 95% 0.68–0.82), but better Sp (0.76, CI 95% 0.68–0.83) and DA (0.76). As for the CDT, a cut point of 5/6 achieved similar results to those of the MMS, with Sn 0.76 (CI 95% 0.68–0.82), Sp 0.78 (CI 95% 0.70–0.85), and DA of 0.76. The MMS + CDT was slightly, but not significantly, superior to the MMS or the CDT alone (Table 4, Figure 2).
|DA: diagnostic accuracy (proportion of correct diagnoses); : kappa index; LR+: positive likelihood ratio; Sn: sensibility; Sp: specificity. |
|aROC: area under curve ROC; Sn: sensibility; Sp: specificity; DA: diagnostic accuracy (proportion of correct diagnoses); : kappa index; MMS: Mini-Mental State; CDT: Clock Drawing Test.|
The Mini-Cog displayed an excellent interrater reliability, with an ICC (±SD) of 0.97 ± 0.01.
We pooled data of two homogeneous studies to ascertain the value of the Mini-Cog for detection of CI (either MCI or dementia) in PC. The score of the Mini-Cog was obtained post-hoc, utilizing data from the MMS and the CDT. The diagnostic utility obtained by the Mini-Cog was low, with an AUC (±SE) of 0.78 ± 0.02 and a percentage of correctly classified subjects of 70%, when the recommended cutoff of 2/3 was applied . A cut point of 1/2 obtained a slightly better yield (73% of cases correctly classified) on the basis of a good specificity (0.90 (CI 95% 0.84–0.95)) but a low sensitivity (0.60 (0.53–0.67)), which was unacceptable for a detection test.
The diagnostic utility of the Mini-Cog was inferior to that of the MMS, which was moderate (AUC 0.84 ± 0.02). When the best MMS cut point was applied (i.e., 22/23), the sensitivity and specificity were modest (0.76 (0.68–0.82)), and the percentage of cases correctly classified did not reach 80% (DA 0.76) (Table 4). These modest results of the MMS coincide with other study conducted in PC  and with the results from two systematic reviews [11, 18]. Albeit also modest, the results obtained by the CDT in the present investigation were superior to the results obtained in previous studies  that led to disregard of the CDT for detection of MCI . A possible explanation for our better results with the CDT is that this test was included, in one of the study sites, in the reference diagnosis. We also evaluated the diagnostic utility of the sum of scores of the MMS and the CDT, but there was no significant gain, with a clear increase in testing time, when compared to the MMS or the CDT alone. That result is also in agreement with a previous investigation .
In previous research, the Mini-Cog displayed good performance for detection of dementia, with figures that were similar  or even superior  to those obtained by the MMS or the CDT. To the authors’ knowledge, only two studies had so far addressed the utility of the Mini-Cog for detection of CI, with conflicting results. In a sample of predominantly male old people, the results of the Mini-Cog were not satisfactory (AUC 0.64) , whereas, in a study conducted in a sample of predominantly “unserved ethnic minorities,” a diagnostic accuracy of 0.83 was reported . The proportion of patients with dementia, that differed dramatically in those studies (3.3% dementia, 39.2% MCI in Holsinger’s study  versus 41.5% dementia, 21.3% MCI in Borson’s study ), could explain the reported differences in the diagnostic yield of the Mini-Cog. Our results, in a sample of intermediate characteristics (20.8% dementia, 36.5% MCI) (Table 1), were also intermediate in terms of Mini-Cog diagnostic utility (diagnostic accuracy 0.73, AUC 0.78) (Table 3 and Figure 2). Clearly, all these data advise against the use of the Mini-Cog for detection of CI in settings where the suspected proportion of unrecognized dementia is low.
We found a significant influence of education in the Mini-Cog performance, which was not in agreement with the original validation study . This discrepancy can be explained in the light of the low educational achievement of our sample (Table 1). In the original validation study, all the participants had completed at least six years of education, whereas in the present study, a proportion of 50.8% of the subjects had not complete primary school, which in Spain usually means less than eight years of formal education, and there was 9.1% of illiteracy (Table 1). There is strong evidence of the influence of education and literacy on CDT performance [31, 35], and reasonably this should be the same with the Mini-Cog. In fact, poor yield of the Mini-Cog was previously reported for dementia detection in a sample of low educational level . Overall, these data suggest a negative influence of education on the Mini-Cog, restricted to the most inferior educational strata.
Our study had some limitations. First, the CDT was part of the reference diagnosis in one of the study sites, a limitation that could have overvalued the utility of the Mini-Cog and therefore would not essentially alter the study conclusions. Second, the score of the Mini-Cog was obtained post-hoc from two different tests. This methodology was used by several researchers with exploratory purposes, including the original descriptions of the Mini-Cog [13, 14] to ascertain the validity of the test . Certainly, the obtained results discourage future prospective studies of the Mini-Cog. Third, the present results were derived from two slightly different studies. However, the results obtained by the Mini-Cog in each of the individual studies were similar (data not shown), which strengthens the consistence and the external validity of the presented global results. As additional strength, the two pooled studies fulfilled the standards of high-quality phase III diagnostic studies , a consecutive and systematic recruitment was conducted, and there were virtually no exclusion criteria.
Simplicity and shortness in time are indeed characteristics of the Mini-Cog, which are necessary for use in PC. However, the Mini-Cog was not a valid choice in the studied sample, and this result, which was consistent with previous research, advise against the use of the Mini-Cog, particularly in settings where the proportion of unrecognized dementia is low. In these settings, the use of tests that offer a wider range of tasks and score, yet maintaining acceptability and feasibility, seems more adequate . In light of the relevance of the detection of CI, more human and technical resources should be implemented in PC.
Conflict of Interests
The authors do not have any direct financial relation with the trademarks mentioned in the paper that might lead to a conflict of interests for any of the authors. Dr. Carnero-Pardo is the author of the Phototest and the Eurotest, two brief cognitive tests under license of Creative Commons. The rest of the authors declare no potential conflict of interests.
The study of Granada was funded by the Agencia de Evaluación de Tecnologías Sanitarias, Instituto de Salud Carlos III (Expdte PI06/90034). The funding source had no role in the study design, data collection, analysis, interpretation, or writing of the report. The authors are grateful to all of the primary healthcare professionals who participated in the study: Pedro Torrero, Esperanza Aparicio, Ana Sanz, Nieves Mula, Garbiñe Marzana, Dionisio Cabezón, Concepción Begué, Samuel López, Carolina Martos, Estrella Mora, Rosa Vílchez, Francisco Romo, Mª José Rodríguez, Francisco Suárez, Félix Rodríguez, Rafael Moya, Concepción López, Bárbara Martínez, Isabel Valenzuela, Victoria Molina, Ana Mª Zamora, José Antonio Henares, Isabel Rodríguez, Carmen Romero, Manuel Jiménez, Francisco Padilla, Miguel Melguizo, Ana Mª de los Ríos, Ascensión Cano, Manuel Alonso, José Luis Martín, José López, Ambrosio Esteva, Francisca Dorador, Dolores Sánchez, José A. López, Mª Ángeles Jurado, Yarmila García, Eduardo Fernández, Consuelo Pérez, José Antonio Castro, Araceli Zambrano, and Hassan Mahmoud.
- T. Luck, M. Luppa, S. Briel, and S. G. Riedel-Heller, “Incidence of mild cognitive impairment: a systematic review,” Dementia and Geriatric Cognitive Disorders, vol. 29, no. 2, pp. 164–175, 2010.
- A. J. Mitchell and M. Shiri-Feshki, “Rate of progression of mild cognitive impairment to dementia—meta-analysis of 41 robust inception cohort studies,” Acta Psychiatrica Scandinavica, vol. 119, no. 4, pp. 252–265, 2009.
- R. T. Woods, E. Moniz-Cook, S. Iliffe et al., “Dementia: issues in early recognition and intervention in primary care,” Journal of the Royal Society of Medicine, vol. 96, no. 7, pp. 320–324, 2003.
- D. L. Weimer and M. A. Sager, “Early identification and treatment of Alzheimer's disease: social and fiscal outcomes,” Alzheimer's & Dementia, vol. 5, no. 3, pp. 215–226, 2009.
- T. Etgen, D. Sander, H. Bickel, and H. Förstl, “Mild cognitive impairment and dementia: the importance of modifiable risk factors,” Deutsches Arzteblatt International, vol. 108, no. 44, pp. 743–750, 2011.
- J. Olazarán, “Can dementia be diagnosed in primary care?” Atencion Primaria, vol. 43, no. 7, pp. 377–384, 2011.
- M. S. Rafii and D. Galasko, “Primary care screening for dementia and mild cognitive impairment,” Journal of the American Medical Association, vol. 299, no. 10, p. 1132, 2008.
- H. Brodaty, L.-F. Low, L. Gibson, and K. Burns, “What is the best dementia screening instrument for general practitioners to use?” The American Journal of Geriatric Psychiatry, vol. 14, no. 5, pp. 391–400, 2006.
- M. F. Folstein, S. E. Folstein, and P. R. McHugh, “'Mini mental state'. a practical method for grading the cognitive state of patients for the clinician,” Journal of Psychiatric Research, vol. 12, no. 3, pp. 189–198, 1975.
- P. Pezzotti, S. Scalmana, A. Mastromattei, and D. Di Lallo, “The accuracy of the MMSE in detecting cognitive impairment when administered by general practitioners: a prospective observational study,” BMC Family Practice, vol. 9, no. 29, 2008.
- A. J. Mitchell, “A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment,” Journal of Psychiatric Research, vol. 43, no. 4, pp. 411–431, 2009.
- J. C. Newman and R. Feldman, “Copyright and open access at the bedside,” The New England Journal of Medicine, vol. 365, no. 26, pp. 2447–2449, 2011.
- S. Borson, J. Scanlan, M. Brush, P. Vitaliano, and A. Dokmak, “The Mini-Cog: a cognitive ‘vital signs’ measure for dementia screening in multi-lingual elderly,” International Journal of Geriatric Psychiatry, vol. 15, no. 11, pp. 1021–1027, 2000.
- S. Borson, J. M. Scanlan, P. Chen, and M. Ganguli, “The Mini-Cog as a screen for dementia: validation in a population-based sample,” Journal of the American Geriatrics Society, vol. 51, no. 10, pp. 1451–1454, 2003.
- J. R. McCarten, P. Anderson, M. A. Kuskowski, S. E. McPherson, and S. Borson, “Screening for cognitive impairment in an elderly veteran population: acceptability and results using different versions of the Mini-Cog,” Journal of the American Geriatrics Society, vol. 59, no. 2, pp. 309–313, 2011.
- G. Kamenski, T. Dorner, K. Lawrence et al., “Detection of dementia in primary care: comparison of the original and a modified Mini-Cog assessment with the mini-mental state examination,” Mental Health in Family Medicine, vol. 6, no. 4, pp. 209–217, 2009.
- J. A. Lonie, K. M. Tierney, and K. P. Ebmeier, “Screening for mild cognitive impairment: a systematic review,” International Journal of Geriatric Psychiatry, vol. 24, no. 9, pp. 902–915, 2009.
- D. Kansagara and M. Freemam, Systematic Evidence Review of the Signs and Symptoms of Dementia and Brief Cognitive Tests Available in VA, VA-ESP Project #05-225, 2010.
- C. B. Cordell, S. Borson, M. Boustani et al., “Alzheimer's association recommendations for operationalizing the detection of cognitive impairment during the medicare annual wellness visit in a primary care setting,” Alzheimer's & Dementia, vol. 9, no. 2, pp. 141–150, 2012.
- A. Milne, A. Culverwell, R. Guss, J. Tuppen, and R. Whelton, “Screening for dementia in primary care: a review of the use, efficacy and quality of measures,” International Psychogeriatrics, vol. 20, no. 5, pp. 911–926, 2008.
- S. Borson, J. M. Scanlan, J. Watanabe, S.-P. Tu, and M. Lessig, “Simplifying detection of cognitive impairment: comparison of the Mini-Cog and mini-mental state examination in a multiethnic sample,” Journal of the American Geriatrics Society, vol. 53, no. 5, pp. 871–874, 2005.
- T. Holsinger, B. L. Plassman, K. M. Stechuchak, J. R. Burke, C. J. Coffman, and J. W. Williams Jr., “Screening for cognitive impairment: comparing the performance of four instruments in primary care,” Journal of the American Geriatrics Society, vol. 60, no. 6, pp. 1027–1036, 2012.
- D. L. Sackett and R. B. Haynes, “The architecture of diagnostic research,” British Medical Journal, vol. 324, no. 7336, pp. 539–541, 2002.
- C. Carnero-Pardo, B. Espejo-Martinez, S. Lopez-Alcalde et al., “Effectiveness and costs of phototest in dementia and cognitive impairment screening,” BMC Neurology, vol. 11, no. 92, 2011.
- I. Cruz-Orduna, J. M. Bellon, P. Torrero et al., “Detecting MCI and dementia in primary care: efficiency of the MMS, the FAQ and the IQCODE,” Family Practice, vol. 29, no. 4, pp. 401–406, 2012.
- R. Blesa, M. Pujol, M. Aguilar et al., “Clinical validity of the 'mini-mental state' for Spanish speaking communities,” Neuropsychologia, vol. 39, no. 11, pp. 1150–1157, 2001.
- P. R. Solomon, A. Hirschoff, B. Kelly et al., “A 7 minute neurocognitive screening battery highly sensitive to Alzheimer's disease,” Archives of Neurology, vol. 55, no. 3, pp. 349–355, 1998.
- American Psychiatry Association, Diagnostic and Statistical Manual of Mental Disorders, Text Revision, DSM-IV-TR, American Psychiatric Association, Washington, DC, USA, 4th edition, 2000.
- P. M. Bossuyt, L. Irwig, J. Craig, and P. Glasziou, “Comparative accuracy: assessing new tests against existing diagnostic pathways,” British Medical Journal, vol. 332, no. 7549, pp. 1089–1092, 2006.
- J. A. Hanley and B. J. McNeil, “A method of comparing the areas under receiver operating characteristic curves derived from the same cases,” Radiology, vol. 148, no. 3, pp. 839–843, 1983.
- E. Pinto and R. Peters, “Literature review of the clock drawing test as a tool for cognitive screening,” Dementia and Geriatric Cognitive Disorders, vol. 27, no. 3, pp. 201–213, 2009.
- L. Ehreke, M. Luppa, H.-H. König, and S. G. Riedel-Heller, “Is the clock drawing test a screening tool for the diagnosis of mild cognitive impairment? A systematic review,” International Psychogeriatrics, vol. 22, no. 1, pp. 56–63, 2010.
- J. Cacho, J. Benito-León, R. García-García, B. Fernández-Calvo, J. L. Vicente-Villardón, and A. J. Mitchell, “Does the combination of the MMSE and clock drawing test (mini-clock) improve the detection of mild Alzheimer's disease and mild cognitive impairment?” Journal of Alzheimer's Disease, vol. 22, no. 3, pp. 889–896, 2010.
- M. Milian, A. M. Leiherr, G. Straten, S. Müller, T. Leyhe, and G. W. Eschweiler, “The Mini-Cog versus the mini-mental state examination and the clock drawing test in daily clinical practice: screening value in a German Memory Clinic,” International Psychogeriatrics, vol. 24, no. 5, pp. 766–774, 2012.
- H. Kim and J. Chey, “Effects of education, literacy, and dementia on the clock drawing test performance,” Journal of the International Neuropsychological Society, vol. 16, no. 6, pp. 1138–1146, 2010.
- R. A. Lourenço and S. T. Ribeiro Filho, “The accuracy of the Mini-Cog in screening low-educated elderly for dementia,” Journal of the American Geriatrics Society, vol. 54, no. 2, pp. 376–377, 2006.
- P. F. Whiting, A. W. S. Rutjes, M. E. Westwood et al., “Quadas-2: a revised tool for the quality assessment of diagnostic accuracy studies,” Annals of Internal Medicine, vol. 155, no. 8, pp. 529–536, 2011.
- C. Carnero-Pardo, B. Espejo-Martínez, S. López-Alcalde et al., “Diagnostic accuracy, effectiveness and cost for cognitive impairment and dementia screening of three short cognitive tests applicable to illiterates,” PLoS ONE, vol. 6, no. 11, Article ID e27069, 2011.
Copyright © 2013 Cristóbal Carnero-Pardo et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.