CAPTCHA as a Visual Performance Metric in Active Macular Disease

Vangipuram, Gautam; Lee, Aaron Y.; Rezaei, Kasra A.; Olmos De Koo, Lisa C.; Chee, Yewlin E.; Chao, Jennifer R.; Egan, Catherine; Lee, Cecilia S.

doi:https://doi.org/10.1155/2019/6710754

Journal of Ophthalmology

On this page

Abstract Introduction Materials and Methods Results Discussion Conclusions Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2019 | Article ID 6710754 | https://doi.org/10.1155/2019/6710754

CAPTCHA as a Visual Performance Metric in Active Macular Disease

Gautam Vangipuram,¹Aaron Y. Lee,¹Kasra A. Rezaei,¹Lisa C. Olmos De Koo,¹Yewlin E. Chee,¹Jennifer R. Chao,¹Catherine Egan,²and Cecilia S. Lee¹

Academic Editor: Achim Langenbucher

Received25 Sept 2018

Revised25 Apr 2019

Accepted16 May 2019

Published09 Jun 2019

Abstract

Purpose. CAPTCHA (completely automated public turing test to tell computers and humans apart) was designed as a spam prevention test. In patients with visual impairment, completion of this task has been assumed to be difficult; but to date, no study has proven this to be true. As visual function is not well measured by Snellen visual acuity (VA) alone, we theorized that CAPTCHA performance may provide additional information on macular disease-related visual dysfunction. Methods. This was designed as a pilot study. Active disease was defined as the presence of either intraretinal fluid (IRF) or subretinal fluid (SRF) on spectral-domain optical coherence tomography. CAPTCHA performance was tested using 10 prompts. In addition, near and distance VA, contrast sensitivity, and reading speed were measured. Visual acuity matched pseudophakic patients were used as controls. Primary outcome measures were average edit distance and percent of correct responses. Results. 70 patients were recruited: 33 with active macular disease and 37 control subjects. Contrast sensitivity was found to be significantly different in both the IRF () and SRF groups (). No significant difference was found comparing the odds ratio of average edit distance of active disease (IRF, SRF) vs. control (OR 1.09 (0.62, 1.90), 1.10 (0.58, 2.05), ) or percent correct responses of active disease vs. control (OR 0.98 (0.96, 1.01), 1.09 (0.58, 2.05), ) in CAPTCHA testing. The goodness of fit using logistic regression analysis for the dependent variables of either IRF or SRF did not improve accounting for average edit distance () or percent correct (). Conclusions. Distance VA and contrast sensitivity are positively correlated with the presence of IRF and SRF in active macular disease. CAPTCHA performance did not appear to be a significant predictor of either IRF or SRF in our pilot study.

1. Introduction

Computer automated public turing test to tell computers and humans apart (CAPTCHA) has gained popularity in recent years as an electronic visual challenge-response test of human authenticity (Figure 1). The test involves a user correctly entering all alphanumeric symbols displayed in a text prompt. Although CAPTCHA testing is useful in thwarting potential hackers and spamming, passing it can prove challenging to those with visual impairment [1, 2]. Responders with diseases affecting central vision, in particular the macula, may have even more difficulty completing these tasks. To this end, quantifying patients’ level of performance on CAPTCHA testing may provide a noninvasive, sensitive, and easy way to perform method of potentially monitoring active macular disease.

To evaluate macular disease, CAPTCHA performance may provide an advantage over other visual acuity metrics methods in assessing central visual acuity. Traditionally, Snellen visual acuity is the standard metric for assessing vision and treatment outcomes [3–9]. Nonetheless, Snellen testing often fails to address aspects of important visual impairment to patients for daily functioning and quality of life. Elliott et al. reported on a series of thirty-three patients with a history of visually significant cataract in at least one eye. The authors measured each patient’s visual acuity, brightness acuity, and binocular contrast sensitivity and correlated these findings to a questionnaire of vision-related quality of life [10]. Binocular contrast sensitivity correlated more strongly to visual quality of life than Snellen visual acuity alone. Similarly, Hazel et al. found that low-contrast visual acuity, contrast sensitivity, and binocular reading speed had the strongest correlation with visual quality of life in 28 patients with active macular disease from either presumed ocular histoplasmosis syndrome (POHS) or exudative macular degeneration [11].

Snellen visual acuity also fails to address visual symptoms in individuals with noncenter involving macular disease. Quantifying a patient’s CAPTCHA performance may provide a functional endpoint that is noninvasive, inexpensive, and easy to perform compared to structural makers of disease activity such as optical coherence tomography (OCT) and optical coherence tomography angiogram (OCTA). To our knowledge, CAPTCHA performance has not been studied in patients with visual disability, and this is the first study to examine CAPTCHA performance in patients with active macular disease [12–20]. Our study hypothesized that patients with macular disease, defined as active diabetic macular edema (DME) or exudative age-related macular degeneration (AMD), would perform poorly on a series of CAPTCHA prompts compared to visually matched pseudophakic controls.

2. Materials and Methods

The study was a prospective cohort study. Patients were recruited from the University of Washington Eye Institute, Seattle, WA. Institutional review board approval was obtained, and the study followed the World Medical Association Declaration of Helsinki Ethical Principles for medical research involving human subjects. Inclusion criteria for the macular disease cohort were defined as having ICD-10 code diagnoses of diabetic macular edema (E11.311) or exudative age-related macular degeneration (H35.32), OCT confirmed intraretinal fluid (IRF) or subretinal fluid (SRF), and English as the primary spoken language. Exclusion criteria for the macular disease cohort were defined as patients with concomitant pathology affecting the central visual axis (e.g., visually significant cataract, central corneal scar, and amblyopia), best near corrected visual acuity worse than 20/50, and the presence of both IRF and SRF. Control patients were pseudophakic, spoke English as their primary language, and had a best-corrected visual acuity of at least 20/20 with no other diseases affecting the central visual axis. Only one eye from each patient was tested. If both eyes met inclusion and exclusion criteria, a coin flip was used to determine the testing eye.

All patients were asked to complete a series of visual tasks. Control patients were blurred to a predetermined visual acuity (20/20–20/50) using positive power spherical lenses in an attempt to create a 1 : 1 match of visual acuity to the macular disease cohort. Distance acuity was tested using a standard Snellen chart while near distance was tested using a Rosenbaum near point acuity card at 14 inches. CAPTCHA testing was performed at the same near distance, 14 inches, as the Rosenbaum near point test by using an iPAD Air (Model A1566, Apple Inc. Cupertino, CA) tablet. A series of 10 CAPTCHA images were presented to each patient in a randomized order. Typed responses were recorded in a separate database. Contrast sensitivity (logcontrast) was tested using a Pelli–Robson chart. Reading speed in seconds was tested at near distance using a 156-word paragraph.

The primary outcome measure was average edit distance. In computational science, edit distance is defined as the minimum number of operations needed to transform one string of letters into another. The average edit distance of each subject’s responses to the ten randomized CAPTCHA prompts was calculated. The secondary outcome measure was percent correct. For example, if a patient answered all of one CAPTCHA prompt exactly as displayed, this was marked as correct, and any deviation from a prompt was marked incorrect. Thus, if a patient answered five CAPTCHA prompts correctly out of ten, the value of this outcome variable would be 50%.

2.1. Statistical Methods

All statistical analyses were completed using SPSS version 24 (IBM corporation, New York, United States) and independently verified by a biostatistician. Logistic regression analysis was used to determine the goodness of fit (Nagelkerke R²) of a base model consisting of the dependent variable (IRF or SRF) and covariates (visual acuity, reading speed, and contrast sensitivity). Average edit distance and percent correct were independently added to this base model, and chi-square analysis was used to determine significance in model fit.

3. Results

Seventy patients were recruited for this study (Table 1). Thirty-three patients had active macular disease comprising either subretinal fluid or intraretinal fluid. Thirty-seven patients were recruited as controls. Of the active disease cohorts, 14 (42.4%) had DME while 19 (57.6%) had AMD. There was no statistical difference in age (), gender (), or race () between the active disease and control groups. When near visual acuity was stratified into 5 groups (20/20, 20/25, 20/30, 20/40, and 20/50), no significant difference was found between active disease and control cohorts.

The active disease cohort was further stratified into patients with IRF () and SRF (). On univariate analysis, there was no significant difference in average edit distance between either the IRF group or SRF group vs. control (Table 2). Likewise, no significant difference was found in percent correct between either the IRF group or SRF group vs. control. As demonstrated in Figure 2, no distinct trend was seen in comparing the IRF group or SRF group vs. control for average edit distance, percent correct, or reading speed. However, contrast sensitivity performance was noted to significantly differ between both IRF vs. control () and SRF vs. control ().

(a)

(b)

(c)

(d)

Logistic regression analysis is shown for both IRF and SRF cohorts (Table 3). The base model (A or B) assumes either IRF (A) or SRF (B) as the dependent variable with covariates of visual acuity, reading speed, and contrast sensitivity. The likelihood, which can be interpreted as a pseudo R² for this model, was calculated. For both the IRF and SRF base models, covariates of average edit distance and percent correct were added to determine if this improved the goodness of fit of the model. In neither IRF (x² = 0.07, ) nor SRF (x² = 0.15, ), did the addition of average edit distance improve the goodness of fit of the model. A similar result was found when the secondary outcome variable of percent correct was added to the model: IRF (x² = 0.68, ) and SRF (x² = 0.02, ).

4. Discussion

This was a prospective cohort study to determine whether macular intraretinal or subretinal fluid correlated with CAPTCHA performance. The “goodness of fit” of the logistic regression model predicting either IRF or SRF did not improve when average edit distance was added to the base model. Similarly, no improvement was seen in comparing percent correct to its respective base model. These results likely suggest that CAPTCHA performance is not an independent predictor of intraretinal or subretinal fluid in active macular disease. Our findings show that patients with macular disease perform poorly on contrast sensitivity assessment in spite of relatively preserved central visual acuity as previously reported [21]. Both the IRF and SRF cohorts performed significantly worse on contrast testing compared to pseudophakic visual acuity-matched controls.

CAPTCHA images contain a series of variably separated optotypes that create a challenging visual prompt to the examiner. Although CAPTCHA images do not fit a standardized visual testing method, it mostly resembles ascertaining a subject’s minimum legible acuity. Both Snellen distance and Rosenbaum near point testing also assess a subject’s minimum legible acuity by differentiating increasingly smaller optotypes to determine a final visual acuity [22]. Unlike these tests, however, CAPTCHA does not employ strong serifs in its font, making its text more difficult to discern. Furthermore, given the variation in CAPTCHA letter size within each image, it is difficult to determine the equivalent visual acuity with Snellen testing. However, the majority of letters in each test image were between 10- and 14-point font or 20/70 and 20/100 Snellen equivalent.

In addition, all CAPTCHA prompts in our study employed high contrast, black print on white background. A tablet computer was chosen to administer the CAPTCHA prompts to most realistically simulate real-life testing conditions and facilitate a standardized near acuity. However, the increased background luminance during testing has been shown to improve a subject’s visual acuity and small letter contrast sensitivity [23, 24]. Thus, the heightened background luminance of the tablet may have aided the subject compared to the standardized photopic room testing conditions (90 lux) of the Pelli–Robson examination. In addition, the enhanced contrast of CAPTCHA images may be above the threshold of how much intraretinal or subretinal pathologies affects one’s visual function, which could explain why CAPTCHA performance was not significantly different in patients with active disease versus control subjects.

The exact pathophysiology of decreased contrast sensitivity has not been fully elucidated. Poor performance on contrast sensitivity testing is seen in a variety of diseases including glaucoma, optic neuritis, and macular degeneration [21]. Arend et al. reported a series of 20 patients with diabetes mellitus without clinically significant macular edema with better than 20/25 vision. The authors found a significant correlation between foveal avascular zone area and contrast sensitivity, suggesting parafoveal capillary dropout may have contributed to a decrease in contrast sensitivity [25]. García et al. performed a retrospective study on 62 multiple sclerosis patients and found a significant correlation with optic nerve disease activity and contrast sensitivity performance. Our study adds to the existing literature demonstrating poor contrast delineation in patients with intraretinal fluid or subretinal fluid [26, 27].

The strengths of this study are its prospective nature, which allowed for well-defined inclusion and exclusion criteria. In addition, OCT images obtained at the time of testing allowed for determination of disease activity in real time. Limitations include the lack of quantification of center vs. noncenter, involving fluid on OCT imaging and the absence of OCT images from pseudophakic controls.

Additionally, the technique of visual blurring using positive convex lenses has been described in the literature to assess its effect on the completion of visual tasks and the comprehension of visual cues [28, 29]. Although visual blurring effectively matched the control population’s visual acuity to the disease cohort, this technique is only meant to simulate retinal pathology and does not entirely mimic the pathology seen in DME and AMD.

In summary, visual function tests other than Snellen visual acuity are important in evaluating ocular diseases. In particular, contrast sensitivity should be considered in the clinical evaluation of active macular disease. Although visually impaired patients can struggle with CAPTCHA task completion, our study results show that neither intraretinal fluid nor subretinal fluid correlate with worsened CAPTCHA performance. We would expect, therefore, that using CAPTCHA to prevent hacking attempts would not exclude patients affected with active macular disease if their visual acuity allowed them to perform the test adequately. Further studies are needed to determine if CAPTCHA performance correlates to other structural markers of ocular pathology.

5. Conclusions

CAPTCHA performance is not a significant predictor of intraretinal fluid or subretinal fluid in patients with active macular disease. Ancillary tests of visual function, particularly contrast sensitivity, correlate to macular disease. Further studies are needed to determine whether CAPTCHA performance is associated with other features of macular diseases.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper. Leona Ding is a biostatistician at the University of Washington.

Acknowledgments

This study was supported by NIH/NIA K23EY024921, P30-EY001730, and unrestricted core grant from Research to Prevent Blindness; Dr. I. Melvin and Nanette D. Freeman endowed fund in ophthalmology.

References

L. von Ahn, M. Blum, N. J. Hopper, and J. Langford, “CAPTCHA: using hard AI problems for security,” in Lecture Notes in Computer Science, pp. 294–311, Springer, Berlin, Germany, 2003.
View at: Publisher Site | Google Scholar
L. von Ahn, B. Maurer, C. McMillen, D. Abraham, and M. Blum, “reCAPTCHA: human-based character recognition via web security measures,” Science, vol. 321, no. 5895, pp. 1465–1468, 2008.
View at: Publisher Site | Google Scholar
C. M. Mangione, R. S. Phillips, J. M. Seddon et al., “Development of the activities of daily vision scale,” Medical Care, vol. 30, no. 12, pp. 1111–1126, 1992.
View at: Publisher Site | Google Scholar
R. Kington, J. Rogowski, L. Lillard, and P. P. Lee, “Functional associations of “trouble seeing”,” Journal of General Internal Medicine, vol. 12, no. 2, pp. 125–128, 1997.
View at: Publisher Site | Google Scholar
M. Lundstrom, G. Fregell, and A. Sjoblom, “Vision related daily life problems in patients waiting for a cataract extraction,” British Journal of Ophthalmology, vol. 78, no. 8, pp. 608–611, 1994.
View at: Publisher Site | Google Scholar
J. C. Javitt, M. H. Brenner, B. Curbow, M. W. Legro, and D. A. Street, “Outcomes of cataract surgery,” Archives of Ophthalmology, vol. 111, no. 5, pp. 686–691, 1993.
View at: Publisher Site | Google Scholar
C. M. Mangione, P. P. Lee, J. Pitts et al., “Psychometric properties of the national eye Institute visual function questionnaire (NEI-VFQ),” Archives of Ophthalmology, vol. 116, no. 11, pp. 1496–1504, 1998.
View at: Publisher Site | Google Scholar
J. Lau, J. J. Michon, W.-S. Chan, and L. B. Ellwein, “Visual acuity and quality of life outcomes in cataract surgery patients in Hong Kong,” British Journal of Ophthalmology, vol. 86, no. 1, pp. 12–17, 2002.
View at: Publisher Site | Google Scholar
E. L. Lamoureux, R. M. Maxwell, M. Marella, M. Dirani, E. Fenwick, and R. H. Guymer, “The longitudinal impact of macular telangiectasia type 2 on vision-related quality of life,” Investigative Opthalmology & Visual Science, vol. 52, no. 5, pp. 2520–2524, 2001.
View at: Publisher Site | Google Scholar
D. B Elliott, M. A Hurst, and J. Weatherill, “Comparing clinical tests of visual function in cataract with the Patient’s perceived visual disability,” Eye, vol. 4, no. 5, pp. 712–717, 1990.
View at: Publisher Site | Google Scholar
C. A. Hazel, K. L. Petre, R. A. Armstrong, M. T. Benson, and N. A. Frost, “Visual function and subjective quality of life compared in subjects with acquired macular disease,” Investigative Opthalmology & Visual Science, vol. 31, no. 6, pp. 1309–1315, 2000.
View at: Google Scholar
P. C. Issa, F. G. Holz, and H. P. N. Scholl, “Metamorphopsia in patients with macular telangiectasia type 2,” Documenta Ophthalmologica, vol. 199, no. 2, pp. 133–140, 2009.
View at: Publisher Site | Google Scholar
J. Holman, J. Lazar, J. H. Feng, and J. D’Arcy, “Developing CAPTCHA for blind users,” in Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility, Assets 2007, Tempe, AZ, USA, October 2007.
View at: Google Scholar
C. W. Baker, T. Almukhtar, N. M. Bressler et al., “Macular edema after cataract surgery in eyes without preoperative central-involved diabetic macular edema,” JAMA Ophthalmology, vol. 131, no. 7, pp. 870–879, 2013.
View at: Publisher Site | Google Scholar
I. Perente, Z. Alkin, A. Ozkaya et al., “Focal laser photocoagulation in non-center involving diabetic macular edema,” Medical Hypothesis, Discovery & Innovation Ophthalmology Journal, vol. 3, no. 1, pp. 9–16, 2014.
View at: Google Scholar
L. von Ahn, M. Blum, N. J. Hopper, and J. Langford, “CAPTCHA: using hard AI problems for security,” in Advances in Cryptology, Lecture Notes in Computer Science, vol. 2656, Springer, Berlin, Heidelberg, Germany, 2003.
View at: Google Scholar
S.-Y. Wang and J. L. Bentley, “CAPTCHA challenge tradeoffs: Familiarity of strings versus degradation of images,” in Proceedings of the 18th International Conference on Pattern Recognition (ICPR’06), Hong Kong, China, August 2006.
View at: Publisher Site | Google Scholar
P. T. V. M. de Jong, “Age-related macular degeneration,” New England Journal of Medicine, vol. 355, pp. 1474–1485, 2006.
View at: Publisher Site | Google Scholar
Usage statistics for CAPTCHA, https://www.drupal.org/project/usage/captcha.
D. J. Browning, M. M. Altaweel, N. M. Bressler, S. B. Bressler, and I. U. Scott, “Diabetic Macular Edema: what is focal and what is diffuse?” American Journal of Ophthalmology, vol. 146, no. 5, pp. 649–655.e6, 2008.
View at: Publisher Site | Google Scholar
R. W. Young, “Pathophysiology of age-related macular degeneration,” Survey of Ophthalmology, vol. 31, no. 5, pp. 291–306, 1987.
View at: Publisher Site | Google Scholar
J. M. Evans, “Standards for visual acuity,” 2006, A Report Prepared for Elena Messina Intelligent Systems Division National Institute for Standards and Technology.
View at: Google Scholar
J. Rabin, “Luminance effects on visual acuity and small letter contrast sensitivity,” Optometry and Vision Science, vol. 71, no. 11, pp. 685–688, 1994.
View at: Publisher Site | Google Scholar
B. Rassow, “Influence of luminance on contrast and glare sensitivity in the mesopic region,” Clinical Monthly Sheets for Ophthalmology, vol. 214, no. 6, pp. 401–406, 1999.
View at: Google Scholar
O. Arend, A. Remky, D. Evans, R. Stüber, and A. Harris, “Contrast sensitivity loss is coupled with capillary dropout in patients with diabetes,” Investigative Opthalmology & Visual Science, vol. 38, no. 9, pp. 1819–1824, 1997.
View at: Google Scholar
J. Richman, G. L. Spaeth, and B. Wirostko, “Contrast sensitivity basics and a critique of currently available tests,” Journal of Cataract and Refractive Surgery, vol. 39, no. 7, pp. 1100–1106, 2017.
View at: Publisher Site | Google Scholar
A. S. García, A. G. Gómez, L. C. Figueroa-Ortiz, A. García-Ben, and J. García-Campos, “Relación entre el test de sensibilidad al contraste y el nivel de gravedad en pacientes con esclerosis múltiple,” Archivos de la Sociedad Española de Oftalmología, vol. 89, no. 9, pp. 347–351, 2014.
View at: Publisher Site | Google Scholar
I. Legault, J.-P. Gagné, W. Rhoualem, and P. Anderson-Gosselin, “The effects of blurred vision on auditory-visual speech perception in younger and older adults,” International Journal of Audiology, vol. 49, no. 12, pp. 904–911, 2010.
View at: Publisher Site | Google Scholar
A. Bertone, L. Bettinelli, and J. Faubert, “The impact of blurred vision on cognitive assessment,” Journal of Clinical and Experimental Neuropsychology, vol. 29, no. 5, pp. 467–476, 2007.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2019 Gautam Vangipuram et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

781

Downloads

723

Citations