Abstract
Purpose. To evaluate macular thickness, agreement, and intraclass repeatability in three optical coherence tomography (OCT) devices: the time domain (TD) Stratus OCT and two spectral domain (SD) OCTs, Spectralis and Cirrus SD-OCT, in eyes with macular edema secondary to diabetic retinopathy (DR) and retinal vein occlusion (VO). Methods. In a prospective observational study at a university-based retina practice, retinal thickness tomography was performed simultaneously for fifty-eight patients (91 eyes) with DR and VO employing a time domain and two spectral domain OCTs. Agreement in macular measurements was assessed by constructing Bland-Altman plots. Intraclass repeatability was assessed by intraclass correlation coefficients (ICCs). Results. Based on the Bland-Altman plots for central macular thickness, there was low agreement between the measurements of Cirrus SD-OCT and Stratus OCT, Spectralis OCT and Stratus OCT, as well as Spectralis OCT and Cirrus SD-OCT among DR and RVO patients. All three devices demonstrated high intraclass repeatability, with ICC of 98% for Stratus OCT, 97% for Cirrus SD-OCT, and 100% for Spectralis OCT among DR patients. The ICC was 97% for Stratus OCT, 79% for Cirrus SD-OCT, and 91% for Spectralis OCT among RVO patients. Conclusion. There are low agreements among interdevice measurements. However, intraclass repeatability is high in both TD and SD-OCT devices.
1. Introduction
Automated measurements of retinal thickness and volume using optical coherence tomography (OCT) and imaging-processing software are commonly used in the diagnosis and management of retinal diseases such as diabetic retinopathy and age-related macular degeneration [1–4]. Time domain Stratus OCT (Carl Zeiss Meditec, Inc., Dublin, CA, USA) is the most widely used device both in clinical and research settings. Stratus OCT acquires images at a rate of 400 axial scans per second with an axial resolution of 10 μm, and an ability to generate measurements with high repeatability [1]. The new generation of OCT devices employing spectral domain technology provides an axial resolution of approximately 3 μm and improves visualization of retinal normal anatomy as well as microstructural changes in the neurosensory layer [5–8]. There are some differences in the intrinsic software algorithms that each device applies to calculate retinal thicknesses. A thickness map is calculated based on the data collected from an array of A-scans distributed across the macula. The number of A-scans used in Stratus OCT is fewer and weighted more towards the center of the scan. The method to correct for this effect, as well as different anatomical landmarks that each device uses to specify the outer retinal boundary, contribute to a different absolute value for thickness measurement [1].
There are similar differences among various spectral domain OCTs. These differences highlight the importance of repeatability testing for each device. The evaluation of association among thickness measurements by different devices is also an important issue to consider prior to using them interchangeably in clinical and research setting. The repeatability of macular measurements by time domain Stratus OCT and spectral domain Cirrus SD-OCT (Carl Zeiss Meditec, Inc.) has been tested in the context of diabetic macular edema (DME) [1]. However, to our knowledge, the repeatability of macular measurements obtained by Spectralis HRA/OCT (Heidelberg Engineering, Inc., Heidelberg, Germany) has not been determined in patients with different maculopathies [9]. The goal of the index study is to evaluate the repeatability and agreement of macular measurements by time domain Stratus OCT versus spectral domain Spectralis OCT and Cirrus SD-OCT in two retinal vascular diseases, diabetic retinopathy and retinal vein occlusion.
2. Materials and Methods
Patients who had macular edema secondary to diabetic retinopathy (DR) and retinal vein occlusion (RVO), which was confirmed with fundus photography as well as fluorescein angiography, were eligible to be enrolled in the study. The diagnosis of macular edema secondary to DR or RVO was made by the two retina specialists (QDN and DVD). Eyes with other conditions resulting in macular thickening such as epiretinal membrane, vitreal macular traction, and neovascular age-related macular degeneration were excluded. Research design and patient deidentification were Health Insurance Portability and Accountability Act (HIPAA) compliant. Institutional Review Board (IRB) approval from the Johns Hopkins Medical Institutions was obtained.
All study subjects were scanned with each of the three OCT machines in random order on a single visit by a single operator. Macular thickness measurements of nine standard subfields were analyzed. The fast macular thickness map protocol was used to scan patients with the Stratus OCT. Within a scan time of 1.9 seconds, the device scans six evenly spaced 6 mm radial lines consisting of 128 A-scans per line that intersect at the fovea (total of 768 sampled points) [1]. The scan pattern was applied to obtain scans with Cirrus SD-OCT. In a scan time of 2.4 seconds, the device scans a mm area of the retina with 128 horizontal lines, each consisting of 512 A-scans per line (total of 65,536 sampled points) [1].
The Spectralis OCT has a transverse (in tissue) resolution of 14 microns (μm) and an axial (in tissue) resolution of 3.9 μm. With a speed of 40,000 A-scans per second, the device scans a mm area ( degrees) of the retina with 19 horizontal raster scans and 240 μm between every two lines, each line consisting of 428 A-scans (total of 8,143 sampled points). The thickness values in between the lines are calculated using sophisticated interpolation algorithms. The device employs its speed to rescan each line several times and averages the data points acquired (this is only possible with the Spectralis OCT because it is able to track the lines and change the scan position according to the eye movements, automatically correcting for motion artifacts) (information obtained from the Spectralis HRA/OCT Manual).
We obtained two high-quality scans with each device. Scans without artifacts caused by eye movement and papillary shadowing were selected as the high-quality scans on Spectralis OCT as well as Stratus OCT and Cirrus SD-OCT. We did not exclude any poor scans due to media opacity and those without clear delineation of the retina layers. Such strategy provided us data to compare scans taken by different devices on eyes of different characteristics—those that were and were not ideal to undergo high-quality scan. We compared the scan results on different devices based on the thickness of the macula. During OCT scanning, we used an intrinsic fixation target to center the macular grid, along with manual centering of the grid by the operator for those images for which the grid outside fovea was not allowed.
To calculate retinal thickness, we defined an internal and external retinal layer position by using the intrinsic retinal segmentation algorithms in each device. All three devices used the protocol from the Age-Related Eye Disease Study (AREDS) [10] to average retinal thickness within nine retinal subfields in a 6 mm diameter circle centered on the fovea. Retinal subfields were reported as central, inner superior, inner nasal, inner inferior, inner temporal, outer superior, outer nasal, outer inferior, and outer temporal. For Cirrus SD-OCT and Spectralis OCT, when OCT device defined the retinal segmentation algorithms on an incorrect anatomical site, we corrected it manually and separately reported the values after correction.
To make pair-wise comparisons of nine subfield retinal measurements between each pair of two devices, we used the two-tailed paired t-test with Bonferroni adjustment for multiple comparisons. By constructing Bland-Altman plots, we assessed the agreement in macular measurements between each device [11]. Intraclass repeatability was assessed by intraclass correlation coefficients (ICCs) for each of the two measurements of each patient on each of the three devices. We calculated the coefficient of repeatability (CRW) to assess and compare the intrasession repeatability of the three devices. CRW is defined as 1.96 * SW, where SW stands for the intrasession within-subject standard deviation [1].
3. Results
Fifty-eight patients (91 eyes) were included in this study: 28 patients (61 eyes; seven patients had more than one visit, and the eyes were scanned and considered independently at each visit) had macular edema secondary to DR in one or both eyes, and 30 patients (30 eyes) had macular edema from RVO in one eye. The age range among patients with DR was 38.3 to 81.9 years (median 69.5 years) and 15 (53.6%) of them were male. The age range among patients with RVO was 40.6 to 87.8 years (median 68.3 years) and 20 (66.7%) of them were male. Central retinal vein occlusion (CRVO) was diagnosed in 13 (43.3%) of the patients and the remaining were diagnosed with branch retinal vein occlusion (BRVO).
3.1. OCT Characteristics among DR Patients
The mean central macular thickness was 289 μm (standard deviation [SD]: 99 μm) on Stratus OCT device, 323 μm (SD: 108 μm) on Cirrus SD-OCT and 344 μm (SD: 116 μm) on Spectralis OCT. These values changed to 322 μm (SD: 108 μm) on Cirrus SD-OCT and 340 μm (SD: 108 μm) on Spectralis OCT after manual correction of algorithm. Table 1 presents the mean values and SDs for nine subfields on the three OCT devices as well as after-correction values for Cirrus SD-OCT and Spectralis OCT devices. The mean central macular thickness value was 21 μm (95% confidence interval (CI): 9, 33 μm) higher on Spectralis OCT compared to Cirrus SD-OCT and 61 μm (95% CI: 51, 72 μm) higher on Spectralis OCT compared to Stratus OCT. The differences decreased to 17 μm (95% CI: 5, 28 μm) between Spectralis OCT and Cirrus SD-OCT and to 58 μm (95% CI: 45, 70 μm) between Spectralis OCT and Stratus OCT values after manual correction of algorithms. The mean central macular thickness was 41 μm (95% CI: 29, 53 μm) higher on Cirrus SD-OCT compared to Stratus OCT. The difference was the same (41 μm, 95% CI: 28, 53 μm) after manual correction of algorithms. Table 2 shows the mean difference and 95% CIs between retinal thickness values on the three OCT devices for nine different subfields before and after manual correction of algorithms. The Bland-Altman plot showed a poor agreement for central macular thickness measurements between each device before and after manual correction of algorithms (Figure 1). There was an ICC of 98% (95% CI: 97, 99%) among two measurements of central thickness values for each eye on the Stratus OCT device. ICC was reported as 97% (95% CI: 96, 99%) on Cirrus SD-OCT and 100% (95% CI: 99, 100%) on Spectralis OCT for central thickness values before manual correction of algorithms and 97% (95% CI: 96, 99%) on Cirrus SD-OCT and 99% (95% CI: 99, 100%) on Spectralis OCT for central thickness values after correction. Table 3 shows the ICC for retinal thickness values on the three OCT devices for nine different subfields. For Cirrus SD-OCT and Spectralis OCT values, ICC is reported after manual correction of algorithms as well. For central thickness values on Stratus OCT, CRW was reported as 27 μm. CRW was 36 μm before and after manual correction of algorithms on Cirrus SD-OCT and 5 μm before and 18 μm after manual correction of algorithms on Spectralis OCT. Table 4 presents CRWs for retinal thickness values on the three OCT devices for nine different subfields. For Cirrus SD-OCT and Spectralis OCT values, CRW is also reported after manual correction of algorithms.
(a)
(b)
(c)
(d)
(e)
(f)
3.2. OCT Characteristics among RVO Patients
The mean central macular thickness was 283 μm (SD: 149 μm) on the Stratus OCT device, 315 μm (SD: 156 μm) on Cirrus SD-OCT, and 359 μm (SD: 198 μm) on Spectralis OCT. The value changed to 353 μm (SD: 241 μm) on Cirrus SD-OCT and 376 μm (SD: 245 μm) on Spectralis OCT after manual correction of algorithm (Table 1). The mean central macular thickness value was 47 μm (95% CI: 7, 86 μm) higher on Spectralis OCT compared to Cirrus SD-OCT, and 76 μm (95% CI: 48, 104 μm) higher on Spectralis OCT compared to Stratus OCT. The difference decreased to 26 μm (95% CI: 13, 38 μm) between Spectralis OCT and Cirrus SD-OCT and 93 μm (95% CI: 45, 142 μm) between Spectralis OCT and Stratus OCT values after manual correction of algorithms. The mean central macular thickness was 31 μm (CI: 1, 64 μm) higher on Cirrus SD-OCT compared to Stratus OCT. The difference increased to 68 μm (95% CI: 20, 117 μm) after manual correction of algorithms (Table 2). There was poor agreement for central macular thickness measurements between each device before and after manual correction of algorithms (Figure 2). There was an ICC of 97% (95% CI: 95, 99%) among two measurements of central thickness values for each eye on the Stratus OCT device. ICC was reported as 79% (95% CI: 66, 93%) on Cirrus SD-OCT and 91% (95% CI: 84, 97%) on Spectralis OCT for central thickness values before manual correction of algorithms, and 99% (95% CI: 99, 100%) on Cirrus SD-OCT and 100% (95% CI: 100, 100%) on Spectralis OCT for central thickness values after correction (Table 3). For central thickness values on Stratus OCT, CRW was reported as 54 μm. CRW was 146 μm before and 30 μm after manual correction of algorithms on Cirrus SD-OCT and 119 μm before and 7 μm after manual correction of algorithms on Spectralis OCT (Table 4).
(a)
(b)
(c)
(d)
(e)
(f)
4. Discussion
Our study is one of the inaugural investigations comparing the OCT findings in patients with macular edema from DR and RVO between a time domain OCT, Stratus OCT, and two spectral domain OCTs, Spectralis OCT and Cirrus SD-OCT [1, 9]. The SD OCTs reported higher values for macular thickness in all nine standard subfields in DR and RVO patients compared to TD Stratus OCT. The values obtained from the two spectral domain OCTs remained higher after manual correction of algorithms. Spectralis OCT detected higher macular thickness in all nine standard subfields compared to Cirrus SD-OCT. The same difference was detected by Forooghian et al. [1] comparing Cirrus SD-OCT to Stratus OCT, as well as by Lammer et al. [9] comparing Spectralis OCT, Cirrus SD-OCT, and Stratus OCT in patients with DME.
The discrepancy among the three devices in terms of macular thickness values reflects differences in defining retinal segmentation algorithms. While Stratus OCT measures the thickness of the retina as the distance between the inner limiting membrane (ILM) and junction of the outer segment (OS) and inner segment (IS) of the photoreceptors, Cirrus SD-OCT reports it as the distance from the anterior border of the retinal pigment epithelium (RPE) to the ILM [1], while Spectralis OCT measures the distance from the posterior border of the RPE to the ILM. Therefore, the macular measurements are larger on Spectralis OCT compared to Cirrus SD-OCT and Stratus OCT. The mean difference between macular thickness measurements obtained by the Cirrus SD OCT and Stratus OCT in nine different subfields was close to 50 μm in most of the subfields, which is the length of the outer segment of human photoreceptors [1, 12]. Bland and Altman invented the method of evaluating agreement between measurements of two devices by plotting their difference against their mean. The measurements can be used interchangeably when the 95% CI of agreement is within a clinically acceptable range [11]. Figures 1 and 2 present the Bland-Altman plots for the central subfield values on different devices. The 95% CIs for agreement between each two machines in DR and VO patients were very wide. The 95% CIs remained very large after manual correction of the algorithms. Therefore, macular thickness measurements obtained with each of the three devices cannot be used interchangeably with the measurements from the other device due to poor agreement among devices [1, 9].
We evaluated the repeatability of the macular thickness measurements of each device by performing two series of images with each device on each patient. The ICC was reported high for all the nine subfields on all three devices. ICC remained high after manual correction of algorithms for measurements by Cirrus SD-OCT and Spectralis OCT. Although different methods were applied to measure the repeatability of each OCT device, our results were consistent with other reports on Stratus OCT [13, 14], as well as Cirrus SD-OCT [1] and Spectralis OCT [9] for patients with DR.
Thus far, there have not been any published reports on the repeatability of measurements of macular edema in RVO. ICC was reported less than 90% on Stratus OCT, Cirrus SD-OCT, and Spectralis OCT for some of the segments (81–100% among DR patients and 79–100% among those with RVO) before and after manual correction of algorithms. The low reported ICC could be due to chance. There is no physiological or anatomical basis to explain the difference in macular thickness measurements in these segments compared to the rest of the scanned area [1]. Since the 95% CI for all these measurements is wide and greater than 90%, the low ICC could be due to some outliers in the measurements for ICC.
The CRW was reported as less than 40 μm for Stratus OCT in DR patients and less than 55 μm in RVO patients. Such index presents the half-width for the 95% CI in the thickness measurement variation from one OCT measurement to another for each patient. Our finding is similar to the Diabetic Retinopathy Clinical Research Network report on Stratus OCT in DME patients [2]. The CRW ranged from 11 μm to 52 μm for Cirrus SD-OCT in DR patients before and after manual correction of algorithms. It ranged from 10 μm to 146 μm before correction to 10 μm to 49 μm after correction of algorithms in RVO patients. The wide range of CRW before manual correction of algorithms reflects the inability of the device to recognize anatomical landmarks and define algorithms in the presence of extensive macular edema. The CRW was large for patients with ≥800 μm in macular thickness, for which Cirrus SD-OCT was not able to recognize anatomical landmarks. The CRW range decreased dramatically after manual correction of algorithms.
The CRW ranged from 4 μm to 20 μm for Spectralis SD in DR patients before and from 7 μm to 39 μm after manual correction of algorithms. It ranged from 9 μm to 119 μm before correction to 7 μm to 28 μm after correction of algorithms in RVO patients. The same problem of not being able to recognize anatomical landmarks and define algorithms in the presence of extensive amount of macular edema was detected in Spectralis OCT. The CRW range decreased dramatically after manual correction of algorithms.
The current study provides valuable information regarding the performance of different OCT devices in macular edema from two different types of retinal vascular diseases. To the best of our knowledge, our OCT analysis in DR patients is one of the few reports with a large sample size on the comparison of SD OCTs among each other as well as with the TD OCT. Our analysis in RVO patients is the first report of SD OCTs in patients with RVO. The index study has evaluated the performance of OCT devices in a real-time clinic without omission of impaired images due to inability of the device to recognize anatomical landmarks. Unlike other reports on this subject, we also reviewed all images taken with SD OCTs on different types of patients and ocular and media condition, and made manual corrections of algorithms when necessary. Therefore, the index study also presents the performance of each device in actual, clinical situation where there are protean factors that may affect quality of scans.
5. Conclusions
Although each OCT device has a unique method of defining algorithms and cannot be used interchangeably, each device has the ability to measure the retinal thickness accurately and repeatedly. The advantage of one SD OCT over the other or to a TD OCT might be in its ability to provide images with more details of different retinal layers and detect small changes in the normal anatomy of retina. Additional studies to evaluate the quality of images by each device to define subtle anatomical changes in different layers of retina are necessary to make informed decision to employ which of the available OCTs in a specific clinical setting.
Authors’ Contribution
Dr. E. Hatef, Dr. A. Khwaja, and Mr. Z. Rentiya have contributed similar amount of efforts toward the conduct of the index study and serve as cofirst authors.
Acknowledgments
Research discussed in this paper was presented in part at the Annual Meeting of the Association for Research in Vision and Ophthalmology (ARVO), Fort Lauderdale, FL, USA in 2010. Dr. Q. Nguyen serves on the Scientific Advisory Board for Heidelberg Engineering, Inc. The Johns Hopkins University has received research support from Heidelberg Engineering, Inc. However, Heidelberg has had no inputs in the design, conduct, or analyses of the index study. None of the other authors has a proprietary interest in this paper.