Abstract

The optical coherence tomography (OCT) is useful in viewing cross-sectional retinal images and detecting various forms of retinal disorders from those images. Image processing methods and computational algorithms underlying this paper try to detect the shadowing region beneath exudates automatically. This paper presents a novel method for detecting hard exudates from retinal OCT images, often associated with macular edema near or within the outer plexiform layer. In this paper, an algorithm can automatically detect the presence of hard exudates in retinal OCT images, and these exudates appear as highly reflective spots. Still, they do not appear as noticeable bright spots because of their minute sizes in predevelopment phases. In the proposed work, we are using a method to detect the presence of hard exudates by analyzing their shadowing effect instead of focusing on brightness spots. The raster scanning operation is performed by traversing the retina horizontally, and noting up any change in normalized summation of brightness intensity (summing up the intensity from top to bottom retinal layers and normalized concerning retinal width) leads to the detection of minute as well as the presence for the detection of large exudates detection by differentiating this brightness intensity graph. The shadow region helps identify the hard exudates; in our proposed method, the output for three input images has been shown. There is an excellent agreement between the results generated by the proposed algorithm and the diagnostic opinion made by the ophthalmologist. The proposed method automatically detects the hard exudates using shadow regions, and it does not need any parameter settings or manual intervention. It can yield significant results by giving the position of shadow regions, which indicates the presence of exudates.

1. Introduction

Optical coherent tomography (OCT) is being widely used nowadays for the detection of several ocular diseases because of its multiple advantages, such as no-contact, no-invasive, real-time imaging, <10 μm axial resolution, cross-sectional image view, and high contrast [1]. The OCT technology uses infrared light to output pseudocolor images formed by varying degrees of light scattering from different retina layers having other refractive and reflective indices. OCT has proven successful in detecting various vitreal interfaces, such as choroidal and retinal, macular edema types, hard exudates, shadow regions, presence of drusen, cotton wool spots, and serous retinal detachment [2, 3]. The current studies involve OCT to diagnose diabetic patients that have reported the problem of fluid accumulation in the macula edema caused by diseases such as age-related macular degeneration (AMD) and diabetic retinopathy (DR). The detection of diabetic macula edema uses the traditional method, which is time-consuming and needs the help of an experienced clinician. Diabetic macula edema occurs in diabetic patients, and screening of such patients in itself is a challenging task. Diabetic patients develop a visual loss in advanced stages until no symptoms are spotted, and the treatment is less efficient. In the early detection stages, hard exudates are the most common lesions. The automated detection module will help clinicians regular independent monitoring in reduced time.

The early clinical sign is important for detecting DME in age-related macular degeneration and diabetic retinopathy. The changes in the retina occur due to various factors such as retinal hemorrhages (red lesions) and dot-blot hemorrhages, which are caused by the bleeding in the inner retinal layer. The retinal exudates (white lesions) include hard exudates (lipid deposition) and soft exudates (also known as cotton wool spots that appear due to ischemia of the nerve fibers). The researchers proposed various image processing algorithms for the DME using fundus imaging and OCT. The methods include morphological operations, segmentation, active contours, thresholding, edge detection, mixture modeling, and support vector machine. The fundus imaging and OCT can be used as promising indications for the early detection of diabetic retinopathy. However, the imaging from the OCT scan is considered to be better due to its high-quality images and its ability to measure the thickness of the retina, which can be helpful for the disease that causes a buildup of fluid, such as in the case of DME.

The earliest retinal OCT imaging technology used for studying neovascular age-related macular degeneration (AMD) was based on first-generation time-domain OCT technology (TD-OCT). TD-OCT is used to measure retinal thickness by measuring the distance from the internal limiting membrane to the highest hyper-reflective band, but it has limitations of speed and sensitivity. To eliminate the drawback of TD-OCT, new technology has been used for retinal thickness measurement, known as the spectral-domain OCT (SD-OCT). SD-OCT has greatly improved speed and sensitivity and can detect small changes in the morphology of the retinal layers and CNV activity in neovascular AMD by setting the threshold value in the retinal pigment epithelium (RPE) hyper-reflective band [46]. Diabetic macula edema (DME) and choroidal neovascularization (CNV) lead to a vision problem in age-related macular degeneration. Figure 1 shows a normal OCT image representing the macula. Choroidal neovascularization is the process of developing new blood vessels in the choroid. It can cause hemorrhage, fluid exudation, and fibrosis, resulting in photoreceptor damage and vision loss. These vessels grow through Bruch’s membrane (BM) and extend into the subretinal pigment epithelial (RPE) or subretinal space. The retinal layers can be automatically located with good accuracy with the aid of local coherence information of the retinal structure. OCT images are processed using the ideas of texture analysis utilizing the structure tensor combined with complex diffusion filtering. Speckle noise removal, enhancement, and segmentation of the various cellular layers of the retinal image were performed. Previously, researchers focused on the OCT analysis in which retinal layer thickness measurement and retinal layer segmentation were done, and comparisons were made with the standard OCT images for making any diagnostic conclusion.

In diabetic patients, macula edema is generally related to the hard exudates. Diabetic macular edema (DME) affects the vision of the retina and is most commonly found in patients who are suffering from the disease known as diabetic retinopathy. The DME can occur at any stage of diabetic retinopathy, but generally, it appears at the moderate nonproliferative stage [710]. Hard exudates can be seen as diagnostic features for diabetic retinopathy, coat disease, and choroidal neovascularization. They appear as highly reflective lesions in OCT images and are found in abundance with a reasonable range of colors, shapes, and sizes. These are lipid and proteinaceous materials leaked from retinal vessels, and their deposition causes significant visual loss when deposited in the foveal region. But unfortunately, hard exudates go untreated in most cases because of the absence of precise treatment guidelines. Hence, they go unresolved. Clinically, the hard exudates are identified by their size greater than 30 µm and their shadowing effect in the outer retina. These exudates block the penetration of the light rays from OCT, thus causing the formation of shadow regions [3]. The “shadowing effect” on the OCT images can be exploited to detect the presence of hard exudates. However, there are some other methods for detecting DME using hard exudates. Some of the techniques are discussed herein in Table 1.

This paper presents a new technique for the automated detection of hard exudates by investigating their shadowing effects. The shadow effect in the OCT image occurs due to a highly reflective surface that produces a blockage and reduces the visibility in identifying the deeper structure of the retinal thickness. Due to the blockage effect, these shadows could play a major role in determining a progressive disease. Few researchers have discussed the shadow’s formation in OCT. MJA Girard et al. [16] have proposed an algorithm to remove the shadow and improve the quality of an OCT image. Camino et al. [17] have used shadow in extracting vessel information. Vupparaboina et al. [18] have proposed a method on ten healthy subjects, including patients suffering from diabetic retinopathy, uveitis, and age-related macular degeneration, to identify a shadow artifact and then compare the performance of the retinal diseases. Most of the reported algorithms for detecting hard exudates used neural network-based techniques that required a large dataset and provided limited accuracy.

Moreover, no algorithm has been reported for the detection of hard exudates using the shadowing effect. The proposed study aims to detect the hard exudates using the shadowing effect in the OCT image so that diabetic macula edema can be diagnosed early. This paper detects hard exudates in macular edema by identifying the shadow region within the layer by using computational and image processing techniques. This paper is organized as follows: Section 2 presents the material and methods. Section 3 discusses the algorithm in detail. The section is subdivided into parts: preprocessing OCT image, marking the upper and lower layer, and detecting hard exudates using shadow regions. Section 4 illustrates the experimental results, and finally, Section 5 concludes the work.

2. Material and Methods

This section focuses on the various steps involved in detecting hard exudates indicating macular edema using OCT images. The mean thickness of the normal retina is generally around 220–280 μm and the foveal depression range from 170 to 190 μm. Edema is the thickening or swelling of the retina’s central part (macula). OCT is widely used in measuring retinal thickness to detect macular edema (ME) caused by diseases such as hereditary retinal degenerations, diabetic retinopathy, retinal vein occlusion, macular degeneration, epiretinal membrane (ERM), and postcataract surgery. Diabetic macula edema is one of the leading causes of visual disability in people with diabetes, and it is a severe complication of diabetes mellitus (DM). The DME causes abnormal increases in thickness of the retinal layers due to abnormal leakage and accumulation of fluid in the macula from damaged blood vessels in the nearby retina. Due to abnormalities in the macula, minute details in vision get affected. Edema generally occurs if the thickening of the retina covers the area around 500 μm or almost near the center. This is an essential aspect because, in most cases of vision loss, it involves the center of the retina, and this vision loss may vary from blur vision to complete vision loss.

This paper presents an automatic image processing-based technique for the detection of hard exudates, which are clinically present within the macular edema [1937]. The block diagram in Figure 2 outlines the method used to detect hard exudates. These hard exudates are difficult to be recognized at the initial stage because of their minute size. Still, as size progressively increases later, it is hard to eliminate them, as exudative plaque is localized within the retina. Therefore, removal becomes difficult. Exudates within subretinal space cause significant visual loss, and it is treated by retinotomy surgery with gentle washout, which involves major surgical complications. However, exudates appear as highly reflective spots. Still, they do not appear as noticeable bright spots because of their minute sizes in predevelopment phases. Hence, we are using an innovative method to detect their presence by analyzing their shadowing effect instead of focusing on brightness. Since several highly reflective bright layers are present in the OCT image, intermixing these layers with exudates cannot yield fruitful results. As per international nomenclature, eighteen anatomic landmarks have been standardized to classify retinal layers. We will use bright layers for classifying our region of interest here.

The upper reflective layer is an internal limiting membrane (ILM). At the bottom end, we have two layers as interdigitation zone and RPE/Bruch’s complex, which often seems merged under moderate resolution conditions as one thick, the highly reflective band [3841]. In retinal OCT images, three brightness levels and six layers are present between the RPE boundary and the ILM layer. The ILM is the boundary between the retina and the vitreous body, formed by astrocytes and the endfeet of Muller cells. It is separated from the vitreous humor by a basal lamina. And the retinal pigment epithelium (RPE) is the pigmented cell layer outside the neurosensory retina that nourishes retinal visual cells and is firmly attached to the underlying choroid and overlying retinal visible cells. Edges of different strengths are formed because of the transitions between these layers. These edges are the different retina layers, and each of them has a specific significance in ophthalmology. The vitreous anterior of the retina is a nonreflective region represented as a dark space. The fovea is the region where thinning of retinal layers is identified. The vitreoretinal interface is the interface between the nonreflective and the backscattering retinal layers. The retinal nerve fiber layer (NFL) is highly reflective. Retinal pigment epithelium (RPE) and choriocapillaris are the posterior boundaries of the retina marked as a hyper-reflective layer. The plexiform and the nuclear layers are medium-reflective, while photoreceptors are low-reflective layers. We need to segment three layers of the retina: the retinal-vitreous interface, retinal pigment epithelium (RPE), and inner-outer photoreceptor segment junction [4245]. Considering above-mentioned layers, the following section depicts algorithm stages along with their associated figures. This method does not need any parameter settings or any human intervention or manual intervention. It can yield significant results by locating the position of shadow regions, which indicates the presence of exudates at these positions. This method was implemented in Python with the help of image processing libraries. The performance of this method is tested on ten different images, and results were checked manually to determine the limitations.

3. Algorithm

This section presents the following image processing steps for hard exudates shadow region detection within the retinal layers.

3.1. Image Preprocessing

To get the position of shadow regions within retinal layers, first of all, find the upper and lower layers of the retina. Then, we can get differentiated intensity regions within these two layers. These operations can better be performed on binary images filtered with noise. To prepare for these, the following steps are performed:(a)Initially, the original image is taken, which is generated from the OCT machine. Then, this image is converted into grayscale to obtain image , which is shown in Figure 3 with the labeling of the presence of hard exudates.(b)Histogram equalization is performed on the grayscale image () for contrast enhancement of the image, and furthermore, Otsu’s binarization is applied to convert the image into a binary image (Ib). Otsu’s binarization is applied to automatically detect the threshold value without manually inserting it for different images having different brightness levels.(c)Opening and closing operations are performed on the binary image (Ib) by taking a small -pixel window. These operations are applied to remove small noise and fill small holes so that the image becomes clear, as shown in Figure 4.

Instead of converting them to grayscale, we can even extract a single layer or a combination of two layers from an RGB image to obtain a binary image (Ib). Some particular color layers may show more prominent features than other layers for further processing.

3.2. Finding Upper Bright Layer of Retina (ILM Layer)

This step aims to mark the topmost layer in the OCT image, which is referred to as the ILM layer mentioned earlier. Following operations are performed on the grayscale image , which is illustrated as:(a)Raster scanning is done in column 1 (leftmost) of the image, starting from the top. When a white pixel appears in Ib, it is marked (let’s say P1). From P1, a small square ( pixels) is taken to the left of P1, used to find connected pixels. Near most pixel is marked if it lies in that selected window. The same operation is performed for the next marked pixel. Likewise, the chain of the connected pixel is plotted.(b)If any pixel is not found in that window, it indicates that the layer is broken here, and we need to start afresh to join the chain further. For this, leaving a gap of a few columns, raster scanning is again performed from the top to find a pixel of this top layer [42, 46]. If located, then the last marked pixel is connected to this, and our chain is further propagated likewise as in step a. If the pixel is not again found within some top and bottom margin of the last marked pixel, the next column is taken to repeat the same until some valid pixels of the top layer are found. The chain is connected until it reaches the rightmost limit of the image. An adequately connected chain overlapped on Ii is shown in Figure 5. Let this connected chain of pixels be taken as Ct.

3.3. Finding Lower Bright Layer of Retina (RPE Layer)

This step aims to mark the most hyper-reflective layer of the retina, which is the RPE layer. This lies approximately at the bottom, and we aim to limit our region of the search for shadow regions. Following steps involved in doing so:(a)Previous filtered binary images cannot be used here as it is required to do binarization to separate high-intensity pixels of this layer. This is done to do away with interference that may have been created because of the low-intensity cloud-like structure below the RPE layer. Hence, a new binary image Ib2i is created by taking a greater threshold value to get pixels in the uppermost intensity band. Threshold may also be determined by getting the image histogram and the high-intensity peak value of the histogram.(b)Opening and closing operations are performed on image Ib2i to get Ib2 similar to Step I to remove small pixel groups. This is shown in Figure 6.(c)To begin with, first rightmost pixel is taken from image as this is not the first pixel from bottom (as used in finding top layer); instead, it is the pixel from most intensity pixel group. To mark this layer, an approach similar to Step II is used here to connect pixel chain by first raster scanning from bottom to top in Ib2 and joining neighborhood pixels. But this lower layer seems to be much more irregular than the top layer, as shown in Figure 3. We can use raster scanning here when breakage occurs because many large white pixel groups occur under RPE and RPS regions are not clean as above-top layer region; hence, raster scanning will be obstructed.(d)Because of the presence of shadow regions, breaks in this layer occur very frequently, but we cannot say that break in this layer indicate the presence of shadow regions as this would not be accurate, and also the width of shadow regions cannot be determined from this shadow length, which is obvious from Figure 3. Now whenever the connected chain of pixels fails to find a white pixel in the neighborhood window, then the slope of this chain is determined (which is done by taking a pixel coordinate of a few pixels back of chain array and then determining slope from the current pixel and this pixel) and next pixel is predicted by getting an estimate from the slope. This pixel is projected at a gap of a few columns toward the left as projecting in the adjacent column will permanently mark an adjacent pixel of the same row. Suppose a white pixel is already present at the neighborhood of that pixel (minimum Euclidean distance from projected pixel). In that case, that pixel is marked to connect chain else, the projected pixel is added in chain, and the same further process is repeated to propagate chain up to a rightmost point. This connected chain is marked in Figure 7 along with C1, and this can be referenced as C2. Because of using this approach, we can see a few sharp turns in this chain.

3.4. Finding Normalized Intensity Curve

Wherever exudates occur, there is the presence of shadow under them as these obstruct waves of OCT instruments and hide the region behind them. Hence, determining the shadow region is the core idea of this paper. Shadow represents less intensity, so if we determine intensity change, shadow will be identified. This step aims to plot the intensity summation of the pixel between two layers in a normalized mode as follows:(a)From image , the intensity is summed up for each column from top to bottom for only those pixels, which lie between coordinates of C1 and C2 at that particular column. But before that coordinates of C1 and C2 are filled up for all columns. This is because C1 and C2 were until now storing only marked pixel and not intermediate chain pixels.(b)This intensity summation array needs to be normalized concerning the gap between the two layers (ILM and RPE). The minimum and maximum gaps between the two layers are marked in Figure 7. The intensity normalization is an important step because the changes in the gap between the layers will result in the exact proportional change in intensity summation. That could lead to detecting false exudates due to intensity changes. So, we need to consider these two layers as parallel, and for that, normalization of intensity along each column will suffice. For normalization, for each intensity summation along the horizontal axis, the summation value is multiplied by the maximum gap between two layers C1 and C2, and divided by the gap between these two layers at that particular horizontal location. This normalized intensity plot is shown in Figure 8.(c)The normalized intensity curve needs to be smoothed out to filter out small sharp peaks. These small differentiated peaks or fluctuations are not because of the presence of exudates instead, and they may be due to discontinuity in C1 or C2 or little extra white pixels. Convolution of an array of normalized intensity summation is performed to result in the smooth plot as shown in Figure 9. All points of the plot are stored in array Ai.

3.5. Finding Shadow Regions from Gradient Change of Smooth Normalized Intensity Summation Plot

In this final step of the algorithm, shadow regions are located by determining the change of intensity with some extra manipulations to filter out erroneous detection. Following are the steps:(a)From a smooth normalized intensity plot, differentiation is plotted by subtracting two nearby intensity values and dividing them by the horizontal gap. This plot is plotted in Figure 10. With differentiation, we can find out peak that depicts the change in intensity summation from high to low or low to high wherever shadow region occurs due to the presence of exudates. A threshold value is taken, above which if the absolute value of differentiation plot lies, that may be treated as the beginning or end of shadow region. Such points are stored in an array termed as As. It is also to be noted that many points nearby will be crossing the threshold, but if they are close enough, they are clubbed together to form a single point. All such points of Ad are marked as green dots in Figure 11.(b)Only start and endpoints of exudates regions have been located but not the areas between them. However, we cannot simply predict that whole region between two consecutive points is of Ad as the shadow region. This may happen because the start point is the end point of the previous region whose start point was not detected. This may happen that the end point detected is the start point of the next shadow region, and the corresponding start point detected is the end point of the previous region or may have been falsely located. Many such combinations may present, but to give a foolproof ability to algorithm, this algorithm performs several checks and manipulations to do away with this.(c)One solution applied here is that whenever a negative peak appears in the differentiation curve, the intensity summation value (from Ai) is stored before that peak location. Now we move along the intensity curve (instead of the differentiation curve) to find next such nearby intensity summation value. When we locate this, then this indicates the end of the shadow region (as getting same intensity summation value after shadow region ends), but this needs to be additionally verified with differentiations curve positive peak. Again, this indicates the end of the shadow region. If we do not find the next positive peak or nearby intensity summation value or both, this start of shadow region is discarded as this would have been false. Apart from this, several other manipulations are involved, which will be detailed here. The final image with shadow portions marked in yellow is shown in Figure 11, and differentiation peaks are observed in green dots.

4. Results

The proposed automated algorithm is used to automatically detect the hard exudates that indicate the presence of macular edema’s by localizing the positions of the shadow region within the retinal layers. The proposed method is very resilient and versatile and can be applied to various images taken from various sources. Due to the nonavailability of the standard dataset, we have collected a few images from different sources that would have been captured from different OCT instruments. They were tested for the proposed algorithm subjected to the tweaking of a threshold value for classification, and results have been illustrated in Figures 12(a) to 12(c). Each resultant image is a stack of four individual images to give clarity of algorithm stages. The topmost image shows the original input image. The second image shows the top inner limiting membrane layer and retinal pigment epithelium layer of retina classification, along with the minimum and maximum gap between layers used for intensity normalization concerning the gap between these two layers. The third image from the top shows a mix-up of the smooth intensity summation graph and differentiation graph. The bottom-most image shows an output image where the marker has been placed with hard exudates/drusen underneath it. The reader can make a correlation between differentiation absolute peaks and marker placed. These output images are still not perfect and can miss a few exudates or detect a very few false positions, but they provide a futuristic solution to such problems. Exact exudate’s location along the y-axis and accurate detection is a problem for us on which we are working. These outputs have been validated with ophthalmologists, and major agreements have been found between automatic and manually marked exudates. Providing any efficiency parameter or any index of severity is not possible for us for a small dataset. The limitation of the small dataset is also found in classifying shadow detection for other pathologies, and we would not ignore the fact that this method would identify other minute diseases that may cause shadowing effects. This may be considered either advantage or disadvantage of this method. However, the subclassification of different pathologies is in the scope of future work. This may be done either through proper thresholding or shape-based classification or through other dependency parameters that can be detected in OCT images responsible for various shadow-causing pathologies.

Figures 12(a)12(c) illustrate intermediate steps to detect the hard exudates: (i) input image, (ii) marking of ILM and RPE layers with the minimum gap and maximum gap, (iii) the combination of intensity curve and differentiation curve, (iv) the final output image.

5. Conclusion

In this paper, we have proposed an algorithm to automatically detect the hard exudates that indicate the presence of macular edema’s by localizing the positions of the shadow region within the retinal layers. During the process, several iterations were performed to find the actual layer, that is, inner limiting membrane and retinal pigment epithelium, using the neighborhood search approach. Furthermore, it has been observed that there are certain constraints in extracting the details between the layers. First, removing information about the layers is challenging, particularly for the low contrast area. Second, it is sensitive to the inner limiting membrane and retinal pigment epithelium layer boundaries. Finally, we have analyzed the intensity summation gradient change between two hyper-reflective layers along the horizontal axis through thresholding to find the shadow region, which shows the presence of hard exudates. Future work includes developing retinal OCT image datasets of patients suffering from DME; the dataset will be used to develop algorithms for other diagnostic feature extractions/detections, which will allow further subclassification of DME diagnosis.

Data Availability

The data are available upon request to the authors.

Ethical Approval

This article does not contain any studies with human participants. No animal studies were involved in this review.

Conflicts of Interest

The authors declare no conflict of interest.

Authors’ Contributions

All authors contributed equally to this work. In addition, all authors have read and approved the final manuscript and given their consent for publication of the article.

Acknowledgments

A small part of this paper appears in https://een.ec.europa.eu/tools/services/EVE/Event/DownloadAttachment?attachmentID=124c0677-99fb-4b13-9488-9f215c46ed6a (CARS 2019 Computer Assisted Radiology and Surgery 33rd International Congress and Exhibition Le Couvent des Jacobins, Rennes, France, June 18–21, 2019 http://www.cars-int.org/http://www.cars2019.org). This study was not funded by an internal, external, or international agency, institute, or organization.