Abstract
Objective. To evaluate NOVA View with focus on reading archived images versus microscope based manual interpretation of ANA HEp-2 slides by an experienced, certified medical technologist. Methods. 369 well defined sera from: 44 rheumatoid arthritis, 50 systemic lupus erythematosus, 35 scleroderma, 19 Sjögren’s syndrome, and 10 polymyositis patients as well as 99 healthy controls were examined. In addition, 12 defined sera from the Centers for Disease Control and 100 random patient sera sent to ARUP Laboratories for ANA HEp-2 IIF testing were included. Samples were read using the archived images on NOVA View and compared to results obtained from manual reading. Results. At a 1 : 40/1 : 80 dilution the resulting comparison demonstrated 94.8%/92.9% positive, 97.4%/97.4% negative, and 96.5%/96.2% total agreements between manual IIF and NOVA View archived images. Agreement of identifiable patterns between methods was 97%, with PCNA and mixed patterns undetermined. Conclusion. Excellent agreements were obtained between reading archived images on NOVA View and manually on a fluorescent microscope. In addition, workflow benefits were observed which need to be analyzed in future studies.
1. Introduction
The antinuclear antibody (ANA) test is a standard screening assay for detecting multiple antibodies that may be produced by a patient with an autoimmune or ANA associated rheumatic disease (AARD). Although there are several methodologies available to screen ANA, the American College of Rheumatology (ACR) issued a statement declaring HEp-2 indirect immunofluorescence (IIF) as the preferred method for ANA screening [1]. This declaration was based on the findings of a task force which investigated and collected information from physicians to evaluate nonstandardization of the various methodologies on the market for evaluating ANA. Using HEp-2 as a substrate allows the detection of more than 100 autoantibodies to different nuclear and cytoplasmic antigens [2]. These include antibodies to dsDNA, SS-A, Ro52, SS-B, RNP, centromere, Scl-70, Jo-1, ribosomal P, RNA Polymerase III, PM/Scl, Ku, Th/To, and Mi-2 to term the most important antigens. There are 5 to 6 indirect immunofluorescence (IIF) nuclear patterns that are commonly reported by most laboratories, namely, homogeneous, speckled, nucleolar, centromere, peripheral/rimmed, and proliferating cell nuclear antigen (PCNA). Laboratories performing the ANA IIF test typically report a positive result with a pattern and titer. This aids the physician when deciding what tests to order next, if any.
Performing IIF test is labor intensive, subjective, and prone to reader bias [3–7]. Many other variables affect the IIF test result such as the HEp-2 substrate, conjugate, microscope, type of bulb, and bulb life [2, 8–14]. Over the past few decades as newer technologies emerged for testing ANAs, there were fewer and fewer laboratorians with the knowledge and skill to perform ANA IIF testing. As a reference laboratory, ARUP continues to offer and perform HEp-2 IIF for ANA testing. Extensive time is required to train a technologist to be competent in reading and interpreting ANA IIF testing. In addition, there is a need for standardization and automation in ANA testing [1–3, 15].
Autoimmune laboratories have made strides in automation over the past decade but are still far behind other areas of the laboratory, such as chemistry with their fully automated instrumentation. Automated pattern interpretation of HEp-2 ANA was first described in 2002 by Perner et al. [16] Since then, there have been several studies of automated or digital IIF interpretation for positive and negative discrimination. Some systems incorporate pattern recognition algorithms. All conclude that automated IIF analysis will improve inter- and intralaboratory results [17–25]. The NOVA View instrument (INOVA Diagnostics, Inc., San Diego, CA) has been designed to address this need. NOVA View is an automated digital image analysis system, which is used for acquiring, analyzing, and interpreting ANA testing on HEp-2 cells, based on measured Light Intensity Units (LIU) and pattern recognition. NOVA View results are expressed in LIU and interpreted as negative or positive based on a preset cutoff. The cutoff intensity is preset by INOVA and may be adjusted for the customer based on their patient population and performance goals. The patented process produces three to five images per patient sample. The automated scan is followed by visual verification of the digital images, allowing for either confirmation or revision of results by the operator. NOVA View software recognizes five basic patterns: homogeneous, speckled, centromere, nucleolar, and nuclear dots. Pattern recognition is based on a software algorithm that analyzes the intensity and distribution of the fluorescent light over the area of the nuclei based on specific criteria. Mixed patterns may not be recognized by the software and may be reported as “unrecognized.” In these cases the final pattern is determined by the user during the revision and confirmation of the digital images.
Based on the recommendation of the ACR for the use of HEp-2 IIF to test for ANA, we aimed to compare the agreement of the NOVA View archived images to the interpretation of the same samples on a manual fluorescent microscope interpreted by a certified medical technologist, with emphasis on agreement of end point titer. In addition, the data were used to calculate ANA titers and positivity rate in various AARD.
2. Materials and Methods
2.1. Clinical Samples
Clinically defined serum samples from patients suffering from SLE (), rheumatoid arthritis (RA, ), SSc (), Sjögren’s syndrome (SjS, ), and polymyositis (PM, ) were included. Diagnoses were established as previously described or according to the respective disease classification criteria [26]. In addition, 99 healthy adult donor sera which consisted of 70% female and 30% male between the ages of 19 to 59 years of age were tested.
2.2. CDC ANA Reference Panel
International reference serum panel (CDC ANA #1–12) was obtained from the Centers of Disease Control and Prevention (CDC) (http://asc.dental.ufl.edu/ReferenceSera.html). (Biological Reference Reagents, NCID/SRP/BRR, Mailstop C-21, Centers for Disease Control and Prevention (CDC), 1600 Clifton Rd. N.E., Atlanta, GA, U.S.A).
2.3. Consecutive Routine Samples
Lastly, 100 consecutive samples from an individual client, sent to ARUP Laboratories for ANA IIF testing, were reviewed. All patient samples included in the study were deidentified according to the University of Utah Institutional Review Board-approved protocol number 7275 to meet the Health Information Portability and Accountability Act Patient Confidentiality Guidelines.
2.4. Microscopes and Indirect Immunofluorescence Reagents
NOVA Lite HEp-2 IgG ANA with DAPI kit and the NOVA View instrument with 1.0.2 software containing a cut-off value of 100 LIU for positive results (INOVA Diagnostics, San Diego, CA). The conjugate used in this assay contains the usual FITC fluorophor along with diamidino-2-phenylindole (DAPI), a blue nuclear stain that selectively binds to double stranded DNA. DAPI allows the instrument to “find” the cells at a 400 nm wavelength. If the cell density is insufficient or there are no cells in the well, the instrument will not switch to FITC but will produce an “,” indicating an inadequate number or that no cells were found. Once the correct number of cells has been identified, the instrument switches to a 490 nm wavelength for FITC identification and quantification of antibody in the sample. NOVA View has 5 preselected fields where it collects a digital image producing 5 images on the screen when the sample is positive. These five preselected fields mimic the areas where a technologist would read with a manual microscope. If the sample is negative, three images are produced.
For the manual reading, a Nikon Eclipse 400 with an LED light source (ARUP Laboratories, Salt Lake City, Utah) was used.
All samples were processed manually and read on both the Nikon microscope and NOVA View, archived images with software version 1.0.2 by a board certified medical technologist. The technologist was blinded to sample classification and has 5 years of reading IIF daily at ARUP laboratories. Intensive training and continuous reading are needed for a technologist to accurately interpret HEp-2 ANA. At ARUP Laboratories, and other facilities, people who interpret HEp-2 ANA on clinical sera must be board certified. In order to read ANA IIF accurately and consistently they read daily and are challenged by internal and external surveys. Patterns recorded at ARUP include speckled, homogeneous, centromere, nucleolar, and nuclear dots, PCNA, and NuMA along with comments on cytoplasmic fluorescence observed.
3. Results
3.1. Agreement between Manual and NOVA View Interpretation
At a 1 : 40/1 : 80 dilution the resulting comparison demonstrated 94.8%/92.9% positive, 97.4%/97.4% negative, and 96.5%/96.2% total agreements (Tables 1 and 2). The majority of discrepant results between the manual and the archived based interpretation were ±1 dilution difference. The highest fluctuation between results was seen at the 1 : 40 dilution. A total of 13 samples that were called positive by one method of reading and negative by the other all had titers of 1 : 40 or 1 : 80 and <1 : 40 (Table 1). One sample demonstrated a PCNA pattern, a pattern which is not recognized by the NOVA View system. Therefore we aimed to evaluate if this pattern can be identified as PCNA pattern by the technologist reading the NOVA View archived images. The archived image of the PCNA pattern was clear and easy to interpret.
Of the 100 samples sent to ARUP Laboratories for routine ANA testing, 63% were negative and 37% were positive. Titers were within plus or minus a doubling dilution between the manual and the NOVA View archived image results, and ranged from 1 : 40 to 1 : 2560 (Figure 1). Patterns matched 100%.
(a)
(b)
(c)
Good agreement and correlation between manual and NOVA View archived based reading were found. Results of the manual IIF interpretation were grouped into positive and negative. Subsequently, the titers obtained from the NOVA View archived image based interpretation were used to generate a receiver operating characteristic (ROC) curve showing very good agreement (Figure 2(a)). Spearman’s correlation (all samples, ) between IIF interpretation and NOVA View showed excellent correlation of rho = 0.96 (Figure 2(b)).
(a)
(b)
3.2. CDC Samples
The 12 CDC samples produced excellent correlation for pattern and titer (data not shown). All samples with ANA were positive with titers ranging from 1 : 40 to 1 : 320. The patterns match their original description of the antibody specificity.
3.3. Clinically Defined Samples
The ROC analyses revealed similar discrimination between AARD and controls using the manual and the NOVA View archive reading (Table 3, Figure 3). A comparative descriptive analysis (Figure 4) demonstrated positivity in 56.0% of SLE, in 68.4% of SjS, in 74.3% of SSc, and in 30.0% of PM patients. In the control groups, 18.2% of the RA and 5.1% of the healthy individuals were ANA positive. Two of the 99 healthy donors demonstrated a result of 1 : 160 speckled pattern by manual microscopy and NOVA View archived image.
3.4. Workflow Analysis
Although not the focus of our study, we investigated the impact of the NOVA View instrument in the laboratory workflow. We found that the system is a walk-away platform, with user friendly software, and the ability to interface with bar coded slides for positive patient identification.
4. Discussion
The recommendation of the ACR to use IIF as the preferred method has triggered the development and validation of automated systems for ANA determination. Although the ANA IIF test is the recommended method for ANA testing, the method has significant limitations, including a high degree of subjectivity [27]. With the availability of novel digital imaging systems, this limitation can be overcome [27]. However, careful evaluation and validation of those systems are required to ensure that the ANA results do not sacrifice clinical accuracy. One of the systems, the NOVA View, was evaluated in our study. In the beta software version 1.0.2 of NOVA View we found the instrument produced high quality images and excellent agreement with manual IIF testing. Properly comparing the archived NOVA View results to the manual results and the negative and positive sera, along with titer outcomes, demonstrated 97% concordance, in this study.
The NOVA View has a walk-away platform, user friendly software, and the ability to interface with bar coded slides for positive patient identification. In addition, the patient images are stored for later viewing without fluorescent burnout. However, the impact on the workflow might vary from laboratory to laboratory and needs to be quantified in further studies. The good agreement between interpretation using a microscope and using archived images on a screen holds promise to avoid the dark room, which is a source of transcription errors of results.
The specificity against healthy individuals in this study was in keeping with recent recommendations for the determination of anticellular antibodies [28]. However, the prevalence in SLE patients was somewhat lower than expected. This might be explained by the SLE population used and the relatively small cohort.
Since we did not analyze the performance of the NOVA View in terms of positive/negative discrimination and pattern recognition, further studies are needed. In recent years, several of those studies have already been performed [17–23, 25, 27, 29]. The internal LIU cutoff value causes the instrument to display the term “negative” for a sample that produces less than 100 LIU, whereas a positive result is displayed if the LIU is 100 or greater. The preset LIU cutoff of 100 does not always correlate between instrument generated outcome and manual microscopy on low positive/negative samples. The LIU cutoff can be adjusted to closely match the laboratory’s manual reading during the validation process if desired. This does not change the image produced by NOVA View. Among many other systems, the NOVA View is an automated image recognition instrument.
Since this study, NOVA View has had two software updates. The current version, 1.0.3.1, contains a Single Well Titer (SWT) application that utilizes the LIU and assigns pattern to produce a calculated titer from one well. A recent study by Schouwers et al. concluding the estimation of fluorescent intensity offers clinically useful information and value added reporting [29].
Further studies are desired to underline the clinical utility of the NOVA View system in diagnostic specimens.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This study was supported by funding from the ARUP Institute for Clinical and Experimental Pathology, Salt Lake City, Utah. All kits used in this study were kindly provided free of charge by the manufacturer who had no role in the collection, analysis, and interpretation of the data. The authors thank Marvin Fritzler, PhD, MD, Alberta Research & Innovation Authority, and Rufus Burlingame, VP of Diagnostic Assay Development, Genalyte Inc., for the use of their well-documented disease serum samples.