Abstract
Background. Standard protocol to detect melanoma relies on visual assessment in which the sensitivity and specificity are low for early melanomas. Pigmented lesion assay (PLA) offers objective metrics to aid in determining suspicious lesions; however, the literature on the diagnostic accuracy is controversial. Methods. To assess the performance of PLA, we retrospectively assessed the diagnostic accuracy using two different cohorts, total (n = 426) and biopsied (n = 96), modifying the definition of true negative for each. Sensitivity, specificity, negative predictive value, positive predictive value, prevalence, and Cohen’s kappa were calculated. Results. 370 PLA (−) lesions and 56 in the PLA (+). Of the 40 PLA (−) lesions that were biopsied, 5 (12.5%) were diagnosed as melanoma and 16 (40.0%) were atypical melanocytic nevus. Of the 56 PLA (+) lesions, 14 (25.0%) were melanoma, of which 10 (71.4%) were double positive, 3 (21.4%) were PRAME only, and 1 (7.1%) was LINC only. For the total cohort (n = 426), sensitivity of 73.7%, specificity of 89.7%, NPV of 98.6%, PPV of 25.0%, prevalence of 4.5%, accuracy of 89.0%, and kappa agreement of 0.329 were calculated. The biopsied cohort revealed the same sensitivity and PPV; however, specificity was 45.5%, accuracy was 51.0%, NPV was 87.5%), and kappa agreement was 0.110. Conclusion. There differences in our study seen between cohorts highlight the importance to recognize that neither findings are perfect. The real value likely falls in between, but further studies are needed.
1. Introduction
Melanoma, like most cancers, is best treated when detected in its earliest stages. Current standard protocol to detect melanoma relies on visual assessment that dictates physician decision to surgical biopsy followed by histopathological analysis that determines final diagnosis. When visual assessment of a lesion is performed by the unaided eye, sensitivity and specificity are considerably low (84% and <30%, respectively) for early-stage melanomas (MIS and stage 1 invasive MM) [1, 2]. This explains the significant number of biopsies performed that are negative for melanoma (>94%) [3, 4] and could have potentially been avoided, sparing patients the additional financial, cosmetic, and psychological implications associated with unnecessary surgical biopsies. Incorporation of dermoscopy, that allows for microscopic examination of cutaneous subsurface structures, as an aid during visual assessment of pigmented lesions has consistently demonstrated significantly increased diagnostic accuracy by up to 49% compared to without log odds ratio of 4.0 (95% CI 3.0 to 5.1) versus 2.7 (1.9 to 3.4); an improvement of 49% () [5]; and reduces unnecessary surgical sampling by 37.7% [6]. Other studies have even demonstrated that sensitivity significantly improves when aided with dermoscopy compared to the naked eye regardless of experience (naked eye 61.9% vs. dermoscopy 74.5%) [7]. However, regardless of whether or not dermoscopy is used as a triaging tool during visual evaluation, the decision to biopsy still relies on subjective pattern recognition and, as such, is vulnerable to uncertainty [8]. New noninvasive technologies that offer objective metrics capable of shifting the diagnostic paradigm away from sole reliance on subjective pattern recognition-based strategies are emerging and highly desirable.
The recently described pigmented lesion assay (PLA), a molecular test that identifies gene expression risk factors for melanoma in clinically suspicious lesions, may fit this profile. The test is based on a new platform technology for noninvasive genomic testing of the skin, relying on sampling using adhesive patches [9–11]. The PLA analyzes the expression of two cancer biomarker genes, LINC (LINC00518), long intergenic noncoding RNA 518, and PRAME (preferentially expressed antigen in melanoma) known to be increased in melanoma [9]. Because of its ease-of-use and potential to reduce the burden of unnecessary biopsies, those who have undergone skin biopsies for melanoma are particularly fond of this technology [12]. However, it is essential to ensure the accuracy of this tool before implementing widespread reliance on its results to prevent catastrophic consequences associated with falsities.
Initial studies evaluating the diagnostic accuracy of PLA appeared promising at first glance, claiming a high sensitivity and specificity. In the initial validation study, Gerami et al. looked at 398 unequivocal melanocytic lesions (confirmed melanomas or nonmelanomas), and LINC00518 and/or PRAME detection was reported to yield a sensitivity of 91% and specificity of 69% [9]. Another study by Ferris et al. measuring real-world clinical utility was even more impressive, reporting a sensitivity of 95% and specificity of 91% [10]; however, these calculations assumed that PLA (−) represented true negatives and the decision to biopsy relied purely on PLA results. A recent health technology assessment evaluated the evidence involving PLA testing and rated the quality of evidence as very low downgrading for high risk of bias, inconsistencies in study design, and publication bias [13]. It is imperative that studies assessing the accuracy of diagnostic testing, especially when involving fatal diseases such as melanoma, make best attempts to minimize bias and are transparent with study limitations.
Granted the level of uncertainty in the existing literature, our study aims to ascertain performance of PLA testing by presenting data calculated similarly to preceding studies [10], in which all PLA (−) tests are assumed to be true negatives and compare results to true negatives confirmed by reference standard histopathology. Furthermore, the decision to biopsy, a suspicious lesion, is often influenced by multiple factors including clinical and dermoscopic presentation, and PLA results should be considered supplemental information that contributes to the patient’s comprehensive clinical picture, rather than a fool-proof means to dictate biopsies. Therefore, to accurately represent real-world clinical practice, we evaluate how PLA testing is used by dermatologists adjunctive to other standard of care procedures, including dermoscopy and reflectance confocal microscopy, to assess real-world performance and utility of the test.
2. Methods
To assess the utility and performance of PLA during routine clinical use, we retrospectively evaluated all pigmented skin lesions that underwent PLA testing. Initially, 490 suspicious pigmented lesions were sampled using PLA; however, 64 returned as “quality not sufficient” and were unable to yield definitive positive or negative PLA results, resulting in a total cohort of 426 cases collected between three dermatologists in a single academic institution. All lesion samples were obtained using an adhesive patch sample collection kit (DermTech, La Jolla, California, USA) and performed only by personnel who had received basic PLA acquisition training. Patients were enrolled under a study approved by the Oregon Health and Science University independent review board (#24367) for cases that underwent PLA testing between May 2021 and May 2022. Similar to previous studies attempting to analyze real-world utility, management decisions were at the discretion of the evaluating dermatologists [10]; however, different in our study the decision to biopsy was not dependent solely on PLA results. Instead, providers considered the entire clinical picture to determine whether a surgical biopsy would be warranted which occurred in a 96 lesions and makes up our biopsied cohort. In this design, each dermatologist evaluates all patient and lesion information and history, including sex, race, and age; personal history of melanoma; first-degree relative with melanoma; history of atypical nevi, basal cell cancer, or squamous cell cancer; more than 5 severe sunburns before 20 years of age; use of tanning beds; UV-A or UV-B treatment; 1 to 10, 11 to 50, or 51 or more moles; Fitzpatrick skin type; location of the lesion; presence of a new lesion; pain or itching; diameter greater than 6 mm; actual diameter 1 to 2 mm; evolving lesion; ulceration, weeping, or oozing; border irregularity; ugly duckling (i.e., a pigmented lesion very different from surrounding pigmented lesions); and patient concern. In addition, close-up, regional, dermoscopic, and select reflectance confocal images were reviewed.
For PLA (+) patients, clinical management (surgical biopsy technique, re-excisions, or follow-up) and histopathologic outcome (as determined by the diagnosis rendered by evaluating dermatopathologists) were recorded. The formula used to calculate the biopsy ratio was [number of biopsied cases minus histopathologically confirmed melanomas] divided by histopathologically confirmed melanomas ([96 − 19]/19 = 4.1). The formula to calculate the number needed to biopsy (NNB) to detect a melanoma was number of biopsied cases divided by histopathologically confirmed melanomas (96/19 = 5.1). PLA (−) patients were either biopsied due to high clinical concern as determined by provider (based-off dermoscopy, reflectance confocal microscopy, and patient history) or monitored and followed up 4–6 months following PLA acquisition. Pathology reports for lesions biopsied in the follow-up period were reviewed and histopathologic diagnoses and histologic features reported were recorded. We decided to assess the diagnostic accuracy using two different cohorts, total (n = 426) and biopsied (n = 96), to showcase the difference in findings when the definition of true negatives was modified. For the total cohort, estimates of diagnostic accuracy were based on the histopathology reports and the assumption that the PLA (−) lesions that were not subjected to follow-up biopsy, due to less concerning comprehensive clinical picture, were true negatives, which follows a similar method as previous reports [10]. We additionally calculated estimates for the biopsied cohort, in which we determined true negatives based-off the gold-standard, histopathological analysis, that received a nonmelanoma diagnosis. Gene expression results for PLA (+) tests (single-positive results, LINC or PRAME and double-positive results, and LINC and PRAME) were correlated with histopathologic outcome. To assess diagnostic accuracy, sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), prevalence (those with the melanoma divided by all patients), accuracy (the sum of true positives and true negatives divided by all patients), and Cohen’s kappa were calculated from 2 × 2 tables using R: A Language and Environment for Statistical Computing [14].
3. Results
A total of 426 real-world use cases were analyzed; 370 PLA (−) lesions and 56 in the PLA (+) lesions, of which 96 were biopsied. Table 1 shows the summary of PLA results for these two cohorts (total and biopsied) and Table 2 summarizes the PLA results corresponding to the lesion pathology results of the biopsied cohort. Forty (10.8%) of the PLA (−) results were managed with biopsy due to high clinical concern by provider considering dermoscopy, reflectance confocal microscopy, and patient history. The remaining 330 (89.2%) of PLA (−) lesions were managed with follow-up surveillance per standard of care, none (0%) of which exhibited concerning clinical changes that would warrant biopsy at 4–6 month follow-ups. Of the 56 PLA (+) lesions, all (100%) were subject to surgical biopsies (57.1% shaves, 39.3% excisional, and 3.5% punch biopsies), and of the 40 PLA (−) lesions that were biopsied, 5 (12.5%) were diagnosed as a melanoma in situ, 16 (40.0%) atypical melanocytic nevus, and 1 (2.5%) by histopathology. Of the 56 PLA (+) lesions, 14 (25.0%) were histopathologically diagnosed as an invasive melanoma or melanoma in situ, of which 10 (71.4%) were double positive for LINC and PRAME, 3 (21.4%) were positive for PRAME only, and 1 (7.1%) was positive for LINC only. Twelve (21.4%) PLA (+) lesions were diagnosed as an atypical melanocytic nevus, and 25 (44.6%) received completely benign pathology results including 3 (5.4%) nonmelanocytic lesions such as lichen planus like keratosis and solar lentigos. Our results indicate increased specificity for melanoma amongst PLA (+) lesions positive for both LINC00518 and PRAME (12/27; 44.4%) compared to PLA (+) lesions positive for only one gene (4/27; 14.8%). Of the 19 total melanomas found in this cohort (2 invasive melanomas and 17 melanoma in situs), the NNB to detect one melanoma was calculated as 5.1 (96/19) and a biopsy ratio of 4.1 benign lesions for each melanoma detected. For the total cohort (n = 426) in which we assumed PLA (−) results without a biopsy (due to low comprehensive clinical concern) are true negatives, sensitivity of 73.7%, specificity of 89.7%, NPV of 98.6%, PPV of 25.0%, prevalence of 4.5%, accuracy of 89.0%, and kappa agreement of 0.329 were calculated. Similar calculations using the biopsied cohort with true negatives determined by histological diagnosis of nonmelanoma revealed the same sensitivity and PPV; however, specificity nearly halved (45.5%), and substantial decreases in accuracy (51.0%), NPV (87.5%), and kappa agreement (0.110) were found. It would have been preferrable to biopsy all PLA (−) lesions in order to obtain more accurate sensitivity and specificity; however, given the retrospective and real-world clinical nature of this study, our calculations were reliant on experts’ decision to biopsy. Diagnostic accuracy results with 95% confidence intervals for both cohorts are seen in Table 3 and Figure 1.
4. Discussion
Melanoma is considered a multifactorial disease arising from an interaction between genetic susceptibility and environmental exposure that varies case by case [15]. Therefore, a physician’s decision to biopsy a lesion suspicious for melanoma is influenced by multiple factors including the patient’s comprehensive risk for melanoma and the clinical-dermoscopic features of the lesion of interest. Historically, the biopsy sensitivity and specificity of this method are 75–90% and <30%, respectively, meaning majority of melanomas will be detected and appropriately biopsied, however, at the cost of a significant number of biopsies that are ultimately benign on histology [1–4]. Several prebiopsy tools have been developed to help inform a physician’s decision to biopsy with increased sensitivity and specificity compared to standard procedures. However, often times these tools struggle with a tradeoff of high sensitivity for low specificity; for example, the impedance spectroscopy device Nevisense (Scibase) demonstrated a sensitivity of 96.6% and a specificity of 34.4% in clinical trials [3]. The tradeoff of sensitivity and specificity is best exemplified by MelaFind (STRATA Skin Sciences). This multispectral imaging device used for melanoma early detection reported a high sensitivity of 98.3% but a strikingly low specificity of 9.9% [16]. More promising tools with higher sensitivity and specificity compared to standard procedures include RCM (84% sensitivity and 95% specificity) [17]; however, there are inherent limitations due to the required expertise and subjectivity of pattern/image recognition. The noninvasive PLA gene expression test is an objective prebiopsy tool that reports a high sensitivity (91%–98.6%) with a relatively high specificity (69%–91%) [9–11, 13, 18] compared to other tools, intended to be an effective rule-out method for melanoma and minimize unnecessary biopsies. However, critics have raised a number of concerns regarding the validity of existing PLA-related studies that calls into question the true diagnostic accuracy of the test.
The largest concern that accounted for the low-grade assessment of the overall body of evidence regarding PLA studies was the propensity for patient selection bias. Three out of four studies that reported on diagnostic accuracy of PLA provided unclear or no details regarding patient selection leading to uncertainty for risk of bias, which is evidence when examining the diagnostic odds ratio of these three studies [10, 11], that is, significantly higher than reported in the low risk for bias Gerami et al. study [9]. Although patient selection in our study was at the provider’s discretion (in order to be representative of real-world practice), we detailed the characteristics that providers considered when selecting a lesion to test with PLA and biopsy. The best-quality evidence available that introduced little to no possible source of bias suggested that PLA has a sensitivity of 79% (95% confidence interval (CI) 58%–93%) and specificity of 80% (95% CI 73%–85%) [9]. The sensitivity is comparable to our study (73.7%) which is the same for our total and biopsied cohort. However, for specificity, our total cohort that assumed PLA (−) as true negatives is higher (89.7%), while our biopsied cohort was drastically lower (45.5%), although this may be due to the low number of biopsied PLA (−) lesions. In one of the high risk for bias studies, Ferris et al. compared the diagnostic accuracy of visual inspection alone compared with PLA which claimed only 0.2% of melanomas would be missed when using PLA [10]. These findings can be deceiving leading providers to believe reliance on PLA results would lead to very low risk of missing a melanoma. However, our study findings show the risk of missing a melanoma to be significantly higher at 1.1–12.5% (total vs. biopsied cohort). Although this is a large range due to the differences between the total vs. biopsied cohort, we believe the true risk of missing a melanoma to exist somewhere in between, and since this risk represents the most detrimental consequences, it is important for studies to be more transparent in reporting this data. Notably, a majority of the PLA-related publications declared a conflict of interest, either due to being industry sponsored or because authors were employees or consultants of the manufacturer, which may explain the deceiving claims of these studies.
5. Conclusion
Our findings are important to consider when using PLA testing as relying on the high sensitivity reported in previous studies to dictate biopsy decisions may lead to missed melanomas, which can have dire consequences for patients. Although there are stark differences in our study seen between cohorts, mostly with specificity (89.7% vs. 45.5%) and accuracy (89% vs. 51%), it is important to recognize that neither of the definitions for true negatives used were perfect representations, and the real value lies somewhere in the middle. We wanted to highlight how modifying these factors lead to drastic differences in the test’s efficacy metrics to emphasize that how previous studies may have reported misleading findings that could have serious consequences to patient outcomes. To date, there are no studies that reported on the impact of pigmented lesion assay on patient-important health outcomes, such as survival or melanoma progression. The relatively high NPV seen in our study supports the notion that PLA testing may still be a valuable objective prebiopsy tool that can help reduce unnecessary biopsies. However, PLA should still be considered an adjunctive test that provides additional genetic insight and not a fools-proof means of dictating biopsy.
Data Availability
The data used to support these findings can be released on application to the Oregon Health and Sciences University institutional review board by contacting Claudia Lee ([email protected]).
Conflicts of Interest
The authors declare that they have no conflicts of interest.