Objective. This systematic review evaluates the accuracy of the mRNA HPV biomarker in cervical smears to identify cervical intraepithelial neoplasia (CIN) 2 or 3 and cervical cancer. Data Source. Eligible studies were identified by performing a search of electronic databases on Medline via Pubmed, Lilacs, Cochrane Library, Embase, and Grey literature for papers published between January 1990 and June 2018. Study Eligibility Criteria. As no randomized studies were identified, this review focuses on observational studies in which the mRNA HPV diagnostic test was compared to a histopathology reference standard. We analyzed studies that included women screened for cervical cancer using mRNA HPV. Study Appraisal and Synthesis Methods. After screening, 61 studies including 29,674 patients met the inclusion criteria and were analyzed. Dichotomization was performed by defining CIN2 or worse (CIN2+) versus CIN1, HPV infection, and normal (CIN 1-). The analysis was discriminated by the following tests: Aptima, PreTect HPV Profeer, NucliSens EasyQ HPV, OncoTect, and Quantivirus. Results. Analyzing by technique, Aptima, with 28 studies, exhibited superior performance, showing for the outcomes CIN2+ and CIN3+ an AUC of 0.88 (0.82-0.95) and 0.91 (0.84-0.99), a pooled sensitivity of 92.8% (95%CI 91.9-93.7) and 95.6% (95%CI 94.5-96.5), and a pooled specificity of 60.5% (95%CI 59.8-61.3) and 61.9% (95%CI 61.1-62.7), respectively. Conclusion. This study supports the current hypothesis that the mRNA HPV assay is an adequate tool for secondary cervical cancer screening.

1. Introduction

Cervical cancer is the third most common malignancy in women and fourth in mortality worldwide. In 2012, there were 406,210 diagnosed cases and 265,672 deaths [1]. In the United States, there were 12,578 new cases and 4,115 deaths in 2014 [2]. Of note, screening tests for cervical cancer make this disease one of the most easily preventable malignant tumors. Worldwide, cervical cancer screening is accomplished using the Papanicolaou test, which looks for cytological abnormalities. If identified, the patient will be referred for colposcopy and targeted biopsies. Given consensus regarding the causal role of high-risk human papillomavirus (HR HPV) in the development of cervical cancer [3], DNA hrHPV assays have been incorporated as a screening method in some developed countries [46]. HPV is the number one most common infectious agent related to cancer development in women, and it is estimated that 570,000 cases of cancer arose from this infection in 2012, including anogenital and oropharynx cancers. Currently, the following HPV strains are considered high risk with respect to cervical cancer development: 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, and 59 [1, 7].

Screening strategies should balance potential benefits and potential harm from intervention. DNA hrHPV tests exhibit high sensitivity with low specificity when the outcome is a precancerous lesion [4, 6]. Maintaining a 3-year interval between screening visits is a good safety measure, but it increases unnecessary routing to colposcopy with a potential rise in cost and overtreatment [4, 6]. As a result, some countries are adopting a 5-year interval [4, 6]. In this scenario, an assay with good accuracy and improved specificity should be associated with or used alone in primary screening. Previous studies reported that mRNA HPV tests, which reveal current HPV oncogene expression and evidence of its deregulation per detection of viral proteins, possess these characteristics [66, 67].

The present systematic review assesses the accuracy of mRNA HPV tests globally that have been submitted to sensitivity analysis and, when available, compared with the DNA hrHPV test and cytology. The prespecified hypothesis is that mRNA HPV exhibits acceptable accuracy and high specificity for detection of high-grade squamous intraepithelial lesion (HSIL) or cervical intraepithelial neoplasia (CIN) 2 or 3, precancerous lesions, and cervical cancer.

2. Methods

We performed a systematic review according to a prospective protocol using PRISMA statement guidelines. This review protocol is registered at PROSPERO (International prospective register of systemic reviews, http://www.crd.york.ac.uk/prospero; CRD 2015: CRD42015020232).

2.1. Identification of Studies

Eligible studies were identified by performing a search of electronic databases on Medline via Pubmed, Lilacs, Cochrane Library, Embase, and Grey for papers published from January 1990 to October 2017. A search on clinical trials was not performed because this database includes intervention trials and is used primarily for intervention systematic reviews and not for diagnostic reviews. The medical subject headings (MeSH) and text words for the terms: “cervical cancer”, “cervical dysplasia”, “squamous intraepithelial lesion”, “cervical intraepithelial neoplasia”, “CIN”, “screening” and “RNAm HPV” were entered. No language restrictions applied. Reference lists of all available primary studies were reviewed to identify additional relevant citations.

2.2. Study Selection

As no randomized studies were identified, this review focused on observational studies in which the mRNA HPV diagnostic test was compared to a histopathological reference standard. All included studies were cross-sectional or, if cohort study, it was included only if biomarkers, cytology, and histopathology have been available in baseline, to characterize a cross-sectional data.

2.3. Patients

We analyzed studies that included women who were screened for cervical cancer in secondary settings, that is, testing performed after someone has had an abnormal result by cytology or HPV testing. When the study was originally from primary screening, only the sample with abnormalities and that had been forwarded to colposcopy was considered. Additionally, when only considering samples submitted for colposcopy, whenever possible, only biopsied samples were included. These variables were subsequently considered in the sensitivity analysis.

2.4. Index Test

The index test was an mRNA HPV test from a sampling of a cervical smear. Positive and negative reads were assigned according to the cut-off points proposed by the manufacturers.

As alternative tests, the accuracy of DNA hrHPV tests was extracted when applied to the same sample used for the mRNA test.

The exclusion criteria for index tests applied in tissue fragments. Studies in which all specimens were diagnosed as cancer were excluded, since there were no false positives or true negatives.

2.5. Reference Standard

The reference test was histologic evaluation of tissue in paraffin-embedded sections using the same Bethesda System classification.

2.6. Data Extraction

This study was independently reviewed by two investigators (MIR, ACM). Disagreements with regard to study inclusion or exclusion were initially resolved by consensus. When consensus was not attained, disagreements were resolved by a third reviewer (JCG).

2.7. Assessment of Methodological Quality

Methodological quality assessment of studies for diagnostic accuracy was performed according to criteria from the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2). These criteria assess the quality of included studies in terms of risk of bias and concerns regarding applicability over four domains [68].

2.8. Statistical Analysis

A 2 x 2 contingency table was constructed for each selected study. Rates were calculated as true positive (TP), false positive (FP), true negative (TN), and false negative (FN). When any cell containing “0" was present in the contingency table, 0.5 was added to all cells in all studies to facilitate calculations. Dichotomization of the contingency tables was performed by defining two categories: (1) CIN2 or worse versus CIN 1 and normal and (2) CIN 3 or worse versus CIN1 and normal (excluding CIN2 from the analysis, since we do not believe that CIN2 can be seen as a false positive).

For all studies, we calculated the true-positive rate (TPR; sensitivity), specificity, false-positive rate (FPR; 1 – specificity), and the diagnostic odds ratio (DOR). The DOR, which relates to different combinations of sensitivity and specificity, was calculated by (sensitivity/(1-specificity))/((1-sensitivity)/specificity)) [69]. A DOR > 1 indicated the assay had discriminative power. The DOR describes the odds of the positive test results in participants with disease compared with the odds of positive test results in those without disease. Bivariate analysis was used to calculate pooled estimates of sensitivity, specificity, and DOR with 95% confidence intervals (CIs) for summary estimates [70].

To analyze the accuracy of HPV mRNA, the area under the curve (AUC) was calculated from the hierarchical summary receiver-operator curves (HSROC). AUC values ≥ 0.5, 0.75, 0.93, and 0.97 were considered to represent fair, good, very good, and excellent accuracy, respectively [71].

Heterogeneity of both sensitivity and specificity across the studies was tested using a χ2 analysis, with a χ2 p-value < 0.05 considered heterogeneous. As an alternative method to explore heterogeneity, the I2 index was also utilized. The I2 index presents the percentage of total variation across studies due to heterogeneity rather than chance; I2 values of 75% or greater were considered substantial heterogeneity [70].

To analyze publication bias, inverted funnel plots of the logarithmic odds ratio (OR) of individual studies were plotted against the sample size. The robustness of the results was tested by repeating the analysis with a different statistical model (random effects model). The meta-analysis was performed using Metadisc® and Review Manager® (RevMan) version 5.2 software [72, 73].

3. Results

3.1. Study Identification and Eligibility

Among the 2,052 studies identified from electronic database searches and reference lists, we excluded 1,868 published studies through title and abstract screening (Figure 1). One hundred seventy-six full-text studies were then retrieved. Of those, 107 studies were excluded after further scrutiny. A complete list of excluded studies is available from the authors.

3.2. Study Descriptions

Sixty-one primary studies were included [865, 7476] in cytology secondary analyses. Of the main analysis, 60 studies informed the major outcome, CIN1- vs. CIN2+, and 39 studies have shown CIN1- vs. CIN3+. A total of 29,674 patients met the criteria for inclusion and were analyzed. The main characteristics of the included studies are shown in Table 1. Table 2 shows the sum contingency tables with regard to the different techniques applied for CIN1- vs. CIN2+ and CIN1- vs. CIN3+. The contingency tables per study may be requested from the authors.

3.3. Quality Assessment

QUADAS-2 was performed considering the following categories: index and reference test, flow, and timing (Figure 2). For the index and reference test, most studies did not mention blinding of the pathologists and were classified as “unclear." In 37.7%, the verification of the histopathological examination was partial; that is, women with normal colposcopy were not biopsied, as shown in Table 1. In addition, all included studies used a histopathological test as a reference, and the index tests were clearly cited. Therefore, “concern” with these items was low. For flow and timing, six studies did not cite the interval between the index and referenced tests [12, 24, 25, 35, 37, 75], and in one, the interval was considered inadequate because it was from a cohort that did not show separate baseline and follow-up results [27]. In cohort studies, we considered the results of the baseline whenever possible.

3.4. Accuracy of HPV mRNA

The accuracy (sensitivity, specificity, AUC, DOR, and sum contingency tables) of HPV mRNA tests stratified by kit identified in this systematic review is discriminated in Table 2.

Different techniques are available, based on identification of HPV mRNA transcription, mainly of E6 and E7 oncogenes. In this systematic review, five main tests were identified. Aptima (Hologic Gen-Probe, San Diego, CA, USA) is a target amplification assay utilizing transcription-mediated amplification (TMA) for qualitative detection of viral polycistronic E6/E7 mRNA from 14 high-risk HPV types [77]. PreTect HPV-Proofer (NorChip AS, Klokkarstua, Norway) is a real-time multiplex assay that uses nucleic acid sequence-based amplification (NASBA), a sensitive transcription-based amplification system (TAS) for the specific in vitro replication of mRNA. NucliSens EasyQ HPV (bioMérieux, The Netherlands) is based on the original PreTect Proofer assay with the addition of the NucliSENS hardware platform and the software for NASBA measurements and data analysis, both identifying the same five most frequently recognized HPV types [78]. OncoTect (IncellDxTM, Inc. Menlo Park, CA, USA) combines two techniques, called in situ hybridization and flow cytometry. Finally, the Quantivirus HPV E6/E7 RNA 3.0 assay (DiaCarta, Hayward, CA, USA) detects E6/E7 mRNA of 13 high-risk and 6 low-risk types and is a sandwich nucleic acid hybridization procedure using chemiluminescent detection of mRNA molecules that are hybridized to DNA probes [65]. Aptima, with 28 studies, exhibited superior performance, with the best sensitivity, near from Hybrid Capture 2, and higher specificity, comparing to this assay, as shown ahead. Its SROC is shown in Figure 3.

We considered the importance of describing the results divided by age; however, few studies [13, 49] discriminated between the over and under 30 years of age category, and there were no important differences in this small sample (data not shown).

3.5. Comparing HPV mRNA to hrHPV DNA

Some studies applied two or more assays to the same sample, making it possible to compare them. In the outcome CIN1- vs. CIN2+, comparing Aptima to Hybrid Capture 2 (HC2, Qiagen, Gaithesburg, MD, USA), a DNA hrHPV test, fourteen studies were available [8, 1214, 19, 20, 2529, 31, 32, 35]. The pooled sensitivity identified was 93.9% (95%CI 92.8-94.8) and 94.3% (95%CI 93.3-95.2), pooled specificity of 61.5% (95%CI 60.6-62.7) and 51.3% (95%CI 50.2-52.4), the DOR was 15.96 (95%CI 10.14-25.17) and 12.55 (95%CI 92.33-17.07), and the AUC was 0.90 (0.80-1) and 0.91 (0.88-0.95), respectively, for Aptima and Hybrid Capture 2 (Table 3).

3.6. Sensitivity Analysis

Discerning by complete verification of the reference test or partial verification, we identified that all samples were biopsied in 38 studies, whereas in 23 studies, they were not (Table 1). In the completely biopsied sample group, the pooled sensitivity was 86.9% (95%CI 85.4-88.2) and the pooled specificity 64.8% (95%CI 63.7-65.8). The DOR was 10.49 (95%CI 6.94-15.85), and the AUC was 0.85 (95%CI 0.79-0.92). In contrast, in the partially biopsied sample group in which women with normal colposcopy were not biopsied, the pooled sensitivity was 80.2% (95%CI 78.8-81.5) and pooled specificity 72.6% (95%CI 71.7-73.5). The DOR was 13.96 (95%CI 9.798-19.91), and the AUC was 0.86 (95%CI 0.82-0.90). This difference is potentially caused by the higher frequency of Aptima studies in the “all biopsied” group, 55.2% vs. 30.4%, as in comparison, this assay has superior sensitivity, as shown above.

4. Discussion

The aim of this systematic review was to evaluate the accuracy of the biomarker HPV mRNA as a means to identify CIN and cervical cancer, a disease with a high prevalence, primarily in low-resource countries. In this analysis, we show 60 studies with the same outcome, making this the most extensive review on the topic to our knowledge.

Two systematic reviews have already been performed analyzing the HPV mRNA test accuracy. Burger et al., in 2011, conducted a systematic review predominately including studies from nonspecific secondary screening [67], and Verdoodt et al., in 2013, included studies with minor abnormal cervical cytology [66]. The first one included 11 studies and concluded that sensitivities ranged from 41.0% to 86.0% and from 90.0% to 95.0% for the PreTect Proofer/NucliSENS Easy Q and Aptima assay, respectively. Specificities ranged from 63.0% to 97.0% and from 42.0% to 61.0% for the same assays, respectively. In our study, the greater number of primary studies led to a wider range of results but maintained the same trend. In a study by Verdoodt et al., which included 10 studies using PreTect Proofer/NucliSENS Easy Q, they concluded that the pooled sensitivity was 75.4% and 76.2% and the pooled specificity was 77.9% and 74.2%, for the triage of ASC-US and LSIL, respectively. These are very close to our results, except that, in our sample, NucliSens EasyQ HPV exhibited a lower specificity.

One of the most promising algorithms is in effect primary screening with the hrHPV DNA test, which has superior sensitivity, and use of the HPV mRNA test, due to its high specificity and the possibility to perform the test with the same sample without the need for patient return. Another possibility is to substitute hrHPV DNA and cytology for HPV mRNA testing. Zappacosta et al., 2015, published a prospective study that compared the cost and effectiveness of three strategies for management of ASC-US and LSIL cytology patients: immediate colposcopy, triage with the hrHPV DNA test, and the HPV mRNA test [79]. They concluded that the HPV mRNA test exhibited overall percentage agreement with histological diagnosis of 89.8%, and as to the AUC, the hrHPV DNA test was 0.79 and the HPV mRNA test 0.92. Cotesting with HPV DNA and mRNA, in comparison with immediate referral, reduced colposcopy referral by 77.5% and by 54.5% in comparison with hrHPV DNA alone. An American study comparing cotesting cytology and hrHPV DNA (n=1,856) or HPV mRNA (n=1,651) in ASC-US cytology samples concluded that the change in the hrHPV detection methodology from HC2 to Aptima has led to a 21% reduction in colposcopy referrals and is more cost-effective for patient care [80]. A multicenter trial with 5,006 women undergoing routine screening in France comparing an HPV mRNA test (Aptima), an hrHPV DNA test (HC2), PCR genotyping, and cytology (LBC) already illustrated that Aptima exhibits the highest absolute risk of both histological endpoints and detected 5% to 15% more CIN3+ and CIN2+ lesions, respectively, than did cytology. Compared with the HC2 assay, the relative risk of Aptima was 24% to 29% higher, with a significant difference in CIN2+ detection, concluding that Aptima is a suitable option for primary cervical cancer screening [81]. In our study, the accuracy was greater for Aptima, when compared to hrHPV DNA tests, suggesting that this could be an adequate substitute, especially considering improvements in specificity. In secondary screening, a test with improved specificity would be more useful, like OncoTect or PreTect HPV Profeer.

Great heterogeneity in sensitivity and specificity was found among studies. This could be explained by different samples and different frequencies of CIN in each population. We performed sensitivity analysis using different screening criteria and studies with partial or complete verification of the reference test, to try and detect confounding factors, but the results retained high heterogeneity (data not shown).

In conclusion, this study supports the current hypothesis that HPV mRNA assays are an adequate tool in the secondary screening of cervical cancer.

Additional Points

Recommendations. Although this systematic review clearly shows the accuracy of HPV mRNA for cervical cancer screening, additional prospective and randomized studies are necessary in order to establish cost-effectiveness and possible changes in screening guidelines.


The present manuscript had been presented at 6th International Congress on Gynecology & Gynecologic Oncology (DOI: 10.4172/2161-0932-C2-028), thanks for the contribution and visibility made to this study.

Conflicts of Interest

The present study has no conflict of interest.


The authors acknowledge generous funding from the University of Extremo Sul Catarinense, Criciúma, SC, Brazil. MIR is a recipient of a CNPQ (Brazil) Productivity Fellowship.