- About this Journal ·
- Abstracting and Indexing ·
- Advance Access ·
- Aims and Scope ·
- Annual Issues ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
Journal of Biomedicine and Biotechnology
Volume 2012 (2012), Article ID 303192, 8 pages
Identification of Women for Referral to Colposcopy by Neural Networks: A Preliminary Study Based on LBC and Molecular Biomarkers
1Department of Cytopathology, Medical School Attikon University Hospital, University of Athens, 12462 Athens, Greece
23rd Department of Obstetrics and Gynecology, Medical School Attikon University Hospital, University of Athens, 12462 Athens, Greece
3Department of Obstetrics and Gynecology, University Hospital of Ioannina, 45500 Ioannina, Greece
4West London Gynaecological Cancer Center, Queen Charlotte's and Chelsea, Hammersmith Hospital, Department of Obstetrics and Gynaecology, Imperial Healthcare NHS Trust Division of Surgery and Cancer, Faculty of Medicine, Imperial College, London W12 0HS, UK
52nd Department of Pathology, Medical School Attikon University Hospital, University of Athens, 12462 Athens, Greece
Received 5 May 2012; Accepted 30 May 2012
Academic Editor: P. J. Oefner
Copyright © 2012 Petros Karakitsos et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Objective of this study is to investigate the potential of the learning vector quantizer neural network (LVQ-NN) classifier on various diagnostic variables used in the modern cytopathology laboratory and to build an algorithm that may facilitate the classification of individual cases. From all women included in the study, a liquid-based cytology sample was obtained; this was tested via HPV DNA test, E6/E7 HPV mRNA test, and p16 immunostaining. The data were classified by the LVQ-NN into two groups: CIN-2 or worse and CIN-1 or less. Half of the cases were used to train the LVQ-NN; the remaining cases (test set) were used for validation. Out of the 1258 cases, cytology identified correctly 72.90% of the CIN-2 or worst cases and 97.37% of the CIN-1 or less cases, with overall accuracy 94.36%. The application of the LVQ-NN on the test set allowed correct classification for 84.62% of the cases with CIN-2 or worse and 97.64% of the cases with CIN-1 or less, with overall accuracy of 96.03%. The use of the LVQ-NN with cytology and the proposed biomarkers improves significantly the correct classification of cervical precancerous lesions and/or cancer and may facilitate diagnosis and patient management.
Approximately 7-8% of the total population screened in the UK will have an abnormal smear [1, 2]; of those, approximately 1.5–2% will present with high-grade and 5% with low-grade cytology. Only a small proportion of women with low-grade cytology has underlying high-grade histology. Even the cytological diagnosis of HSIL does not necessarily reflect the histological presence of CIN2+ lesions. HPV DNA test [3, 4] has proven its value in detecting women with ASCUS cytology that may have an underlying high-grade histology; however its high-positivity rates in women with low-grade abnormalities fail its use as a triage tool in that population. Accurate triage methods and tests for women with both low- and high-grade cytology are lacking. New emerging technologies and biomarkers such as HPV DNA genotyping, E6&7 mRNA testing, and P16 immunostaining are continuously being investigated [5–12].
Various classification techniques such as neural networks [13–18], discriminant analysis [16, 19, 20], decision trees [21, 22], or genetic algorithms  have been used in medicine and, particularly, in diagnostic cytology. The implementation of new diagnostic tools and molecular techniques that are increasingly used in the diagnostic cytology laboratory  may improve the accuracy of the final diagnosis in comparison to that of cytology alone. The application of neural networks on modern diagnostics might be helpful in that respect.
This study aims to investigate the potential role of learning vector quantizer neural networks (LVQ NN) on various diagnostic variables used in the modern cytopathology laboratory and build an algorithm that may facilitate the classification of individual cases.
2. Materials and Methods
2.1. Inclusion Criteria and Interventions
This was a multicentric diagnostic study conducted at the Attikon University Hospital and the University Hospital of Ioannina from 2007 till 2010. The population included a consecutive sample of women with cytology taken as part of screening or during colposcopy. A liquid-based cytology (LBC) sample was obtained and was routinely prepared for cytological assessment and the remaining material was used for testing of specific biomarkers. These included the following tests: cytology using the revised Bethesda classification system (TBS2001 system) [25, 26], HPV arrays using the CLART HUMAN PAPILLOMAVIRUS 2 (GENOMICA) that allows simultaneous detection of 35 different HPV genotypes by PCR amplification of a fragment within the highly conserved L1 region of the virus , NASBA assays  (NucliSENS EasyQ HPV v1.0) that are used for the identification of E6/E7 mRNA of the HPV types: 16, 18, 31, 33, and 45, the PermiFlow (Invirion Diagnostics, LLC, Oak Brook, IL) that allows the identification of E6/E7 mRNA expression of high-risk HPV using FLOW cytometry technique , and finally the immunocytochemical expression of p16 using the CINtec Cytology Kit . All these tests produce results that can be used in a classification process and assess the whole cytologic sample.
The histological diagnosis was the gold standard. All women had histological diagnosis with colposcopically-directed biopsies or LLETZ, apart from those with negative cytology, colposcopy and HPV DNA test (clinically negative cases). These were considered as having negative histology; random biopsies were not taken.
2.2. Data Description and Preprocessing
The LVQ NN was designed to classify the cases into two groups: GROUP 1 (clinically negative and CIN-1 or less at histology—these are considered as negative) or GROUP 2 (CIN-2 or worse at histology—these are considered as positive).
Before applying the LVQ NN, data have been processed as shown in Table 1. Prior to feeding the data to the NN, all variables were scaled at the same range (0 to 10) in order to give all the same significance when processed by the NN. Additionally, the dataset was randomly divided into two sets: the training set used to train the NN, and the test set, used for the NN evaluation. Stratified random sampling was used to select approximately 50% of the cases from each diagnostic category (GROUP 1 and GROUP 2) and form the training set in order to preserve the structure of the diagnostic groups in the divided sets. The LVQ NN is developed by using the training set data; the trained model was subsequently assessed by feeding the test set data and evaluating its performance.
2.3. LVQ Neural Networks Basics
LVQ  is a supervised neural network classifier. The available data is divided into two sets, namely, the training and test sets and additionally the category that these data belong to should be known in advance.
During the training phase, the data is used along with their allocated class and the classifier learns from this specific data set. The LVQ NN creates partitions of the feature space. Each partition is characterised by a vector in its center, called the codebook vector; the class of this vector characterises also the class of the complete partition. Only the training set is used during the training phase, while the codebook vectors are modified to represent the complete feature space. Consequently, a passing of all the training vectors to the classifier initiates the codebook vectors and subsequently the training algorithm is applied during each of these passes. Approximately 50–200 passes for all the training-set data are required for the training of the classifier.
During the test phase, the trained NN is evaluated. Each unknown case, being represented by a data vector is presented to the network. The case class is determined by the class of the partition, where the data vector resides. The partition is defined by two steps, initially it is found the codebook vector that is nearest to the unknown case data vector and subsequently the partition of this nearest codebook vector is considered to be similar to the partition of the data vector.
LVQs can be expressed as NNs composed of two layers: the first layer is a competitive layer having as inputs the feature vectors and follows a second linear layer that produces the NN output. The competitive layer learns and subsequently classifies input vectors into subclasses as described previously . The linear layer assigns the classes produced by the competitive layer into the target classification classes as required by the specific problem. The classes represented by the competitive layer represent various subclasses formed in the feature space however belonging to the same target class, the classes produced by the linear layer (target classes) group the subclasses and produce from them a single class. A schematic diagram of the LVQ NN structure is presented in Figure 1. The competitive layer can be trained with various versions of the LVQ algorithm, namely, LVQ1, LVQ2.1, LVQ3, and optimized LVQ1 “OLVQ1” [32, 33]. The optimal number of codebook vectors, the number of data passes and the optimal LVQ training algorithm variant are determined according to the classifier performance on the training set, that is, when there are satisfactory results on the training set. The interested reader for the details of the LVQ algorithm may consult the on line resources [32, 34].
2.4. Statistical Analysis
The performance of cytology and the LVQ NN was assessed; parameters such as sensitivity (), specificity (Sp), positive (PPV), negative predictive value (NPV), and overall accuracy (OA) were calculated.
The accuracy of the LVQ NN in comparison to cytology was assessed by comparing the areas under curve on the ROC curves; the -test was used for estimation of the standard error.
In order to produce comparable results for cytology and the LVQ NN, cases with cytology of ASC-H or worse and histology of CIN-2 or worse were considered as true positive, while cases with cytology of ASC-US or less and histology of CIN-1 and less were considered true negative.
The Cochrane Q test  was applied on the results of five new classifiers, trained and tested from scratch on different dataset splits. This test was used for the assessment of the robustness of the proposed methodology,
3. Results and Discussion
A total of 1258 samples were analyzed. The correlation of the cytological diagnosis with the histological result is shown in Table 2.
A total of 155 of those (12.3%) had CIN-2 or worse histology, while 1103 (87.7%) CIN-1 or less. The detailed distribution of the samples for the training and test sets appear in Table 3. The correlation of the cytology and histology is presented in Table 1. The ROC curve using the CIN-1 histological result as a cutoff point is presented in had an area under curve (AUC) 0.913 with standard error (S.E.) 0.014. The accuracy parameters for cytology and the results for the training and test set from the LVQ NN along with the combined results for both sets appear in Table 4.
The LVQ training was based on the LVQ1 algorithm, the number of neurons of the competitive layer was initially started from 2 and, with a step of 1, increased up to 50. The best results for classification of the training set were obtained using 17 neurons and no significant improvement was achieved for more neurons. The addition of more neurons in the competitive layer incorporates the danger of over-training, as a result the NN may not perform well in the test set. The output layer (linear layer) had two neurons; one for each class. The optimal number of data passes was 50 (during each data pass, all available data of the training set have been used for a training cycle), for more than 50 training cycles there was not improvement of the NN performance on the training set. The other variations of the LVQ algorithm (LVQ2.1, LVQ3, and OLVQ1) have been tested to further improve already trained NNs however there were produced similar or worst results.
The combined results for both the training and test set indicate a sensitivity of 85.16%, a specificity of 98.01%, a PPV of 85.71%, a NPV of 97.92% with an overall accuracy of 96.42%. All these results favor the LVQ NN in comparison to cytology alone, both for the complete (Table 4) as well as for the individual training and test datasets (Table 3). The comparison of the two ROC curves for cytology () and the LVQ () using histology as the gold standard, demonstrated that the LVQ NN classifier results are superior to the cytological diagnosis alone (, ). The comparison on the overall accuracy of standalone cytology versus that of the LVQ classifiers favored the LVQ (, ) as well.
In addition the LVQ NN system may identify more accurately women that require immediate referral to colposcopy but also reduce the number of women seen with clinically insignificant lesions. For the LVQ NN, 2% will require unnecessary colposcopy, whilst only 15% with a potentially significant lesion will fail to be referred appropriately for further colposcopic assessment. Using a cutoff of ASCUS+ cytology for referral to colposcopy the rates are 23% and 2.6%, respectively, and 2.6% and 27% using for a cutoff of ASC-H+.
The stability of the method was subsequently assessed using the statistics on the two ROC curves (classifier versus histology for each one) for the LVQ training versus the test set. The lack of significant difference in the classifier’s performance for the training and test set in the ROC curves (Training Set , Test set , with ) and in the overall accuracy of the training and test set ( with ) proved the system’s stability.
The robustness of the method was further evaluated with five new experiments. The dataset was de novo divided into “new” training and test sets. The stability in the performance of the “new” classifiers was evaluated after de nuovo retraining each time. The results are shown in Table 5. The probability that all the five classifiers would provide similar outcomes for an individual case was high (, ) as calculated by the Cochrane test.
This study shows that LVQ NN has superior performance than cytology alone for the detection of high-grade lesions in a mixed population (women attending screening and colposcopy). It has higher sensitivity and specificity than cytology at the threshold of ASC-H + and much higher specificity than cytology at the threshold of ASCUS, even though its sensitivity was lower. Our results from the comparison of the training and test set were nonsignificant and suggest that the use of LVQ NN appears to provide an accurate prediction of the histological outcome that could guide further management. It therefore seems that the LVQ NN is a reliable tool for the diagnosis of high-grade lesions. The challenge is finding an appropriate role for its use in the cervical cancer prevention efforts. The high costs of the biomarkers and the reduced sensitivity over cytology at the commonly used threshold for screening (ASCUS+) makes the LVQ NN probably unsuitable for screening purposes. However its high specificity and its overall superiority over higher cytological thresholds suggest a possible role in triage of minor or even major cytological abnormalities.
Despite the advances of the last decade, there is still lack of consensus on the optimal management of women presenting with a low-grade cytological abnormality (LSIL) [36–39]. A substantial proportion of these women may actually harbor a high-grade lesion (HSIL). In our population, 14.6% of women with LSIL had underlying CIN2+ at histology. The so far available management options for low-grade abnormalities include either conservative surveillance with repeat cytology or immediate referral to colposcopy . Surveillance has an inherent risk of noncompliance and default from further surveillance , which may put women with clinically significant lesions at potential risk of invasive cervical cancer. Immediate colposcopy can conversely lead to overloading of colposcopy clinics with financial consequences to clinical and health resources as well as overintervention and overtreatment with long-term adverse future pregnancy outcomes or even increased perinatal mortality in women of reproductive age [2, 42, 43]. A more accurate assessment of the underlying risk would be undoubtedly beneficial for patients as well as health economies.
Similarly, current practice and guidelines advocate immediate referral and commonly histological confirmation and treatment for all high-grade lesions. A small, albeit significant, proportion of those cases (up to 30%) may, however, have a low-grade or even normal histology. In our study 20.4% of women with ASC-H+ cytology had CIN-1 or less. Reliable triage methods are lacking. An attempt in that scientific direction is the development of the concept of a “scoring system”, in order to identify individual risk for CIN2+ regardless of the cytologic and colposcopic findings. A combination of the proposed HPV-related biomarkers as well as other epidemiological data, available from women’s history, such as demographic characteristics, sexual behavior, and potential cofactor information (smoking, condom use, etc.), in addition to cytology and colposcopy, could identify individual risk estimation for CIN2+, CIN3+ or even invasive disease [40, 44].
Although HPV DNA testing has been proposed as a reliable method of triage of ASCUS cytology [3, 4], new biomarkers, such as HPV DNA status, mRNA E6/E7 expression of HPV -16, -18, -31, -33, and -45, E6/E7 mRNA expression of high-risk HPV types and p16 immunocytochemistry may prove useful and improve the accuracy of diagnosis [36, 38, 39]. The incorporation of those new tests in neural networks may allow easier and more accurate interpretation of the results.
Our study has certain limitations. The population under study is mixed, that is, women attending for screening and women referred for colposcopy because of abnormal smears. Secondly in our study a large number of variables was used, which although validated as relevant in the diagnosis of CIN2+ in the literature, some of them might eventually prove to have minimal weight in the final diagnosis and could be excluded.
Therefore a future research direction could be studying the performance of LVQ NN in a strictly defined population consisting of women with LSIL smears only and assess its accuracy indices in triage. Also as the use of all the biomarkers in all situations will increase costs to a nonacceptable level, research should be directed at finding different combinations with a selection of those tests with similar accuracy at reduced cost. Studies assessing also the cost-effectiveness of such an approach on a screening program are required.
These preliminary results suggest that the incorporation of new tests and combinations of biomarkers using artificial intelligence methods end especially the LVQ NN may significantly improve the accuracy of diagnosis. Such an approach may reduce the overload of colposcopy clinics and guide tailored management and intervention. The results should be further assessed in larger datasets in order to confirm the reproducibility of those findings and the applicability in situations of specific interest.
This study was funded in part by the Greek Ministry of Development (GSRT), Project “AKAKOS (code ATT 95)” Research code: 8925 and the Greek Ministry of Health, Project “Study and evaluation of the methods, for the prognosis of success for vaccination against HPV infections-Ygeia Proneia 2000–2006” Research code: 9594. The study sponsors had no involvement in this study. This work has been approved by the ethical committee of “ATTIKON” University Hospital and subjects gave informed consent to the work.
- NHS, “Statistical bulletin: cervical screening programme 2005–6,” 2006, http://www.cancerscreening.nhs.uk/.
- E. Paraskevaidis, M. Kyrgiou, and P. Martin-Hirsch, “Have we dismissed ablative treatment too soon in colposcopy practice?” An International Journal of Obstetrics and Gynaecology, vol. 114, no. 1, pp. 3–4, 2007.
- M. Arbyn, F. Buntinx, M. van Ranst, E. Paraskevaidis, P. Martin-Hirsch, and J. Dillner, “Virologic versus cytologic triage of women with equivocal pap smears: a meta-analysis of the accuracy to detect high-grade intraepithelial neoplasia,” Journal of the National Cancer Institute, vol. 96, no. 4, pp. 280–293, 2004.
- M. Arbyn, E. Paraskevaidis, P. Martin-Hirsch, W. Prendiville, and J. Dillner, “Clinical utility of HPV-DNA detection: triage of minor cervical lesions, follow-up of women treated for high-grade CIN: an update of pooled evidence,” Gynecologic Oncology, vol. 99, no. 3, supplement 1, pp. S7–S11, 2005.
- W. P. Soutter, S. Wisdom, A. K. Brough, and J. M. Monaghan, “Should patients with mild atypia in a cervical smear be referred for colposcopy?” British Journal of Obstetrics and Gynaecology, vol. 93, no. 1, pp. 70–74, 1986.
- J. A. Giles, A. Deery, J. Crow, and P. Walker, “The accuracy of repeat cytology in women with mildly dyskaryotic smears,” British Journal of Obstetrics and Gynaecology, vol. 96, no. 9, pp. 1067–1070, 1989.
- E. M. Walker, J. Dodgson, and I. D. Duncan, “Does mild atypia on a cervical smear warrant further investigation?” Lancet, vol. 2, no. 8508, pp. 672–673, 1986.
- L. Contreras-Melendez, A. Herbert, G. H. Millward-Sadler et al., “Assessment of the accuracy of cytology in women referred for colposcopy and biopsy: the results of a 1 year audit,” Cytopathology, vol. 3, no. 5, pp. 267–274, 1992.
- E. Paraskevaidis, A. Kaponis, V. Malamou-Mitsi et al., “The natural history of HPV infection of the uterine cervix. Long-term observational and histological data,” Anticancer Research, vol. 22, no. 2 B, pp. 1177–1181, 2002.
- M. I. Shafi, D. M. Luesley, J. A. Jordan, J. A. Dunn, T. P. Rollason, and M. Yates, “Randomised trial of immediate versus deferred treatment strategies for the management of minor cervical cytological abnormalities,” British Journal of Obstetrics and Gynaecology, vol. 104, no. 5, pp. 590–594, 1997.
- M. Kyrgiou, G. Koliopoulos, P. Martin-Hirsch et al., “Management of minor cervical cytological abnormalities: a systematic review and a meta-analysis of the literature,” Cancer Treatment Reviews, vol. 33, no. 6, pp. 514–520, 2007.
- G. Koliopoulos, G. Valasoulis, and E. Zilakou, “An update review on HPV testing methods for cervical neoplasia,” Expert Opinion on Medical Diagnostics, vol. 3, no. 2, pp. 123–131, 2009.
- P. Karakitsos, E. B. Stergiou, A. Pouliakis et al., “Potential of the back propagation neural network in the discrimination of benign from malignant gastric cells,” Analytical and Quantitative Cytology and Histology, vol. 18, no. 3, pp. 245–250, 1996.
- C. Markopoulos, P. Karakitsos, E. Botsoli-Stergiou et al., “Application of the learning vector quantizer to the classification of breast lesions,” Analytical and Quantitative Cytology and Histology, vol. 19, no. 5, pp. 453–460, 1997.
- D. Pantazopoulos, P. Karakitsos, A. Iokim-Liossi, A. Pouliakis, E. Botsoli-Stergiou, and C. Dimopoulos, “Back propagation neural network in the discrimination of benign from malignant lower urinary tract lesions,” Journal of Urology, vol. 159, no. 5, pp. 1619–1623, 1998.
- A. M. Marchevsky, J. A. Tsou, and I. A. Laird-Offringa, “Classification of individual lung cancer cell lines based on DNA methylation markers: use of linear discriminant analysis and artificial neural networks,” Journal of Molecular Diagnostics, vol. 6, no. 1, pp. 28–36, 2004.
- P. Karakitsos, A. Kyroudes, A. Pouliakis, E. Botsoli Stergiou, Z. Voulgaris, and C. Kittas, “Potential of the learning vector quantizer in the cell classification of endometrial lesions in postmenopausal women,” Analytical and Quantitative Cytology and Histology, vol. 24, no. 1, pp. 30–38, 2002.
- P. Karakitsos, B. Cochand-Priollet, A. Pouliakis, P. J. Guillausseau, and A. Ioakim-Liossi, “Learning vector quantizer in the investigation of thyroid lesions,” Analytical and Quantitative Cytology and Histology, vol. 21, no. 3, pp. 201–208, 1999.
- P. Karakitsos, T. M. Megalopoulou, A. Pouliakis, M. Tzivras, A. Archimandritis, and A. Kyroudes, “Application of discriminant analysis and quantitative cytologic examination to gastric lesions,” Analytical and Quantitative Cytology and Histology, vol. 26, no. 6, pp. 314–322, 2004.
- E. S. McHugh, A. P. Shinn, and J. W. Kay, “Discrimination of the notifiable pathogen Gyrodactylus salaris from G. thymalli (Monogenea) using statistical classifiers applied to morphometric data,” Parasitology, vol. 121, no. 3, pp. 315–323, 2000.
- L. Breiman, J. Friedman, C. J. Stone, and R. A. Olshen, Classification and Regression Trees, Wadsworth International Group, 1984.
- P. Karakitsos, A. Pouliakis, C. Meristoudis et al., “A preliminary study of the potential of tree classifiers in triage of high-grade squamous intraepithelial lesions,” Analytical and Quantitative Cytology and Histology, vol. 33, no. 3, pp. 132–140, 2011.
- F. Saiti, A. A. Naini, M. A. Shoorehdeli, and M. Teshnehlab, “Thyroid disease diagnosis based on genetic algorithms using PNN and SVM,” in Proceedings of the 3rd International Conference on Bioinformatics and Biomedical Engineering (iCBBE'09), June 2009.
- F. C. Schmitt, A. Longatto-Filho, A. Valent, and P. Vielh, “Molecular techniques in cytopathology practice,” Journal of Clinical Pathology, vol. 61, no. 3, pp. 258–267, 2008.
- M. R. Henry, “The Bethesda system 2001: an update of new terminology for gynecologic cytology,” Clinics in Laboratory Medicine, vol. 23, no. 3, pp. 585–603, 2003.
- J. H. F. Smith, “Bethesda 2001,” Cytopathology, vol. 13, no. 1, pp. 4–10, 2002.
- J. J. Gomez-Roman, C. Echevarria, S. Salas et al., “A type-specific study of human papillomavirus prevalence in cervicovaginal samples in three different Spanish regions,” APMIS, vol. 117, no. 1, pp. 22–27, 2009.
- S. Tyagi, D. P. Bratu, and F. R. Kramer, “Multicolor molecular beacons for allele discrimination,” Nature Biotechnology, vol. 16, no. 1, pp. 49–53, 1998.
- R. Narimatsu and B. K. Patterson, “High-throughput cervical cancer screening using intracellular human papillomavirus E6 and E7 mRNA quantification by flow cytometry,” American Journal of Clinical Pathology, vol. 123, no. 5, pp. 716–723, 2005.
- R. Klaes, A. Benner, T. Friedrich et al., “p16INK4a immunohistochemistry improves interobserver agreement in the diagnosis of cervical intraepithelial neoplasia,” American Journal of Surgical Pathology, vol. 26, no. 11, pp. 1389–1399, 2002.
- T. Kohonen, Self-Organization and Associative Memory, Springer, 3rd edition, 1988.
- T. Kohonen, J. Hynninen, J. Kangas, J. Laaksonen, and K. Torkkola, “LVQ_PAK the learning vector quantization program package,” 1995, http://www.cis.hut.fi/research/som-research/nnrc-programs.shtml.
- T. Kohonen, J. Kangas, J. Laaksonen, and K. Torkkola, “LVQPAK: a software package for the correct application of learning vector quantization algorithms,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN'92), vol. 1, pp. 725–730, June 1992.
- MATLAB web site, http://www.mathworks.com/.
- H. Motulsky, Intuitive Biostatistics, Oxford University Press, 1995.
- G. Valasoulis, I. Tsoumpou, C. Founta et al., “The role of p16INK4a immunostaining in the risk assessment of women with LSIL cytology: a prospective pragmatic study,” European Journal of Gynaecological Oncology, vol. 32, no. 2, pp. 150–152, 2011.
- F. Carozzi, S. Cecchini, M. Confortini et al., “Role of P16(INK4A) expression in identifying CIN2 or more severe lesions among HPV-positive patients referred for colposcopy after abnormal cytology,” Cancer, vol. 108, no. 2, pp. 119–123, 2006.
- M. Nasioutziki, A. Daniilidis, K. Dinas et al., “The evaluation of p16ink4a immunoexpression/immunostaining and human papillomavirus DNA test in cervical liquid-based cytological samples,” International Journal of Gynecological Cancer, vol. 21, no. 1, pp. 79–85, 2011.
- I. Tsoumpou, G. Valasoulis, C. Founta et al., “High-risk human papillomavirus DNA test and p16INK4a in the triage of LSIL: a prospective diagnostic study,” Gynecologic Oncology, vol. 121, no. 1, pp. 49–53, 2011.
- M. Kyrgiou, G. Valasoulis, C. Founta et al., “Clinical management of HPV-related disease of the lower genital tract,” Annals of the New York Academy of Sciences, vol. 1205, pp. 57–68, 2010.
- M. Kyrgiou, G. Koliopoulos, P. Martin-Hirsch, M. Arbyn, W. Prendiville, and E. Paraskevaidis, “Obstetric outcomes after conservative treatment for intraepithelial or early invasive cervical lesions: systematic review and meta-analysis,” Lancet, vol. 367, no. 9509, pp. 489–498, 2006.
- M. Kyrgiou, I. Tsoumpou, T. Vrekoussis et al., “The up-to-date evidence on colposcopy practice and treatment of cervical intraepithelial neoplasia: the cochrane colposcopy & cervical cytopathology collaborative group (C5 group) approach,” Cancer Treatment Reviews, vol. 32, no. 7, pp. 516–523, 2006.
- M. Arbyn, M. Kyrgiou, C. Simoens et al., “Perinatal mortality and other severe adverse pregnancy outcomes associated with treatment of cervical intraepithelial neoplasia: meta-analysis,” BMJ, vol. 337, article a1284, 2008.
- G. Valasoulis, G. Koliopoulos, C. Founta et al., “Alterations in human papillomavirus-related biomarkers after treatment of cervical intraepithelial neoplasia,” Gynecologic Oncology, vol. 121, no. 1, pp. 43–48, 2011.