Table of Contents Author Guidelines Submit a Manuscript
BioMed Research International
Volume 2017, Article ID 2563910, 9 pages
https://doi.org/10.1155/2017/2563910
Research Article

Interaction of MRE11 and Clinicopathologic Characteristics in Recurrence of Breast Cancer: Individual and Cumulated Receiver Operating Characteristic Analyses

1Department of Electronic Engineering, National Kaohsiung University of Applied Sciences, Kaohsiung, Taiwan
2Department of Chemical Engineering & Institute of Biotechnology and Chemical Engineering, I-Shou University, Kaohsiung, Taiwan
3Cancer Center, Kaohsiung Medical University Hospital, Kaohsiung Medical University, Kaohsiung, Taiwan
4Translational Research Center, Kaohsiung Medical University Hospital, Kaohsiung Medical University, Kaohsiung, Taiwan
5Institute of Clinical Medicine, Kaohsiung Medical University, Kaohsiung, Taiwan
6Kaohsiung Municipal Hsiao-Kang Hospital, Kaohsiung Medical University, Kaohsiung, Taiwan
7Department of Anatomy, School of Medicine, College of Medicine, Kaohsiung Medical University, Kaohsiung, Taiwan
8Institute of Medical Science and Technology, National Sun Yat-sen University, Kaohsiung, Taiwan
9Department of Biomedical Science and Environmental Biology, Kaohsiung Medical University, Kaohsiung, Taiwan
10Department of Medical Research, Kaohsiung Medical University Hospital, Kaohsiung Medical University, Kaohsiung, Taiwan

Correspondence should be addressed to Yi-Chen Lee; wt.ude.umk@38nehciy and Hsueh-Wei Chang; wt.ude.umk@whgnahc

Received 12 April 2016; Accepted 28 November 2016; Published 4 January 2017

Academic Editor: Franco M. Buonaguro

Copyright © 2017 Cheng-Hong Yang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

The interaction between the meiotic recombination 11 homolog A (MRE11) oncoprotein and breast cancer recurrence status remains unclear. The aim of this study was to assess the interaction between MRE11 and clinicopathologic variables in breast cancer. A dataset for 254 subjects with breast cancer (220 nonrecurrent and 34 recurrent) was used in individual and cumulated receiver operating characteristic (ROC) analyses of MRE11 and 12 clinicopathologic variables for predicting breast cancer recurrence. In individual ROC analysis, the area under curve (AUC) for each predictor of breast cancer recurrence was smaller than 0.7. In cumulated ROC analysis, however, the AUC value for each predictor improved. Ten relevant variables in breast cancer recurrence were used to find the optimal prognostic indicators. The presence of any six of the following ten variables had a high (79%) sensitivity and a high (70%) specificity for predicting breast cancer recurrence: tumor size ≥ 2.4 cm, tumor stage II/III, therapy other than hormone therapy, age ≥ 52 years, MRE11 positive cells > 50%, body mass index ≥ 24, lymph node metastasis, positivity for progesterone receptor, positivity for epidermal growth factor receptor, and negativity for estrogen receptor. In conclusion, this study revealed that these 10 clinicopathologic variables are the minimum discriminators needed for optimal discriminant effectiveness in predicting breast cancer recurrence.

1. Introduction

Breast cancer is the most common cancer in women worldwide and is diagnosed in one in three of all women with cancer. Reported risk factors for breast cancer include age, family history, genetic specificity, and lifestyle [14]. Local and/or systematic treatments for breast cancer now enable a high survival rate, especially when breast cancer is diagnosed at an early stage [5]. However, breast cancer recurrence or metastasis (i.e., the spread of tumor cells from the original site) can reduce survival time [6].

Prognostic indicators of breast cancer complication and recurrence can be used to predict survival after diagnosis of breast cancer [7]. Estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2) are reportedly accurate and independent prognostic indicators of breast cancer recurrence risk [6], and combining these independent indicators can improve accuracy in predicting recurrence. Representative prognostic indicators can also be used to evaluate the effectiveness of adjuvant therapy and to estimate the risk of tumor recurrence [8].

Our previous study [9] reported the important role of meiotic recombination 11 homolog A (MRE11) in cell proliferation, tumor invasion, and DNA repair in patients with breast cancer. The MRE11 is considered an oncoprotein because it is overexpressed in colorectal cancer [10] and in highly malignant breast cancer [9]. However, no studies have evaluated the use of breast cancer tumor marker MRE11 as a diagnostic or prognostic indicator in breast cancer. Specifically, no studies have evaluated whether MRE11 interacts with clinicopathologic variables associated with breast cancer recurrence.

Receiver operating characteristic (ROC) analysis is widely used method of evaluating the performance of a diagnostic test according to a continuous spectrum of results [11, 12]. By graphically depicting the quantitative analysis results, the ROC curve reveals the discriminant thresholds based on the probability of positive results (sensitivity against 1 − specificity) in individual subjects [13]. The area under the curve (AUC) is a measure of the overall accuracy of the dichotomous methods of the measurements. Recently, ROC has been used as a tool for comparing the accuracy of various models for predicting cancer diagnosis, prognosis, and survival [1420].

This study developed a scoring system based on ROC analysis to identify patient characteristics associated with susceptibility to breast cancer recurrence. Thus, aims of this study were (i) to assess breast cancer recurrence based on MRE11 expression and clinicopathologic characteristics and (ii) to identify the patient characteristics that are risk factors for breast cancer recurrence.

2. Methods

2.1. Study Participants

After obtaining IRB approval, this study enrolled 254 female breast cancer patients who had received surgical treatment for pathology-confirmed invasive ductal carcinoma at the Department of Surgery, Kaohsiung Medical University Hospital, during 2006–2010. Informed consent was obtained from all patients. The ethics statement, laboratory procedures, and other study procedures were identical to those in our previous study [9]. The dataset for all clinicopathological variables is available online at https://wp.kmu.edu.tw/changhw/files/2015/10/ROC_MRE11_DATASET.xlsx.

2.2. Criteria for Breast Cancer Recurrence

Patients with and without breast cancer recurrence were classified into a recurrence group and a nonrecurrence group, respectively. In the recurrence group, breast cancer recurrence was defined as a local/regional recurrence with or without distant metastasis diagnosed according to symptoms observed in clinical examination, pathology study, or imaging study. Patients who remained disease-free for 60 months after diagnosis or who were disease-free at the end of the follow-up period were classified into the nonrecurrence group.

2.3. Dichotomous Results of ROC Analysis for Each Clinicopathologic Variable

The 13 clinicopathologic variables included in the ROC analysis included MRE11 positive cells (%), tumor stage, tumor grade, age, body mass index (BMI), tumor size, lymph node (LN) metastasis, estrogen receptor (ER) status, progesterone receptor (PR) status, human epidermal growth factor receptor 2 (HER2) status, radiotherapy (RT), chemotherapy (CT), and hormone therapy (HT). First, the clinicopathologic variables were dichotomized by ROC curve analysis. The ROC curve is a graphical plot of the true positive rate (sensitivity) against the false positive rate (1 − specificity) at various threshold settings. Each cut-off point estimated by ROC analysis indicates the distinguishing characteristic of each clinicopathologic variable used to classify participants into the recurrence group. The area under the ROC curve (AUC) is used to calculate the accuracy of dichotomous results.

2.4. AUC of Cumulated ROC Analysis

A cumulated ROC analysis was performed to detect the combined effects of the clinicopathologic variables used to predict recurrence. Variables that had a strong association with recurrence (i.e., values larger than the cut-off point in AUC from the individually dichotomous results) received a score of 1. The remaining variables received a score of 0 (i.e., values less than the cut-off point in AUC from the individually dichotomous results). The indicators were then ranked by cumulated AUC results for individually dichotomous results. Positive changes in the cumulated AUC values for these variables were tracked until the addition of other variables no longer increased the AUC values. Accordingly, clinicopathologic variables that contributed to positive changes in AUC were selected for further analysis.

2.5. Cut-Off Point for Cumulated Scoring System

The cumulated scoring system was then used to rank the clinicopathologic variables as breast cancer predictors. The cumulated score for each subject was obtained by adding the risk factors to the recurrence score (1 or 0). The correctly classified rate for each possible cut-off point within the range of cumulated scores was dependent on the number of clinicopathologic variables selected for ROC analysis. For example, if “” variables were selected according to a positive change in AUC value in the previous step, cumulated scores ranging from 0 to “” were generated. The last step was calculating the specificity, sensitivity, and correctly classified rate for each cut-off point in the cumulated score range.

2.6. Risk Relationship of the Selected Variables in the Cumulated Scoring System

For each subject, the cumulated score represented the total number of clinicopathologic variables that were breast cancer risk factors. For instance, a score of 3 was interpreted as the presence of three risk factors that were more relevant to breast cancer recurrence compared to the selected clinicopathologic variables.

2.7. Statistical Analyses

Differences in the distributions of clinicopathologic variables between the recurrence and the nonrecurrence groups were estimated by frequency tables and the test. The AUC represents the accuracy of the dichotomous results for a single clinicopathologic variable for predicting breast cancer recurrence. The dichotomous results with high AUC values were considered better predictors of breast cancer recurrence.

In cumulated ROC analysis, the likelihood ratio was used to assess recurrence status in subjects with different cumulated scores. The likelihood ratio for a positive test result [LR+: sensitivity/(1 − specificity)] represents the ratio of the probability of a positive test in the recurrence subjects to the probability of a positive test in the nonrecurrence subjects. Comparatively, the likelihood ratio for a negative test [LR−: (1 − sensitivity)/specificity] result represents the ratio of the probability of a negative test in the recurrence subjects to the probability of a negative test in the nonrecurrence subjects. All statistical analyses were performed by STATA version 11.0.

3. Results

3.1. Clinicopathologic Characteristics and Recurrence of Breast Cancer

During the 5-year follow-up period, recurrence developed in 34 (13.39%) of the 254 breast cancer patients. Table 1 compares clinicopathological variables between the recurrence and the nonrecurrence groups. Compared to the nonrecurrence group, the recurrence group had significantly larger proportions of patients with MRE11 positive cells > 50% (85.29%), breast cancer stage II/III (94.12%), age ≥ 52 years (67.65%), tumor size ≥ 2.4 cm (67.65%), LN metastasis (58.82%), negative expression of ER (58.82%), negative expression of PR (64.71%), triple negative breast cancer (35.29%), RT (79.41%), and no-HT treatment (61.76%).

Table 1: Clinicopathologic characteristics of breast cancer patients in recurrence status.
3.2. AUC for Clinicopathologic Variables for Breast Cancer Recurrence

Table 2 shows the AUCs obtained when these 13 variables were considered in estimates of recurrence risk. The clinical criteria for high and low risk of breast cancer recurrence were identical to those used in our previous study [9]. Notably, although the dichotomized tumor size yielded the highest AUC value (0.679 with sensitivity of 0.677 and specificity of 0.682), it did not meet the criterion of AUC ≥ 0.7 [21] for classification of recurrence.

Table 2: AUC of clinicopathologic characteristics for recurrence status.
3.3. Cumulated ROC Analysis of Breast Cancer Recurrence

No single dichotomized variable showed satisfactory performance in predicting breast cancer recurrence (defined as AUC < 0.700). Therefore, this study developed an improved scoring system that considered the combined effects of these variables. Table 3 shows the ROC analysis results obtained for the developed scoring system with cumulated top-ranked predictors. The scoring system obtained a good AUC value (0.806) when 6 dichotomized variables (tumor size, tumor stage, ER, HT, LN metastasis, and age) were used. The AUC values showed further positive changes (range, 0.806 to 0.821) when the scoring system consisted of the cumulated top tenth rank of dichotomized variables (tumor size, stage, ER, HT, LN metastasis, age, PR, MRE11 positive cells, BMI, and HER2). When the number of variables exceeded ten cumulated top-ranked variables, however, the AUC value slightly decreased. That is, the scoring system obtained the best AUC values when ten dichotomized variables were used.

Table 3: Cumulated top-ranked prediction results using ROC analysis.
3.4. Cut-Off Point for Cumulated Scoring System

Hence, the performance of possible cut-off points ranging from score 0 to 10 were compared in the scoring system. Table 4 compares the results. For predicting recurrence, a cut-off point of 0 had a sensitivity of 100% (all recurrence patients correctly classified) but had a specificity of 0% (no recurrence patients correctly classified). In contrast, a cut-off point of 100% had a specificity of 10 (all nonrecurrence subjects correctly classified) but a sensitivity of 0 (no recurrence patients correctly classified). The cut-off point for 6 dichotomized variables had the highest sensitivity and the highest specificity. This cut-off point correctly classified 71.3% with the best combination of sensitivity (0.794) and specificity (0.700; LR+ 2.647 and LR− 0.294).

Table 4: Cut-off point identified by ROC analysis.
3.5. Risk Relationships of the Selected Variables in the Cumulated Scoring System

Table 5 lists the clinicopathologic variables that contributed the five largest possible changes in AUC values (0.799 to 0.821) in Table 3. The scores for 10 dichotomized variables were then computed into these five clinicopathologic variables as indicated in Table 5. In the 254 patients analyzed, scores were ≤ 5 in most (63.39%; 161) patients. In the 161 patients with scores of ≤ 5, most (63.98%, 103) patients were HER2 negative. The score was 6 in 16.93% (43/254) of the patients. In patients with a score of 6, most (74.42%; 32/43) had BMI ≥ 24. The score was 7 in 9.45% (24/254) of the patients. In patients with a score of 7, all (100.00%) had BMI ≥ 24. The score was 8 in 6.69% (17/254) patients. In patients with a score of 8, 88.24% (15) had BMI ≥ 24. The score was 9 in 3.15% (8/254) patients. All patients who had a score of 9 were negative for PR expression and had a BMI ≥ 24. Only 0.39% (1/254) patients had a score of 10, that is, high values for all five clinicopathologic variables that were risk factors for breast cancer recurrence.

Table 5: Risk relationship of scores with selected variables.

4. Discussion

Conventional statistical methods used to estimate probability of breast cancer recurrence include logistic regression, Cox-proportional hazard regression model, Kaplan-Meier estimator, and log-rank test [22]. Our previous work using similar statistical methods revealed that MRE11 is associated with breast cancer malignancy [9]. However, possible interactions between MRE11 and clinicopathologic variables for breast cancer recurrence have not been reported in the literature.

An ROC analysis is a simple and powerful approach to discriminant analysis. In this study, dichotomization of clinicopathologic variables by AUC enabled quick and easy differentiation of variables associated with high and low recurrence risk. However, variations in breast cancer recurrence are rarely affected by a single factor. Similarly, we found that the contribution of each variable may be too weak (<0.7) in terms of AUC.

A recent study developed a cumulated ROC analysis strategy for assessing outcomes of orthodontic surgery [23] and for diagnosing metastasis in breast cancer [24] and other cancer types [25, 26]. In our study, a similar scoring system was applied in multivariate cumulated ROC analysis to evaluate diagnostic indicators of breast cancer recurrence. Table 3 shows that 10 variables were the minimum number of discriminators required to obtain the optimum discriminant effectiveness. Table 4 further shows that a cut-off point of 6 had the best combination of sensitivity and specificity for predicting breast cancer recurrence. These results suggest that the cumulated ROC analysis strategy also improves accuracy in predicting breast cancer recurrence.

Moreover, these data suggest that both gene-environment and environment-environment interactions have important roles in predicting recurrence when clinicopathologic variables are regarded as the environmental factors. Similar interactions between these variables have been reported. For example, ER and HER2 are reportedly both interdependent and independent prognostic indicators of breast cancer recurrence [27]. The Nottingham prognostic index considers tumor size, lymph node metastasis, and tumor grade to obtain an estimate of recurrence risk in patients with breast cancer [28, 29]. In contrast, the breast cancer severity score (BCSS) is a prognostic scoring system based on tumor size, number of metastatic lymph nodes, and HER2 status [30]. The BCSS may be the best predictor of both overall survival and disease-free survival. A common feature of these prognostic scoring systems is that all of the breast cancer characteristics mentioned in the current study are considered simultaneously.

Comparisons of these variables further showed that most subjects with a high risk of breast cancer recurrence had a high BMI (Table 5), which is consistent with reports that high BMI is associated with aggressive tumor characteristics in premenopausal [31] and postmenopausal women [32]. Another randomized trial showed that lymph node metastasis, ER-negativity, and HER2 negativity are associated with breast cancer risk and prognosis [33]. Breast cancer patients with the triple negative subtype (i.e., negativity for ER, PR, and HER2) have a high risk of disease progression [34]. Supplementary Table  1 (in Supplementary Material available online at https://doi.org/10.1155/2017/2563910) shows that MRE11 expression did not significantly differ among ER-positivity, ER-negativity, PR-positivity, or PR-negativity in the current study. Supplementary Table  2 further shows that BMI level did not significantly differ among ER-positivity, ER-negativity, PR-positivity, PR-negativity, HER2-positivity, or HER2-negativity.

Some limitations of this study should be noted. First, our previous work [9] suggested a survival analysis of the current dataset using Cox-proportional hazard model, Kaplan-Meier curve, and log-rank test. However, survival was not considered in the follow-up analyses of subjects in the current study. Another limitation is that, although the cumulated ROC analysis revealed potential joint effects of the selected clinicopathologic variables, the complex interactions of all possible combinations of clinicopathologic variables were not analyzed because the cumulated ROC was initially based on the highest ranked AUC for a single variable. In subsequent analyses, the variable with the lowest AUC was not considered. Moreover, this study did not validate the signature identified by ROC analysis in a testing dataset. Therefore, the use of a testing dataset and intelligent computational algorithms [3544] is warranted in future studies of the complex high-order interactions between these clinicopathologic variables and breast cancer recurrence.

5. Conclusions

This study evaluated the contributions of 13 clinicopathologic variables in predicting breast cancer recurrence. In individual ROC analysis, each variable had a weak AUC for predicting breast cancer recurrence. In cumulative ROC analysis, however, each variable had an improved AUC. Finally, this study revealed that 10 clinicopathologic variables is the minimum number of discriminators needed for optimum accuracy in predicting breast cancer recurrence by discriminant analysis.

Competing Interests

The authors have no competing interests to declare.

Acknowledgments

This work was supported by grants from the Ministry of Science and Technology (MOST 104-2320-B-037-013-MY3, MOST 103-2221-E-151-029-MY3), the National Sun Yat-sen University-KMU Joint Research Project (#NSYSU-KMU 105-p022), the Chimei-KMU jointed project (105CM-KMU-06), and the Health and Welfare Surcharge of Tobacco Products, the Ministry of Health and Welfare, Taiwan (MOHW105-TDU-B-212-134007, MOHW104-TD-B-111-05, and MOHW104-TDU-B-212-124-003).

References

  1. E. M. Ibrahim, M. E. Al-Foheidi, M. M. Al-Mansour, and G. A. Kazkaz, “The prognostic value of tumor-infiltrating lymphocytes in triple-negative breast cancer: a meta-analysis,” Breast Cancer Research and Treatment, vol. 148, no. 3, pp. 467–476, 2014. View at Publisher · View at Google Scholar · View at Scopus
  2. Z.-Y. Yang, M.-Y. Di, J.-Q. Yuan et al., “The prognostic value of phosphorylated Akt in breast cancer: a systematic review,” Scientific Reports, vol. 5, article no. 7758, 2015. View at Publisher · View at Google Scholar · View at Scopus
  3. K. N. Stevens, C. M. Vachon, and F. J. Couch, “Genetic susceptibility to triple-negative breast cancer,” Cancer Research, vol. 73, no. 7, pp. 2025–2030, 2013. View at Publisher · View at Google Scholar · View at Scopus
  4. W.-P. Chang, M.-E. Liu, W.-C. Chang et al., “Sleep apnea and the subsequent risk of breast cancer in women: a nationwide population-based cohort study,” Sleep Medicine, vol. 15, no. 9, pp. 1016–1020, 2014. View at Publisher · View at Google Scholar · View at Scopus
  5. A. Goldhirsch, W. C. Wood, R. D. Gelber, A. S. Coates, B. Thurlimann, and H. J. Senn, “Progress and promise: highlights of the international expert consensus on the primary therapy of early breast cancer 2007,” Annals of Oncology, vol. 18, no. 7, pp. 1133–1144, 2007. View at Publisher · View at Google Scholar
  6. N. J. Bundred, “Prognostic and predictive factors in breast cancer,” Cancer Treatment Reviews, vol. 27, no. 3, pp. 137–142, 2001. View at Publisher · View at Google Scholar · View at Scopus
  7. L. Ohno-Machado, “Modeling medical prognosis: survival analysis techniques,” Journal of Biomedical Informatics, vol. 34, no. 6, pp. 428–439, 2001. View at Publisher · View at Google Scholar · View at Scopus
  8. J. A. Ludwig and J. N. Weinstein, “Biomarkers in cancer staging, prognosis and treatment selection,” Nature Reviews Cancer, vol. 5, no. 11, pp. 845–856, 2005. View at Publisher · View at Google Scholar · View at Scopus
  9. S.-S. F. Yuan, M.-F. Hou, Y.-C. Hsieh et al., “Role of MRE11 in cell proliferation, tumor invasion, and DNA repair in breast cancer,” Journal of the National Cancer Institute, vol. 104, no. 19, pp. 1485–1502, 2012. View at Publisher · View at Google Scholar · View at Scopus
  10. C.-J. Wang, H. Frånbergh-Karlson, D.-W. Wang, G. Arbman, H. Zhang, and X.-F. Sun, “Clinicopathological significance of BTF3 expression in colorectal cancer,” Tumor Biology, vol. 34, no. 4, pp. 2141–2146, 2013. View at Publisher · View at Google Scholar · View at Scopus
  11. E. A. Robertson and M. H. Zweig, “Use of receiver operating characteristic curves to evaluate the clinical-performance of analytical systems,” Clinical Chemistry, vol. 27, no. 9, pp. 1569–1574, 1981. View at Google Scholar · View at Scopus
  12. M. H. Zweig and G. Campbell, “Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine,” Clinical Chemistry, vol. 39, no. 4, pp. 561–577, 1993. View at Google Scholar · View at Scopus
  13. J. Gu, S. Ghosal, and A. Roy, “Bayesian bootstrap estimation of ROC curve,” Statistics in Medicine, vol. 27, no. 26, pp. 5407–5420, 2008. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  14. C.-Y. Yen, C.-H. Chen, C.-H. Chang et al., “Matrix metalloproteinases (MMP) 1 and MMP10 but not MMP12 are potential oral cancer markers,” Biomarkers, vol. 14, no. 4, pp. 244–249, 2009. View at Publisher · View at Google Scholar · View at Scopus
  15. C.-Y. Yen, C.-Y. Huang, M.-F. Hou et al., “Evaluating the performance of fibronectin 1 (FN1), integrin α4β1 (ITGA4), syndecan-2 (SDC2), and glycoprotein CD44 as the potential biomarkers of oral squamous cell carcinoma (OSCC),” Biomarkers, vol. 18, no. 1, pp. 63–72, 2013. View at Publisher · View at Google Scholar · View at Scopus
  16. I.-M. Shih, “Application of human leukocyte antigen-G expression in the diagnosis of human cancer,” Human Immunology, vol. 68, no. 4, pp. 272–276, 2007. View at Publisher · View at Google Scholar · View at Scopus
  17. H. Jin, D. S. Daly, J. R. Marks, and R. C. Zangar, “Oxidatively modified proteins as plasma biomarkers in breast cancer,” Cancer Biomarkers, vol. 13, no. 3, pp. 193–200, 2013. View at Publisher · View at Google Scholar · View at Scopus
  18. D. Baskic, S. Popovic, D. Bankovic et al., “Evaluation of inflammatory biomarkers as helping diagnostic tool in patients with breast cancer,” Cancer Biomarkers, vol. 14, no. 6, pp. 401–408, 2014. View at Publisher · View at Google Scholar · View at Scopus
  19. Y. Zhao, M. Wang, C. Cui et al., “Significance of combined tests of serum golgi glycoprotein 73 and other biomarkers in diagnosis of small primary hepatocellular carcinoma,” Cancer Biomarkers, vol. 15, no. 5, pp. 677–683, 2015. View at Publisher · View at Google Scholar · View at Scopus
  20. M. Qin, G. Liu, X. Huo et al., “Hsa_circ_0001649: a circular RNA and potential novel biomarker for hepatocellular carcinoma,” Cancer Biomarkers, vol. 16, no. 1, pp. 161–169, 2016. View at Publisher · View at Google Scholar · View at Scopus
  21. J. A. Hanley and B. J. McNeil, “A method of comparing the areas under receiver operating characteristic curves derived from the same cases,” Radiology, vol. 148, no. 3, pp. 839–843, 1983. View at Publisher · View at Google Scholar · View at Scopus
  22. C. Sotiriou, S.-Y. Neo, L. M. McShane et al., “Breast cancer classification and prognosis based on gene expression profiles from a population-based study,” Proceedings of the National Academy of Sciences of the United States of America, vol. 100, no. 18, pp. 10393–10398, 2003. View at Publisher · View at Google Scholar · View at Scopus
  23. Y.-C. Tseng, C.-Y. Pan, S.-T. Chou et al., “Treatment of adult Class III malocclusions with orthodontic therapy or orthognathic surgery: receiver operating characteristic analysis,” American Journal of Orthodontics and Dentofacial Orthopedics, vol. 139, no. 5, pp. e485–e493, 2011. View at Publisher · View at Google Scholar · View at Scopus
  24. H. A. El-Mezayen, F. M. Metwally, and H. Darwish, “A novel discriminant score based on tumor-associated trypsin inhibitor for accurate diagnosis of metastasis in patients with breast cancer,” Tumor Biology, vol. 35, no. 3, pp. 2759–2767, 2014. View at Publisher · View at Google Scholar · View at Scopus
  25. R. Rozalski, D. Gackowski, A. Siomek-Gorecka et al., “Urinary 5-hydroxymethyluracil and 8-oxo-7,8-dihydroguanine as potential biomarkers in patients with colorectal cancer,” Biomarkers, vol. 20, no. 5, pp. 287–291, 2015. View at Publisher · View at Google Scholar · View at Scopus
  26. M. De Paoli, P. Perco, I. Mühlberger et al., “Disease map-based biomarker selection and pre-validation for bladder cancer diagnostic,” Biomarkers, vol. 20, no. 5, pp. 328–337, 2015. View at Publisher · View at Google Scholar · View at Scopus
  27. R. A. Walker and A. M. Thompson, Prognostic and Predictive Factors in Breast Cancer, Informa Healthcare, London, UK, 2008.
  28. G. D'Eredita', C. Giardina, M. Martellotta, T. Natale, and F. Ferrarese, “Prognostic factors in breast cancer: the predictive value of the Nottingham Prognostic Index in patients with a long-term follow-up that were treated in a single institution,” European Journal of Cancer, vol. 37, no. 5, pp. 591–596, 2001. View at Publisher · View at Google Scholar · View at Scopus
  29. R. S. Rampaul, S. E. Pinder, C. W. Elston, and I. O. Ellis, “Prognostic and predictive factors in primary breast cancer and their role in patient management: The Nottingham Breast Team,” European Journal of Surgical Oncology, vol. 27, no. 3, pp. 229–238, 2001. View at Publisher · View at Google Scholar · View at Scopus
  30. R. Jimenez-Lee, B. Ham, J. Vetto, and R. Pommier, “Breast cancer severity score is an innovative system for prognosis,” American Journal of Surgery, vol. 186, no. 4, pp. 404–408, 2003. View at Publisher · View at Google Scholar · View at Scopus
  31. A. Yanai, Y. Miyagawa, K. Murase et al., “Influence of body mass index on clinicopathological factors including estrogen receptor, progesterone receptor, and Ki67 expression levels in breast cancers,” International Journal of Clinical Oncology, vol. 19, no. 3, pp. 467–472, 2014. View at Publisher · View at Google Scholar · View at Scopus
  32. N. Biglia, E. Peano, P. Sgandurra et al., “Body mass index (BMI) and breast cancer: impact on tumor histopatologic features, cancer subtypes and recurrence rate in pre and postmenopausal women,” Gynecological Endocrinology, vol. 29, no. 3, pp. 263–267, 2013. View at Publisher · View at Google Scholar · View at Scopus
  33. S. Loi, N. Sirtaine, F. Piette et al., “Prognostic and predictive value of tumor-infiltrating lymphocytes in a phase III randomized adjuvant breast cancer trial in node-positive breast cancer comparing the addition of docetaxel to doxorubicin with doxorubicin-based chemotherapy: BIG 02-98,” Journal of Clinical Oncology, vol. 31, no. 7, pp. 860–867, 2013. View at Publisher · View at Google Scholar · View at Scopus
  34. I. Cetin and M. Topcul, “Triple negative breast cancer,” Asian Pacific Journal of Cancer Prevention, vol. 15, no. 6, pp. 2427–2431, 2014. View at Publisher · View at Google Scholar · View at Scopus
  35. C.-H. Yang, H.-W. Chang, Y.-H. Cheng, and L.-Y. Chuang, “Novel generating protective single nucleotide polymorphism barcode for breast cancer using particle swarm optimization,” Cancer Epidemiology, vol. 33, no. 2, pp. 147–154, 2009. View at Publisher · View at Google Scholar · View at Scopus
  36. T. N. Wang, C. H. Cheng, and H. W. Chiu, “Predicting post-treatment survivability of patients with breast cancer using Artificial Neural Network methods,” in Proceedings of the 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2013), pp. 1290–1293, Osaka, Japan, July 2013. View at Publisher · View at Google Scholar · View at Scopus
  37. C.-H. Yang, Y.-D. Lin, L.-Y. Chuang, J.-B. Chen, and H.-W. Chang, “MDR-ER: balancing functions for adjusting the ratio in risk classes and classification errors for imbalanced cases and controls using multifactor-dimensionality reduction,” PLoS ONE, vol. 8, no. 11, Article ID e79387, 2013. View at Publisher · View at Google Scholar · View at Scopus
  38. C.-H. Yang, L.-Y. Chuang, Y.-J. Chen, H.-F. Tseng, and H.-W. Chang, “Computational analysis of simulated SNP interactions between 26 growth factor-related genes in a breast cancer association study,” OMICS, vol. 15, no. 6, pp. 399–407, 2011. View at Publisher · View at Google Scholar · View at Scopus
  39. H.-W. Chang, L.-Y. Chuang, C.-H. Ho, P.-L. Chang, and C.-H. Yang, “Odds ratio-based genetic algorithms for generating SNP barcodes of genotypes to predict disease susceptibility,” OMICS A Journal of Integrative Biology, vol. 12, no. 1, pp. 71–81, 2008. View at Publisher · View at Google Scholar · View at Scopus
  40. C.-H. Yang, Y.-D. Lin, C.-Y. Yen, L.-Y. Chuang, and H.-W. Chang, “A systematic gene-gene and gene-environment interaction analysis of DNA repair genes XRCC1, XRCC2, XRCC3, XRCC4, and oral cancer risk,” OMICS, vol. 19, no. 4, pp. 238–247, 2015. View at Publisher · View at Google Scholar · View at Scopus
  41. C.-H. Yang, L.-Y. Chuang, Y.-H. Cheng et al., “Single nucleotide polymorphism barcoding to evaluate oral cancer risk using odds ratio-based genetic algorithms,” Kaohsiung Journal of Medical Sciences, vol. 28, no. 7, pp. 362–368, 2012. View at Publisher · View at Google Scholar · View at Scopus
  42. L.-Y. Chuang, H.-W. Chang, M.-C. Lin, and C.-H. Yang, “Chaotic particle swarm optimization for detecting SNP-SNP interactions for CXCL12-related genes in breast cancer prevention,” European Journal of Cancer Prevention, vol. 21, no. 4, pp. 336–342, 2012. View at Publisher · View at Google Scholar · View at Scopus
  43. L.-Y. Chuang, Y.-D. Lin, H.-W. Chang, and C.-H. Yang, “An improved PSO algorithm for generating protective SNP barcodes in breast cancer,” PLoS ONE, vol. 7, no. 5, Article ID e37018, 2012. View at Publisher · View at Google Scholar · View at Scopus
  44. K. Kourou, T. P. Exarchos, K. P. Exarchos, M. V. Karamouzis, and D. I. Fotiadis, “Machine learning applications in cancer prognosis and prediction,” Computational and Structural Biotechnology Journal, vol. 13, pp. 8–17, 2015. View at Publisher · View at Google Scholar · View at Scopus