- About this Journal ·
- Abstracting and Indexing ·
- Advance Access ·
- Aims and Scope ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
International Journal of Biomedical Imaging
Volume 2011 (2011), Article ID 846312, 10 pages
Conditional Random Fields and Supervised Learning in Automated Skin Lesion Diagnosis
1Department of Computing Science, Simon Fraser University, Burnaby, BC, Canada V5A 1S6
2Department of Dermatology and Skin Science, Photomedicine Institute, University of British Columbia and Vancouver Coastal Health Research Institute, Vancouver, BC, Canada V5Z 4E8
3Cancer Control Research Program and Integrative Oncology Department, BC Cancer Research Centre, Vancouver, BC, Canada V5Z 4E6
Received 2 March 2011; Revised 21 May 2011; Accepted 23 July 2011
Academic Editor: Fei Wang
Copyright © 2011 Paul Wighton et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Many subproblems in automated skin lesion diagnosis (ASLD) can be unified under a single generalization of assigning a label, from an predefined set, to each pixel in an image. We first formalize this generalization and then present two probabilistic models capable of solving it. The first model is based on independent pixel labeling using maximum a-posteriori (MAP) estimation. The second model is based on conditional random fields (CRFs), where dependencies between pixels are defined using a graph structure. Furthermore, we demonstrate how supervised learning and an appropriate training set can be used to automatically determine all model parameters. We evaluate both models' ability to segment a challenging dataset consisting of 116 images and compare our results to 5 previously published methods.
Incidence rates of melanoma are increasing rapidly in the western world, growing faster than any other cancer . Since there is no effective therapy for patients with advanced melanoma , educational campaigns attempt to encourage high-risk individuals to undergo routine screening so that melanomas can be identified early while they are still easily treatable . While worthwhile, these educational campaigns generate a large amount of referrals to dermatologists, whose services are already undersupplied .
Automated skin lesion diagnosis (ASLD) is expected to alleviate some of this burden. By acting as a screening tool, ASLD can reject obviously benign lesions, while referring more suspicious ones to an expert for further scrutiny. Most ASLD methods adopt the standard computer-aided diagnosis (CAD) pipeline illustrated in Figure 1. First an image is acquired with a digital dermoscope. Next, undesirable artifacts (such as hair or oil bubbles) are identified and, if necessary, replaced with an estimate of the underlying skin color. After this, the lesion is segmented, and discriminative features are then extracted. Finally, supervised learning is used to classify previously unseen images.
Our previous work demonstrated how the use of supervised learning, under the proper generalization (of assigning labels to pixels), was able to solve several tasks in this pipeline including detecting occluding hair, segmenting the lesions, and detecting the dermoscopic structure pigment network . Our method was relatively simple; it labeled pixels in an image independently using modest features, linear discriminant analysis (LDA) for supervised dimensionality reduction, and maximum a-posteriori (MAP) estimation. Nevertheless, in spite of its simplicity, our model was able to perform comparably to other previously published, nongeneral methods for lesion segmentation [6–10] and hair detection .
In this paper, we seek to expand on this generalization by replacing the per-pixel (PP) estimation model with a conditional random field (CRF) model. The largest criticism levied at the PP approach is that pixels are labeled independently, regardless of the label assigned to their neighbors. This assumption of independence is clearly not valid, as there is a high degree of correlation between neighboring pixels in any image (any image other than pure noise, i.e.). CRF-based models attempt to relax this assumption of independence by creating a graphical model which defines the dependencies between pixels.
In order to apply a CRF model, a parameter vector specifying the relative contribution of the input features is required. Often, these parameters are determined in an ad hoc fashion via trial and error. Since our goal is a general method, easily applicable to a variety of problems, it is crucial that these parameters be determined automatically based on observations. We, therefore, apply the maximum likelihood estimator for the parameter vector and describe a gradient-based method for its computation. We also address many practical considerations encountered during the implementation.
The paper is organized as follows: in Section 2, we briefly review previous work. In Section 3, we formulate the generalization in Section 3.1, review our previous PP model  in Section 3.2, and present our CRF model in Section 3.3. In Section 4, we present results. Finally, we conclude in Section 5.
2. Previous Work
Our original PP model was based on the work by Debeir et al.  who also attempts to generalize many tasks in ASLD. Our model was found to perform comparably to many published lesion segmentation algorithms including K-means++ (KPP) , J-image segmentation (JSEG) , dermatologist-like tumor area extraction algorithm (DTEA) , statistical region merging (SRM) , and threshold fusion (FSN) . It also performed comparably to DullRazor  for detecting occluding hair and was able to identify the dermoscopic structure pigment network. Our PP model is briefly reviewed in Section 3.2; however, we refer readers to our previous study for further details, as well as a more comprehensive review of previous work in ASLD, including the methods against which we compare .
We defer the review of the CRF model until Section 3.3, where we examine it in detail.
This section is divided into 3 parts. We begin in Section 3.1 by formalizing the generalization that is capable of performing a variety of tasks in ASLD. In Section 3.2, we briefly review our previous PP model . Finally, in Section 3.3, we outline our CRF model.
3.1. The Generalization
We are given a set of observations , consisting of images () and corresponding ground truths labeling (). Using the notation of Szummer et al. , the superscript or indexes a specific image/labeling in the set and the subscript or indexes a specific pixel. Let represent the number of images, and represents the number of pixels in image . An imageset can contain any number of channels (or features), which we denote by . Valid values for each entry in the label field () are defined by the label set , where is the number of possible labels.
Given our training set , we use supervised learning to predict the label fields for previously unseen images.
Formally, we are given
And our task is to use the information in to infer the function that produces the best possible label field.
3.2. The PP Model
In this section, we briefly review our per-pixel (PP) estimation model . An overview of the training and testing phases of the model is illustrated in Figures 2 and 3, respectively. Under this model, we assign the most probable label to each pixel independently
Applying Bayes’ rule and simplifying, we arrive at the standard maximum likelihood formulation
We model the posterior as a set of multivariate normal distributions , whose parameters () are estimated using the training set . We model as a discrete distribution. Let represent the number of elements in that assume the value , then
We also normalize the probabilities across the label set, which are later used as features in the CRF model. The normalized likelihood that a pixel is associated with the label is
In order to examine the model’s performance across the entire sensitivity/specificity range, we consider many thresholds on over the range and label pixels accordingly.
As the number of channels () in the images grows, we perform supervised dimensionality reduction on the observations to focus the predictive power of our dataset onto a smaller subset of parameters. Linear discriminant analysis (LDA) is used to determine the subspace of which best separates the labels .
LDA performs an eigenvalue decomposition of a scatter matrix representing the ratio of between-class covariance to within-class covariance. It returns a matrix of eigenvectors which projects observations () from dimensions to where is the overall mean of across all images and classes. Once the projection is determined, the posteriors are estimated, likelihoods are computed, and inference is performed in this subspace () where the superscript () is used to differentiate the label means/covariances in this subspace from the original space in which the observations were performed ().
3.3. The CRF Model
In this section, we seek to improve upon the PP model developed in previous work  and described in Section 3.2. We present an overview of conditional random fields (CRFs) in Section 3.3.1. In Section 3.3.2, we describe how the CRF parameters can be determined empirically using maximum likelihood estimation (MLE) . In Section 3.3.3, we discuss practical considerations for finding these parameters, including how to estimate the partition function  and how to regularize the likelihood expression . In Section 3.3.4, we solve the MLE formulation via gradient-based methods. An overview of the training and testing phases of our CRF model is illustrated in Figures 4 and 5, respectively.
The CRF model is an undirected graphical model that is naturally suited to represent and exploit the dependencies between observations, such as neighboring pixels in an image . The probability that a label field is associated with the image under model parameters is given by where the function , known as the partition function, is used to normalize the probabilities for given values of and
The energy function represents the linear of combination of features employed by the model and is parameterized by the weight vector
Given an undirected graph , where represents the nodes (i.e., pixels) of an observation, represents the dependencies between nodes (throughout this document, is the 4-connected set of neighboring pixels), and the energy function is the weighted sum of features . Features can either operate over the nodes of the graph (), or over its edges ()
In order for the model to be tractable, edge features , and their corresponding weights must adhere to certain constraints. Let represent the set of edge features. The following constraints must be satisfied 
Strictly speaking, the second constraint can be replaced with the more general constraint that edge feature functions be submodular . However, throughout this document, we will impose this stricter constraint which can be interpreted as “an edge cost is only incurred across nodes with differing labels.”
A CRF solver is one that, given observations and parameters , can find the most likely labeling
3.3.2. Determining MLE Parameters
Since the emphasis of our work is on a general model capable of performing a variety of tasks, it is crucial that model parameters () be determined automatically from training data via empirical means. In this section, we derive the partial derivatives of the likelihood function which can be used by gradient-based methods to compute .
Since the observations are assumed to be independent. The likelihood of the data, given the set of parameters, is equal to the product of the probabilities in the observed set, under those parameters
The maximum likelihood estimator is then
If we can find the partial derivatives , we can optimize using gradient-based methods. We begin by expressing the likelihood function in terms of
Solving for the partial derivatives, we get the following expression for the gradients of the likelihood function: However, we now come to an impasse. The second term of (18) would have us iterating over all possible label fields . For a binary classification task over a modestly sized image of 256 × 128, this would require a summation over labelings. Clearly this is intractable, and we must resort to estimating this second term.
3.3.3. Practical Considerations
In order to derive CRF parameters with grid-structured models for even modestly sized images, a method to estimate the partition function is required. Inspired by , we employ one of the simplest estimation methods and approximate the partition function using saddle-point approximation (SPA) 
We also introduce an additional practical consideration. Since gradient-based methods will be used to determine , we regularize the likelihood function () by the squared L2 norm of the parameters  to penalize large weight vectors (since scalar multiples of a weight vector produce identical results). This makes the resulting likelihood function strictly convex. The regularized likelihood is then And the gradients become Which under SPA becomes
We are now ready to implement a gradient-based method to estimate the CRF parameter vector . Given an initial weight vector , the gradients of the likelihood function are estimated as per (22). These gradients are used to update the weight vector, which in turn is used to estimate a new set of gradients. This process is repeated until convergence.
We have observed (as does ) that gradient methods using saddle point approximation lead to oscillating behavior. Therefore, we keep a record of the best empirical set of parameters found, rather than the parameters of the final iteration. We also enforce the constraint from (13) that weights for edge-based features must remain positive.
In addition to the training set (), the algorithm also requires an initial weight vector (), a regularization factor ), a step size (), and termination conditions (convergence criteria: ; maximum number of iterations: ). The algorithm has been found to be robust to these additional parameters. Pseudocode of our implementation is presented in Algorithm 1.
Previous work has demonstrated our model’s ability to generalize to many applications . Here, we focus on a single application (lesion segmentation) and present results for our two models. We also compare our results to 5 previously published methods (KPP , JSEG , DTEA , SRM , and FSN ).
The dataset consists of 116 images from dermoscopy atlases [22, 23], which were acquired by a several dermatologists in separate practices using differing equipment. The images have not been properly color calibrated. Since the goal was to create a difficult dataset, 100 of the 116 lesions were selected to be particularly challenging to segmentation algorithms . We intentionally chose a simplistic featureset to emphasize the power of the models under consideration.
The features employed were 5 Gaussian, and 5 Laplacian of Gaussian filters applied a various scales () in each channel of the image in CIE L*a*b* space. The responses of these filters represent the observations (where ). Each image was expertly segmented by a dermatologist. These ground truth labelings are denoted as .
For all experiments, 10-fold cross-validation was employed. The dataset was randomly divided into 10 groups, and label fields for each group of images were determined using model parameters which were estimated from the observations in the 9 other groups. In both the PP and CRF models, all steps after the computation of features (refer to Figures 2 and 4) were included within the cross-validation loop including determining the projection , estimating the prior/posteriors, determining CRF parameter vector , and so forth.
4.1. The PP Model
We begin by summarizing previous results on how our PP model faired on this dataset. A more detailed analysis, including the relative contribution of various aspects of the model (including features, dimensionality reduction, and classification method), can be found in our previous work .
Since that time, we have discovered that we can partially compensate for the lack of color calibration by subtracting the mean of the L* channel before computing features. While not as desirable as full color and lighting calibration , this procedure at least compensates for various camera exposure levels, as can be seen in the resulting PP likelihood maps in Figure 6 (as calculated by (8)). Figure 7 illustrates a ROC curve comparing the performance of our PP model (before and after normalization) to the segmentation algorithms KPP , JSEG , DTEA , SRM , and FSN . Our method performs comparably to JSEG, DTEA, SRM, and FSN and outperforms KPP although only KPP, DTEA, and FSN algorithms were able to generate results for all 116 images. Table 1 summarizes the results.
4.2. The CRF Model
As described in Section 3.3, the CRF model operates over an undirected graph and consists of node features () and edge features (). The graph structure employed was the 4-connected set of neighboring pixels. Our featureset contains 2 features: one over the nodes and one over the edges. The node features are the likelihoods as computed by (8) of the PP model as in Section 4.1, and the edge features are set to the CIE L* intensity difference between neighboring pixels, if the labels of said pixels differ where we use to denote the indicator function (i.e., evaluates to 1 if ; 0 otherwise)
While the method described in Section 3.3 is general enough to handle an arbitrary number of node and edge features, there are 2 reasons why we chose only one of each. To begin, we seek to make the comparison between the PP model and the CRF model as meaningful as possible. Using the likelihoods from the PP model as the node feature is an elegant way to evaluate the improvements realized by the CRF model. Note that with this choice of features, the CRF model with weight vector gives identical results to the PP model. Additionally, the saddle-point method for approximating the partition function seems to degrade as the number of features increases. We note, however, that even in studies where the partition function can be computed exactly (because the CRF graph contains no loops), the loss incurred by such piecewise training methods is negligible .
Figure 8 compares some segmentations produced by the PP and CRF model. By relaxing the assumption of independence in the PP model, the CRF model is able to smooth over small areas of discontinuity, filling in “gaps” in segmentations, and removing noise. In Figures 8(a) and 8(c), the “holes” in the resulting PP segmentations do not manifest in the CRF segmentations (Figures 8(b) and 8(d)) due to the model’s holistic search for the best label field, rather than best individual label. Additionally, while the PP model is already fairly robust to occluding hair (Figure 8(e)), the CRF model is even more robust, able to smooth over misclassifications due to artifacts.
We also tested the stability of the CRF model with respect to regularization and the hyperparameter . Varying (to assume values in the range ) had little effect on performance of the model on this particular dataset. In spite of the seemingly ineffectual nature of this parameter, we do not remove it from the model since the emphasis of this work is on general models for ASLD. The effect of in general (over many tasks in ASLD) has yet to be determined.
While subjectively, the CRF model offers substantial improvements; objectively, the CRF model is a marginal improvement over the PP model. Figure 9 shows an ROC curve comparing the CRF’s performance to that of the PP model and previously published methods, and Table 1 summarizes the results.
In this paper, we have generalized several common problems in ASLD into a single formulation. We also presented 2 probabilistic models capable of solving the formulation, and described how supervised learning can be used to determine all model parameters. Since the parameters for the resulting models can all be determined automatically from training data, it is hoped that these models can be applied quickly and effectively to a variety of relevant tasks in ASLD.
While both methods perform comparably to previously published methods, the qualitative improvements realized by CRF model aren’t reflected in the quantitative score. Unlike the PP model, the CRF model does not assign labels to pixels independently. Rather, the CRF model selects the best label field to assign to an image. This allows the CRF model to fill in “holes” and smooth out noise that would otherwise appear.
The discrepancy between the objective and subjective performance of the CRF model implies that our evaluation metric (pixel-wise sensitivity and specificity) may be less than ideal. Therefore, future work will explore the use of alternate evaluation metrics [26, 27].
Even though the models presented are competitive, there are many potential directions in which they can be improved upon even further. In our grid-structured CRF model, we must resort to approximating the partition function due to the computational complexity of calculating it exactly. Imposing a tree-based structure over the image  would enable the exact computation of the partition function via dynamic programming and should lead to more reliable CRF parameters. Replacing our gradient-based method for determining CRF parameters with a max-margin formulation  is another possible way to increase the reliability of the resulting parameters. We can also induce non-linearities into the model by replacing the linear dimensionality reduction step (LDA) with its nonlinear counterparts (i.e., KLDA ). Finally, the use of semi-supervised learning techniques may be used to decrease the cost of acquiring expertly annotated datasets .
This work was supported in part by the Canadian Institutes of Health Research (CIHR) Skin Research Training Centre, the Canadian Dermatology Foundation, and Collaborative Health Research Projects Program, a grant jointly funded by the Natural Sciences and Engineering Research Council of Canada and the Canadian Institutes of Health Research. The Authors would like to thank Dr. M. Emre Celebi for making available results of the alternative segmentation algorithms.
- C. Erickson and M. Driscoll, “Melanoma epidemic: facts and controversies,” Clinics in Dermatology, vol. 28, no. 3, pp. 281–286, 2010.
- M. Lens and M. Dawes, “Global perspectives of contemporary epidemiological trends of cutaneous malignant melanoma,” British Journal of Dermatology, vol. 150, no. 2, pp. 179–185, 2004.
- R. MacKie and D. Hole, “Audit of public education campaign to encourage earlier detection of malignant melanoma,” British Medical Journal, vol. 304, no. 6833, pp. 1012–1015, 1992.
- J. Resneck and A. B. Kimball, “The dermatology workforce shortage,” Journal of the American Academy of Dermatology, vol. 50, no. 1, pp. 50–54, 2004.
- P. Wighton, T. Lee, H. Lui, D. McLean, and M. Atkins, “Generalizing common tasks in automated skin lesion diagnosis,” IEEE Transactions on Information Technology in BioMedicine, vol. 15, no. 4, pp. 622–629, 2011.
- H. Zhou, M. Chen, L. Zou et al., “Spatially constrained segmentation of dermoscopy images,” in Proceedings of the 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro (ISBI '08), pp. 800–803, Paris, France, 2008.
- M. E. Celebi, Y. Aslandogan, W. Stoecker, H. Iyatomi, H. Oka, and X. Chen, “Unsupervised border detection in dermoscopy images,” Skin Research and Technology, vol. 13, no. 4, pp. 454–462, 2007.
- H. Iyatomi, H. Oka, M. E. Celebi et al., “An improved Internet-based melanoma screening system with dermatologist-like tumor area extraction algorithm,” Computerized Medical Imaging and Graphics, vol. 32, no. 7, pp. 566–579, 2008.
- M. E. Celebi, H. Kingravi, H. Iyatomi et al., “Border detection in dermoscopy images using statistical region merging,” Skin Research and Technology, vol. 14, no. 3, pp. 347–353, 2008.
- M. E. Celebi, S. Hwang, H. Iyatomi, and G. Schaefer, “Robust border detection in dermoscopy images using threshold fusion,” in Proceedings of the 17th IEEE International Conference on Image Processing (ICIP '10), pp. 2541–2544, Hong Kong, 2010.
- T. Lee, V. Ng, R. Gallagher, A. Coldman, and D. McLean, “Dullrazor®: a software approach to hair removal from images,” Computers in Biology and Medicine, vol. 27, no. 6, pp. 533–543, 1997.
- O. Debeir, C. Decaestecker, J. Pasteels, I. Salmon, R. Kiss, and P. van Ham, “Computer-assisted analysis of epiluminescence microscopy images of pigmented skin lesions,” Cytometry, vol. 37, no. 4, pp. 255–266, 1999.
- M. Szummer, P. Kohli, and D. Hoiem, “Learning CRFs using graph cuts,” in Proceedings of the 10th European Conference on Computer Vision (ECCV '08), pp. 582–595, Marseille, France, 2008.
- A. Martínez and A. Kak, “Pca versus lda,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 228–233, 2002.
- C. Sutton and A. McCallum, “An introduction to conditional random fields for relational learning,” in Introduction to Statistical Relational Learning, p. 93, 2007.
- D. Geiger and F. Girosi, “Parallel and deterministic algorithms from MRFs: surface reconstruction,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 13, no. 5, pp. 401–412, 2002.
- H. Ishikawa, “Exact optimization for Markov random fields with convex priors,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 10, pp. 1333–1336, 2003.
- V. Kolmogorov, “Primal-dual algorithm for convex Markov random fields,” Microsoft Research MSR-TR-2005-117, 2005.
- N. Komodakis and G. Tziritas, “Approximate labeling via graph cuts based on linear programming,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 8, pp. 1436–1453, 2007.
- N. Komodakis, G. Tziritas, and N. Paragios, “Performance vs computational efficiency for optimizing single and dynamic MRFs: setting the state of the art with primal-dual strategies,” Computer Vision and Image Understanding, vol. 112, no. 1, pp. 14–29, 2008.
- S. Kumar, J. August, and M. Hebert, “Exploiting inference for approximate parameter learning in discriminative fields: an empirical study,” in Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 153–168, Springer, New York, NY, USA, 2005.
- G. Argenziano and H. Soyer, Interactive Atlas of Dermoscopy (Book and CD-ROM), Edra medical publishing and new media, Milan, Italy, 2000.
- H. Soyer and G. Argenziano, Dermoscopy of Pigmented Skin Lesions. An Atlas based on the Consesnsus Net Meeting on Dermoscopy, Edra medical publishing and new media, Milan, Italy, 2000.
- P. Wighton, T. Lee, H. Lui, D. McLean, and M. Atkins, “Chromatic aberration correction: an enhancement to the calibration of low-cost digital dermoscopes,” Skin Research and Technology, vol. 17, no. 3, pp. 339–347, 2011.
- S. Nowozin, P. Gehler, and C. Lampert, “On parameter learning in CRF-based approaches to object class image segmentation,” in Proceedings of the 11th European Conference on Computer Vision (ECCV '10), pp. 98–111, Crete, Greece, 2010.
- M. Celebi, G. Schaefer, H. Iyatomi, W. Stoecker, J. Malters, and J. Grichnik, “An improved objective evaluation measure for border detection in dermoscopy images,” Skin Research and Technology, vol. 15, no. 4, pp. 444–450, 2009.
- T. Lee, D. McLean, and M. Atkins, “Irregularity index: a new border irregularity measure for cutaneous melanocytic lesions,” Medical Image Analysis, vol. 7, no. 1, pp. 47–64, 2003.
- D. Cai, X. He, and J. Han, “Efficient kernel discriminant analysis via spectral regression,” in Proceedings of the 7th IEEE International Conference on Data Mining (ICDM '07), pp. 427–432, Omaha, Neb, USA, 2007.
- C. Lee, S. Wang, F. Jiao, D. Schuurmans, and R. Greiner, “Learning to model spatial dependency: semi-supervised discriminative random fields,” Advances in Neural Information Processing Systems, vol. 19, pp. 793–800, 2007.