Abstract
Biscuits and cookies are one of the major parts of Indian bakery products. The bake level of biscuits and cookies is of significant value to various bakery products as it determines the taste, texture, number of chocolate chips, uniformity in distribution of chocolate chips, and various features related to appearance of products. Six threshold methods (isodata, Otsu, minimum error, moment preserving, Fuzzy, manual method, and k-mean clustering) have been implemented for chocolate chips extraction from captured cookie image. Various other image processing operations such as entropy calculation, area calculation, parameter calculation, baked dough color, solidity, and fraction of top surface area have been implemented for commercial KrackJack biscuits and cookies. Proposed algorithm is able to detect and investigate about various defects such as crack and various spots. A simple and low cost machine vision system with improved version of robust algorithm for quality detection and identification is envisaged. Developed system and robust algorithm have a great application in various biscuit and cookies baking companies. Proposed system is composed of a monochromatic light source, and USB based 10.0 megapixel camera interfaced with ARM-9 processor for image acquisition. MATLAB version 5.2 has been used for development of robust algorithms and testing for various captured frames. Developed methods and procedures were tested on commercial biscuits resulting in the specificity and sensitivity of more than 94% and 82%, respectively. Since developed software package has been tested on commercial biscuits, it can be programmed to inspect other manufactured bakery products.
1. Introduction
Automating visual inspection based quality control processes are highly desirable for manufacturing companies as it significantly reduces manufacturing costs and can provide for greater accuracy in the monitoring of their manufacturing processes for different biscuits varieties [1]. Physical features present on cookies can be examined by employing various image processing algorithms to extract various features of importance [2]. A wide array of image acquisition technology is available that helps make this automation efficient and cost effective [3]. Developed algorithm also greatly simplifies the image processing techniques used in an automation process by handling at acquisition level common problems such as light reflection and color variation in an acquired image. Various physical parameters have been extracted using developed algorithm such as convex area, solidity, change in color, and convex hull. Higher level of image processing has been implemented for integration of quality features for commercial biscuits collected in database. Very few researchers contributed their work in this area. Some of the reviewed work has been shared as follows. Chocolate chips counting by multiple image operations has been reported by Chang [4]. He used various image processing operations for quality parameters extraction from cookies such as HSI color extraction and various morphological operations (dilation, erosion, and filtering). Automated Tallying of Chocolate Chips Using Color Segmentation has been reported by James Eastman.
Author implemented color segmentation, dynamic threshold, and various morphological operations for extracting and counting the number of chocolate chips from cookies [5]. Roberts and Merat reported about surface feature detection for quality assurance with cookies. Various surface features have been extracted related to quality parameters using color segmentation and morphological operations [6]. Johannsen reported a different methodology for counting the number of chocolate chips from cookies using various morphological operators such as erosion, dilation, and boundary extraction [7]. Presented work has been tested on commercial biscuits but it can be used for other bakery products. Guillard et al. reported a model for change in diffusivity calculation due to moisture transfer at 20°C in various dry biscuits [8]. A prototype artificial neural network system for assessment of bake level of biscuits has been implemented by Yeh and Hamey [9]. A robust automatic crack detection method for nonuniform color distributions on texture images is proposed by Abdullah et al. [10]. In this method, a new image segmentation technique is developed where the Hough transform is used for feature extraction. A simple and low cost machine vision system for fault detection and identification has been developed by Ganesan et al. [11].
2. Material and Methods
2.1. Image Acquisition and Database Preparation
The cookies and commercial biscuits used are from a study [12] investigating the various characteristics about baking time, organic flours, change in color, and so forth. 100 circular KrackJack biscuit pieces (area = 50.26 cm2 and 10 gram) with 50 different cookies (area = 38.48 cm2 and 14 gram) are collected from local market and preserved in different environmental conditions such as moist, open, and packed. A 10.0 megapixel USB based camera has been interfaced with ARM-9 based embedded system for image acquisition. Total twenty cookies and commercial KrackJack biscuit digital images were captured using 10.0 megapixel cameras and saved in JPEG format. Image acquisition experimental setup (Figure 1) is composed of USB based camera with ARM-9 based developed embedded system, monochromatic light source, color filters, and light controlling slits and is installed in a black color aluminum chamber. Color filters have been used for validation of extracted color index using developed algorithm and also for detailed color analysis in RGB domain separately.
Preliminary captured images (Figure 2) brightness and lightness are to be adjusted to −1 and 50 for each color plane (red, green, and blue (RGB)), respectively. Standard image tool software (UTHSCA Image Tool v.2.0) was cropped to 535 535 pixels (representing 40 × 40 mm2 of frame area) and was finally converted into gray-level images (8 bits). Collected biscuits samples have been preserved in different kind of environmental conditions such as controlled ( = 40°C and RH = 80%), moist ( = 15°C and RH = 30%), household refrigerator ( = 8°C and RH = 70%), and open ( = 27°C and RH 50%) (see Figure 3).
(a)
(b)
(c)
(d)
2.2. Proposed Algorithm
A digital image is a two-dimensional discrete function which has been digitized both in spatial coordinates and in magnitude of feature value. Acquired image processed through implemented segmentation process follows through the preprocessing steps like noise removal, image calibration, and so forth. Hundreds of segmentation techniques are present in the literature and almost are all ad hoc in nature [12]. However, neither single method can be considered better enough for all images nor all methods do equally good for a particular type of image [12] (see Figure 4). The calculated entropy is extracted from gray-level cooccurrence matrix (GLCM) for texture quality inspection. Entropy is used to characterize the texture of the segmented image by statistical measure of randomness. Other quality parameters related to physical appearance of the biscuits and cookies have been extracted using area and parameter calculation. Red, green, and blue (RGB); hue, saturation, and intensity (HSI), and gamma (YCbCr) color models have been used for extraction of color with respect to properties for cookies and commercial biscuits (see Table 1).
(a)
(b)
(c)
(d)
(e)
(f)
2.2.1. Segmentation Techniques
Seven standard techniques of threshold have been implemented and compared for segmentation purpose. The exact value of the threshold has considerable effect on effect on the boundary position and therefore on the overall size of the extracted objects. -mean clustering also has been implemented with manual threshold method for segmentation of commercial biscuits and cookies.
(a) Isodata Thresholding Algorithm [13]. This irrelative technique for choosing a threshold was developed by Dias Velasco. The histogram is initially segmented into two parts using a starting threshold value such as , which is half of the maximum dynamic range. Consider where corresponds to various gray levels present in gray-level images.
The sample mean () value of gray values of converted image associated with the foreground pixels and the sample mean () of the gray values associated with the background pixels have been computed. In order to calculate new threshold , system calculates average of the two sample means. This process is repeated until the new calculated threshold value remains unchanged. Consider
(b) Otsu’s Thresholding Algorithm [14]. Otsu’s method [15] has been used to automatically perform clustering based image threshold or the reduction of a gray-level image to a binary image. This technique assumes that the image contains two classes of pixels following bimodal histogram (foreground and background pixels). System then calculates the optimum threshold separating the two classes, so that their combined spread (intraclass variance) is minimal. In this method, algorithm exhaustively extract for the threshold that minimizes an intraclass variance (within the class) can be defined as a weighted sum of variances of two classes, such that where is the probability of the two classes separated by a threshold and is variance of these classes: Otsu shows that minimizing an intraclass variance is the same as maximizing interclass variance: where and correspond to class probabilities and means, respectively. Class probabilities have been computed from the extracted histogram as , while class means are as follows:
(c) Minimum Error Thresholding Algorithm [16]. In this algorithm developed by Kittler and Illingworth, the gray-level histogram is viewed as an estimate of the probability density function of the mixture population comprising the grey levels of the objects and background (). Each element of GL image is normally distributed with mean and standard deviation and probability of . These parameters are generally not usually known and instead a creation function is used: where The optimal threshold is obtained by
(d) k-Mean Clustering Algorithm [17]. -mean clustering is a method of vector quantization, cluster analysis in a given input image consisting of various clusters. This algorithm aims to partition observations into clusters in which each observation belongs to the cluster with the nearest mean, serving as a prototype of the various clusters. This technique calculates Euclidian distance from a random selected mean, each presenting in a cluster in an observation and trying to minimize the calculated Euclidian distance by shifting the point to its nearest cluster. Distance between different points and cluster has been calculated using the following relations:
(e) Fuzzy Thresholding Algorithm [18]. Fuzzy set theory has been used for partitioning an image space by minimizing the measure of fuzziness of an image. This measurement can be expressed as entropy, which is index of fuzziness and nonfuzziness. In the notation of fuzzy set, an image set of size can be expressed as
The membership function levels can be viewed as characteristics equation that can extract the fuzziness of a pixel in given image : Average grey levels and can be regarded as target values of the background and the object for a given threshold value ; and is a constant value such that : The measure of fuzziness used in this work was the entropy by using Shannon’s function . The optimal threshold can be extracted by minimizing the measure of fuzziness .
(f) Proposed Manual Method. Morphological analysis has been implemented to extract the chocolate chips from cookie input images. This approach is especially designed to extract chocolate chips perfectly. The processing starts with a gray-level thinning, followed by a gray-level opening, so that region segmentation is obtained. Morphological erosion operation has been used for extracting the chocolate chips from binary images of the cookies. In order to perform erosion, a 3 3 structuring element has been used over the binary segmented image. Boundary extraction also has been performed for area calculation of each chocolate chip existing in cookies. It has been performed by subtracting the eroded image from original binary image. Detailed process of morphological operations is demonstrated in Figure 5. It shows that all morphological operations have been performed using 3 3 cross-shaped structural elements. Various morphological operations used in proposed manual method have been performed using kernel movement over the gray-level image:
2.2.2. Color Extraction
Various Color Models such as RGB, HSV, and HIS have been implemented for extraction of color regarding information from cookies and biscuit input images. Change in color of biscuits and cookie during baking is a dynamic process in which certain color transitions occur as the baking proceeds. Developed algorithm also discriminates between the changes in color due to various environmental conditions. Sample biscuits are preserved in three different conditions such as packed, moist, open. It has been observed that there is considerable change in color due to change in environment: HSI and HSV color models have been used for extraction of color regarding information for various chocolate cookies because HSI and HSV color models are designed in such a way that humans can see color (see Figure 6). The particular color such as chocolate can be easily recognized by HSI color model. Various statistical methods have been implemented for extraction of perfect chocolate color ranges using HSI color models: HIS color model can be derived using , , and :
2.2.3. Physical Parameters Extraction
Various physical parameters related to quality of the biscuits and cookies such as number of chocolate chips per cookie, area of the chocolate chips and cookies, area of the biscuits, chocolate chips density, solidity, convex hull, convex area, and entropy have been extracted using various image processing operations (see Tables 2 and 3). Entropy is used to characterize texture of the input image by statistical measure of randomness: Area of each sample biscuit and cookie has been measured using different algorithms. In this process, system first applies binary image conversion on input grayscale image. After binary image conversion, system calculates approximate area of chocolate chip. Calculated area is a scalar whose value corresponds roughly to the total number of pixels in the image but might not be exactly the same because different patterns of pixels are weighted differently. Basically system estimates the area of all pixels in the converted binary image by summing the areas of each pixel in the image. The area of an individual pixel is determined by looking at its 2-by-2 neighborhood. There are six patterns, each representing a different area. Each pixel is part of four different 2-by-2 neighborhoods:
(i)patterns with zero on pixels (area = 0);(ii)patterns with one on pixel (area = 1/4);(iii)patterns with two adjacent on pixels (area = 1/2);(iv)patterns with two diagonal on pixels (area = 3/4);(v)patterns with three on pixels (area = 7/8);(vi)patterns with four on pixels (area = 1).Convex hull and its area also have been calculated using -by-2 matrix that specifies the smallest convex polygon that can contain the region. Each row of the matrix contains the - and -coordinates of one vertex of the polygon. This property is supported only for 2D input label matrices.
Various physical parameters also have been extracted for different cookies samples such as chocolate chips count, area of chocolate chips, and chocolate chips density. Having discrete, uninterrupted blocks is important, but a process still needs to be defined so that each block can be counted up effectively. Towards this purpose, an algorithm involving a special spatial-domain filter was devised to reduce each block of white to a single white pixel. Once each block is reduced as such, it becomes trivial to count. The filter operates a window of a certain size for counting the number of white pixels. With the direction of its movement basically, if more than one white pixel group is present in input binary image, it will reduce it to one pixel of white in corner of the window corresponding with the direction of its movement across the sample image. System also calculates the area of each sample cookie and it can be further helpful in chocolate density calculation.
2.2.4. Crack and Fault Detection
Crack and fault detection also have been implemented using Hough transform implementation on converted binary image of commercial biscuit digital image. The objective is to enhance the crack portion coming in input image. Several methods have been used to segment defects like crack, ranging from threshold methods to different advanced methods that combined background subtraction [19], histogram threshold technique [20], adaptive threshold [21], and so forth. In all these methods, low threshold value provides a seed point of a crack and less noise and high threshold provide an entire crack region with high amount of noise. Therefore, to extract the entire crack region in captured input image, a soft threshold method based on fuzzy -means clustering has been proposed for binary image conversion from given target image. Segmented image proceeds for crack detection in segmented biscuit sample digital image (see Figure 8). The key component of a robust crack detection system is crack feature extraction and detection. The objective of this section of implemented algorithm is to detect cracks in commercial biscuits and cookies and find out whether cracks are able to be avoided based on the extracted features. Hough transform technique is basically a robust global method used for fitting lines and different curves such as circle and parabola. It has been used extensively for a wide range of applications with excellent empirical performance [22]. Hough transform method basically identifies a specific class of shapes based on voting procedure. The aggregation of pixels belonging to various cracks in each sample approximately forms a line. Therefore, Hough transform has been implemented for crack detection and feature extraction. The method performs a mapping from the space to the space, using parameters to represent solution of the line equation which can be defined as the following equation: where () is the coordinate of a pixel and () is corresponding distance angle parameter curve (see Figure 7).
(a)
(b)
(c)
(d)
(e)
(f)
(g)
To extract crack’s features in commercial biscuit image, every pixel in the original image is mapped to the Hough space using all values of [23]. This leads to a sine wave in Hough space for each individual pixel. The detection is based on a voting procedure. It is carried out in a parameter out in a parameter space or accumulator. Here, accumulator value is updated continuously. In this algorithm, an accumulator array is used to count the number of intersects of various and values. For every input image, the concentricity measure is computed by finding the maximum values of for each angle that can be expressed as the following mathematical equation: Crack segmentation and detection are implemented in faulty biscuit sample digital image.
Various features of the cracks have been extracted using histogram analysis. Histogram [24] of various biscuit samples has been extracted and it has been observed that each histogram consists of two major peaks. Second peak signifies about crack features. Overshoot of second peak has been inversely related with crack width. As the overshoot of second peak decreased, width of the crack increases. It can be observed in Figures 9(a) and 9(d) that second peak achieves maximum height, and as crack width increases, height of the second overshoot decreases. This analysis extracts information regarding various crack features. This analysis can be discriminating between commercial biscuit samples having large cracks, minor cracks, and no cracks.
(a)
(b)
(c)
(d)
(e)
(f)
3. Results and Discussion
3.1. Comparison of Proposed Segmentation Techniques with Preexisting Thresholding Algorithms
Developed algorithm has been tested on 50 cookies and 100 commercial KrackJack biscuits regarding various quality parameters such as chocolate chip count, area of chocolate chips, proper baking time, texture features, solidity, and various faults occurring in commercial biscuit and cookies digital images. Various threshold techniques for image as mentioned earlier have been implemented for cookies feature extraction. Error index has been calculated based on various extracted features as mentioned earlier. It has been observed that proposed morphological based threshold technique performance is for better compared to other threshold techniques as shown in Figure 10. Error index has been calculated by calculating the difference between automatic chocolate chips count and visual count.
3.2. Characterization of Sample on the Basis of Environmental Conditions
Developed algorithm can also characterize various biscuit and cookie samples on the basis of various environmental conditions such as moist, open, controlled, and packed. It has been observed from entropy calculation that it can discriminate between various texture properties such as open, controlled, frozen, and packed environment conditions. Entropy relates with the texture properties influenced due to various environmental conditions. It can be observed from Figure 11 that entropy can differentiate between various environmental conditions.
3.3. Samples Discrimination Based on Defects and Baking Index
Quality of biscuits and cookies also relates with the proper baking time (see Figure 12). Proper baked sample has been judged based on the color extraction using RGB and HSI color models. It has been observed that extracted color index can differentiate between various biscuit and cookie samples such as underbaked, medium baked, perfect baked, and overbaked. Developed algorithm also differentiates between the faulty and perfect biscuit and cookies samples based on the second peak overshoot calculation extracted from histogram of the gray-level digital images of the various input samples.
(a)
(b)
4. Conclusion
Quality evaluation of commercial biscuits and cookies has a major role in the food industry. The results from this study indicate that developed robust machine vision algorithm can depict various quality parameters such as baking stage, chocolate chips count, influence of environmental condition on the texture of samples, solidity, and various kinds of problems such as major and minor cracks. Experiment on biscuit and cookies products demonstrates effectiveness of proposed threshold algorithm for proper segmentation and Hough based features for crack detection and extraction. Developed algorithm is invariant to rotation and robust from the effect of nonuniform color or different intensity distributions. It was discovered that the developed robust machine vision algorithm resulted in highest correctness classifications averaging at 95% in the sense of prediction of about various quality parameters and crack detection. Therefore, this artificial technique has potential for use in routine inspection of biscuit, cookies, and other bakery products.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
Authors are grateful to CSIR-CEERI, Director Dr. Chandra Shekhar for giving chance to develop such kind of project in CSIR-CEERI as well as allowing them to publish this work. Thanks are due to AEG team members who helped them in successful completion and installation of machine vision system for quality extraction of bakery products such as biscuits and cookies.