Abstract
With the rapid rising of living standard, people gradually developed higher shopping enthusiasm and increasing demand for garment. Nowadays, an increasing number of people pursue fashion. However, facing too many types of garment, consumers need to try them on repeatedly, which is somewhat time- and energy-consuming. Besides, it is difficult for merchants to master the real-time demand of consumers. Herein, there is not enough cohesiveness between consumer information and merchants. Thus, a novel fashion evaluation method on the basis of the appearance weak feature is proposed in this paper. First of all, image database is established and three aspects of appearance weak feature are put forward to characterize the fashion level. Furthermore, the appearance weak features are extracted according to the characters’ facial feature localization method. Last but not least, consumers’ fashion level can be classified through support vector product, and the classification is verified with the hierarchical analysis method. The experimental results show that consumers’ fashion level can be accurately described based on the indexes of appearance weak feature and the approach has higher application value for the clothing recommendation system.
1. Introduction
The fashion industry occupies a significant position in the global economy and involves large industrial chain, including garment design, production, and sales. In fact, in the recent years, there has been an expanding demand for clothing all over the world. Since 2008, the garment sales have increased by $3.3 billion every year, and the global garment sales reached $1.25 trillion in 2012 [1]. According to a report of Euromonitor International, in 2015, the growth rate of clothing sales was 4.5%, and the industry gross reached $1.6 trillion. The global clothing sales enhanced by 3.8% and the industry gross rose to $1.7 trillion in 2016. The above data show that the garment industry is developing at a rapid rate.
In fashion sales, the recommendation technology, as an emerging technology, has attracted wide attention of scholars. As is widely known, the traditional garment recommendation depends on manual operation. To be specific, salesmen need to recommend garment to customers in order to arouse their interest in purchasing. However, it is very difficult for salesmen to understand customers’ real thoughts and then recommend the targeted garment as there is no sufficient cohesiveness between customer information and merchants. Therefore, it is essential and meaningful to find a set of objective indicators, instead of subjective opinions, to evaluate the fashion level in the clothing recommendation technology.
As the Internet technology continues to develop rapidly, virtual fitting and other clothing intelligent equipment have enjoyed great popularity in the fashion industry. Cordier et al. (2003) [2] first applied the 3D graphics technology to create and simulate the virtual store. Subsequently, Li et al. (2011) [3] proposed the interactive 3D virtual fitting room system, in which the model’s hairstyle and accessories can be changed according to customers’ preferences and customers’ matching degree can be evaluated to guide people to choose the suitable clothes. Nevertheless, the virtual fitting research products are constantly innovating and developing. In fact, today’s systems are mainly used to display garment, and customers can only have a preview of the fitting effect. If the store does not have an efficient recommended method, the search will be tedious and frustrating. Zhang et al. (2008) [4] presented an interaction clothes fitting system that can recognize what human eyes perceive in terms of the clothing similarity through the frontal-view outfit images. Limaksornkul et al. (2014) [5] put forward the Closet Application to record the clothing statistics and accessories that are frequently used to recommend clothes to customers according to the statistics of their purchasing history. The recommended technology not only allows customers to quickly find the right clothes in the fitting process but also helps businesses increase sales. Nonetheless, the above methods are mainly based on the subjective views that ignore the objective data. To address this problem, this paper proposes a fashion level evaluation method for clothing recommendation based on the weak appearance feature.
This paper is organized as follows. Section 2 analyzes and summarizes the current situation of recommendation technologies as well as their advantages and disadvantages. Section 3 (Figure 1) contains the description of the definition of fashion, classifies the fashion index, extracts the weak features of human appearance, and describes the SVM classification. The experimental method and the result of the experimental analysis are given in Section 4. Last but not least, Section 5 presents the conclusion and summarizes the contents of this paper.
2. Related Works
There are lots of methods achieved in garment recommendation. For instance, customer ratings and clothing are utilized as considerations for garment recommendation [6]. Similarly, user’s personal preference and the history of clothing items have been tried [7]. Furthermore, some scholars found that the past statistics of clothes and accessories and current weather conditions as well as special occasions can provide a relevant recommendation on garment [8]. In order to meet different needs, an intelligent clothing recommendation system based on the principles of wearing fashion and aesthetic is studied [9]. In addition to the above work, Iwata et al. (2011) [10] offered a recommender system, utilizing fashion magazines’ full-body photographs. In the same way, Sha et al. (2016) [11] extracted multiple features from images to analyze their contents in different attributes, such as fabric pattern, collar, and sleeve. Some garment system integrates the fashion themes and shapes professional designers’ knowledge and perception to help them choose the most relevant garment design scheme for a specific customer [12].
In the computer realm, the concept of the recommendation technology was first introduced in the middle 1990s [13]. So far, different advanced algorithms have been developed. The following is a review of the relevant methods. The first method is the content-based recommendation algorithm. For example, the CRESA combined textual attributes, visual features, and human visual attention to compose the clothes profile in the recommendation [14]. Ajmani et al. (2013) [15] present a novel method for content-based recommendation of media-rich commodities with the use of probabilistic multimedia ontology. Li et al. (2012) [16] utilized the HMM of recommended items to match customers’ model according to customer data. The second method is the collaborative filtering-based recommendations algorithm. For instance, Nogueira et al. (2015) [17] presented a new collaborative filtering strategy that utilizes the visual attention to characterize images and alleviate the new item cold-start problem. The rule-based recommendation algorithm is the third method. Hwang et al. (2016) [18] put forward a method to generate the automatic rules with the user's items and made a suggestion on the best rule. The fourth method is the utility-based recommendation. For instance, Scholz et al. (2015) [19] found that exponential utility functions are better geared to predicting optimal recommendation ranks for products, and linear utility functions perform much better in estimating customers’ willingness.
2.1. Conclusions on the Literatures
From the review of the related literatures, the following conclusions can be drawn.
() The recommendation method based on customer rating and personal interests, to some extent, has backward features. In practical conditions, most customers would judge the vogue of clothing according to the subjective feelings and matching degree. In particular, most people will become confused when selecting clothes. Actually, it is the representation of customers’ ambiguity regarding their personal conditions.
() As for the recommendation method based on the contents, it is applicable for multiple regions. Typically, it will recommend new projects to users according to the individual browsing records. The recommendation results have been proven to be explicit and accessible. However, the content-based recommendation method is relatively improper when applied in the fashion industry, which can be ascribed to the data cold-start problem. That is to say, new users without any browsing record could not obtain recommendations. In addition, it is rather difficult to process clothing products with the relatively complicated attributes.
() The retrieval means of customer data are of importance. In the past, businessmen would retrieve customers’ data through membership cards and questionnaires. Such method fails to guarantee the authenticity of data, since the data could not be timely upgraded in case of any physiological and psychological changes of customers. At present, e-commerce businessmen mainly make recommendations through registering virtual members, tracking consumption, and browsing information. However, the recommendation results are sometimes not ideal, since some customers doubt if their private information will be exposed. Confronted by the difficulty in data retrieval, this paper utilizes the camera as the output equipment of image and video to retrieve customers’ data. After building the image database and analyzing the image data, this paper subsequently classifies customers according to their fashion level. This classification will be upgraded with the change of customers’ data. In the end, garment recommendation will become easy according to customers’ classification data and clothing classification data.
3. Our Proposed Method
The word “fashion” is originated from the translation of VOGUE, a famous US fashion magazine. Fashion level is a way of life and the awareness of pursuing the real, good, and beautiful things. Different individuals tend to have different pursuits of fashion. This paper characterizes individuals based on different fashion levels. To quantize the fashion level, customers’ fashion levels are divided into four degrees (Table 1), namely, wonderful, great, good, and common. Such classification aims to provide objective clothing recommendations to customers.
Furthermore, the garment fashion level is classified (Table 2) based on the data from fashion designers, buyers, vendors, and producers. The classification process needs to consider the quarter sales, clothing style, and other factors. As a result, garments can be divided into four categories.
The classification of fashion level is a subjective method that needs subjective evaluation on the image characters through the expert group. The subjective method is closely related to the subjective factors, such as knowledge background and psychological motivation of the experts involved in the evaluation. Actually, this is time-consuming, taking up lots of resources and causing unstable data classification results. As for the researches of visual psychological characteristics, there has been no quantitative description method by which the objective evaluation results can represent the subjective evaluation results.
This dissertation aims to find out a set of objective indexes which can be used to assess fashion level. The fashion level is defined as the individual degree of fashion. Fashion level shows people’s appearance, dress, act, and so forth. Considering all the factors that affect the evaluation of personnel’s scoring, this paper regards the weak appearance feature as an important index that can influence the fashion level. The weak appearance feature here means that the individual features have low characterization degrees. There are many weak appearance features related to the individual fashion level.
Under these conditions, if the camera can recognize the person (Figure 2), effective image data can be accessed to. Three major categories, namely, make-up, accessories, and hair colors, of the weak appearance feature are extracted, including eye shadow, blush, lip color, eyebrow color, hat, accessories on hand and neck, nails, brooches, and hair color. By utilizing the SVM classification method, as shown in Table 3, fashion level is evaluated based on whether human body has weak appearance features. The garment recommendation is established on fashion level classification; the comparison result of fashion level category and ground truth validate the correctness of fashion level classification results. There is no effective way to establish fashion level database. Nonetheless, the fashion level image database established in the paper is the basis of follow-up studies. Effective fashion image database is of vital significance to the training and testing of algorithm.
3.1. Establishment of Database
3.1.1. The Establishment of Image Library
Image Source. Images are selected from the pictures collected through work, study, entertainment, and rest or from the Internet.
Object Definition (Table 4). The research objects are limited to Asian women, aged 15~55, excluding the ones younger than 15 and older than 55.
Photo Regions. The image should cover all the weak feature regions, including the three types of indicators which are make-up, accessories, and hair colors. The images (shown in Figure 3) should contain the frontal face images of the objects, while the images of sides and rear are invalid. The JPG format is adopted in image data.
Image Quality Requirements. The efficiency is greater than 100 dpi and the requirement of image sharpness is related to the resolution size. The images taken by camera during the daytime generally can meet the requirements; in the evening, shadow occlusion area will be generated under the light. For example, the shadow of the neck area is blocked by the head so the images taken in the evening are excluded in the data source. Image database consists of high-resolution and high-quality color source images.
Image Grouping. The images are randomly divided into the training group (800 images) and test group (200 images), and many tests have been conducted for verification.
3.1.2. The Establishment of Ground Truth
The Expert Group. It consisted of 60 experts in the fashion field (20 fashion designers, 20 garment buyers, and 20 garment salesmen) and 40 experts in other fields (20 nonartistic experts and 20 students of nonart major).
Expert Scoring. The expert group judges the fashion level from the perspective of professional knowledge according to the images under the premise that they are not informed of the feature indexes. The highest and lowest scores offered by the expert group for the same image should be removed, and the average of the rest of the scores is regarded as the standard scores of the image fashion level.
3.2. The Extraction of Weak Feature Index
The current face detection methods mainly include two categories: the knowledge-based ones and statistics-based ones. To extract the weak facial feature, the facial feature points are located first and then the face recognition is started (Figure 4). This paper adopts the Adaptive Boosting method for facial feature positioning. Adaptive Boosting method, proposed by Freund (1995) [20], is a statistical learning method that integrates the weak classifier with the strong classifier [21, 22]. The basic idea is to endow large weight to the unsuccessful training samples, make learning algorithm focus on the difficult training samples in the subsequent study, and finally weight and add a number of weak classifiers selected by the algorithm to strong classifier.
The input of the algorithm is a set of data matrix, , where belongs to a sample space and belongs to a sample space . This paper assumes that . Subsequently, the learning-based algorithm is used for times, , so as to maintain the weight distribution of training data set. The data are set in the round weight value of . All the weights at the beginning of the training are given values. However, at the end of each round, the weight value of the data which are wrongly classified will rise. As a result, the weak learning device focuses on the classification of a more difficult part. The weak learning device is utilized to find the weak assumption , and the superiority of the weak assumptions is decided by its level of error .
The indexes are judged according to the different characteristics of the weak feature indexes and through different methods or steps. Figure 5 shows weak appearance feature identification framework.
3.2.1. The Make-Up Index Includes Eyebrows, Eye Shadow, Cheek Color, and Lip Color
(1) Eyebrows. It is important to first determine whether consumers pencil eyebrows nor not. The facial images of volunteers who pencil the eyebrows and those who do not pencil the eyebrows are selected, with an image processing process shown in Figure 6. First of all, the Laplace operator is used to detect the eyebrows edge. The four boundary points of eyebrows area are assumed as follows: , , , and . The eyebrows area is set by the rectangle which is connected by the four boundary points, and the rectangle image is clipped. Secondly, the gray-scale image processing is conducted to obtain gray image. Later, the binarization processing is executed on the images. It is visible that the images of the ones who pencil the eyebrows show dense and uniform black distribution, while the images of the ones who do not pencil the eyebrows have sparse and uneven black distribution. Since there are obvious differences between the black distribution of the binarization images of the ones who pencil the eyebrows and the ones who do not, the black pixel density, namely, the black area in the rectangle area, is calculated. The black pixel density threshold of the eyebrows can be obtained through the statistical analysis, and it is regarded as the judgment to determine whether people pencil the eyebrows.
(2) Eye Shadow. As indicated in Figure 7, the rectangle in the middle area of eyes and eyebrows can be defined as the eye shadow area; the area above eyes and the lower edge of hair constitute the forehead area; features are extracted from the eye shadow and forehead areas. The maximum likelihood estimates that RGB of the two areas can be compared. If the maximum likelihood estimation is almost the same, it can be concluded that one does not use eye shadow; on the other hand, if it is different, one uses eye shadow.
(3) Cheek Color. As indicated in Figure 8, the triangle area below eyes is set to be cheek color area; the cheek and forehead areas are extracted. The maximum likelihood estimates that RGB of the two areas can be compared. The contrast threshold is determined by many experiments. When the contrast ratio is at range of the threshold, it is considered that one does not use the blusher. Otherwise, it can be considered that one uses the blusher.
(4) Lip Color. Figure 9 shows lip structure. In order to determine whether consumers apply lipstick, 100 color photographs are selected; lip color samples are chosen; RGB values of four sample points of upper lip bead and lower lip bead are manually selected. Then, RGB values are converted to HSV value (color, purity, and brightness). The four groups of HSV value are calculated. Everyone has four detection areas; a total of 12 HSV data value is classified as lip color features.
3.2.2. The Accessories Index Includes Neck and Hand Accessories and Bracelet and Nails
(1) Neck and Hand Accessories. The edge detection technique can be used to finger the gesture recognition [23]. People’s wrist, finger edge, and neck edge are detected. If the obvious edge cannot be detected in the areas, it shows that people wear neck or hand accessories; otherwise, it can be considered that people do not wear neck or hand accessories.
(2) Brooch. Brooch is highly reflective, and its image area color space has obvious characteristics. Thus, brooch can be identified through the extraction of the color space [24]. The area below neck and above waist is extracted as the chest area. The RGB values of the chest area are extracted, and then the RGB values are converted to the HSV value. If the highlighted area is judged through statistical analysis and the size is between 1 cm2 and 9 cm2, it can be argued that the people wear brooch. Otherwise, if there is no highlighted area or the highlighted area is too small or too large, it is thought that people do not wear brooch.
(3) Nail. As shown in Figure 10, after machine identifies nails, the nail color threshold of nails without nail polish can be obtained through statistical analysis [25]. If consumers’ nail color is within the threshold, it can be regarded that they do not apply nail polish; if not, it is considered that they apply nail polish.
(4) Hat. Edge detection is conducted on head position. If shape or semicircle closed graph is measured, hair color or environmental color is extracted for further comparison; if the color is obviously different, it is argued that consumers wear hat; otherwise, it is argued that consumers do not wear hat.
3.2.3. Hair Color
A polar coordinate system is established with two-eyed center as the center of the circle and each line extends from the origin. The first detection edge pixel point is marked with ; the second detecting edge pixel point is marked with ; the area marked between and is hair area. The RGB maximum likelihood estimator of the region is extracted as the hair color values.
3.3. SVM Algorithm
After reading the images in the database and identifying the weak feature, the support vector product is used with training error as the optimization problem constraint and with the minimized incredible range as the optimization goal. SVM is developed from the statistical pattern recognition theory. SVM can be utilized to solve two kinds of classification problems by finding an optimal separating hyperplane and separating two classes of data with maximum intervals mainly based on the structural risk minimization principle. Finally, solving of a linearly constrained convex quadratic programming (QP) is involved. Thus, the solution of support vector machine is unique and optimal [26].
In the SVM training model, 1000 images are selected as experimental data from database. In the second place, 800 groups of data are randomly selected as the training groups and trained for 100 times. The paper selects the appearance weak features, including eye shadow, cheek color, lip color, eyebrow color, hats, neck and hand accessories, nails, and brooches and red, yellow, green, blue, brown, black, gray, and white colors. After determining the SVM model as C-SVC, the type of kernel function as the radial basis function (RBF), the relevant punishment coefficient, and the radius of kernel function, a 12 d linear hyperplane is established to divide samples. Later, the remaining 200 groups of data are used as the validation group and the number of verifications is 100 times. Subsequently, the test results are compared with the ground truth score and the accuracy of the model is tested.
3.4. Validation with the Analytic Hierarchy Process
Analytic hierarchy process (AHP) is used for the validation of SVM algorithm. AHP regards the research object as a system, which does not cut out the influence of factors on the results. The results can be affected by each layer weight setting in AHP directly or indirectly. Besides, the influence level of each factor on the results is clear and quantitative [27]. The specific modeling process is as follows.
3.4.1. The Individual Fashion Level of Each Type of Feature Is Calculated Based on AHP
() The criterion function of weak feature is established for weak feature in the classification of make-up. If customers have the feature, the corresponding function assignment of the feature is “1”; if not, the corresponding function assignment is “0,” as shown in the following formula:
Through the combined form of expert suggestion and extensive survey, matrix of the feature in the classification of make-up is given as follows:
The weight of the weak feature is calculated based on the feature matrix and the following are the results: , , , and . According to the consistency check, . Table 5 indicates that .
The judgment matrix is found to be valid. Next, the individual fashion level in the classification of make-up is solved with the equation as follows:
() The criterion function of the weak feature is established for weak feature in the classification of collocation. If customers have the feature, the corresponding function assignment of the feature is “1”; if not, the corresponding function assignment is “0”; namely,
In the combined form of expert advice and extensive survey, matrix of the feature in the classification of make-up is given as follows:
The weight of the weak feature is calculated based on the feature matrix and the following results can be obtained: , , , and . According to the consistency check, . by referring to Table 4.
The judgment matrix is found to be valid. Then, the individual fashion level in the classification of collocation is solved with the following equation:
() The criterion function of the weak feature is established for weak feature in the classification of hair color. If customers have the feature, the corresponding function assignment of the feature is “1”; if not, the corresponding function assignment is “0,” as indicated in the following formula:
By the means of expert advice and extensive survey, matrix of feature in the classification of hair color is given as follows:
The weight of the weak feature is calculated based on the feature matrix, and the following results are obtained: , , , , , , , and . According to the consistency check, . by referring to Table 4.
It can be found that the judgment matrix is valid. Subsequently, the individual fashion level in the classification of hair color is solved with the following equation:
The overall fashion level is calculated according to the fashion level of each type of individual feature. Based on AHP, the weights of make-up, hair accessories, and hair color are relative to the weight of individual fashion levels: : : : : .
200 women are randomly selected as the validation object for model validation. Compared with the calculation results of the AHP model, the fashion level is obtained through experts scoring.
4. Analyses of Experimental Results
() As shown in Table 6 and Figure 11, the average learning accuracy through the SVM classification experiment is as high as 92.08%. It suggests that the weak feature index is feasible and rational. However, there are three accuracies that are lower than 83% among the 200 validation groups; this is because of evaluation difficulties caused by unprecise image screening. In contrast to the two images with low and high accuracies in the database, the research found that the database needs to be optimized. Thus, the weak feature shadow caused by the natural light should be adjusted. The dpi of image should be increased to 150%. In the future, the system data update should continue to be researched and analyzed.
() The validation results through AHP model show that the accuracy of fashion level index reaches 92.37% and the average matching degree of the classification results is 93%, proving the effectiveness of AHP. Table 7 shows 20 groups of results randomly; there are 16 values which are beyond 90%, and 4 values are below 86%. The AHP experimental result, as the verification method for SVM algorithm, indicates that the SVM algorithm is effective. To further illustrate the results, is set as the inaccuracy; the equation is as follows:
The change of is shown in Figure 12; fluctuation of is in the range of plus or minus 15%. More than 700 pieces of data are fluctuated within 10%. It is certificated that the AHP is effective. But there is a small amount of data with a higher inaccuracy. Research found that it is mainly due to the different psychological factors and physiological factors of the ground truth. Also, there is some data quality which is below standard. Later, it should be improved; on the whole, the weak appearance features are suitable for the fashion clothing recommendation.
The SVM classification model and AHP model are used to explore the relationship between the appearance weak feature and fashion level. As can be seen from the results of the model validation, both models can correctly characterize the fashion level, with the accuracy of being above 92%. Thus, it is a feasible scheme to classify the fashion level of different people through the appearance weak feature, and the rationality of weak feature index is proven.
5. Conclusion
The paper uses the appearance weak feature to characterize consumers’ fashion level and draws the following conclusions by comparing the science experiment and expert evaluation: the fashion level of consumers can be accurately determined on the basis of make-up, accessories, and hair color. In the paper, the support vector product is used to establish and classify the fashion level model. According to the model test results, the accuracy is more than 92%, and consumers’ fashion level is accurately classified. For verification from the perspective of the hierarchy by utilizing AHP, the accuracy is as high as 92.37%, demonstrating the effectiveness of the appearance weak feature index. Customers can obtain the appropriate clothing recommendation based on their classification results of the fashion level. By this means, fashion level evaluation method based on the appearance weak feature can be successfully applied in the intelligent garment recommendation system, and it has essential significance for fashion recommendation.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
The research is supported by the National Natural Science Foundation of China (intelligent recommendation computation for electronic commerce based on eye fixation detection and emotion analysis, no. 61461021).