Abstract
Personalized interactive image recommendation has several issues, such as being slow or having poor recommendation quality. Therefore, we propose an image personalized recommendation algorithm (IPRA) using deep learning to improve the time and quality of personalized interactive image recommendations. First, the feature subimage is obtained and converted into a one-dimensional vector using the convolution neural network model. Single input and single output functional and dual input and single output generalized functional network model are integrated into the model to improve the learning ability of nonlinear mapping and avoid overfitting during the training process; second, a one-dimensional vector is clustered using the fuzzy k-means approach and then translated into hyperbolic space; Finally, the Poincare map model is used to map the updated vector, the transformed vector is mapped using the PM model, and the image information is fed back to the two-dimensional plane, and the image recommendation set is formed based on the ranking of similarity, and the visual recommendation is presented to the user. The results show that the size of the convolution kernel is 2 × 2, and the image one-dimensional vector clustering can be better completed. The optimal value of F1 is 0.92, and the optimal value of average time is 11 s. The image recommendation quality is better, and the image recommendation can be formed according to the photographic similarity, which has good application value.
1. Introduction
At the moment, with the rapid development of Internet technology and multimedia [1], image interactive personalized recommendation is an important research in the field of image application to obtain the required images accurately and quickly and transmit them to users in time [2]. The personalized recommendation is a clear feature of content distribution, which has a high performance, high availability level, which can recommend it related content to the user in a short time, recommended content reliability, and high accuracy. Personalization recommendation avoids the filling information push of traditional recommendation services, which can set the push time and the number of pushes and can push the relevant information to the user according to the user’s interest and preferences [3, 4]. The main contributions of this article are as follows: (1) image features are obtained through the training of the image recommendation process model to visualize the image recommendation results. (2) To realize interactive image personalized recommendation, hyperbolic space and the Poincare map (PM) are used to complete the visualization of images.
For the personalized recommendation problem of the image, many scholars put forward related learning methods. Zhou W et al. [5] utilized image structures for the recommendation, and image-based recommendation methods focus on capturing the user’s preferences and using the image model to exploit the relationships between different entities in the image, and a new-based ranking recommended algorithm is proposed, which utilizes the user’s explicit and implicit feedback. Yin P et al. [6] proposed a recommended algorithm based on deep learning, the algorithm utilizes a user preference three-dimensional model and improved resource allocation processes, matching target users with similar preferences, and performing personalized recommendations. The principle of the additional preference layer is to capture the user’s pair preference, providing detailed information for the user for further recommendation. The results show that this algorithm has better performance than other image based and ranking oriented benchmark algorithms. Chen S et al. [7] calculated the semantic similarity between learners and all learning resources based on the similarity measurement based on meta path, combined with knowledge transformation probability and learning feedback information. According to the similarity ranking, the learning resources ranked Top-k are recommended to learners relevant experiments show that it effectively realizes the accurate recommendation of learning resources in adaptive learning. Ye Junmin et al. [8] proposed a research on Learning Resource Recommendation Algorithm Based on HIN. On the basis of similarity measurement based on meta path, combined with knowledge transformation probability and learning feedback information, semantic similarity is calculated, learners are sorted according to the similarity, and learning resources ranked Top-k are recommended to teachers and students. Yang X et al. [9] proposed a novel Translation-based Neural Fashion Compatibility Modeling (TransNFCM) framework, which jointly optimizes fashion item embeddings and category-specific complementary relations in a unified space via an end-to-end learning manner. Extensive experiments demonstrate the effectiveness of TransNFCM over the state-of-the-art on two real-world datasets. Jian M et al. [10] proposed a semantic manifold modularization-based ranking (MMR) for image recommendation. Experimental results demonstrate that user-consumed visual correlations play actively to capture users’ interests, and the proposed MMR can infer user-image correlations via visual manifold propagation for image recommendation. Qiu Ningjia et al. [11] proposed the research on recommendation algorithm based on user preference optimization model and constructed user preference matrix for project type by using user project scoring matrix and project type information; Then, the linear regression model is used to calculate the user’s weight for each type; finally, the prediction score is combined with slope one algorithm to improve the quality of user preferred recommendation algorithm. Although the above-given methods have made some progress, in the process of recommendation, the recommendation of the image set cannot be completed according to the similarity of the images. Therefore, this paper proposes an image personalized recommendation algorithm (IPRA) under deep learning. The IPRA cannot only complete image recommendations, but also realize image visualization and image interaction.
2. Methodology
2.1. Method Framework
In this paper, in order to realize the personalized recommendation of the interactive image, the IPRA based on a convolution neural network (CNN) is proposed. The method includes two parts: the model training process and the image recommendation process. The method structure is shown in Figure 1.
As can be seen from Figure 1, the method framework is mainly completed by three steps, and steps 1 and 2 belong to the model training process, and Step 3 is the recommended process and visualization. In step 1, it is necessary to determine the input of the model. In this paper, we used the features of images as input to build an association model between users and images, determine whether the images meet the needs of users, and judge whether the images are recommended. Step 2 is to construct a CNN model according to the association between the user and the image, and complete the model training. The model expression iswhere represents the weight of Gaussian distribution . represents the mean value of Gaussian distribution when the time is . Equation (1) shows that the probability density function of the convolution neural network model is established by using the three-dimensional Gaussian function with the quantity of .
Step 3 is to input the training data into the training completed model, complete the personalized recommendation, and complete the visualization of the recommendation results. The training process of the model is to realize the algorithm design as well as to obtain the image features; the image personalized recommendation process is to realize the visualization of image recommendation results based on the training process [12, 13].
2.2. Interactive Image Feature Extraction from the Perspective of Deep Learning
The structure of hierarchical generalized networks in interactive image features from a deep learning perspective is basically the same as that of artificial neural networks, and not all hierarchical generalized networks can be described by a universal structure, nor can all hierarchical generalized networks be represented using a unified generalized equation [14, 15]. Based on these features, interactive image features are classified into single input and single output type and dual input and single output type, and the two models are used as basic components in extracting interactive image features of hierarchical generalized networks [16].
2.2.1. Construction of Single Input and Single Output Generalized Function Network Model
Figure 2 shows the specific structure of the single input and single output type generalized network model.
Figure 2 shows that the structure of the single input and single output functional network model shows that it can continuously map between input and output, thus overcoming the problems of network continuity and approximation and can be well applied to various complex problems.
The output expression for a single input and single output generalized function network is
In hierarchical generalized networks, the expression uniqueness problem is an important issue to be solved.
2.2.2. Two-Input Single-Output Generalized Network Model Construction
The structure of the two-input single-output flooding network model is shown in Figure 3.
It can be seen from Figure 3 that, it can perform a continuous mapping between input and output. Its form is different from the single input and single output functional network model, and the selection of functional neuron function in the functional network is not fixed, which makes the model more adaptable.
Assuming that and represent the input and output vectors of a two-input single-output generalized network, respectively [17, 18], then the output of the generalized network can be expressed as follows:
When Equation (3) holds the generalized network can be defined as a separable generalized network.
Based on the above-given single input and single output generalized network model and dual input and single output generalized network model, the specific steps for extracting interactive image features from a deep learning perspective are [19, 20].
Step 1. In the first layer of the hierarchical generalized function network, assume that there exist input variables . When , the output of the first layer of the hierarchical generalized network iswhere the expression for is
Step 2. Let , then there are input variables in the th basic level generalized network, then the output of the basic level generalized network iswhere the expression for is
Step 3. If the relation exists, return to Step 2, and vice versa build the hierarchical generalized function network [21, 22].
According to the above-given steps, interactive image feature extraction from a deep learning perspective is completed.
2.3. Proposed Algorithm
To achieve image personalized recommendation, clustering is used to cluster the one-dimensional vectors of mapped images, and it is done based on the degree of similarity of visual content, which is done in this paper using the fuzzy K-means method. The interactive image personalization recommendation algorithm is described as follows:
Input: introduction of the affiliation function . The function Equation of the method iswhere represents the -th sample; is the constant; represents the cluster center.
Output: the result of interactive image personalized recommendation.
In order to realize interactive image personalized recommendation, hyperbolic space and PM are used to complete the visualization of images. The specific steps are as follows:(1)The one-dimensional feature vector after clustering is transformed into hyperbolic space; The expression iswhere is the hyperbolic space coefficient. represents hyperbolic space conversion coefficient.(2)PM model is used to map the transformed feature vector, and the image information is fed back to the two-dimensional plane to realize image interaction. The expression of the PM model is:where is the offset function, is the excitation function. indicates the output characteristic.(3)Hyperbolic space can maximize the retention of image feature similarity information and display the nonlinear growth trend of an image, which belongs to feature similarity. The expression iswhere represents the maximum load of image nonlinear growth trend.(4)Since the space cannot present the image on the two-dimensional plane, Poincare disk mapping is used to map the coordinate points of the space and present them on the two-dimensional plane to realize the presentation of the image on the two-dimensional plane.(5)The PM model is also known as a codisc model, which consists of a hyperplane geometric model, and the dimension is . During mapping, if is a point on the hyperboloid and located in this space, the definition of a point in the hyperboloid model can be completed, the point can be connected with , and the connecting line between them can be mapped to hypersurface , so as to obtain the corresponding point in the PM model.
2.4. Experimental Analysis and Results
In order to verify the effectiveness and validity of the IPRA from the deep learning perspective, the experimental environment is built in MATLAB simulation software, and the operating system required for the experiment is Windows 10. The algorithm of Zhou W et al. [5], the algorithm of Yin P et al. [6], the algorithm of Chen S et al. [7], and the algorithm of Ye Junmin et al. [8] are used as experiments to compare the IPRA. The LIDC-IDRI data set and the LUNA16 data set are used as experimental objects and the images in two images are named L image and C image, respectively, and real-time tracking of images in two datasets using the IPRA to verify the effectiveness of the recommendations of the IPRA. The LIDC-IDRI data set was collected at the initiative of the National Cancer Institute (NCI) to study the early detection of pulmonary nodules in high-risk populations. In this data set, a total of 1018 study instances are included. For each instance, the images were diagnostically annotated in two stages by four experienced chest radiologists. The data set consists of chest medical image files (e.g., CT and radiographs) and corresponding diagnostic outcome lesion annotations. The LUNA16 data set contains 888 CT images with 1084 tumors, with a desirable range of image quality and tumor size. The data were divided into 10 subsets, and the subset contained 89/88 CT scans. The CT images of LUNA16 were taken from the LIDC/IDRI data set, and the annotation with more than three radiologists’ agreement was selected, and the tumors smaller than 3 mm were removed, so the data set does not contain tumors smaller than 3 mm, which is convenient for training. 500 images are randomly selected from each image data set for experimental analysis. 70% of the images will be selected as the training set, and the other images will be used as the test set.
The evaluation indexes are as follows:(1)Convolution Kernel Size. In the training process of IPRA, the size of the convolution kernel has a direct impact on the training results. Therefore, it is necessary to determine the optimal size of the convolution kernel and take the value of the loss function as the measurement standard.(2)Image Feature Clustering Effect. Taking adjusted Rand index (ARI), Macro-F1 measure (F1), and average time as evaluation criteria, the calculation equations of the first two are expressed by Equations (13) and (14), and the larger the values of the two, the better the performance of the IPRA.where represents the average distance, and represent macroprecision and macrorecall, respectively.(3)Personalized Recommendation Effect. the Normalize Discounted Cumulative Gain (nDCG) is adopted as the evaluation index, which is standardized, and its calculation equation iswhere represents the -th image. and represent the quality and weight of each image recommendation result, respectively. The larger the NDCG value, the better the quality of the recommended image.(4)In order to intuitively measure the image recommendation effect of the five algorithms, the images in the verification set of the five algorithms are used for recommendation, and one image is randomly extracted from the Y image as the target image to obtain the image recommendation results of the five algorithms.(5)Expect Average Overlaprate (EAO). The algorithm from Zhou W et al. [5], Yin P et al. [6], Chen S et al. [7], and Ye Junmin et al. [8] is selected as the comparison algorithm of the IPRA, and the images within the recommended dataset using the five algorithms are analyzed for the EAO in the process of recommending 10 images with different attributes. EAO belongs to the comprehensive evaluation index of tracking accuracy and robustness, and the value of EAO is proportional to the tracking effect.
3. Results and Discussion
The function value results are tested under different convolution kernel sizes are shown in Figure 4.
According to Figure 4, with the increase of the number of iterations, the value of the loss function of the convolution kernel changes significantly when the size of the convolution kernel is 2 × 2, the loss function value shows a slow downward trend with small fluctuation. The other two convolution kernels are different in size and the loss function value is higher. Therefore, the convolution kernel size is determined to be 2 × 2.
In order to analyze the advantages of IPRA, Zhou W et al. [5], Yin P et al. [6], Chen S et al. [7], and Ye Junmin et al. [8] algorithms are used as the comparison algorithms of IPRA. The test results of the five algorithms on the training set are obtained according to Equations (13) and (14), as shown in Table 1.
According to the test results of Table 1, among the test results of the five algorithms with the different number of images and three evaluation indexes, the results of each index of IPRA are better than the other four algorithms, and the best value of ARI is 0.62, the best value of F1 is 0.92, and the best value of average time is 11 s, which are significantly better than the four comparison algorithms. Therefore, the clustering effect of IPRA is good.
According to equation (6), the recommended nDCG results for the verification set are obtained, as shown in Figure 5.
According to Figure 5, in the test set, with the gradual increase of the number of images, the nDCG values of the five algorithms fluctuate to a certain extent, and rise slowly and slightly; in which, the nDCG values of the IPRA are above 0.8, and the nDCG values of the other four comparison algorithms are between 0.5 and 0.8, which is significantly lower than the IPRA. Therefore, the image recommendation quality of the IPRA is better.
The images in the validation set of the five algorithms are used for recommendation, and one image is randomly extracted from the Y image as the target image, and the image recommendation results of the five algorithms are obtained as shown in Figures 6 and 7.
(a)
(b)
(c)
(d)
(e)
According to the test results of Figure 7, the five algorithms can complete the personalized recommendation of the target image. However, the IPRA can form an image recommendation set according to the photographic similarity. Users can click any recommendation set to view all the images in it; the other four algorithms cannot form the image recommendation set, cannot complete the formation of the image recommendation set according to the photographic similarity and can only present the recommendation results of relevant images. Therefore, the recommendation effect of IPRA is better than the four comparison algorithms.
The algorithm from Zhou W et al. [5], Yin P et al. [6], Chen S et al. [7], and Ye Junmin et al. [8] was chosen as the comparison algorithm for the IPRA. Five algorithms are used to recommend interactive images in the data set. The EAO analysis results are shown in Figure 8.
According to Figure 8, when the five algorithms recommend interactive images, the EAO values of this algorithm are significantly higher than those of the other four algorithms, and the change range of EAO values of IPRA in image personalized recommendation is small, and the fluctuation range of EAO values of the other five algorithms is large. Experiments show when recommending interactive images, the EAO values of IPRA are high, which has a better personalized recommendation effect.
4. Conclusions
In order to realize interactive image personalized recommendation and provide users with more reliable and higher quality images, this paper proposes an IPRA from the perspective of deep learning. The conclusions are as follows: (1) the IPRA combines the CNN model, hyperbolic spatialization, PM, and fuzzy k-means algorithm to realize image feature extraction, processing, transformation, and clustering. (2) The IPRA realizes image recommendation and visualization. (3) With the gradual increase of the number of images, the nDCG values of this algorithm fluctuate to a certain extent, and rise slowly and slightly. The nDCG values are above 0.8, and the image recommendation quality is better. (4) The IPRA has good image feature clustering performance, and the recommended image quality is high. At the same time, it can form the recommended set of images required by users according to the similarity, and present the image recommendation results. (5) The EAO value of IPRA is high and has a better personalized recommendation effect.
From the perspective of deep learning, IPRA has shortcomings. In future works, it is necessary to continuously optimize the algorithm according to the development of the image, so as to truly and accurately provide the basis for the image algorithm. The security and omnipotence of the personalized recommendation process are studied to further optimize the performance of interactive image personalized recommendation. (1) The recommended algorithm is fully considering different features, and the new metric is performed according to the actual development. (2) Improving the performance of recommendation capability prediction in the recommendation process.
Data Availability
The data used to support the findings of this study are included within the article. Readers can access the data supporting the conclusions of the study from LIDC-IDRI data set and the LUNA16 data set.
Conflicts of Interest
The authors declare that there are no conflicts of interest.
Acknowledgments
This work was supported by the Natural Science Foundation of Heilongjiang Province of China under Grant no. LH2021F049.