Abstract
With the development of ubiquitous computing, recommendation systems have become essential tools in assisting users in discovering services they would find interesting. This process is highly dynamic with an increasing number of services, distributed over networks, bringing the problems of cold start and sparsity for service recommendation to a new level. To alleviate these problems, this paper proposes a hybrid service recommendation prototype utilizing user and item side information, which naturally constitute a heterogeneous information network (HIN) for use in the emerging ubiquitous consumer wireless world (UCWW) wireless communication environment that offers a consumer-centric and network-independent service operation model and allows the accomplishment of a broad range of smart-city scenarios, aiming at providing consumers with the “best” service instances that match their dynamic, contextualized, and personalized requirements and expectations. A layered architecture for the proposed prototype is described. Two recommendation models defined at both global and personalized level are proposed, with model learning based on the Bayesian Personalized Ranking (BPR). A subset of the Yelp dataset is utilized to simulate UCWW data and evaluate the proposed models. Empirical studies show that the proposed recommendation models outperform several widely deployed recommendation approaches.
1. Introduction
With the rapid development of ubiquitous computing, people today are able to access any services anytime and anywhere. Many studies have been done in exploiting wireless communications models for use in ubiquitous network, for example, NGMN (Next Generation Mobile Network) [1] and MUSE (Mobile Ubiquity Service Environment) [2]. Among them, the ubiquitous consumer wireless world (UCWW) [3, 4] brings a different approach to the current global wireless environment, setting out a generic network-independent and consumer-centric techno-business model (CBM) foundation for future wireless communications. The primary change the UCWW brings is that the users become consumers instead of subscribers and thus potentially are able to use the mobile service of any service provider (SP) via the “best” available access network of any access network provider (ANP). Figure 1 depicts a high-level view of the UCWW [3].
One of the key UCWW features is related to the provision of a personalized and customized list of preferred mobile services to consumers by taking into account their preferences as well as the current network and service context [5]. The following are some possible scenarios for utilizing the UCWW within the smart-city paradigm [6]:(i)Smart parking service: when a consumer in her/his car enters a university/hospital campus or a similar facility, s/he will automatically get a recommendation for the “best” car parking spaces, with allocation and reservation options subject to her/his profile preferences and campus parking policies. The recommendation will come with enhanced functions and information options, if required by the consumer profile, for example, reservation fee payment scheme and detailed directions to that parking space on a standard navigator app or other proprietary app. Options for provision of all or part of this service, for example, the key parking space reservation, can be made under other conditions, for example, as a “yes” response to “reserve parking at my work-place” pop-up on a mobile device first thing in the morning, even before leaving from home to go to work.(ii)Personal-health location reminders: the goal of this service is to present the consumer with up-to-date notifications about lowest priced consumer-prescribed drugs in drugstores/pharmacies within the geographic location of the consumer. There would be matching service descriptions (SDs) for apps to collect and collate the information, for example, as part of a cloud-based service recommendation system, from cooperating drugstores. In the SD for such an app, alerts or reminders may be set manually through profile policy, when the consumer is within easy reach of a drugstore with the lowest priced drug. There are many consumer-oriented variations of such a kind of service, leading to many ways personal-health location reminders may work for different people. Also, this service can potentially support other smart-city healthy living applications, for example, targeted profile-based real-time alerts about areas of high and low pollen count, pollution, air quality index (AQI), and so on or more specific alerts about consumer moves around the city.
In order to support consumer requirements in scenarios such as those described above, recommendation techniques become essential tools assisting consumers in seeking the best available services. The services in the UCWW are divided into two broad categories: access network communication services (ANCSs) and teleservices (TSs) [7]. ANCSs are used by the consumer to find and use the best access network available in the current location, while TSs are more complex, containing all non-access-network services, from e-learning to online Internet shopping, email, and multimedia services [4]. In this work, we only focus on TSs recommendation problems. The terms “services” and “items” are used to refer to TSs, and “users” is used to refer to consumers in the rest of the paper.
In this paper, a hybrid recommendation prototype for TSs advertising is proposed, working as a platform to assist service providers to reach their valuable targeted users, while at the same time offering each user a list of ranked service instances they may be interested in. To alleviate the cold start and sparsity problems, we propose to leverage the rich side information related to users and services, constructed as a heterogeneous information network (HIN), to build the proposed recommendation models. The proposed models can be potentially also utilized in other recommendation systems. The contributions of this paper are summarized as follows:(i)First, we design a layered recommendation framework for use in the UCWW, consisting of an offline modeling part and an online recommendation part.(ii)Second, we propose to leverage HIN to model the information related to users and services, from which rich entity relationships can be generated. The rich relationships are combined with implicit user feedback in a collaborative filtering way to alleviate the cold start and sparsity problems. Recommendation models are defined at both global and personalized level in this paper and are estimated by the Bayesian Personalized Ranking (BPR) optimization technique [8].(iii)Third, we select a subset of the Yelp dataset to construct the HIN which is complementary to the UCWW service recommendation scenario. Based on this dataset, extensive experimental investigations are conducted to show the effectiveness of the proposed models.
The remainder of the paper is organized as follows. Section 2 presents some related work in this area. Section 3 introduces the background and preliminaries for this study. Section 4 presents the layered configuration of the recommendation prototype architecture. The proposed global and personalized recommendation models are presented in Section 5, with parameters estimated in Section 6. Section 7 presents and analyses the experimental results. Finally, Section 8 concludes the paper and suggests future research directions.
2. Related Work
2.1. Collaborative Filtering with Additional Information
Collaborative filtering (CF) is the most successful and widely used recommendation approach to build recommendation systems. It focuses on learning user preferences by discovering usage patterns from the user–item relations [9]. CF recommendation algorithms are typically favored over content-based filtering (CBF) algorithms due to their overall better performance in predicting common behavior patterns [10]. In the past few decades, huge amount of work was done on exploiting user–item rating matrices to generate recommendations [11–14].
In recent years, there is an increasing trend in exploiting various kinds of additional information to solve the cold start and sparsity problems in CF as well as to improve the recommendation quality of CF models. With the prevalence of social media, social networks have been popular resource to exploit in order to improve recommendation performance. Ma et al. [15] introduce a novel social recommendation framework fusing the user–item matrix with users’ social trust networks using probabilistic matrix factorization. Guo et al. [16] propose a trust-based matrix factorization approach, TrustSVD, which takes both implicit influence of ratings and trust into consideration in order to improve the recommendation performance and at the same time to reduce the effect of the data sparsity and cold start problems. User and item side information is also a popular information source for incorporation into CF models in the form of tags [17, 18], user reviews [19, 20], and so on.
To further improve the recommendation performance, HINs have been used to model information related to users and items, in which entities are of various types and links represent various types of relations [21]. Yu et al. [22] introduce a matrix factorization approach with entity similarity regularization, where the similarity is derived from metapaths in a HIN. Luo et al. [23] proposed a social collaborative filtering method, HeteCF, based on heterogeneous social networks. Zheng et al. [24] propose a new dual similarity regularization to enforce the constraints on both similar and dissimilar objects based on a HIN. Majority of the works related to HINs are based on explicit feedback data; few works have been done exploiting implicit feedback data. Yu et al. [25] propose to utilize implicit feedback data to diffuse user preferences along different metapaths in HINs for recommendation generation. However, there are some limitations to this work. Firstly, the authors learn a low-rank representation for the diffused rating matrix under each metapath, which makes the computational complexity of the model training stage relatively high. Secondly, the authors make personalized recommendation based on a group of users obtained by clustering. However, finding a suitable number of clusters for a dataset is a challenging problem and the recommendation performance heavily depends on the quality of the clusters.
In this study, we propose to use item similarities along different metapaths in a HIN directly to enrich the item-based CF. Recommendation models are defined at both global and personalized level, where different metapath weights are learned for each user avoiding the use of user clusters.
2.2. Top-N Recommendation with Implicit Feedback
Every recommendation algorithm relies on the past user feedback, for example, the user profiling in CBF and the user similarity analysis in CF. The feedback is either explicit (ratings, reviews, etc.) or implicit (clicks, browsing history, etc.) [26]. Although it seems more reliable to make recommendations using the information explicitly supplied by users themselves, the users are usually reluctant to spend extra time or effort on supplying such information, and sometimes the information they provide is inconsistent or incorrect [27]. Compared to explicit feedback, implicit feedback can be collected in a much easier and faster way and at a much larger scale, since it can be tracked automatically without any user effort. For this reason, there has been an increasing research attention to the task of making recommendations by utilizing implicit feedback as opposite to explicit feedback data [28].
Along with recommendation, based on implicit feedback, in the last few years, great attention was paid to the top- recommendation problem. Many works have been published addressing both tasks [29, 30]. While rating prediction attempts to predict unrated values for each user as accurate as possible, top- recommendation aims at discovering a ranked list of items which are the most interesting for the user.
In the UCWW recommendation scenario, with consumers feedback available, the proposed hybrid recommendation methods should be able to provide a list of top- services for each active consumer.
3. Background and Preliminaries
3.1. Heterogeneous Information Network
Most entities in the real world are interconnected, which can be represented with information networks, for example, social networks and research networks. The entity recommendation problem also exists in an information network environment, with items recommended by mining different type of relations from resources that are related to users and items.
In real-world recommendation scenarios, multiple-type objects and multiple-typed links are involved. Thus, the recommendation problem could be modeled with heterogeneous information networks (HINs) [21]. The following definition of an information network was adopted from [21].
Definition 1 (information network). An information network is defined as a directed graph with an object type mapping function and a linked type mapping function . Each object belongs to one particular object type , and each link belongs to a particular relation [21].
When the number of object types is greater than 1 or the number of relation types is greater than 1, the network is called a heterogeneous information network (HIN); otherwise, it is a homogeneous information network.
In a HIN, an abstract graph is used to represent the entity and relation-type restrictions as per the following definition.
Definition 2 (network schema). A network schema of is the directed graph defined over the object type with edges from , denoted as , [21].
The definition of the network schema sets the rules on what types of entities exist and how they are connected in an information network. The network schema designed for use in the UCWW service recommendation is shown in Figure 2. Links between a consumer and a service denote their interactions; links between a service and a tag, or a service and a category, denote the corresponding attributes for a service; and links between a consumer and a group, or a consumer and another consumer, denote their social relationships.
In a HIN, two entity types could be connected via different types of relationships following the network schema, thus generating a metapath.
Definition 3 (metapath). A metapath is a path defined on a network schema [21].
Each metapath can be considered as a type of a path in an information network, representing one relation between entity pairs in a HIN. An example of service recommendation in the personal-health location reminder scenario mentioned in Section 1 is described in Example 1.
Example 1. A drug sale reminder service, which advertises a healthcare product, will belong to the “personal-health” category and will have tags like “sale,” “healthcare,” and so on which are supposed to be defined by the service providers. For a consumer , if the recommendation system found that some of this consumer’s friends used the same service in the last two weeks, this service will be in the rank list for recommendation to consumer under a consumer-consumer-service metapath.
3.2. Metapath Based Similarity
In a HIN, rich similarities between entities can be generated following different metapaths. Different metapaths represent different semantic meanings; for example, user-user denotes social relation between two users and user-service-user means that two users are similar because they have similar service-usage histories. The network mining approaches used in homogeneous information networks, such as the random walk used in personalized PageRank [31] and the pairwise random walk used in the SimRank [32], are not suitable for HINs because they are biased to either highly visible or highly concentrated objects [33]. In this study, the PathSim approach is utilized to quantitatively measures the same-type objects’ similarity in a HIN along symmetric metapath [33]. Given two entities belonging to the same type in a HIN, the PathSim is defined as follows [33]: where denotes the path type and denotes the number of path instances between and along metapath .
4. UCWW Service Recommendation Architecture
The service recommendation system in the UCWW [34, 35] works as a platform for connecting service providers with consumers. The service recommendation architecture consists of three layers (Figure 3). The data layer and the computing layer belong to the offline modeling part, in which the similarities between services along different metapaths and their corresponding weights are precomputed. In the online recommendation part, the top- services for the active user are computed at the recommendation layer, based on the results provided by the offline modeling part.
4.1. Data Layer
Information related to users and services is collected and extracted at this layer to construct a HIN, which works as both a service repository and a knowledge base. Compared to most semantic-based recommendation approaches utilizing existing knowledge base or ontology [36], recommendation using a HIN as a knowledge base is more flexible, as it is able to define its own rules (network schema in HIN) for different recommendation requirements.
As shown in Figure 3, in the UCWW, information about consumers and services is collected from three different sources:(i)A central registry, where service descriptions (SDs) are stored, including attributes such as category, quality of service (QoS), biding price, and consumers package [37](ii)A third-party monitoring platform, which provides information about the number of clicks/requests made by consumers for services(iii)User interactions with services in the past, or social relations between users extracted from other social resources, and so on (details about data collection and data management platform can be found in [34, 35]).
4.2. Computing Layer
In a HIN, items could be similar via different types of relations, which represent different reasons for similarity. Therefore, similarity between items in a HIN could be computed from a combination of different relations rather than only from the rating distributions as in the traditional item-based CF. The main task of this layer is to compute service similarities along different metapaths in the HIN and learn the weights for each metapath in both global and personalized recommendation models.
4.3. Recommendation Layer
This is the most external user-facing layer, presenting system facade to the consumers. All the queries are performed through this layer. When a user has a request for finding the “best” instance of a particular service, a ranked list (computed according to a certain recommendation model) is provided as a response back to him/her.
5. Semantic Recommendation Model
In the UCWW recommendation scenario, the number of services and consumers is relatively high, which can cause even more serious cold start and sparsity problems in service recommendation. In this section, we propose to exploit the side information related to services and consumers to alleviate this problem. The side information is first constructed as a HIN, from which rich service similarities under different semantics are calculated. The proposed models incorporate these similarities into item-based CF to improve the prediction accuracy. For each user, the recommendation system will first calculate the prediction score for each unrated service and then recommend the top- services with the highest scores to that user.
5.1. Global Recommendation Model
The item-based CF approach tries to find similar items to the target item, based on their rating pattern. However, with an additional data source related to items and users, items could be similar because of different reasons, based on different features of items. In the UCWW context, within the scope of the HIN, services could be similar due to different reasons via different metapaths. For instance, service-consumer-service represents the relation used in the traditional item-based CF, denoting that two services are similar because they are used by a group of consumers, while service-category-service means that two services are similar because they share the same category. If one can understand the underlying semantic relations between services and discover services based on rich relations, then potentially more accurate recommendations can be provided to the consumers. Based on this observation and the background knowledge presented in Section 3, a global recommendation model [38] is proposed, which utilizes metapaths with the following format: service––service.
Given a metapath with and , the predicted value of a consumer for service could be defined as follows:where is the PathSim value between service and service along the th metapath, is the number of different metapaths considered, is the weight of the th metapath among all metapaths (since different types of metapaths represent different relationship semantics and naturally have different importance in the recommendation model), and denotes the set of services with user interactions in the past.
5.2. Personalized Recommendation Model
With the global recommendation model proposed in the previous subsection, consumers are provided with potentially interesting (for them) services, based on both different types of service relations with rich semantic meanings and service-rating patterns from consumer feedback. However, in real-world UCWW scenarios, consumers’ interests in particular features may differ from each other. For instance, taking the online shopping case as an example, the price of a photo camera is usually much more important criterion for buying than its color, which could be learned from the global recommendation model. However, it may happen that one consumer simply wants a camera of a certain color regardless of the price, which means that the metapath, which includes the corresponding tag (a certain color), should have higher importance. In this case, the accuracy of the global recommendation model may not be sufficient because it only considers the overall weights of features without taking into consideration the consumers’ individual preferences. In order to better capture the consumer preferences and interests, a fine-grained personalized recommendation model is also elaborated in this work, with consideration of every consumer’s interests. It allows a higher degree of personalization compared to the global recommendation model. The personalized recommendation model applied to consumer and service is defined as follows:where represents the weight of interest of consumer in the pth feature (metapath) and is the vector representing the consumer’s preferences for all features (metapaths).
Compared to the global recommendation model with parameters to learn, the personalized recommendation model needs to learn parameters, where is the set of customers.
For both the global and personalized recommendation models, given a consumer, one can calculate the recommendation scores for all services by utilizing either (2) or (3), and then the top- services can be returned to that consumer as the recommendation result. Parameter estimation methods for both models are introduced in the next section.
6. Recommendation Models Optimization
The objective of the recommendation task is to recommend unrated items with the highest prediction score to each user. A large number of previous studies concentrate on predicting unrated values for each user as accurately as possible. However, the ranking over the items is more important [39]. Considering a typical UCWW recommendation scenario, with only a binary consumer feedback available, a rank-based approach, Bayesian Personalized Ranking (BPR) [8], could be utilized to estimate parameters in the proposed recommendation models. The assumption behind BPR is that the user prefers a consumed item to an unconsumed item, aiming to maximize the following posterior probability: where is the rating matrix, represents the likelihood of the desired preference structure for all users according to , and is the parameter vector of an arbitrary model. Thus, BPR is based on pairwise comparisons between a small set of positive items and a very large set of negative items from the users’ histories. BPR estimates parameters by minimizing the loss function defined as follows [8]:where is the sigmoid function of , is the set of available consumers, and are the predicted scores of consumer for items and j, and is the set of items without user ratings yet. Parameters are estimated by means of minimization.
6.1. Global Recommendation Model Learning
In the global recommendation model, the parameter for estimation is which represents the global weights of all metapaths considered.
The gradient descent (GD) approach [40] could be used to estimate this parameter. The gradient with respect to can be calculated as follows:where . For each in , the gradient of isThe process of learning the global recommendation model is presented in Algorithm 1.
| ||||||||||||||||||||||
6.2. Personalized Recommendation Model Learning
In the personalized recommendation model learning process, one need to learn parameters, with a weighted vector of metapaths for each consumer. Considering the large number of consumers and services in the UCWW and the corresponding huge number of parameters to learn, we employ the stochastic gradient descent (SGD) [41] approach to estimate the parameters for the personalized recommendation model.
Similar to (6), for each triple , the update step with respect to is based on BPR and for each triple it is computed as follows: For each , the gradient for is estimated as
The learning algorithm for the personalized recommendation model is presented in Algorithm 2.
| ||||||||||||||||||||||
7. Experiments
7.1. Experiment Setup
In order to simulate a typical UCWW recommendation scenario, we define the network schema for the proposed recommendation prototype as shown in Figure 2. We select a subset of the Yelp dataset (https://www.yelp.ie/dataset_challenge), which contains user ratings on local business and attributes information related to users and businesses. After preprocessing, the new dataset consists of five matrices, representing different relations. The details of the dataset are shown in Table 1. In this dataset, the consumer-service matrix contains 2000 consumers with 8757 service binary interactions on 5000 services, which leads to an extremely sparse matrix with a sparsity of 99.91%.
We randomly take 70% of the consumer-service interaction dataset as a training set and use the remaining 30% as a test set. Six different types of metapaths were utilized for both models in the information network, in the format of service––service as shown in Table 2. For BPR parameter estimation, fifty triples are randomly generated for each consumer in the training set.
7.2. Evaluation Metrics and Comparative Approaches
In the proposed service recommendation prototype, a ranked list of services with top- recommendation score is provided to the consumers. Precision, recall, and -Measure are used to measure the prediction quality [42]. In the UCWW service recommendation prototype, precision indicates how many services are actually relevant among all selected/recommended services, whereas recall gives the number of selected/recommended services among all relevant services.
In the evaluation of the adopted top- recommendation model, precision is normally inversely proportional to recall. When increases, recall increases as well, whereas precision decreases. Therefore, the -Measure, which is the harmonic mean of precision and recall [43], was also used as per the following definition:
For all the three evaluation metrics, a higher score indicates better performance of the corresponding approach.
To demonstrate the effectiveness of the proposed models, we evaluated and compared them with the following widely deployed recommendation approaches:(i)Item-based CF (IB-CF): this is the traditional and widely used item-based collaborative filtering that recommends items based on the item’s -nearest neighbors [11].(ii)BPR-SVD: this method learns the low-rank approximation for the user feedback matrix based on the rank of the items, with model learning by BPR optimization technique [8].
We use Hybrid-g and Hybrid-p to denote the proposed global and the personalized recommendation models, respectively.
7.3. Experimental Results
To examine the effectiveness of the proposed recommendation models, we experimentally computed the top- list, containing items with the highest top- recommendation score for each consumer in the test set. The evaluation and comparison results are shown in Figure 4 (precision), Figure 5 (recall), and Table 3 (-Measure), from which several observations can be drawn.(i)First, IB-CF outperforms BPR-SVD for small values of for both precision and recall, but BPR-SVD achieves better results when increases .(ii)Second, the two proposed recommendation models (Hybrid-g and Hybrid-p) sufficiently outperform the other two methods over a wide range of values of .(iii)Third, the global model Hybrid-g shows overall better recommendation accuracy than the personalized model Hybrid-p, which may be due to the sparsity of the rating matrix as a relatively small number of rated items cannot truly reflect the true interests of consumers.
Similar to the IB-CF, the rich similarities generated from the HIN in proposed models can be also precomputed and updated periodically offline, as well as the learned weights for both models. Given consumers and services, for an active consumer, the upper bound of the computational complexity for top- recommendation among all algorithms addressed in this paper is shown in Table 4 where denotes the number of services the active consumers already used, t is the number of iterations for learning parameters and is the number of latent features in the matrix factorization approach, and is the number of metapaths considered in the proposed models.
As and are much smaller than m, we can assume that the proposed global recommendation model has similar computational complexity to both the traditional IB-CF and BPR-SVD approaches in the online recommendation stage but higher computational complexity in the offline modeling stage for achieving better effectiveness. The computational complexity of the personalized recommendation model is higher than the global recommendation model in both the offline and online stages, with a different set of weights for each user to learn and combine. Between the proposed models, the global recommendation model provides better results than the personalized model and achieves this with lower computation complexity in both the offline modeling stage and the online recommendation stage.
8. Conclusion
Mobile phones are currently the most popular personal communication devices. They have formed a new media platform for merchants with their anytime-anywhere accessible functionalities. However, the most important problem for merchants is how to deliver a service to the right mobile user in the right context efficiently and effectively. The proposed service recommendation prototype can potentially provide a platform to assist service providers to reach their valuable targeted consumers.
The integration of the proposed service recommendation system prototype into the ubiquitous consumer wireless world (UCWW) has the potential to create an infrastructure in which consumers will have access to mobile services, including those supporting smart-cities operation, with a radically improved contextualization. As a consequence, this environment is expected to radically empower individual consumers in their decision making and thus positively impact the society as a whole. It will also facilitate and enable a direct relationship between consumers and service providers. Such direct relationship is attractive for the effective development of smart-city services since it allows for more dynamic adaptability and holds the potential for user-driven service evolution. Besides benefiting consumers, the UCWW opens up the opportunity for stronger competition between service providers, therefore creating a more liberal, more open, and fairer marketplace for existing and new service providers. In such a marketplace, service providers can deliver a new level of services which are both much more specialized and reaching a much larger number of mobile users.
The recommendation prototype proposed in this paper could be potentially employed for discovering the “best” service instances available for use to a consumer through the “best” access network (provider), realizing a consumer-centric always best connected and best served (ABC&S) experience in UCWW. In line with the layered architecture of the service recommendation prototype, two hybrid recommendation models which leverage a heterogeneous information network (HIN) are proposed at a global and personalized level, respectively, for exploiting sparse implicit data. An empirical study has shown the effectiveness and efficiency of the proposed approaches, compared to two widely employed approaches. The proposed recommendation models also have the potential to work under other recommendation scenarios effectively.
However, for service recommendation in the UCWW, we only provided the basic recommendation models in this paper, without considering real-time context information. Also, the similarity matrices computed from different metapaths are still sparse, which may cause some inaccurate rating predictions. As a future work, we intend to conduct further study on context aware recommendations with a real application operating with big data. We also intend to explore the study of matrix factorization approach on similarity matrices derived from different metapaths.
Disclosure
This paper is extended from the paper entitled “Hybrid Recommendation for Sparse Rating Matrix: A Heterogeneous Information Network Approach,” presented at the IAEAC 2017.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This publication has been supported by the Chinese Scholarship Council (CSC), the Telecommunications Research Centre (TRC), University of Limerick, Ireland, and the NPD of the University of Plovdiv, Bulgaria, under Grant no. ΦΠ17-ΦMИ-008.