Abstract
Due to the large amount of opinions available on the websites, tourists are often overwhelmed with information and find it extremely difficult to use the available information to make a decision about the tourist places to visit. A number of opinion mining methods have been proposed in the past to identify and classify an opinion into positive or negative. Recently, aspect based opinion mining has been introduced which targets the various aspects present in the opinion text. A number of existing aspect based opinion classification methods are available in the literature but very limited research work has targeted the automatic aspect identification and extraction of implicit, infrequent, and coreferential aspects. Aspect based classification suffers from the presence of irrelevant sentences in a typical user review. Such sentences make the data noisy and degrade the classification accuracy of the machine learning algorithms. This paper presents a fuzzy aspect based opinion classification system which efficiently extracts aspects from user opinions and perform near to accurate classification. We conducted experiments on real world datasets to evaluate the effectiveness of our proposed system. Experimental results prove that the proposed system not only is effective in aspect extraction but also improves the classification accuracy.
1. Introduction
Tourism is a dynamically growing industry and important for many regions and countries as key industry [1]. Hundreds and thousands of tourists visit tourist places every year and share their opinions on various websites such as TripAdvisor and Opinion Table. These opinions give an overall view of an opinion holder regarding the tourist place [2]. However, there are a large number of opinions which are available on a particular place and it is very difficult for a normal user to review/read all these available opinions and decide on whether to visit a place or not. A number of opinion mining methods [3–13] have been proposed to deal with the large number of opinions and these methods help to classify the opinions into positive and negative. However, these previously proposed methods do not deal with various aspects present in an opinion. Instead, these methods just point out the overall expression (positive or negative) of each opinion [14]. Therefore, new aspects based opinion mining methods [15–35] were proposed. These methods allow users to extract different aspects from opinions and classify each aspect in the opinions into positive and negative. For instance in a given sentence, “Food is delicious but service is slow.” The words “food” and “service” refer to aspects and “delicious” is the positive opinion of the food aspect and “slow” is the negative opinion of service aspect.
In this context, these aspects based opinion mining methods consist of two sequential tasks: aspects extraction and aspects based classification. Extracting aspects and classification of extracted aspects is a difficult and challenging task as reported in [8, 36, 37].
In terms of aspects extraction, firstly identifying the implicit aspects is a problem. Implicit aspects do not directly appear in any opinion but it indicates to an important aspect. For instance, in a given sentence “Last night my wife and I visited Hasten restaurant, the taste was awesome,” the tourist did not mention any important aspect in this sentence. But the indication of this sentence is implying a “food” aspect.
Secondly, identifying the coreferential aspects is a difficulty. It is common that people use different words and expressions to describe the same aspect. For instance, in a restaurant opinion, atmosphere and ambience refer to the same aspect and these are coreferential to each other.
Thirdly, identifying the infrequent aspects is also very cumbersome. Due to large amount of explicit aspects available aspect extraction methods discarded the infrequent aspects. However, some infrequent aspects may be coreferential of frequent aspect or may be important for a tourist place; for instance, Air conditioner and Bed are less frequent aspects but these aspects are important for hotels.
In terms of aspects based classification, irrelevant sentences are another problem. Irrelvent sentences include self-introductory lines of opinion holders. Previous histroy of the visit adds noise and dramatically affects the accurate classification and prediction.
There is a need of an efficient aspects extraction and aspect based classification system for tourism domain to extract useful information for tourists about the different aspects of a tourist place. In this paper, we report the a new fuzzy aspect based opinion classification system. In this system, we propose a fuzzy rules based aspects extraction method that can extract explicit, implicit, and infrequent aspects and can also group coreferential aspects.
For aspect based classification we propose a three-stage fuzzy aspect based classification method using fuzzy logic algorithms. In the first stage, the opinion sentences between opinion words and aspects are filtered using Stanford Basic Dependency method proposed by [38].
In the second stage, features are built from filtered opinion sentences like n-grams and Part-Of-Speech tags. In the last stage, fuzzy logic algorithms are applied on the built features and evaluation has been performed using 10-fold cross-validation. 10-fold cross-validation is useful to limit the problems like Overfitting [39]. The aim of our proposed system is to extract the aspects from each opinion and to classify them into positive and negative based on the opinion words expressed in it using fuzzy logic. Fuzzy logic is handy in real life situations where decisions are taken on the basis of interlinked multiple criteria [40]. The same situation exists with aspect based opinion classification process where algorithm decides the class/label of opinions on the basis of multiple aspects and opinion words. For example, in a restaurant review, a reviewer praises the decoration of restaurant but does not approve the service provided by the staff. Hence, the decision about the opinion label, either positive or negative, is dependent on the opinion words or phrases used by the reviewer for each of the aspects.
Experiments were conducted on real world hotel and restaurant reviews taken from TripAdvisor and OpenTable websites. To evaluate the performance and effectiveness of fuzzy aspect based opinion classification system, we examined the effects of dataset size, time, feature size, feature types, and feature weighted methods on the performance of our proposed system.
In this paper, we argue that fuzzy based classification methods are very useful and effective in aspect based opinion classification. Five predominant fuzzy based algorithms, namely, Fuzzy Unordered Rule Induction Algorithm (FURIA) [41], Fuzzy Nearest Neighbors [42], Fuzzy Rough Nearest Neighbors [42], Vaguely Quantified Nearest Neighbors [42], and Fuzzy Lattice Reasoning (FLR) [43], have been compared with other similar supervised learning methods [17, 18, 20–22]. Comparison results indicate noteworthy improvement in aspect based classification. The proposed system effectively performed aspect based opinion classification, achieving an accuracy of 90.12% on restaurants dataset with FURIA and 86.02% on hotels dataset with FLR.
The rest of the paper is organized as follows. Section 2 presents an overview of previous research related to aspect based opinion mining. In Section 3, we present our proposed fuzzy based aspect extraction and classification model. It is followed by Section 4 in which experimental results on real world datasets are given. Section 5 presents the comparative evaluation of fuzzy based classification with the traditional supervised learning methods. Section 6 summarizes the contributions made in this paper.
2. Related Work
In this section, related work of aspect based opinion classification on tourism domain is presented. The purpose of this related work is to study, analyze, and identify the limitations in this area. Our overview of related topics focuses on two tasks of aspect based opinion classification: aspects extraction and aspect based classification.
2.1. Aspect Extraction
Aspect extraction is a major task in aspect based opinion classification. In the last few years vast majority of aspects extraction methods have been proposed for tourism domain. These methods used different ways and mechanisms to extract the important aspects from tourist reviews. We can categorize these methods into four main categories: rules based methods, seeds based methods, sequence models based methods, and topic models based methods [14].
2.1.1. Rule Based Methods
Rule based methods extracted frequent aspects from reviews using extraction rules that based on frequency, importance, appearance, and domain dependence. Extraction of frequent aspects on the basis of extraction rules is easy and effective. Pekar and Ou [15] proposed a rules based method that extracted aspects from hotels reviews using aspects appearance in each review. They applied TermExtractor on reviews and split them into terms and then form these terms into a lexicon. Then they manually extracted most apparent six aspects (single nouns and multiword nouns) from terms lexicon.
Similarly Muangon et al. [16] applied a LexToPus preprocessing on hotels reviews and split all reviews into features. These features contain both aspects and polar words. Using ranked based approach they extracted all high ranked aspects. Using the same ranked based approach de Albornoz et al. [17] also extracted high ranked nouns from hotels reviews applying a shallow preprocessing including POS tagging.
Marrese-Taylor et al. [18] proposed an aspect extraction algorithm to extract the aspects from restaurants reviews. They transferred all reviews sentences into POS tagged sentences using Part-Of-Speech Tagger. Later, they applied aspects extraction algorithm on POS tagged sentences. This algorithm extracted nouns that frequency is more than ten in all sentences. Similarly Marrese-Taylor et al. [19] extend [18] method. In their extended work, authors firstly used combination of POS tagging and chunking to extract aspects such as nouns and noun phrases from reviews. Secondly, they used approach frequent item set to filter out the more frequent and important aspects from all extracted aspects.
Unlike [18, 19], Hai et al. [20] proposed a different method to extract the aspects. In their method, they extracted aspects on the basis of two criteria, namely, domain-specific and not domain-independent. Firstly, they applied syntactic dependence rules to build a candidate aspects list. Secondly, calculate the score of domain-specific and domain-independent of each aspect from candidate aspects list, which they termed the intrinsic-domain relevance (IDR) score and the extrinsic domain relevance (EDR) score, respectively. Thirdly, those candidate aspects are pruned from candidate list which have low IDA and high ERD score. The limitation of the discussed rules based aspects extraction methods is that they extracted only frequent or important aspects. They pruned or discarded the aspects that have low frequent and not important for tourism domain. Table 1 categorizes these methods into four main categories: rules based methods, seeds based methods, sequence models based methods, and topic models based methods [14].
2.1.2. Seeds Based Methods
Seeds based methods extracted aspects about an tourist place from reviews using grammatical relation between seeds words with opinion words. Colhon et al. [21] selected five most debated aspects in reviews and built five seed sets of each aspect. In these seed sets each word is belonged to an important aspect. After building the seed sets they checked the grammatically relation between the terms in reviews sentences with the each aspect seed set and then grouped these terms under this aspect.
In the same context, Mukherjee and Liu [22] grouped semantically related terms in the same aspect that are more specific and related to the seed words. Wang et al. [23] proposed an algorithm to extract the major aspects from review that is based on bootstrapping approach. In this algorithm, firstly they assigned aspect to the each sentence on the basis of maximum overlapping between sentence words and aspect. Secondly, to check the relationship between assigned aspect and sentences words they calculate the basic dependencies between them. Thirdly, which sentences words have high dependency with assigned aspect that is considered to be an aspect and added into list of aspect keywords.
Similarly Zhu et al. [24] proposed bootstrapping framework that used seed information to extract the meaningful aspects. They consider two types of terms which can be used for aspect identification: POS such as nouns, adjective, adverbs, and verb and -grams. They applied -value method on both types of terms to filter out important and meaningful terms. The meaningful terms were extracted on the basis of the frequency of occurrence of each term. As in rules based methods seed words based methods are also extracting frequent aspects but partially extracting low frequent aspects. Because list of seed words are limited that are determined by a human, with the help of these words majority of low frequent aspects could not be extracted.
2.1.3. Topic Model Based Methods
Topic model based methods are widely applied in aspects extraction and entity recognition, which is based on the assumption that each opinion is a mixture of various topics, and each topic is a probability distribution over different words. Wu and Ester [25] proposed a unified probabilistic model on users preferences about different aspects. In this model they assumed that each opinion about hotel and restaurant is connected with an aspect such as food, service, and so forth. Each of opinions described the importance of connected aspect that depends on three factors: global importance, reviewer impotence, and how much probability that aspect will be mentioned in other opinions. Based on these assumptions they used additive generative methods to extract the aspects.
Xianghua et al. [26] proposed a sliding windows based method to extract the aspects from reviews. In this method, firstly, sliding window scans the review from start to end. On each scan those words come in sliding window analyzed as aspects. Secondly, real computation process has been performed to discover the aspects accurately which were not identified in the first step.
Xu et al. [27] proposed a method JAS which adopts the classic Latent Dirichlet Allocation to make the extracted topics correspond to the reviewable aspects, rather than global properties of entities. They extracted the major aspects like food, service, and so on and fined grained aspects like staff, order, and so on of both hotels and restaurants. For the limitation of topic model based methods, they mostly governed by the phenomenon called “higher-order cooccurrence” based on how often terms cooccur in different contexts. This unfortunately results in many “nonspecific” and “irrelevant” aspects being pulled and clustered.
Moreover, there are limitations of above aspects extraction methods; they are not extracting implicit aspects and not dealing with coreferential aspects problem in reviews. Implicit aspects that do not directly appear in review but the indication of the review to a specific aspect. For example, “food” is an implicit aspect in “This restaurants taste is too good” review. In coreferential aspects problem people use different words and expressions to describe the same aspect. For example, environment and atmosphere refer to the same aspect in restaurants reviews. There should be a mechanism that categorized or grouped the similar kind of aspect.
2.2. Aspect Based Opinion Classification
Aspect based opinion classification is the determination of orientation of opinions of given text in two or more classes about aspects. Opinion classification has been performed in various classes like binary, ternary, -ary in the form of stars, and “thumbs-up” or “thumbs-down,” and so forth. We categorize the existing opinion classification of extracted aspects methods into two main categories: lexicon based and machine learning based methods.
2.2.1. Lexicon Based Methods
Lexicon based methods classified the opinions of aspects into classes using external lexicon resource. These lexicons have opinions words with the positive and negative score. Colhon et al. [21] performed binary classification of reviews using lexicon of positive and negative terms. They applied term-counting method that is based on the positive and negative terms in a review which are related to aspects of the object under discussion. In this method, a review is considered positive if it contains more positive than negative terms and negative if there are more negative than positive terms. A review is neutral if it contains (approximately) equal numbers of positive and negative terms.
Similarly Marrese-Taylor et al. [18] and Marrese-Taylor et al. [19] performed binary classification of tourist product reviews which relies on a sentiment word dictionary that contains a list of positive and negative words (called opinion words). They applied terms score method that is based on the positive and negative term scores in a review which are related to aspects. In this method, a review is considered positive if its positive terns score is greater than negative terms score, and a review is considered negative if its negative terms score is greater than positive terms score. In the same context Muangon et al. [16] performed -ary classification of hotel review using the polar words. The term “polar words” means the lexicons which can identify the aspect such as good, bad, and expensive. This approach for extracting them from opinion text is based on syntactic pattern analysis and calculating the scores. Pekar and Ou [15] performed five-point scale classification of hotel reviews using three different lexicons. They applied opinion terms intensity method based on positive and negative opinion words score.
The limitations of lexicon based methods are domain-dependent opinions words and aspect-dependent opinion words that are used in aspect opinion classification. In domain-dependent opinion words many of the opinions words have different opinion score in context of positive and negative in different domains. We take these two reviews “The restaurant service was very cheap” and “The price of hotel dishes was very cheap.” In these reviews the “cheap” opinion word should have positive score in hotel domain and should have negative score in restaurant domain. In aspects-dependent opinion words many of the opinion words have different opinion score in different aspects. We take these two reviews “The restaurant service was very cheap” and “The price of hotel dishes was very cheap.” In these reviews the “cheap” opinion word should have positive score in “price” aspect and should have negative score in “food” aspect.
2.2.2. Machine Learning Based Methods
Machine learning based methods classified the opinion of aspects into classes using different machine learning algorithms.
Wang et al. [23] proposed a novel Latent Rating Regression (LRR) method which aims to classify the opinions about aspects ratings into five-point scale. Proposed method can decompose the overall rating of a given review into ratings on different aspects and reveal the relative weights placed on those aspects by the reviewer. They implemented the proposed method on Support Vector Regression model and performed 4-fold cross-validation on hotels reviews. Results show that the proposed method acheives 78% accuracy in correctly classifying on the given dataset. In the same context Xu et al. [27] proposed aspect level opinion classification method that can predict the opinion about specific aspects (“staff,” “food,” and “ambience”). In order to avoid ambiguity, they only used sentences annotated with “positive” or “negative” opinion. To evaluate the model, they used the two datasets, the restaurant reviews and the hotel reviews, respectively. The restaurant reviews have been preprocessed with sentence segmentation and Part-Of-Speech tagging. For hotel reviews, they used a NLP toolkit to segment reviews into sentences and used the Stanford POS Tagger to conduct Part-Of-Speech tagging over sentences. They applied the state-of-the-art supervised learning approach, Support Vector Machine (SVM) on both datasets. They used the LibSVM to train the classifier based on the annotation information with all default options. The results show that 83.9% accuracy has been achieved using SVM classifier using 7-fold cross-validaiton.
Similarly, Pontiki et al. [28, 29] proposed system that can classify the opinions of aspect and aspect category into positive negative. They trained a SVM classifier with a linear kernel on manually labeled hotels and restaurants datasets. Then they predict the trained classifier on golden dataset that was labeled by the experts in this domain. In this system, firstly they extracted unigram features from the respective sentences of each of the training datasets. In addition, an integer-valued feature that indicates the category of the tuple is used. The correct label for the extracted training feature vector is the corresponding polarity value (e.g., positive). Then, for each tuple of a test sentence, a feature vector is built and it is classified using the trained SVM. The system scores indicate robustness across the two domains, achieving the most stable performance: 79.34% in hotels and 78.69% in restaurants.
de Albornoz et al. [17] proposed the system that aggregate the information to provide an average rating for the review. They translate the review into a Vector of Feature Intensities (VFI). A VFI is a vector of values, each one representing different aspects. They experience the proposed system with two strategies for assigning values to the VFI positions Binary Polarity and Probability of Polarity. In binary polarity, the aspect position is increased or decreased by 1 depending on wether the sentence was predicted as positive or negetive. In Probability of Polarity the aspect position is increased or decreased by the probability of the polarity assigned to the sentence by the polarity classifier. The VFI is used as the input to a machine learning algorithms (logistic regression, Support Vector Machine, and functional tree) that classifies the review into different rating categories. They used manually labeled hotels reviews to evaluate the method. Results indicated that 71.7% accuracy has been achieved by using logistic regression with 10-fold cross-validation.
The limitation of machine learning methods is that they need labeled data for training the classifier. Above methods use two kinds of labeled data. First is manually labeled data that are labeled by some experts who have knowledge about the domain and they assigned classes to each instance by hand. Second is automatically assigned classes data that crawled from the third party website like (TripAdvisor and Booking.com). These data have classes that have been assigned by the review owners. Manually labeled data is expensive because they need some experts to assign classes to each instance. Automatically crawled data have many useless sentences like self-introduction, previous history, and so on that diluted the opinion classification of aspects. Table 2 summarizes the limitations of lexicon based and machine learning based methods.
3. Proposed System
In this section, we describe the fuzzy aspect based opinion classification system using machine learning. The main objectives of our proposal are to extract important aspects from tourist opinions and classify each aspect opinion into positive and negative. We employ fuzzy logic based algorithms to aspects extraction and aspect based classification. Fuzzy logic algorithms are handy in aspects based opinion classification where the data is very noisy and decisions to be taken are based on multiple aspects. We utilized five predominant fuzzy logic based algorithms individual bases, FURIA, FNN, FRNN, VQNN, and FLR for determine effective of these algorithms. Figure 1 depicts the main phases of our proposed system for aspect based opinion classification. In first phase reviews are collected to build datasets about different tourist places from tourism websites. In second phase, preprocessing has been performed on collected datasets to transform reviews into sentences and eliminate the data redundancy and ambiguity in opinion words. In third phase, fuzzy rules are built using FURIA algorithm to extract and assign the aspect to each sentence of preprocessed datasets. In last phase, we performed classification on aspects assigned sentences into positive and negative using fuzzy logic algorithms.
3.1. Data Collection
We collected two datasets of different sizes from restaurants and hotels domains. Restaurants dataset consists of 2000 reviews, out of which 1000 are positive and 1000 are reviews, and hostels reviews consists of 4000 reviews, out of which 2000 are positive and 2000 are reviews that we collected through a crawler from TripAdvisor website. We selected reviews of top five restaurants and top five hotels of London city from TripAdvisor website.
3.2. Data Preprocessing
The data preprocessing of collected reviews includes three steps: in the first step, we removed data redundancy because operators of the hotels and restaurants posted the review with background information about their own hotels and restaurants. If such information is included, the opinions would introduce certain bias into the dataset. So those reviews that are posted by the hotels and restaurants operators must be removed from the collected data. In the second step, we generate sentences from the collected reviews, based on sentence end characters as delimiters (i.e., period, exclamation, and question mark). In the last step, we correct ambiguous words because ambiguous words cannot be identified by classifier. Some examples of ambiguous words are “goooood, yummy, and dreamy” which are not standard English words. These words have vague meanings which may affect the aspect of opinion. So we fixed these words into standard English words like “good, yummy, and dream.” After preprocessing the data, the restaurants dataset consists of 3787 sentences and hostels dataset consists of 7802 sentences.
3.3. Aspect Extraction
The aim of aspect extraction is to extract the aspects from reviews that are relevant to tourist places. We have proposed a fuzzy rules based method to extract explicit and implicit aspects from reviews. The algorithm of the proposed method is shown in Algorithm 1.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||
The basic workflow of the proposed aspect extraction algorithm is as follows: take all the review sentences as input then the algorithm assigns aspect to each sentence. Firstly, we extract explicit aspects from the given sentences using Stanford Part-Of-Speech Tagger [38] shown in lines to of Algorithm 1. In this procedure, we build Part-Of-Speech tags by applying the Tagger on each sentence shown in lines and . Then, filter out the Noun and Noun Phrases as explicit aspects shown in lines to . Secondly, we group all coreferential aspects that have the same meaning or indication to the same aspect by applying WordNet synonym set [44] and select high frequent one as leader of group shown in lines to . In this procedure we match the synonyms relationship between aspects by applying WordNet on each aspect shown in lines and . If relationship exists, then group both aspects and make the high frequent one as a leader of that group shown in lines to . Thirdly, after extracting explicit aspects and grouping the coreferential aspects, we select the frequent aspects basis of frequency of each explicit aspect and combined frequency of each coreferential aspects group. We set ten frequencies of each frequent aspect in all sentences for selection shown in line . Fourthly, we build fuzzy rules using FURIA algorithm on the basis of selected frequent aspects. We generate rules involving each opinion word of a sentence as the condition and a frequent aspect as the consequence, where opinion word and frequent aspect cooccur frequently in sentences shown in line . Fifthly, we apply the generated fuzzy rules on all the sentences to identify the aspects from all sentences and assign identified aspects to each sentence shown in lines to . In this procedure, we match each word of sentence with built fuzzy rules shown in line . If match exists then assign that aspect to the sentence shown in lines to . If match does not exist then discard that sentence from dataset. Lastly, we return all the sentences with assigned aspects shown in line of Algorithm 1.
3.4. Aspect Based Opinion Classification
In aspect based classification phase, we classified the opinions of aspects into positive or negative. For this purpose we proposed fuzzy aspect based classification method that can classify the opinions of aspects into positive and negative using fuzzy logic algorithms. This method consists of three stages: filter opinion sentences, features bulling: and classifier.
3.4.1. Filter Opinion Sentences
When we crawled reviews from third part website (TripAdvisor) there were irrelevant sentences. In these irrelevant sentences, reviewer did not discuss about any aspect making it difficult to remove these sentences from reviews. There are two kinds of irrelevant sentences. One kind of irrelevant sentence exists at the start of review. Reviewers used these sentences to introduce about these trips or talked about why they visited the places, for example, “Been there a few times for lunch with friends and work but it is my first time here for dinner”; in this sentence the reviewer did not discuss about any aspect. We should remove this kind of redundant sentence that only causes the noise in the reviews. In the second kind of irrelevant sentence the reviewer just mentioned aspects but did not provide any opinion about these aspects. For example, “My father and I ordered fish, chicken and desert with the help of my uncle”; in this sentence there is no opinion in the food aspect. So they should be removed too or else they are just noise.
To remove the irreverent and aspect less sentences, we applied Stanford Basic Dependency [6] that checks the dependencies between opinion words (Adjectives) and aspects (Nouns). If an aspect does not have any dependency with opinion words then it will be removed from the reviews sentences.
3.4.2. Feature Building
The reviews sentences dataset is used to extract features that will be used to train our classifier. We built -grams and POS tags features from the datasets. The process of obtaining -grams and POS tags from a review is as follows: in first step of process, we tokenized the review by splitting it, on basis of spaces and punctuation marks, and form a bag of words. However, we make sure that short forms such as “don’t,” “I’ll,” and “she’d” will be considered a single word. For POS tags we extracted only verb, adverb, and adjective from dataset. In second step, we removed stop words (“a,” “an,” and “the”) from the bag of words. In last step, we deal with negation; a negation (such as “no” and “not”) is attached to a word which precedes it or follows it. For example, a sentence “I do not like fish” will form three bigrams: “I do + not,” “do + not like,” “not + like fish.” This last step allows improving the accuracy of the classification since the negation plays a special role in an opinion expression.
3.4.3. Classifier
Fuzzy logic algorithms are handy in real life situations where the decision to be taken is based on multiple criteria with complex interlink among them. It is very true for opinion classification process in which the algorithms must be able to understand the opinion expressed by a tourist in a review based on the opinions about various aspects of the tourist place. For example, in restaurant reviews, the some reviewers may praise the decoration of restaurant and some blame the service and staff. Deciding on the opinion as positive or negative depends on the opinion words or phrases used by the reviewers for each of the aspects. When the number of aspects is more, the complexity in the decision-making gets added and hence the decision-making becomes tough. In such situations, fuzzy logic can be effectively used. We have used five fuzzy logic based algorithms individual bases, FURIA, FNN, FRNN, VQNN, and FLR for determine effective of these algorithms in the opinion about aspects in tourist places reviews.
(1) FURIA. Fuzzy Unordered Rule Induction classifier uses greedy approach to learn rules by implementing separate and conquer strategy [41]. The classes are used to make learning rules, starting with the shortest rule. Later, all the rule instances involved in learning are removed from training data. The process is continued till all target class instances are removed [41]. The propositional version of First-Order Inductive Learner (FOIL) algorithm is used to attain the process of rule growing. An empty conjunction is assigned to the rule to initiate it and features/selectors are added till no more negative instances are covering rule. The choice of prospective feature is such that it maximizes the FOIL’s information gain criterion (); it is a measure for rule improvement compared to default rule for the target class. This measure is given bywhere and represent number of positive and negative instances participating in rule during growing phase, respectively. Similarly, and represent number of positive and negative instances participating in default rule, respectively. The replacement of intervals with fuzzy intervals named fuzzy set with trapezoidal membership function results in a fuzzy rule.
(2) Fuzzy Nearest Neighbors (FNN). This technique of classification is based on similarity to nearest neighbors and these neighbors’ class membership [42]. Consider a set of objects . A test object within is considered to be classifying object. All remaining objects of this set are considered to be training objects [42]. According to the algorithm, measure the fuzzy similarity of all training objects with the test object one by one (fuzzy similarity is basically weighted distance between a training object and a classifying object) [42]. Choose training objects having highest degrees of similarity. All of these chosen objects have specific memberships to existing crisp classes. In simple words, each of these chosen objects belongs to a certain class to a certain degree. Now the test object is to be classified by using the information about class membership of nearest neighbors.
The extent to which an unclassified object belongs to a class is computed as follows: where nearest neighbors -valued similarity of and .
(3) Fuzzy Rough Nearest Neighbors (FRNN). This algorithm combines the approach of FNN algorithm and fuzzy rough approximations [42]. From the FNN approach get nearest neighbors and from the fuzzy rough approximations get fuzzy upper and lower approximations of decision classes. For example, consider a set of objects called [42]. One of the objects is considered test object called and the remaining are training objects. Establish a fuzzy relation between a test object and each of the training objects. Calculate the similarity value of each couple that varies from 0 to 1. Choose the training objects with highest value of similarity as the nearest neighbors. Determine the upper and lower approximation of each class by means of the nearest neighboring objects. Predict the class membership of test object by using upper and lower approximations [42]. Output the decision class with the resulting best combined fuzzy lower and upper approximation memberships. Let = set of decision classes, = training data, and = test object to be classified output class: ; ; ; ; ; .There are two instance algorithms of FRNN named FRNN-FRS and FRNN-VQRS. Both differ in their approximations. FRNN-FRS uses traditional approximations of all and at least one; on the other hand FRNN-VQRS uses VQRS approximations of most and some. Consider a class , so high value of upper approximation reflects that all or most of the neighboring objects belong to class ; similarly high value of lower approximation reflects that at least one or some of neighboring objects belong to class for FRS and VQRS approximations, respectively.
(4) Vaguely Quantified Nearest Neighbors (VQNN). This algorithm is a variant of FRNN (Fuzzy Rough Nearest Neighbor) algorithm [42]. According to the algorithm a test object is classified using VQRS (vaguely quantified rough set) approximations. In contrast to traditional approach, VQRS approach uses “most” and “some” quantifiers for upper and lower approximations, instead of “all” and “at least one,” respectively. The VQRS approximations are favorite compared to the tradition approximations because as opposed to traditional approximations these approximations do not bring any drastic change to the upper and lower approximations by change in a single object [42]. So impact of noise will be less on VQRS approximations compared to traditional approximations. For example, consider a set of objects containing a test object to be classified and training objects. A subset of training objects is considered to be the nearest neighboring objects. Classify test object to a class on basis of upper and lower approximations of a class in these nearest neighbor objects. If we consider a class , high value of upper approximation reflects that most of the neighboring objects belong to class ; similarly high value of lower approximation reflects that some of neighboring objects belong to class . In this algorithm, is replaced with and is replaced with .
(5) Fuzzy Lattice Reasoning (FLR). This algorithm consists of a set of fuzzy lattice rules. These rules are induced from the training data. The classification of testing data is performed on basis of induced rules of classifier [43]. Consider that is a set of data objects including all types of data exist in the universe but in this case focus is on lattices. A fuzzy lattice is designated as . It is couple of a lattice and its valuation function . It is composed of a number of its constituent elements. Each element is associated with a class [43]. Consider as a set of classes that will be assigned to these lattice elements. Now fuzzy lattice rules are induced. Each fuzzy lattice rule is a couple of object and its corresponding class, that is, that implements function . These are the couples of training set. When a new object comes in the universe then existed rules compete over it to categorize this newly coming object to a category. Consider that a new object comes in . Calculate its inclusion measure parameter. Now this is presented to each rule of classifier iteratively. Rules compete to categorize . Eventually is categorized in one of the classes which belong to on basis of its inclusion measure parameter.
4. System Evaluation
In this section we present the fuzzy aspect based opinion classification system evaluation experiments that determine the performance of system on restaurants reviews dataset that we crawled form reviews websites such as TripAdvisor and OpenTable. We presented both tasks: aspect extraction and aspect based classification experiments results.
4.1. Aspects Extraction
In terms of aspects extraction, experiments are conducted to determine the percentage of correctly extracted aspects. Aspect extraction achieved a better performance 79% in restaurants dataset and 88% in hotels dataset. Results show that explicit aspects including frequent and infrequent are the most common aspects. The percentage of extracted explicit aspects for resturant and hotel datasets is 55% and 61%, respectivily. Moreover, the percentage of infrequent aspects from the explicit aspects was 21% and 23%, respectivily. Secondly, implicit aspects, the second important type, have been correctly identified from restaurant and hotel datasets with the respective percentage of 17% and 15%. Thirdly, coreferential aspects that include three or four explicit aspects, representing 7% in restaurants dataset and 10% in hotels dataset.
4.2. Aspect Based Classification
In terms of aspect based classification, experiments are conducted to examine the performance of each algorithm on different sizes of datasets, different feature weighting methods, and different feature types in aspect based classification. We also examine the time taken of each classifier on different sizes of datasets. The results obtained by aspect based classification task using restaurants dataset are presented in Table 3 and Figures 2–9. Table 3 presents the performance of each algorithm on restaurants and hotels datasets. In performance evaluation we record higher accuracy 90.12% with FURIA on restaurants dataset and 86.34% with FLR on hotels dataset.
Figures 2 and 3 present the each algorithm accuracy according to its time taken in labels prediction. In time based experiment we record that FNN takes less time in restaurants dataset and FLR takes less time in hotels dataset due to different sizes of datasets. FLR time in label prediction is quite low compared to FNN. So we can say that FLR is faster than FNN or any other fuzzy algorithms for big datasets.
Figures 4 and 5 present the effect of features types such as Unigrams, Bigrams, Trigrams, and POS on performance of aspect based classification. We run each algorithm on each feature type at the end of this experiment; we record that Unigrams and POS provide the better accuracy with FURIA on restaurants dataset and FLR on hotels dataset.
Figures 6 and 7 present the effect of feature weighting methods such as Presence, TF, and TF-IDF on performance of aspect based classification. As the previous effect of feature type experiment we use the same approach in this experiment; we run each feature weighting method on each algorithm. We record that presence weighting method provides better accuracy with FURIA on restaurants dataset and FLR on hotels dataset.
Figures 8 and 9 present that effect of size of dataset on the performance. We split both datasets into four parts for restaurants such as 500, 1000, 1500, and 2000 and for hotels such as 1000, 2000, 3000, and 4000. As mentioned above we run each algorithm on each dataset, respectively. In this experiment we record that 1000 reviews of restaurants dataset provide the better accuracy with FURIA and 4000 reviews of hotels dataset provide the better accuracy with FLR.
So the overall experiments results show that in restaurants dataset FURIA provides better accuracy and in hotels dataset FLR provides the better accuracy. On smaller dataset (restaurants reviews) where interlinking between opinion words and aspects is less FURIA build more effective rules than FLR. However, on large datasets FURIA rules are not very effective due to the complex interlinking between opinion words. FLR, on the other hand, handles such large datasets interlinkings more effectivily as compared to FURIA.
5. Comparison
In this section we compared our system with other aspects extraction and aspects based classification systems in tourism domain. Tables 4 and 5 compared the results of other systems and our proposed system. Results were computed by simply the results obtained by the best systems on tourism domain datasets. These results show an improvement, being higher than others in terms of aspects extraction and aspects based classification in the tourism domain.
6. Conclusion
In this paper, we proposed an aspect based opinion classification system that can extract aspects from reviews and classify these reviews into positive and negative about extracted aspects. Firstly in this system, we proposed fuzzy rules based method that built rules from frequent nouns and noun phrases using FURIA algorithm and used for aspects identification. Secondly, we proposed a three-stage fuzzy aspect based opinion classification method that classified the opinions of extracted aspects into positive and negative. Finally, evaluation experiments were designed to run on real world datasets taken from restaurants and hotels reviews. The proposed system achieved improved results as compared to already reported results in the literature. FURIA algorithm achieved the better results as compared to other fuzzy classifiers with the 90.12% accuracy on restaurants dataset and FLR algorithm achieved the best result with the 86.02% accuracy on hotels dataset. Resultantly, tourists could easily get meaningful information about any tourist place that would be helpful to take a decision about tour to any tourist place.
Competing Interests
The authors declare that there are no competing interests regarding the publication of this paper.
Acknowledgments
The authors of this paper are thankful to the financial supports of the grant offered with code MYRG2015-00024, called “Building Sustainable Knowledge Networks through Online Communities,” by RDAO, University of Macau.