Abstract
Fingerprinting-based algorithms are popular in indoor location systems based on mobile devices. Comparing the RSSI (Received Signal Strength Indicator) from different radio wave transmitters, such as Wi-Fi access points, with prerecorded fingerprints from located points (using different artificial intelligence algorithms), fingerprinting-based systems can locate unknown points with a few meters resolution. However, training the system with already located fingerprints tends to be an expensive task both in time and in resources, especially if large areas are to be considered. Moreover, the decision algorithms tend to be of high memory and CPU consuming in such cases and so does the required time for obtaining the estimated location for a new fingerprint. In this paper, we study, propose, and validate a way to select the locations for the training fingerprints which reduces the amount of required points while improving the accuracy of the algorithms when locating points at room level resolution. We present a comparison of different artificial intelligence decision algorithms and select those with better results. We do a comparison with other systems in the literature and draw conclusions about the improvements obtained in our proposal. Moreover, some techniques such as filtering nonstable access points for improving accuracy are introduced, studied, and validated.
1. Introduction
Indoor location systems provide a solution for user navigation in places where GPS signals are not available. There are many alternatives available for locating users indoors that can be categorized in several ways according to the mathematical algorithms used, radio technologies required, or hardware components used. Some of the alternatives are based on the user carrying a mobile device and others are based on device free [1]. One classical categorization of indoor location systems divides them into presmartphone and smartphone eras. The former are based on specific hardware components like infrared badges, ultrasound tags, laser rangers, and wireless modules such as RFID [2]. Since they require additional equipment to be carried by users, they tend to be difficult to deploy and use in real scenarios. The latter are based on the use of smartphones as the user device for indoor location [3, 4]. Smartphones are reliable and user friendly tools that allow people to have access to information services anytime and anywhere. Some of these services may require using the location of the user and, consequently, integrating indoor location services in smartphone based systems is a natural way to minimize deployment requirements. Smartphones have access to several radio technologies such as Wi-Fi, Bluetooth, or terrestrial cellular mobile networks. They incorporate built-in sensors such as accelerometer, gyroscope, magnetometer, microphone, or camera. They are, therefore, computing platforms which are capable of sensing the environment and applying the required calculations in order to locate the users in scenarios where the GPS signals are not available.
Smartphone based indoor location systems can be categorized into two major families: on-demand location systems and continuous user location tracking systems. The former normally rely on fingerprinting techniques [5–9] while the latter tend to use inertial embedded sensors to calculate incremental user displacements [10, 11].
This paper focuses on fingerprinting smartphone-based algorithms which are employed to locate users at room level (the exact location of the user inside a room is not that relevant but knowing the exact room the user is in with the highest accuracy is crucial). Using room level granularity will allow us to overcome one of the crucial aspects of indoor location systems when the areas to cover are big: scalability [12]. Training indoor locations systems with a complete grid of training points is time consuming and does not scale on computational resources required for estimating user locations. Some research proposals such as [12] overcome this limitation by using crowdsourcing-based place learning. However, this approach requires the manual involvement of volunteer collaborating users to improve the accuracy of the system and collateral aspects such as data quality should be taken into account. We will focus on optimizing the way by which to select sample points to train the system by considering the geometry of the areas in which to position users.
In summary, this paper contributes in 3 major ways:(1)The way to select the most convenient sampling points to measure training fingerprints in order to increase the accuracy while minimizing the memory and CPU execution requirements. The cost for training the system includes both the number of training points and the number of fingerprints at each point. The paper focuses on reducing the cost associated to the number of points not adding additional constrains on the number of fingerprints at each training point.(2)The analysis and evaluation of the performance of different decision algorithms.(3)The way to filter unstable access points in order to increase accuracy and minimize execution requirements.
The rest of the paper is organized as follows. Section 2 presents a brief review of the major algorithms that have been previously proposed in order to estimate the current user location based on already located fingerprints used when training the system. It also defines the scope of the research in this paper and its major contributions. In Section 3 we propose several options for deciding where to take fingerprints in order to best train the system. Previously used alternatives are presented together with a novel mechanism that will be analysed and validated in Section 6. In Section 4 we present a mechanism designed to select the best WAPs among those available. This section shows how this mechanism provides optimal results in terms of the relative accuracy provided, as well as the computational resources needed. Noise and stability filters are presented. Section 5 introduces different approaches to combine several fingerprints when locating unknown points in order to improve results. This is done thanks to the analysis of the time fluctuations of the received signal strengths. Section 6 captures the validation results using experimental data in two real case scenarios. Conclusions are presented in Section 7.
2. Fingerprinting-Based Algorithms
Several different approaches and technologies have been proposed, analysed, explored, and validated for indoor positioning systems, fingerprinting being the preferred alternative for smartphone-based indoor locations systems [13]. Many approaches have been tried on location fingerprinting [14]. KNN (K-Nearest Neighbor), which converts fingerprints into vectors and chooses the K historical fingerprints with the shortest distance to the measured fingerprint to estimate the location, has been used in algorithms such as Redpin [14]. Authors in [15] provide a general comparison of SVM (support vector machines), KNN, Bayesian modeling, and multilayer perceptrons. Carlotto et al. [16] uses a statistical Gaussian Mixture Model. Authors in [14] propose an enhanced version of the Redpin algorithm by modifying the significance of each AP (access point) at each location according to their correlation and introduce a noise filter to discard less frequently visible APs at each location. The authors compare their results with Naïve Bayes classifiers (NBC) and SVM based algorithms. Authors in [17] propose a Wi-Fi indoor location technology based on the combination of fingerprinting and the use of K-Means algorithm. The different approaches are normally compared using two major metrics: relative accuracy (%) and average errors () [13].
The authors in [18] present an analytical model for analyzing fingerprinting-based indoor location systems. They develop the framework which analyses a simple positioning system that employs the Euclidean distance between a sample signal vector and the location fingerprints of an area stored in a database. They analyse the effect of the number of access points that are visible and radio propagation parameters on the performance of the positioning system and provide some preliminary guidelines on its design. They predict that in optimal circumstances a resolution of a few meters could be expected in such systems. This predicted result for best algorithm’s resolution has been a common fact for the latest developed systems [13]. Authors in [19] propose the idea of using Virtual Access Points (VAPs) in order to improve the location’s accuracy. The idea is particularly significant if the number of available access points is small. The resolution of the systems using a KNN (K-Nearest Neighbors) approach is again of a few meters approximately. A recent experiment comparing 13 infrastructure-based and 9 infrastructure free indoor location technologies showed that best results for Wi-Fi fingerprint based technologies were close to 1 meter when using Bayesian filters [20] but that the localization accuracy degrades as much as 3 meters due to setup and environmental changes, such as human or furniture changes.
Indoor location systems focus on solving the problem of best estimating the user location. The exact user location inside a building is occasionally required. In some occasions, however, the location is only required to be estimated at room or area levels. Smartphone-based indoor location systems which rely on fingerprinting can be used for either finding the best estimate for the user location inside a building or for finding the best room or area in which the user is located. In the first case it was found that best estimate means that an exhaustive training of the system is required in order to measure fingerprints in as many training locations as possible. Trying to estimate the user location at a room level involves, on the other hand, different training requirements for the system. In this paper we focus on best solving this second problem. In the next section we analyse different alternatives for selecting the best training locations for indoor location fingerprinting-based algorithms. The validation results are presented in Section 6.
The major contribution of this paper is the study, analysis, and evaluation of the relative accuracy of the different smartphone fingerprinting-based indoor location algorithms when locating users at room level. This has been done using different strategies for selecting the points used to train the system, applying different values for noise and stability filters in order to select the best WAPs, and taking the average of several measured fingerprints when locating the user. In particular the following algorithms have been used:(i)ADTree, BFTree, Decision Stump, FT, J-48, J-48graft, LADTree, LMT, NBTree, Random Forest, Random Tree, REPTree, SVM, Naïve Bayes, KNN, Redpin, and WASP.
The confidence is also analysed in order to select the best algorithms, the number of WAPs to be considered, the values for the noise and stability filters, and the location of the training points. The results are also validated by comparing them to those presented in papers which deal with similar aspects.
3. Selecting Training Points
Smartphone fingerprinting-based indoor location systems require a training phase to be carried out before they can be used to locate new unknown points. Select the locations where taking measures for the set of training fingerprints is by itself a decision that has to be studied, analysed, and optimized taking into account both the final relative accuracy of the location algorithm and the underlying computational requirements. Two major approaches are examined in previous studies: using a complete grid of points which cover the entire part of the building where users are to be located [18] and using a randomly distributed set of points inside the building [14] which cover the entire area but without the restriction of having to use a regular grid of training locations. The first approach aims to provide a good resolution for the estimated location of unknown points in every part of the building by considering a number of training fingerprints that grows as (being the linear dimension of the building). The amount of time required to take the training fingerprints and the computational cost in terms of memory, CPU, and execution time needed to run the location algorithms are major drawbacks of this approach if big areas are to be covered. Randomly selecting the locations to measure the training fingerprints allows us to provide better resolutions in certain areas of interest and to relieve the requirements on the total amount of fingerprints to be measured. This approach has been used in [14] in order to provide location estimates at room level for certain rooms in a building. The major problem of this approach is that it does not give us a mechanism to optimize the number of points to consider for training the system but a mechanism which simply selects them in the areas of interest.
In this paper, we propose a third approach to minimize the requirements for the number of training fingerprints when using the indoor location algorithms at room level. We propose to only measure the points in the border of each room without fingerprinting the interior part of each area. This approach is aligned with the objective of the system to be designed: to differentiate among rooms without taking into account user movements inside each room. The results in Section 6 show how this approach can provide even better accuracy than exhaustive fingerprinting of each room with a smaller number of training fingerprints required.
One of the parameters to be considered when fingerprinting an area for training the indoor location system is the distance between adjacent points that need measuring. The authors in [18] propose that in order to select distance between the points for optimal sampling for training fingerprints, the compromise between granularity and accuracy should be taken into account. They do not recommend selecting points which are less than 1 meter away from each other. The review done in [13] of the different algorithms used for locating users indoors also shows that optimal resolutions provide an average error of several meters. The Redpin algorithm, for example, provides an average error of 3 meters. Experimental results in Section 6 are carried out taking these results into consideration.
4. Applying Noise and Stability Filters
The temporal stability of the RSSI has a major impact on the results of the algorithms that use it to estimate the location of the user in indoor environments. The authors in [18], when presenting an analytical model for analyzing fingerprinting-based indoor location systems, recognise the impact that the value of the variance of the measured RSSIs has on predicted results. While they develop a model to predict the expected accuracy of a fingerprinting-based system based on the measured parameters, they do not propose mechanisms to improve the accuracy by introducing additional factors to improve nonmodifiable parameters such as the standard deviation of signal strength over time from a single access point. In this section we propose the application of noise and stability filters to improve results when unpredictable and unavoidable changes in the physical setting have to be considered.
Authors in [14] implement a noise filter to remove less frequently visible APs for a given location, considering these signals as noise. They divide the APs into relevant and noisy APs. They also take into account the percentage of times in which relevant APs contribute to the fingerprints in one location and create a correlation index between locations and APs. The obtained results improve those of the basic algorithm and constitute a promising field in order to further optimize the relative accuracy of fingerprinting-based indoor location systems.
In this paper we propose (and validate in Section 6) an improvement of the mechanisms proposed in [14] to reduce the impact of the fluctuations of the measured RSSIs on the location estimation results. We propose to filter the WAPs performing a stability based classification using different thresholds. The more restrictive the threshold is, the more stable the WAPs to be selected are (those with measured RSSIs fluctuating less over time). However, a more restricted threshold also implies a smaller number of WAPs to be selected which may have a negative impact on the results. The results presented in [14] show that a number of 10–15 WAPs is enough to obtain optimal results. The algorithm in which we propose to select this set of optimal WAPs in terms of stability is the following:(1)Take several measures over time in the same location to train the system (ideally on different days and different times).(2)Mark as potentially unstable those WAPs whose number of noisy RSSIs is, for a particular training location, below a noise threshold.(3)Mark as potentially unstable those WAPs whose variance for the RSSI is, for a particular training location, greater than a stability threshold.(4)Finally, those WAPs marked as potentially unstable must be removed if the number of training points where they are marked as potentially unstable is above a threshold.
The selection of the values of these thresholds will provide a mechanism to select either a smaller number of more stable WAPs or a bigger number of less stable WAPs. The results captured in Section 6 show that selecting the best 10–16 WAPs will provide the optimal results.
In more detail, the following four thresholds have been set for the analysis performed in this paper.
(1) Noise Threshold (NT). We set its value to −80 dBm as it is considered the minimum acceptable RSSI value for establishing a connection.
(2) Number of Noisy Samples Threshold (NNST). It is calculated according to the following formula: NNST = Samples factor, where Samples , factor and (Samples factor) .
(1) Variance Threshold (VT). We set its value to 60 dBm2 (~8 dBm). This means that we choose ±4 dBm as the acceptable range of oscillation of a set of RSSI values for a given WAP. Above this threshold we consider that it is not possible to achieve stable transfer rates.
(2) Location Threshold (LT). It is calculated according to the following formula: LT = Locations factor, where Locations , factor and (Locations factor) .
Having established the above thresholds, the first step of the algorithm is to calculate the number of noisy samples by WAP for each location. A sample is considered a noisy one when its RSSI is less than the value set as NT (note that the WAP is not considered noisy if it is not detected at a certain point, only if it is detected with a fluctuating power below the noise threshold a number of times above the NNST).
Then, for each location and wireless access point we check if the result obtained in the previous step equals or exceeds the NNST. If so, the WAP is marked as unstable for that location. If not, the variance of its RSSIs is calculated. If this number exceeds the VT, the WAP is marked as unstable for that location. In summary, we consider that a WAP is unstable at a given location when the number of noisy samples for this access point in that location equals or exceeds the NNST or the variance of the RSSIs for that wireless access point and location exceeds the VT.
Finally, for each wireless access point we add the number of locations where the WAP is considered unstable. If the result equals or exceeds the LT we eliminate this WAP.
Depending on how the previous four thresholds (noise, number of noisy samples, variance, and location) are configured we can be more or less strict when taking into account a smaller or larger number of WAPs. For instance, if we keep fixed NT and VT, we take a number of samples (Samples) and locations (Locations) equal to 10 and 12, respectively, and adjust the factors involved in the calculation of NNST and LT; we would obtain the results shown in Table 1.
In the first row of Table 1, the number of wireless access points that would result after applying our algorithm to delete unstable WAPs, with the configurations indicated for NNST and LT, is shown. As can be appreciated, as we expand both thresholds we are less strict; hence, we obtain a larger number of WAPs.
5. Optimizing the Location Estimations by Fingerprint Averaging
Another mechanism that may be used to improve the performance of the fingerprint based indoor location algorithms is combining several fingerprints measured when trying to estimate the location of an unknown point. The more the fingerprints over a longer period of time the better the impact one expects to obtain in the predicted results. However, there is a trade-off in the number of fingerprints that can be taken into account in a real setting since the user expects to obtain a result from the algorithm in a reasonable amount of time. In this paper we will analyse the impact on the relative accuracy obtained when combining several consecutive fingerprints taken over a few seconds interval. In order to combine the fingerprints we propose two different algorithms (the results will be presented in Section 6):(1)Calculate the average of the measured fingerprints and make an estimate of the location of this average fingerprint.(2)Estimate the location of each of the measured fingerprints and make a majority based decision in order to select the most likely location.
Section 6 shows the results when 2, 3, or 5 fingerprints are used in the average.
6. Experimental Results
6.1. Environment
All the tests described in this paper took place in the first floor of the Torres Quevedo Building (including the hallway and the research lab 4.1.C.03) of the Telematics Engineering Department at the Carlos III University of Madrid. In Figure 1, the map corresponding to the interior of the first floor of the building is shown and the location of the selected points for the two experiments conducted is presented.
For the first experiment 12 post-its (4 for each area) were arranged as training labels and they were separated by a distance of 3 meters. These markers indicated the points that were going to carry out the sampling. In total there were 12 training locations that we grouped and divided into 3 sequential areas: 6 interior area locations and the other remaining 6 were the ones we called “border” locations and we defined the three areas designated on the map. For each of the locations we needed to make 5 consecutive measurements (at intervals of 15 s) of the RSSI of Wi-Fi networks. Once the 12 points had been scanned we repeated the same procedure for each location; hence the training had involved 10 steps in two batches of 5 spaced in time. This made a total of 120 (12 locations 10 measures) fingerprints. The next day we performed the same experiment, which made it in total a final set of 240 fingerprints.
For the second experiment, a more complex setting was selected. We used 3 areas physically separated by walls and doors (two connected aisle sections, separated by a connecting door and a research laboratory wall to wall with one of the aisle sections). We took 6 location points in each of these areas following the proposed approach to sample points in the border between areas. The borders sample points have been measured with a 2-3-meter separation criteria.
6.2. Experimental Methods
To carry out the analysis of the tests we used 10-fold cross-validation to estimate how accurate a model is before implementing it. Using this technique the input data set is divided into 10 equal-sized subsets. Of these, one subset is taken to validate the generated model, while the remaining nine are used to train the system and generate the corresponding model. This process is repeated 10 times, using each of the 10 subsets exactly once to validate the model. The 10 results from the 10 iterations are then averaged to produce a single estimation. The sampling type we have chosen is “stratified” because not only does it generate random subsets but also ensures that the class distribution in these subsets is the same as in the initial set (Figure 2).
In all experiments we simulated, using the RapidMiner Studio 6.0.008 tool, what would be the response of our indoor location system when training with each of the following algorithms?
(1) ADTree. The basic algorithm is based on [21]. This current version only supports two-class problems. The number of boosting iterations needs to be manually tuned to suit the dataset and the desired complexity/accuracy trade-off. The trees’ induction has been optimized, and heuristic search methods have been introduced to speed learning.
(2) BFTree. This algorithm uses binary split for both nominal and numeric attributes. For missing values, the method of “fractional” instances is used. For more information, see [22].
(3) Decision Stump. This algorithm is usually used in conjunction with a boosting algorithm. It does regression (based on mean-squared error) or classification (based on entropy). Missing is treated as a separate value.
(4) FT. Functional Trees are classification trees that could have logistic regression functions at the inner nodes and/or leaves. The algorithm can deal with binary and multiclass target variables, numeric and nominal attributes, and missing values. For more information, see [23, 24].
(5) J-48. We use this algorithm to generate a pruned or unpruned C4.5 decision tree. For more information, see [25].
(6) J-48graft. This algorithm generates a grafted (pruned or unpruned) C4.5 decision tree. For more information, see [26].
(7) LADTree. We use this algorithm to generate a multiclass alternating decision tree using the LogitBoost strategy. For more information, see [27].
(8) LMT. Logistic Model Trees are classification trees with logistic regression functions in the leaves. The algorithm can deal with binary and multiclass target variables, numeric and nominal attributes, and missing values. For more information, see [28, 29].
(9) NBTree. This algorithm generates a decision tree with Naive Bayes classifiers in the leaves. For more information, see [30].
(10) Random Forest. We use this algorithm to generate a forest of random trees. For more information, see [31].
(11) Random Tree. This algorithm generates a tree that considers randomly chosen attributes at each node, performs no pruning, and also has the option to allow estimation of class probabilities based on a hold-out set (backfitting).
(12) REPTree (Fast Decision Tree Learner). It builds a decision/regression tree using information gain/variance and prunes it using reduced-error pruning (with backfitting) and only sorts values for numeric attributes once. Missing values are dealt with by splitting the corresponding instances into pieces (i.e., as in C4.5).
(13) SVM. This learner uses the Java implementation of the support vector machine mySVM by Stefan Rueping. This learning method can be used for both regression and classification and provides a fast algorithm and good results for many learning tasks.
(14) KNN (). The -Nearest Neighbor algorithm is based on learning by analogy, that is, by comparing a given test example with training examples (-Nearest Neighbors) that are similar to it. “Closeness” is defined in terms of a distance metric, such as the Euclidean distance.
(15) Naïve Bayes. A Naïve Bayes classifier is a simple probabilistic classifier based on applying Bayes’ theorem (from Bayesian statistics) with strong (naive) independence assumptions. In simple terms, a Naive Bayes classifier assumes that the presence (or absence) of a class’ particular feature (i.e., attribute) is unrelated to the presence (or absence) of any other feature.
(16) Redpin. This algorithm uses a weighted combination of the vector distance and the AP similarity and chooses to decide the best match. The default configuration is , , and and the value of function is within the range −1 to 1, depending on the contribution of the signal strength of two common Aps. The Redpin source code can be found here [32].
(17) WASP. This algorithm uses Point-Wise Mutual Information (PMI) as the weight for one AP to one location and chooses to decide the best match. It also applied noise filter to remove irrelevant Aps.
6.3. Results
This section captures the results of both experiments carried out. Section 6.3.1 presents the results of the simpler scenario based on the measurements on a single aisle section (divided into 3 areas) and Section 6.3.2 captures the results of the second experiment based on measurements from 3 different wall and door separated areas.
6.3.1. Results from the Linear Configuration
In this section we present a comparison of the algorithms described in Section 6.2 for the case of the linear configuration in the aisle of the building. The metric we used to evaluate them is called “relative accuracy” and is commonly used in fingerprinting-based systems. In our case we aim to measure the number of test fingerprints which are correctly classified according to the area to which they belong.
These algorithms receive as input parameters the area in which the fingerprint was taken and the RSSIs values measured for each WAP. Initially we started with a set of 40 WAPs which involved training the system with 40 (RSSIs for each WAP) + 1 (area) parameters. Our first objective was using our algorithm to delete unstable WAPs (previously described in Section 4), discard those access points that do not provide any relevant information to train the algorithms, and verify that the results improved compared to when the process was performed without filtering.
Our second challenge was to study the behaviour of the algorithms when we trained the system with only border locations, that is, the ones that delimit an area, and to analyse whether the results were better than when all locations were used (or at least comparable, since training the system only with border locations simplifies the training face and increases scalability). In Figure 3, each geometric shape represents an area of the hallway in which the tests were conducted. “Border” locations are the ones that look darker.
Finally, we wanted to analyze how the variations in the measured RSSIs values overnight affect the predictions made by the system and if there was any way to improve them when applying our noise and stability filters (in our case, the building is dedicated to research activities involving wireless networks and some of the WAPs were moved of switched off from one day to the other).
In the following two subsections we show a comparison of the results we obtained when training the algorithms with all locations (12) compared to those obtained when we only use the “border” (6) and validate the models generated considering all locations (interior and borders). These results are presented in conjunction with the ones coming from our algorithm to delete unstable WAPs. This way we get a better overview of the impact of our three major contributions to improve the problem of indoor location based on smartphones and fingerprinting and for the metric “relative accuracy.”
(a) Training the System with All Locations. In Tables 2–4 we show the results we obtained in the first row for the different algorithms when we train the system with all locations (12) and we have not executed our algorithm to delete unstable WAPs. In this case we choose all the WAPs from the initial set (40). Then we show the values obtained when we also train the system with all locations (12) but this time we run our algorithm with the configuration previously detailed in Table 1 to take 5, 8, 10, 16, 20, and 32 access points, respectively.
Comparing the results for the first and fourth rows we can see that out of the 15 algorithms evaluated, the results are maintained or improved significantly for 9 of them (ADTree[+0.84], BFTree, FT, LADTree[+5.84], Random Forest, Random Tree[+3.34], REPTree[+0.84], KNN[+1.66], and NBC[+11.67]) and slightly worse for the remaining 6 (Decision Stump[−2.5], J-48[−2.5], J-48graft[−0.83], LMT[−1.66], NBTree[−0.83], and SVM[−3.34]). Although this decline reaches a peak of 3.34 percentage points for the algorithm of support vector machines (SVM), this number seems negligible since we generally get better results. At the same time we are reducing the computational cost substantially, going from 40 to 10 WAPs.
When we compare the results published for the WASP and Redpin algorithms with noise filtering and 5 access points [11] with the ones we get when we run our algorithm to delete unstable WAPs (and also take 5 WAPs) we can see that we get significant improvements for the following algorithms: ADTree, BFTree, Decision Stump, J-48, J-48graft, LADTree, NBTree, Random Forest, REPTree, and KNN. If we select the algorithm which provides the best resolution with 5 WAPs (LADTree [88.33]) we see that the difference from Redpin and WASP is +6.33 percentage points. For 8 access points we managed to improve the results very significantly with NBTree (91.67) and Random Forest (92.50) compared to results published for WASP with 8 APs [33]. This is an improvement of +2.50 percentage points when using the Random Forest algorithm.
To generate the comparative chart on Figure 4 we have considered the case where our algorithm to delete unstable WAPs gives us 10 access points since for a larger number we did not observe a significant improvement. We have used the green colour to show the results when we have not run our algorithm and thus the number of input parameters that the algorithms receives is equal to the RSSIs corresponding to the initial number of access points (40) plus the area. In blue we show the results we get when we have executed our algorithm with a given configuration to take 10 access points. In this case the number of parameters that will be passed to the algorithms will be equal to the RSSIs corresponding to the 10 WAPs selected plus the area (30 parameters less than the previous case).
Figure 5 shows a comparison of the “relative accuracy” for algorithms evaluated when the system is trained with all locations and by the number of WAPs selected. For 8 access points the algorithm that provides the best resolution is Random Forest while for 10 the FT and Random Forest algorithms are at the same level.
(b) Training the System with “Border” Locations. The results of this second test are presented in the same format as those already discussed in the previous section. The only difference is that now the system has been trained with only half of the locations (6), those we have called “border,” and has been validated with all (internal and border) (Tables 5, 6, and 7).
If we compare the values and reliability intervals obtained in this second test for the metric “relative accuracy” we can see a significant improvement as compared to when all locations were used. The increases/decreases in percentage points in those cases where we did not execute our algorithm to delete unstable WAPs are as follows (we take all the WAPs from the initial set, as indicated in the first row of combined Tables 2–4 and combined Tables 5–7): ADTree[+2.34], BFTree[+0.17], Decision Stump[−0.50], FT[+2.33], J-48[−4.33], J-48graft[−1.16], LADTree[+2.17], LMT[+0.67], NBTree[+0.17], Random Forest[+1.50], Random Tree[+0.67], REPTree[−0.83], SVM[−1.17], KNN[+2.83], and NBC[+2.17].
Although for five (Decision Stump, J-48, J-48graft, REPTree, and SVM) of the fifteen algorithms evaluated the percentages decrease, when we run our algorithm to rule out unstable WAPs these results improve substantially (Decision Stump[+4.17], J-48[+2.50], J-48graft[+3.33], and SVM[+2.34]). Just to REPTree [−0.84] we do not get an improvement. For other algorithms markers would be as follows: ADTree[+2.33], BFTree[+1.83], FT[+1.67], LADTree[+0.16], LMT[+1.16], NBTree[+1.00], Random Forest[+1.00], Random Tree[+3.16], KNN[−3.66], and NBC[+0.33].
Comparing the results of the second test for the first and fourth rows we can see that out of the 15 algorithms tested, the results are maintained or improved significantly for 11 of them (ADTree[+0.83], BFTree[+1.66], Decision Stump[+2.17], J-48[+4.33], J-48graft[+3.66], LADTree[+3.83], NBTree, Random Tree[+5.83], REPTree[+0.83], SVM[+0.17], and NBC[+9.83]) and slightly worse for the remaining 4 (FT[−0.66], LMT[−1.17], Random Forest[−0.50], and KNN[−4.83]). Although this decline reaches a peak of 4.83 percentage points for the algorithm of K-Nearest Neighbors, this number seems negligible since generally we get better results, especially for NBC and Random Tree algorithms. At the same time, again, we are reducing the computational cost substantially, going from 40 to 10 WAPs.
When we compare the results published for the WASP and Redpin algorithms with noise filtering and 5 access points [11] with the ones we get when we run our algorithm to delete unstable WAPs (and also take 5 WAPs) we can see that we get significant improvements for the following algorithms: ADTree, BFTree, Decision Stump, J-48, J-48graft, NBTree, Random Forest, Random Tree, and REPTree. If we select the algorithm which offers the best resolution with 5 WAPs (ADTree [91.33]) we observe that the difference from Redpin and WASP is +9.33 percentage points. For 8 access points we managed to improve the results very significantly with Random Forest (94.50) and BFTree (94.17) compared to results published for WASP with 8 APs [12]. This is an improvement of +4.50 percentage points when using the Random Forest algorithm.
Figure 6 compares the results for the metric “relative accuracy” when we run our algorithm to delete unstable WAPs (shown in blue) versus when we do not (shown in green). For most algorithms (11/15) percentages are maintained or improved significantly.
Figure 7 presents a comparison of the “relative accuracy” for algorithms evaluated when the system is trained only with “border” locations and by the number of WAPs selected. For 8 access points the algorithm that provides the best resolution is Random Forest while for 10 the ADTree and FT algorithms ranked first and second, respectively.
(c) Average Test Fingerprint to Validate the System Trained with “Border” Locations. Out of the 15 evaluated algorithms to conduct for this test we chose Random Forest, as this one gave us the best results overall for 16(99.00), 10(98.50), and 5(90.00) WAPs, respectively, as it is reflected in the previous paragraph “Training the System with ‘Border’ Locations.”
Our goal here was to evaluate the system’s predictions for each of the three areas we have trained and labeled “1” (left side hallway), “2” (central hallway), and “3” (right side hallway). As detailed at the beginning of Section 6 for each area we carried out measurements on two consecutive days in 4 different locations and for each we took 10 fingerprints. Of these 4 locations we choose 2 (only for this test’s purposes, as we are only taking into account “border” locations). This makes a total of 20 fingerprints (10 fingerprints 2 locations) for each area. Thus we have a set of training data of 60 fingerprints (20 fingerprints/area 3 area) per day. To validate the system, we selected as test data set both fingerprints (the ones corresponding to the day when we trained the system and the ones corresponding to the previous/following day) for all locations (indoors and borders). Here we only show the results corresponding to when we train the system with the fingerprints that are taken on the second day and validated with the previous one. As we discussed at the beginning of Section 6.3, we intend to study how changes in the stability of the WAPs from one day to another affect to the predictions of the areas. Also, we aim to study how to adjust our algorithm to delete unstable WAPs with the configuration that we saw in Table 1 such predictions evolve as we take 16, 10, and 5 WAPs.
In Tables 8–13, we have the following: in the first row we point out the number of fingerprints, followed by the corresponding area identifier, the area forecast that the system gives us, and the reliability intervals for area “1,” “2,” and “3.”
When we train the system with the fingerprints collected on the second day and validate with the ones corresponding to the first, adjusting our algorithm to take 16 WAPs, we obtain a relative accuracy of 80% (48/60 successes and 12/60 failures) (Tables 14, 15, 16, and 17).
For 10 access points the results are as follows: area 1 (20/20 successes and 0/20 failures) and area 2 (18/20 successes and 2/20 failures) (Tables 18, 19, 20, 21, 22, and 23).
If we analyze the results for 5 access points (area 1) we can see that the system succeeds 19 times out of 20 and fails once out of 20 (fingerprint 11). For area 2 the system is wrong in its prediction for the fingerprint 39; it succeeds 19 times out of 20 and it fails once out of 20. As for area 3 it succeeds 14 times out of 20 and it fails 6 out of 20 times (fingerprints 46, 49, 51, 52, 56, and 57). In total the system succeeds 52 times out of 60 and fails 8 times out of 60 which means there is a relative accuracy of 86.67%. If we compare this result with those obtained for 16 (80%) and 10 (63.34%) WAPs we get a difference of +6.67 and +23.33 percentage points, respectively. This finding is of special interest since it proves that when changes occur in the WAPs’ measured RSSIs considered day to day we need to be stricter when selecting access points in order to obtain better predictions.
As we saw in the previous tables the system made, among others, the following wrong predictions: for area 1 (fingerprint 11), for area 2 (fingerprint 39), and for area 3 (fingerprint 52). These are the three scenarios in which we will introduce average fingerprinting to help the system make the right predictions for each of the areas. To do this we will calculate the average fingerprint using (a) the previous fingerprint, (b) the previous and subsequent ones, and (c) two previous and two subsequent fingerprints to the current one.
As we see in Table 24, when we use the average fingerprint resulting from these three methods (previous, previous and subsequent, two previous, and two subsequent) the system makes the right prediction with a reliability of an 80% percent, 40% percent, and 50% percent, respectively (Table 25).
For area “2” the system also makes the right prediction when it receives the average fingerprint in the three modalities described before. For 16 WAPs the system says that it belongs to area 2 with 80% confidence whereas for 5 WAPs this percentage increases up to 100% (Table 26).
For area 3, fingerprint 52, the system also makes the right prediction when it receives the average fingerprint for 5 and 16 WAPs scenarios with 50% and 80% percent of reliability, respectively. However, when the system takes 10 WAPs it still fails. Why? This is because whether we calculate the average fingerprint using the previous one (51), one previous fingerprint and subsequent one (51 and 53) or two previous and two subsequent fingerprints (50, 51, 53, and 54) the predictions made by the system for these fingerprints are also wrong so there is no way to get an upgrade.
6.3.2. Results from the Second Experiment
A more complex scenario including 3 wall and door separated areas has also been implemented as presented in Figure 1. In order to validate the accuracy of the algorithms for estimating the area of an unlocated validation point based on the proposed training only with area border points, different validation points from those used for training were selected (randomly selected at each area) as shown in Figure 8.
A similar approach to evaluate the 15 machine learning algorithms as in the previous section was used. The same sampling schema as in Section 6.3.1 was used for the training points. The average results of the estimated areas for each validation point are shown in Table 27.
All validation points are properly located except point 15 which is a point very close to the border between areas 2 and 3. For a moving target to be located, the assessed trajectory information could be used to decide in which area a user is in order to improve the accuracy of the algorithms when approaching the border of two areas.
7. Conclusion
In this paper we have presented several ways to optimize fingerprinting-based techniques for locating users indoors when only room level accuracy is required.
One of the parameters studied is the way to select the most convenient sampling points to measure training fingerprints in order to increase the accuracy while minimizing the memory and CPU execution requirements. Previous studies have focused on either an exhaustive grid of sample points or a random selection of them. This paper concludes that, for room level accuracy, border points between rooms or areas contain the required information for the prediction algorithm in order to provide a similar or even better accuracy that the one provided by exhaustive or random sampling techniques.
The paper also provides a comparative analysis and evaluation of the performance of different decision algorithms identifying under which circumstances each of them work better.
Another contribution presented in this paper is the proposal and evaluation of a new way to filter unstable access points in order to increase accuracy and minimize execution requirements. Taking into account the quality and stability over time of the received signals from the different WAPs, the algorithm selects those maximizing both the signal strength and the minimal fluctuations over time. Several thresholds are defined. The experimental results show that the algorithms find optimal behavior when filters are configured to select between 10 and 16 WAPs.
Finally, the paper has presented that taking several consecutive fingerprints when locating a mobile device can improve the accuracy of the results. A compromise exists between the accuracy obtained and the time required to locate the user. The results show that 3 consecutive measures are a reasonable trade-off.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The research leading to these results has received funding from the “HERMES-SMART DRIVER” Project TIN2013-46801-C4-2-R within the Spanish “Plan Nacional de I+D+I” funded by the Spanish Ministerio de Economía y Competitividad, from the Grant PRX15/00036 for the “Estancias de Movilidad de Profesores e Investigadores Seniores en Centros Extranjeros de Enseñanza Superior e Investigación,” from the Ministerio de Educación Cultura y Deporte, and from the Spanish Ministerio de Economía y Competitividad funded projects (cofinanced by the Fondo Europeo de Desarrollo Regional (FEDER)), IRENE (PT-2012-1036-370000), COMINN (IPT-2012-0883-430000), and REMEDISS (IPT-2012-0882-430000) within the INNPACTO program.