International Journal of Navigation and Observation

International Journal of Navigation and Observation / 2017 / Article

Research Article | Open Access

Volume 2017 |Article ID 6750346 | https://doi.org/10.1155/2017/6750346

Jalal Ibrahim Al-Azizi, Helmi Zulhaidi Mohd Shafri, "Performance Evaluation of Pedestrian Locations Based on Contemporary Smartphones", International Journal of Navigation and Observation, vol. 2017, Article ID 6750346, 10 pages, 2017. https://doi.org/10.1155/2017/6750346

Performance Evaluation of Pedestrian Locations Based on Contemporary Smartphones

Academic Editor: Letizia Lo Presti
Received08 Aug 2016
Revised08 Nov 2016
Accepted29 Nov 2016
Published13 Feb 2017

Abstract

Nowadays, a Global Navigation Satellite System (GNSS) unit is embedded in nearly every smartphone. This unit allows a smartphone to detect the user’s location and motion, and it makes functions, such as navigation, tracking, and compass applications, available to the user. Therefore, the GNSS unit has become one of the most important features in modern smartphones. However, because most smartphones incorporate relatively low-cost GNSS chips, their localization accuracy varies depending on the number of accessible GNSS satellites, and it is highly dependent on environmental factors that cause interference such as forests and buildings. This research evaluated the performance of the GNSS units inside two different models of smartphones in determining pedestrian locations in different environments. The results indicate that the overall performances of the two devices were related directly to the environment, type of smartphone/GNSS chipset, and the application used to collect the information.

1. Introduction

These days many different brands and models of smartphone are available on the market. Most of these smartphones incorporate different types of embedded sensors such as GNSS receivers, Radio-Frequency Identification, a camera, and an inertial platform [1]. These devices could be used to provide seamless integration of three positioning technologies: the Assisted Global Positioning System (A-GPS), WiFi positioning, and cellular network positioning [2]. Thus, positional information could be obtained at almost every moment and in almost every location, which has led to an increase in the number of available navigation and location based applications.

Numerous studies have been conducted testing smartphones as navigational and/or tracking systems. However, less attention has been paid to pedestrian navigation. Klimaszewski-Patterson [3] compared the GPS capabilities between a smartphone (HTC G1 Dream) and dedicated GPS (Trimble Juno SB) device. Gikas et al. [4] concluded that positioning sensors embodied in modern smartphones, such as the HTC One S and iPhone 5S, can provide accuracy metrics of sufficiently high accuracy for a variety of intelligent transportation system applications. Kos and Brčić [5] considered weather and ionospheric conditions when studying the horizontal positioning performance of two identical smartphone devices and found distinct differences in horizontal positioning accuracy.

Bauer [6] proved that the accuracy of smartphone positioning depends on the application. In her study, nine popular applications were run on a single smartphone and used to take measurements on a running track. Bauer’s results showed different outputs for each application, indicating that GPS accuracy is affected by various influencing factors. Menard et al. [7] used three different smartphones (each with an embedded GPS sensor) and analyzed their positional capability when vehicle tracking. Each of the three smartphones was accurate to within 10 m about 95% of the time. Zhang et al. [8] evaluated high sensitivity GPS and mobile phones (iPhone 3GS, iPhone 4, Nokia N95, and HP iPAQ) in three scenarios: urban canyon, shopping center, and an area under dense tree foliage in kinematic mode. One of their conclusions was that the performance of the receivers from different manufacturers varies, whereby a given receiver may perform very well in one aspect, but not as well based on other criteria.

In this study, for the purposes of research and to evaluate the performance of contemporary smartphones in estimating pedestrian location, two popular smartphones (Samsung Galaxy S3 based on the Android platform [9] and iPhone 5S based on the iOS platform [10]) were selected to obtain kinematic and static observations using four different smartphone applications in two different environments: an open outdoor area to represent ideal conditions and inside a shopping center to represent compromised conditions. Figure 1 shows the selected devices.

2. Smartphone GNSS Analysis

The technical specifications of the Samsung Galaxy S3 and Apple iPhone 5S have been studied to assess the positioning accuracy and reliability obtained by these devices [1]. Based on data from the GSMArena website, Table 1 shows the most important technical specifications for these two devices.


NameGalaxy S3iPhone 5S

ReleasedMay 2012Sep 2013
 TechnologyGSM/HSPAGSM/CDMA/HSPA/EVDO/LTE
OS/PlatformAndroidiOS
CPUQuad-core 1.4 GHz Cortex-A9Dual-core 1.3 GHz Cyclone (ARM v8-based)
Memory1 GB RAM1 GB RAM DDR3
A-GPSYesYes
GLONASS (chipset)YesYes
(Broadcom BCM47511)(Qualcomm WTR1605L)
 SensorsAccelerometer, gyro, proximity, compass, barometerAccelerometer, gyro, proximity, compass

http://www.gadgethelpline.com/samsung-galaxy-iii/.
https://discussions.apple.com/thread/6246837?tstart=0.

To evaluate the availability and accuracy of locations obtained by smartphones, four tests were conducted in the two selected environments. Table 2 shows the four types of the conducted tests.


Test numberObservation
mode
Observation
environment

Test AStaticOpen outdoor area
Test BKinematicOpen outdoor area
Test CStaticIndoor (inside the shopping center)
Test DKinematicIndoor (inside the shopping center)

First, suitable smartphone applications [11, 12] that have the capability of storing navigational data in the National Marine Electronics Association (NMEA) format [14] were selected. Many applications were downloaded and tested, but only two applications were chosen for each device. Table 3 provides detailed information about the selected applications.


App 1App 2App 3App 4

Application nameNMEA GPSTracks Logger ProUltra GPS LoggerNMEA Data Logger
SmartphoneiPhone 5SiPhone 5SSamsung S3Samsung S3
Version2.0.62.6.03.118d1.0
Last updateAug 17, 2015Jan 19, 2016May 23, 2016Apr 17, 2013
Application rate4+4+4.73.7
Observation sampling rate1 s1 s1 s1 s

NMEA data logged by Samsung Galaxy S3.

Analysis of the NMEA output (Figure 2) shows that the GNSS sensor output depends on the smartphone platform such as iOS or Android. Table 4 shows the difference in outputs [13].


AndroidiOS

Positioning (lat., long.)
Speed
Altitude
GPS time
Number of tracked

iOS devices do not provide HDOP, number of tracked satellites, and CRN values. Hence, some apps provide constant values.

During collection of the observations, each smartphone recorded its own GNSS positioning data at a certain sampling rate (Table 3) using two different dedicated applications that stored the NMEA sentences [14] in a text format. The observations (NMEA data) of each test were analyzed using applications written in MATLAB®.

2.1. Smartphone GNSS Availability

Test A (Outdoor Environment; Static Mode). A location in a parking area (Figure 3) was selected to conduct this test. The two devices were placed on the ground and left for almost 10 minutes to collect data synchronously.

The analysis of the NMEA outputs revealed that Samsung’s applications were giving different numbers of tracked satellites. However, the availability of a 3D positional solution (tracked with a minimum of four satellites [8]) using the Samsung S3 smartphone in the outdoor environment was almost 100% during the static test. Figure 4 shows the number of satellites that were tracked during the observations.

Test B (Outdoor Environment; Kinematic Mode). This test involved holding the two devices by hand and then walking twice around a loop that started and ended at the same point. Each loop was around 930 m in length and took around 13 minutes to complete. The test track and observational results are shown in Figure 5.

The number of satellites tracked during Test B is shown in Figure 6 in order to assess the availability of 3D positioning when using smartphones in a kinematic mode in an outdoor environment. It is noted that the two Samsung applications were providing different numbers of tracked satellites. However, the availability of 3D positioning [8] using the Samsung S3 in the outdoor environment was 100% during the kinematic test.

Test C (Indoor Environment; Static Mode). The test started outside the shopping center to allow the devices to acquire and receive the GNSS signals. The devices were then taken inside the shopping center to obtain the static measurements. Both devices were placed on a table under the center of the concrete dome and left for around 10 minutes. Figure 7 shows the test location and the observations for each device. Figure 8 displays the number of satellites tracked during this test. Samsung’s applications tracked differing numbers of satellites inside the shopping center.

Test D (Indoor Environment; Kinematic Mode). The same shopping center used in Test C was selected to perform Test D. The acquisition of observations began outside the shopping center under an open sky environment, which allowed the devices to acquire and receive the GNSS signals. After 100 sec, the devices were taken indoors and observations were collected within the shopping center.

The test track inside the mall comprised a rectangular shape that was traced anticlockwise starting from the entrance in three loops. Each loop was around 180 m in length and took 2-3 minutes to complete. Figure 9 shows the test track and the observations that were obtained by the applications on each device.

Samsung’s applications could track some satellites some of the time (Figure 10), which provided 3D positioning for 30% of the observation time (142 observation records out of a total of 440).

2.2. Smartphone GNSS Accuracy

According to NovAtel’s GPS Position Accuracy Measures report [15], there are various statistical methods that can be used to describe the specifications for GPS receivers. Here, the NMEA outputs of the selected smartphones are analyzed using the most commonly used accuracy measures, that is, 2D and Dilution of Precision (DOP).

2.2.1. Accuracy Measures—2D

Precision and accuracy are often used to describe the position acquired by a GPS receiver [15]. Accuracy is the degree of closeness of an estimate to its true, but unknown value, whereas precision is the degree of closeness of the observations to their arithmetic means [15]. To establish the positional accuracy of the horizontal component of the selected smartphones in static mode (Tests A and C), the following procedures were followed:(1)The same observation locations for Tests A and C were measured and centimeter-level positional accuracy was obtained as follows:(i)Test A: obtained using a 7 cm spatial resolution aerial photo (the pink triangle in Figure 3).(ii)Test C: obtained using the shopping center architecture as-built drawing within 3 cm accuracy (the green triangle in Figure 7).(2)All measurements logged by the two devices were converted from geographical coordinates (latitude and longitude) to local projected coordinates (north and east).(3)The differences in the north and east components between the coordinates of the observational point and the observations were calculated.(4)An application was developed in MATLAB to display the outputs shown in Figures 11 and 12.

In general, the performances of the smartphones in the outdoor area (Test A) were good; the maximum differences in the east and north components were 7.48 and −1.78 m and 6.67 and 3.30 m for the Samsung Galaxy S3 and iPhone 5S, respectively. Table 5 shows the detailed statistic values.


App #1App #2App #3App #4

Mean (m)E7.487.482.192.19
N−1.69−1.781.111.11

Std dev (m)E0.020.022.652.65
N0.040.041.231.23

Max diff (m)E7.487.486.676.67
N−1.69−1.783.303.30

RMSEE7.487.483.443.44
N1.691.781.661.66

The results obtained inside the shopping center during Test C (Figure 12) indicate that the sensors of the Samsung device were affected by missing satellites, and the logged observations were intermittent. However, the statistics that were obtained show that the maximum differences in the east and north components were −56.98 m and −26.95 m, respectively. The iPhone device performed in a different way, and it showed almost fixed positioning error values. However, the maximum differences in the east and north components were 4.40 m and 14.36 m, respectively. Table 6 summarize the most significant statistical parameters.


App #1App #2App App

Mean (m)E3.213.42−2.43−2.35
N14.1214.15−1.25−1.27

Std dev (m)E2.371.994.844.43
N0.720.092.672.65

Max diff (m)E4.404.40−56.97−56.97
N14.3514.35−26.94−26.94

RMSEE4.003.965.415.02
N14.1414.152.952.94

the test, Samsung’s applications provided zero records when it was not possible to obtain observations, which were removed from the statistics.

To estimate the positional accuracy of the horizontal component of the selected smartphones in kinematic mode (Tests B and D), the reference track was determined as the centerline of the observation path according to the following steps, and then the estimated horizontal positioning accuracy was calculated.(1)The observation track centerline for Tests B and D were obtained as follows:(i)Test B: digitizing a polyline in ArcGIS (ESRI®) software using a 7 cm spatial resolution aerial photo (the yellow line in Figure 5).(ii)Test D: obtained using the shopping center architecture as-built drawing within 3 cm accuracy (the pink line in Figure 9).(2)All measurements logged by the two devices were converted from geographical coordinates (latitude and longitude) to local coordinates (north and east).(3)For both tests, the horizontal accuracy was defined as the shortest distance in meters between the observed position and the track centerline, using the mathematical model in Figure 13.(4)An application was developed in MATLAB to display the outputs shown in Figures 14 and 15.

The Smartphones Performances in the Outdoor Environment. Kinematic mode (Test B) was good (Figure 14); the maximum estimated horizontal errors were 7.48 m and 6.48 m for the Samsung Galaxy S3 and iPhone 5S, respectively. Table 7 shows the detailed statistic values.


TestApp #1App #2App #App #

Mean (m)B1.851.852.272.27
D7.927.948.018.01

Std dev (m)B1.351.341.851.85
D5.605.575.195.19

Max diff (m)B6.486.4610.6710.67
D23.1623.1917.7417.74

RMSEB2.292.292.932.93
D9.699.689.549.54

Test D, Samsung’s applications provided zero records when it was not possible to obtain observations, which were removed from the statistics.

The Results Obtained inside the Shopping Center. Kinematic mode during Test D (Figure 15) indicates that the sensors of both devices were affected by missing satellites. However, the maximum estimated horizontal errors were 17.74 m and 23.19 m for the Samsung Galaxy S3 and iPhone 5S, respectively. The iPhone device performed in a different way, and it showed almost fixed positioning error values. Table 7 summarizes the most significant statistical parameters.

2.2.2. Dilution of Precision (DOP)

DOP is a term used to describe the strength of the satellite configuration on the accuracy of the data collected by the GPS receivers [3, 14]. It represents the volume formed by the intersection points of the user-satellite vectors with the unit sphere centered on the user. Larger volumes give smaller values of DOP, which represent better positional accuracy. However, a lower DOP value does not automatically mean a low positional error. Positional accuracy is affected by other factors such as multipath and ionospheric effects.

According to NovAtel’s report [15], DOP values can be classified as follows: <1 ideal; 1-2 excellent; 2–5 good; 5–10 position could be provided, but the fix quality could be improved; 10–20 positional measurements should be discarded or used only to indicate a very rough estimate of the current location; and >20 poor positional quality.

DOP can be expressed as a number of separate measurements: horizontal DOP (HDOP), vertical DOP, position (3D) DOP, and time DOP (TDOP). The NMEA sentences include the HDOP records [14], and hence the iOS device does not provide HDOP values [13], whilst the HDOP values for the Android device were analyzed for all four tests. Figure 16 shows that the Android smartphone (Samsung Galaxy S3) displays differing HDOP behaviors over time, which are dependent on the environment. Table 8 summarizes the most significant statistical measurements.


Samsung S3
TestApp #3App #4TestApp #3App #4

Mean (m)A0.900.90Max diff (m)A2.42.40
B0.890.89B4.84.80
C39.7539.76C300300
D3.733.73D9.69.6
Std dev (m)A0.300.28RMSE
A0.950.95
B0.290.29B0.940.94
C98.1298.13C99.9899.98
D1.421.42D3.993.99

3. Conclusions

This research studied the availability and accuracy of GNSS receivers embedded inside two of the most popular smartphones by analyzing the NMEA output logged by different mobile applications in two different environments. The results show that both devices have the capability of providing positioning to within accepted accuracy [1, 3] in both outdoor and indoor environments. Detailed analysis indicates that the overall performances of the two devices are related directly to the environment, type of smartphone/GNSS chipset, and the application used to collect the information. Thus, smartphone devices with embedded GNSS receivers could help users to define their locations, which can be used in different applications.

In the static tests, the iPhone device provided constant positioning values. After investigation we found that the iOS application developers have the option to either use the “precise standard location service” resulting in high energy consumption or use the “significant update feature” with less precision [16, 17]. Hence, we tested more than 13 applications and all of these, including those that were used in this research to log the NMEA data, were developed based on the “significant update feature” method. This method, proposed by Apple to conserve the device battery, means that the applications can be suspended and then awoken automatically should a significant change in location occur [17].

Further tests and research will be conducted to study the positioning performance obtained by(1)other smart devices such as smart glasses and smart watches;(2)an iOS application which will be developed based on “standard location service” method.

Competing Interests

The authors have no conflict of interests to declare in respect of this paper.

References

  1. P. Dabove and M. Petovello, “What are the actual performances of GNSS positioning using smartphone technology?” Inside GNSS, pp. 34–37, 2014. View at: Google Scholar
  2. P. A. Zandbergen, “Accuracy of iPhone locations: a comparison of assisted GPS, WiFi and cellular positioning,” Transactions in GIS, vol. 13, no. 1, pp. 5–25, 2009. View at: Publisher Site | Google Scholar
  3. A. Klimaszewski-Patterson, “Smartphones in the field: preliminary study comparing GPS capabilities between a smartphone and dedicated GPS device,” in Proceedings of the Applied Geography Conferences, vol. 33, pp. 270–279, Fort Worth, Tex, USA, 2010. View at: Google Scholar
  4. V. Gikas, C. Antoniou, T. Mpimis, H. Perakis, and C. Danezis, “Performance evaluation of vehicle location estimation in heavily obscured environments based on contemporary smartphones,” in Proceedings of the International Conference on Indoor Positioning and Indoor Navigation (IPIN '15), p. 2, Banff, Canada, October 2015. View at: Google Scholar
  5. S. Kos and D. Brčić, “Smartphone application GPS performance during various space weather conditions: a preliminary study,” in Proceedings of the 21st International Symposium on Electronics in Transport (ISEP '13), p. 4, Ljubljana, Slovenia, March 2013. View at: Google Scholar
  6. C. Bauer, “On the (in-)accuracy of GPS measures of smartphones: a study of running tracking applications,” in Proceedings of the 11th International Conference on Advances in Mobile Computing and Multimedia (MoMM '13), pp. 335–340, Vienna, Austria, December 2013. View at: Publisher Site | Google Scholar
  7. T. Menard, J. Miller, M. Nowak, and D. Norris, “Comparing the GPS capabilities of the Samsung Galaxy S, Motorola Droid X, and the Apple iPhone for vehicle tracking using FreeSim_Mobile,” in Proceedings of the 14th International IEEE Conference on Intelligent Transportation Systems (ITSC '11), pp. 985–990, IEEE, Washington, DC, USA, October 2011. View at: Publisher Site | Google Scholar
  8. J.-H. Zhang, B.-H. Li, A. G. Dempster, and C. Rizos, “Evaluation of high sensitivity GPS receivers,” in Proceedings of the International Symposium on GPS/GNSS, p. 6, Taipei, Taiwan, October 2010. View at: Google Scholar
  9. Android, Welcome to the Android Open Source Project!, 2016, http://source.android.com.
  10. Apple Inc, “Welcome to Apple Support, iOS,” 2016, https://support.apple.com. View at: Google Scholar
  11. Apple Inc, Apple Store, 2016, https://itunes.apple.com/app/apple-store/id375380948?mt=8.
  12. Google Inc, “Google Play,” 2016, https://play.google.com/store/apps?hl=en. View at: Google Scholar
  13. P. Handel, I. Skog, M. Ohlsson, and J. Ohlsson, “Smartphone instrumentation for insurance telematics,” in Proceedings of the IEEE International Instrumentation and Measurement Technology Conference (I2MTC '15), pp. 1613–1616, IEEE, Pisa, Italy, May 2015. View at: Publisher Site | Google Scholar
  14. National Marine Electronics Association (NMEA), http://www.nmea.org/.
  15. NovAtel, “GPS position accuracy measures,” APN-029 Rev 1, 6 pages, 2003, http://www.novatel.com/assets/Documents/Bulletins/apn029.pdf. View at: Google Scholar
  16. D. Huber, “Background positioning for mobile devices—Android vs. iPhone,” in Proceedings of the Joint Conference of IEEE Computer & Communication Societies, p. 7, 2011. View at: Google Scholar
  17. Apple Inc, “Getting the User's Location, iOS Developer Library,” 2016, https://developer.apple.com/library/ios/documentation/UserExperience/Conceptual/LocationAwarenessPG/CoreLocation/CoreLocation.html#//apple_ref/doc/uid/TP40009497. View at: Google Scholar

Copyright © 2017 Jalal Ibrahim Al-Azizi and Helmi Zulhaidi Mohd Shafri. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


More related articles

 PDF Download Citation Citation
 Download other formatsMore
 Order printed copiesOrder
Views932
Downloads271
Citations

Related articles

We are committed to sharing findings related to COVID-19 as quickly as possible. We will be providing unlimited waivers of publication charges for accepted research articles as well as case reports and case series related to COVID-19. Review articles are excluded from this waiver policy. Sign up here as a reviewer to help fast-track new submissions.