Abstract
The motivations, the design, and some applications of the new Pedestrian Dead Reckoning (PDR) navigation device, ULISS (Ubiquitous Localization with Inertial Sensors and Satellites), are presented in this paper. It is an original device conceived to follow the European recommendation of privacy by design to protect location data which opens new research toward self-contained pedestrian navigation approaches. Its application is presented with an enhanced PDR algorithm to estimate pedestrian’s footpaths in an autonomous manner irrespective of the handheld device carrying mode: texting or swinging. An analysis of real-time coding issues toward a demonstrator is also conducted. Indoor experiments, conducted with 3 persons, give a 5.8% mean positioning error over the 3 km travelled distances.
1. Introduction
More and more Internet of Things applications are based on connected objects carried by human. These objects are not only smartphones but also smartwatches, smartglasses, and the future smart clothes. The processing of signals recorded by the embedded sensors enables monitoring physical activities, providing personal services, and calculating the travelled path. Thanks to this evolution, transport services are able to directly target the travellers along their multimodal journey. This evolution supports novel mobility services for connected travellers thanks to online geolocalized data about on-foot traffic or public transit. Irrespective of the targeted application, access to accurate, continuous, and reliable localization of connected objects in indoor and outdoor spaces is essential to develop new personal mobility services [1] and remains a challenging task [2].
1.1. Personal Navigation Research in the Legal Context of Location Data
A survey published by the Information Systems Audit and Control Association showed that users of Location-Based Services (LBS) are most worried by strangers knowing too much about their activities and their information being shared and used for marketing purposes. Oversees, there exists no specific regulation for researching or using geolocated data. The data is covered by laws only when it is considered private and personal. But whether an individual has a reasonable privacy interest in their own movement is still debated [3]. In Europe, localizing an isolated worker for security reasons became an obligation for answering the specifications of the Emergency Call number 112 [4]. In October 2012, the American Federal Communications Commission (FCC) tasked the Communications Security, Reliability, and Interoperability Council (CSRIC) with running test bed for improving indoor location accuracy for wireless 911 calls [5] because wireless carriers are oftentimes unable to provide a 911 call centre with precise location information in challenging environments. The 2013 final report stated [6] that “very few location technologies are currently available for 9-1-1 use indoors” but the technologies are fast evolving encouraging conducting new tests. European Regulation recommends “privacy by design” technologies.
The European legal context on location data is a guardian of human rights and complicates the development of personal mobility navigation device. Globally, a ubiquitous, anonymous, and accurate universal personal navigation solution is still missing to support multimodal and active transport. A transdisciplinary approach, including engineering, human, ethical, and legal challenges is needed to overcome existing limitations [7].
1.2. Enabling Real-Life Scenarios Testing with a New INS/PDR Device
It is well known that Dead Reckoning (DR) processing of inertial, magnetic, and other signals from self-contained navigation and positioning solutions follows the concept of “privacy by design” since the measurements do not depend on the access to local infrastructure. Furthermore, these signals are available everywhere, supporting the development of a ubiquitous localization technology. This motivated us to develop new hardware to support research on personal navigation which is based on pedestrian DR technologies and meets usage and size factors for conducting real-life experiments.
Despite numerous commercialized connected objects for general public or research purposes [8, 9], there is no commercialized open object in terms of small-size, lightweight, and easy to use features available to assist real-life scenarios testing. Here is a list of the main needed features:(i)The data rate must be above 100 Hz to be able to capture stride and step parameters with enough resolution.(ii)The system must be autonomous. No PC and no wire, which could hinder human movements, should be necessary to run the system.(iii)The device must be held in hand for ULISS and on foot for PERSY. Consequently, no big and heavy inertial system can be used. It has to be small and lightweight to fit in a hand or on a foot.(iv)The system must be easy to use for being able to conduct large survey with general public.(v)The IMU data (accelerations, angular rates, and magnetic fields) must be raw data, that is, unfiltered and without internal compensation.(vi)All data should be precisely time-tagged and synchronized with the GPS time.(vii)The measurement range has to be large enough for capturing human motion with handheld sensors. For example, a really large gyroscope measuring range is required: above 1000°/s.(viii)GNSS receiver and antenna must deliver raw satellites based measurements: pseudoranges, Doppler, phase, and so forth.
Looking for all these features at the same time, we concluded that no off-the-shelf equipment existed and decided to design one.
Two types of product have been designed. The first one targets research activity on personal geolocalization systems and methods with handheld object. It is a unit called ULISS, for Ubiquity Localization with Inertial and Satellite System, which is presented along with its applications in the rest of the paper. The second hardware has been conceived as a self-reliant indoor/outdoor reference system. Among other applications, it will be used to assess the accuracy of ULISS and other indoor navigation systems. It is called PERSY, for PEdestrian Reference System, and its accuracy is about ten times better than ULISS.
2. Design of the Wireless Time Synchronized Navigation Units
PERSY and ULISS systems, shown in Figure 1, embed inertial sensors (triaxis accelerometers and triaxis gyroscopes), triaxis magnetometer, barometer, GNSS receiver/antenna, a battery, and a memory card. Whereas PERSY is a reference system for indoor/outdoor pedestrian navigation, ULISS is conceived as an original handheld unit that facilitates research on personal mobility with naturalistic walking. Here is a presentation of their general technical specifications.
2.1. General Technical Specifications
2.1.1. Time Synchronization
Special attention was paid to the synchronization of all raw data samples to enable research on tight and ultratight coupling of the different sensors’ measurements. Accurate time synchronization is known to be a critical issue for inertial navigation systems that combine both inertial and GNSS signals [10]. Any timing error introduces a position error or a bias in the state vector estimate. The adopted strategy uses the pulse per second (PPS) output of the GNSS receiver embedded in our navigation units to time-tag each sample within one micro second.
Because ULISS raw data samples are all synchronized with GPS time, another interesting aspect of the PPS based synchronization strategy is that data coming from different ULISS units located on different body parts are all time-synchronized without using wires to connect the units in a network. This enables conducting distributed devices based research for pedestrian navigation. This option widens the use of ULISS to other research fields. An example is biomechanics with simultaneous observation of motion from different body parts. The presence of wires in a network of distributed devices will certainly modify pedestrian walking gait. Benefiting from wireless time-synchronized units is undoubtedly an interesting approach to seek for novel research outcomes.
2.1.2. Data Logging
ULISS and PERSY are data-logging systems. All data are recorded on a 4 GB micro SD card in two files. The first file is a “csv” file containing inertial and magnetometer data expressed in standard units, time-stamped with the GPS Time of Week (ToW). The second file contains all GNSS raw data (pseudoranges and phases).
2.1.3. Human Machine Interface
A human machine interface (HMI) has been developed to monitor effective functioning of the unit. It comprises three LEDs shown in Figure 1. The red LED, marked with the label (), indicates the battery charge. The tricolour LED () gives information about the running operation mode: “ready,” “initialization,” or “recording data.” This LED is also blinking when the battery is almost empty. The green LED () indicates when GPS signal has been acquired and GPS ToW is available. Two buttons complete the HMI: a power button () to turn on/off the system and the button () to start/stop the data acquisition. The USB connector () is used to charge the internal battery. The memory card is inserted in the SD card slot, which is marked with the label () in the same figure.
2.2. ULISS Technical Specifications
2.2.1. Sampling Frequency
ULISS sampling frequencies are 200 Hz and 5 Hz for the inertial and magnetic mobile unit and the GNSS receiver, respectively. ULISS is based on VectorNav VN-300 module [11] whose technical specifications are recalled in Table 1.
2.2.2. Calibration and Modelling of the Sensor Errors
The calibration platform, shown in Figure 2, was used to estimate deterministic and stochastic errors of ULISS embedded sensors. A six-position test was used to calibrate the deterministic errors of the inertial sensors [12] with ULISS rigidly attached to the levelled surface of the platform.
The parameters of the stochastic error models have been estimated with an Allan variance analysis on 14-hour static acquisition at 200 Hz. Allan variance results are shown in Figures 3, 4, and 5 for the triaxis accelerometer, the triaxis gyroscope, and the triaxis magnetometer, respectively. Random walk component and bias instability are observed on all figures. The values of the stochastic error models are given in Tables 2, 3, and 4.
3. Applications to Pedestrian Dead Reckoning with Handheld ULISS
ULISS is a useful device that can assist research and development of many innovation geolocalization applications dedicated to intelligent transport systems. A first example is the development of PDR algorithms in postprocessing mode. The complete PDR algorithm scheme is presented in Section 3.1. A second example is a real-time personal navigation device still under development. Software and hardware real-time implementation issues of this demonstrator are presented in Section 3.2.
3.1. Postprocessing PDR Algorithms with Handheld ULISS
The walking trajectories are computed using a PDR strategy, where the current position is derived from the previous position and the displacement vector estimate computed using the magnetic, inertial, and GNSS data. The complete process is illustrated in Figure 6.
3.1.1. Activity Classification
ULISS raw data are first processed to classify the pedestrian locomotion state and the handheld device carrying mode [13–15]. An outcome of the classification outcome is shown in Figure 7. A Short Time Fourier Transform (STFT) based analysis [16] is then applied to compute the walking step frequency () with an example shown in Figure 8.
3.1.2. Step Length Estimation
An adaptive step detection algorithm based on peaks detection in the angular rates or the accelerations depending on the motion state is applied [16–18]. An example of the steps detected with ULISS held in a texting mode is shown in Figure 9. The following parametric step length model is then applied to estimate the horizontal displacement:where is the step length. is the pedestrian’s height and is a set of parameters that can either be fixed using universal values or be tuned using training phases [19].
3.1.3. Walking Direction Estimation
Estimating the walking direction with handheld IMU is a critical issue for PDR algorithms, since hand motions largely differ from the locomotion direction. This is conducted in a three-step process. At first, magnetometers are calibrated on site in a clean environment by rotating the handled device with random motions [20]. The norm of the magnetometer before and after calibration is shown in Figure 10.
(a)
(b)
A Magnetic, Acceleration fields and GYroscope Quaternion- (MAGYQ-) based attitude angles estimation filter [21] is applied to estimate the attitude angles of the handheld sensors. It is based on a gyroscope signal modelling in the quaternion set and two opportunistic updates: magnetic angular rate update (MARU) and acceleration gradient update (AGU).
Last step extracts the walking directions from the accelerations, which have been rotated to the local navigation frame using MAGYQ estimates. Contrary to existing approaches, like principal component analysis, which search for the energy main axis, a statistical model based approach is adopted [22]. During a training phase, horizontal acceleration is modelled by a Gaussian Mixture Model (GMM) knowing the walking direction and motion state. Motion state is an output of the classification and the walking direction can be computed by GPS outdoors or using building constraints. Figure 11 shows the GMM outcome with two modes for one dataset of accelerations. An Expectation and Maximization algorithm is used to build the GMM with a zero-degree misalignment between the pointing direction of the handheld device and the walking direction. Likelihood maximization is used to find the correct GMM and estimate the walking direction knowing the observations. The walking direction is the parameter that optimizes the GMM defined by the probability density function : is the GMM defined by the mean and the variance matric . are weighting factors with .
This approach learns from individual walking gait and is more robust to the variability of hand movements.
3.1.4. Estimation of the PDR Trajectory
Finally, a recursive PDR process combines the step lengths and walking directions to compute the pedestrian footpath:where is the current position and is the previous position.
3.2. Real-Time Implementation Issues toward a Demonstrator
This part presents another application of ULISS. It is dedicated to the development of a real-time demonstrator of personal navigation. It presents the challenges that arise when moving from positioning and navigation algorithms developed in scientific computation software (e.g., Matlab) to real-time implementation. Scientific and hardware issues are presented along with the motivation and methodology adopted to solve the latter.
3.2.1. Real-Time Constraints
Real-time coding of the previously described positioning algorithms requires the use of sliding windows. The implementation raises questions about their sizes and overlaps. To answer these questions, a physical analysis of pedestrians’ activity is needed. The first objective is to at least capture the walking frequency. Literature [23] and experiments [16, 24] show that pedestrian’s step frequency is around 2 Hz leading to the choice of a 0.5 seconds minimum sliding window. Considering this biomechanical result coupled with the 200 Hz output frequency of the inertial and magnetic sensors and the need to set the window size as a multiple of a power of 2 for the filtering optimization, the size of the second sliding window is set to 0.64 seconds, that is, 128 samples. This corresponds to the base cycle.
Figure 12 shows the execution of all processes, presented in Figure 6, including the sizes of the sliding windows and the overlaps. It can be seen that the pedestrian activity classification (process A) is triggered every 0.64 s without overlap. Two other processes, that is, the walking frequency analysis (process B) and MAGYQ attitude estimation filter (process E), are triggered every 0.64 s with a 1.92 s overlap. Finally, the step detection (process D), the step length estimation (process F), the walking direction estimation (process G), and the PDR estimation (process H) are triggered every 1.28 s with a 1.28 s overlap.
Only process A is executed over the elementary cycles 1, 2, and 3. Processes A, B, C, D, E, F, G, and H are executed over cycle 4 according to their different window sizes. This is repeated for all subsequent even cycles. Starting from cycle 5, processes A, B, C, and E are executed every odd cycle.
3.2.2. Transcoding Matlab Code into C/C++
Matlab environment is practical for research purposes but has not been conceived for real-time coding. The choice has been made to keep research activities into a high-level programming language. Unfortunately, high-level languages are not compliant with real-time context and it is necessary to transform the code into another programming language. C/C++ was chosen since it is more appropriate to real-time implementation. Some toolbox can be used to ease the transcoding task.
As mentioned, the first objective is to maintain only the high-level code. Hence, the transcoding task is processed with as much automation as possible with limited manual adjustments. This is, for example, the case with vectors elements that are not accepted in conditional tests such as “if.” Some native Matlab functions require the input variables to be known a priori for generating the transcoded program. Sometimes, complex function such as filtfilt used on a whole-window dataset can be replaced by easier ones like mean or median on smaller sliding windows. The most complex case is when native Matlab functions are not supported by automatic transcoding toolbox as it is the case for Hilbert function [25]. All manual adjustments shall be uploaded and integrated in the high-level code to ease the software maintenance.
Secondly, simplicity shall be targeted. Because the toolbox can produce numerous files, which depend on the options selected for transcode in C/C++, a static or dynamic library is first generated and added to the project to simplify the implementation of the generated code. The function is finally called with all input/output data properly initialized.
It is also important to deploy a validation process. Because transcoding can lead to some unexpected behaviour, each output of a transcoded library shall be compared to the output of its original high-level part. It is even recommended to deploy an automatic validation process.
3.2.3. Choice of the Hardware Platform
Once the original code has been transcoded, an appropriate CPU has to be found to run the latter. Many powerful electronic boards of small size have been developed for the Internet of Things. They offer different options that must be benchmarked to evaluate their good functioning for the computation costs of the positioning and navigation algorithms. As shown in Figure 13, this is performed by measuring the computation time and deriving both the CPU load and idle periods. This defines the choice of the actual electronic board and future developments of the platform in terms of novel algorithm.
A key element is the form factor of the casing which hosts the complete system. Its dimensions are related to the battery choice. A compromise must be found between its size and the battery life. Today, LiPo (lithium polymer) batteries offer very interesting capacities in small volumes. For example, an analysis in terms of consumption of the quad-core CPU and MEMS and HSGNSS embedded sensors shows 0.9 to 1.1 W consumption. A 850 mA·h battery would provide autonomy of approximately 2.5 h.
4. PDR Field Testing with Handheld ULISS
4.1. Description of the Experiment in the Shopping Mall
ULISS was held in hand by three pedestrians (M1, W1, and M2) over a 1 km walk in a shopping mall at comfortable speed (Figure 14). W1 and M2 carried ULISS in a texting mode for the 1st half of the track set and in a swinging mode for the 2nd half. M1 was walking in a texting mode for the entire walk. The reference solution is computed using a geodetic GNSS receiver carried in a backpack whose data are postprocessed in a differential mode with RTKLIB software. The decimetre-level accurate DGPS solution is available outdoors at the entrance of the shopping mall and at the middle of the walk (500 m). Inside, main corridor directions are used to assess the accuracy of the position estimates.
4.2. Performance Assessment of the Postprocessed PDR Trajectories
Figures 15, 16, and 17 show in red/blue the PDR trajectory estimates for three persons and the DGPS reference positions in green. Figure 15 shows one continuous trajectory in red, which corresponds to the texting carrying mode. Because the carrying mode did not change in this test, the walking directions are computed using the yaw estimate from MAGYQ. This choice enables comparing the PDR performances with the trajectories for M2 and W1, where the walking direction is estimated using the GMM approach.
In Figures 16 and 17, the pedestrian’s trajectory is divided into two parts, shown in red and blue. They correspond to the texting carrying mode and the swinging carrying mode, respectively, for the test subjects M2 and W1. The distinction is made because a different statistical GMM (see (2)) is constructed for each carrying mode for the same person. Although GPS signal is available at the beginning, the middle, and the end of the experiments, it is not used to compute the PDR trajectories but only to assess the PDR positioning performances.
It is observed that the pedestrians remain in the corridors inside the shopping mall. The error on the walking direction estimation for all three datasets varies between 1.4° and 15.3° after the 1 km walk. The PDR positioning error, that is, using only inertial and magnetic field records of the handheld unit, varies between 5.6% and 6.1% for all experiments. It is computed by dividing the positioning error at the end of the track-set by the sum of the stride length estimates. The walking direction estimation based on GMM is found to be better for the swinging carrying mode compared to the texting carrying mode. The GMM model was created during the outdoor phases when the GPS signal was available. Possible reason is that there was not sufficient data available to create the GMM in the texting mode where the arm/hand motion induces rather small accelerations.
Globally, the PDR errors are rather small for these 1 km indoor tests and are promising for the development of self-reliant positioning solution based only on ULISS raw data inertial, magnetic, and GNSS data.
4.3. Long Outdoor/Indoor Walk in the Research Facility Site
A longer experimentation has been conducted in IFSTTAR’s facilities in order to test the proposed pedestrian positioning methods and system in another environment. This environment is a mixed indoor and outdoor space. During the data collection, ULISS was held in hand in a swinging mode during a 2 km walk. Similar to the tests conducted in the shopping mall, GPS signals have only been used to compute the first initial location’s coordinates. Similar performances are obtained for this new dataset. The footpath computed with the proposed PDR algorithms is depicted in blue in Figures 18 and 19. The magenta tracks correspond to the GPS-only positions computed in differential mode. Figure 19 focused on the indoor parts, where no GPS solution is available. The error on the walking direction varies between 4.5° and 5.8°. The positioning error is improved with a 1.5% error over the 2 km travelled distance.
5. Conclusion
The motivations, the design, and some applications of a new PDR navigation device (ULISS) are described in this paper. First “privacy by design” approach and technological choices are discussed, leading to the choice of inertial, magnetometer, and GNSS sensors. Processing the signals of these sensors enables characterizing human walking gait and sensors’ small weight and size to maintain natural walking. PDR algorithms are then presented. MAGYQ attitude estimation filter is used to compute roll, pitch, and yaw angles. It is combined with a novel walking direction estimation technique based on individual statistical models. Stride length estimation is based on gait analysis and modelling. 1 km indoor experiments are conducted by 3 persons in a shopping mall with ULISS held in hand. Promising 5.8% mean error on travelled distance and 2° to 15° errors over the walking direction are obtained in postprocessing mode.
Real-time coding issues toward a demonstrator are discussed and a method containing the steps needed to produce the latter is also presented. It includes the choice of sliding windows sizes, automatic transcoding process into C language, and CPU performances. Future work shall lead to a real-time demonstrator enabling new research on multimodal transportation services.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper
Acknowledgments
This research was supported by the European Marie Skłodowska-Curie Actions fellowships for the smartWALK project.