Abstract
Aim. Self-report diaries are a low-cost method of measuring community participation but may be inaccurate, while the “gold standard,” observation is time consuming and costly. This study aimed to investigate the feasibility and validity of a global positioning system (GPS) for measuring outings after stroke. Design. Cross-sectional cohort study. Methods. Twenty ambulant people with stroke wore a GPS device and kept a diary for 7 days, and 18 were observed for half a day. We recorded recruitment rate, user perceptions, and data extraction time. GPS data were analysed against Google maps. Percent exact agreement (PEA) with observation was calculated for GPS and diary. Results. Of 23 eligible participants, 20 consented (mean 3.6 years after stroke). GPS data recovery was high (87%). Some participants had difficulty operating the on/off switch and reading the small screen. Data extraction took an average of 5 hours per participant. PEA with observation was high for number of outings (GPS 94%; diary 89%) but lower for purpose of outings (GPS 71%; diary 82%). Conclusions. The GPS device and diary were both feasible and valid for measuring outings after stroke. Simultaneous use of GPS and diaries is recommended for comprehensive analysis of outings.
1. Introduction
Community participation is an objective indicator of the success and outcome of stroke rehabilitation [1]. The “amount” and “type” of community participation, such as outings, can be quantified, providing valuable information about changes over time due to intervention. While the measurement of participation is increasingly common in health research, the measurement process remains challenging. The criterion standard method is direct observation, which is objective but time-consuming and costly. In addition, the process of observation may influence a person’s behaviour.
Self-report diaries are a simple and cheap method of measuring physical activity. Yet diaries produce unreliable data and are burdensome for participants, especially those with impaired cognitive abilities [2]. When retrospective interviews about events were compared with daily diary recordings of events, fewer recall errors were seen in the diary [3]. Diaries capture minor events and produce higher “incidence” rates, especially for healthy unimpaired populations; however, diaries can also underreport events and lead to lower compliance by people with disabling conditions when compared to retrospective interviews [3].
Collecting information about outings can provide a picture of someone’s ability to get out of their house and into the local community. Wearable sensors are emerging as a potential method of remotely monitoring and objectively measuring participation [4]. Accelerometers are increasingly being used to measure physical activity with adult populations, including people with stroke [5–7]. However, these devices do not adequately measure outings because they do not record where an outdoor activity occurred or its duration. In contrast, wearable tracking devices such as global positioning systems (GPS) have the potential to give specific information about the timing and location of outings.
A GPS device receives triangulated signals from satellites orbiting the earth. The location is identified by calculating the difference in time between when a satellite signal was sent and received. A minimum of four satellites are required to estimate the position and elevation of the receiver [8]. A GPS device can measure active transport and movement patterns in outdoor areas, but requires an unobstructed line of sight to the sky. If the signal is blocked by dense urban development with narrow streets, close buildings, urban “canyons” [8] or large objects, the accuracy of the GPS may be degraded or the signal not received at all.
GPS technology has been used in health research to track the position of children [9] and older adults with dementia who wander [10, 11]. A GPS device was also used recently to measure distance walked outdoors by people with stroke during the 6-minute walk test [12] while supervised by a therapist. The device was valid and feasible for measuring distance walked in local streets, potentially removing the need for therapists to use a measuring wheel during an escorted trip. Potential benefits of using a GPS device in health research include the provision of objective and accurate information about time, distance, and place [13, 14]. Contextual information can also be collected, such as the time and frequency of visits, providing important insights into a person’s weekly or monthly routine [8, 15, 16].
Additional problems with GPS technology have been reported, including limited battery life and time required by researchers to download and analyse hours of electronic data [9]. In one study involving people with dementia, five of the 11 participants stopped using the GPS device because it was too heavy, uncomfortable, or awkward to use, or they did not want to be tracked [10]. For these reasons, many researchers and clinicians prefer to use self-report methods such as a diary to measure participation, physical activity, and outings.
Few studies have compared the advantages and disadvantages of self-report methods with GPS technology. A GPS device may potentially be less effortful and more reliable than a diary for recording participation. On the other hand, people with stroke often have impaired hand function and older adults may be unfamiliar with this type of technology. Validity was recently reported for daily walking trips taken by healthy adults who kept a self-report diary and used a GPS device [17]. Agreement between the number and duration of walking trips was high, with 90% of GPS-recorded trips reported in the diaries.
No study has yet trialled a GPS device to electronically capture participation by community-dwelling stroke survivors. Therefore, the aim of this study was to determine whether a wearable GPS device was feasible and valid for recording outings when used by people with stroke. Our research questions were as follows.(1)Is wearing a GPS device feasible for community-dwelling stroke survivors?(2)Is this recording method valid?(3)Is the GPS device superior compared with a self-report diary for measuring number and purpose of outings?
GPS and diaries were compared with direct observation, the “gold standard” for measuring community participation.
2. Methods
2.1. Design
A cross-sectional cohort study design was used. Three procedures were used to measure the number and purpose of outings: the GPS device, a self report diary, and direct observation. Participants were asked to carry the GPS device and record outings prospectively and concomitantly in a diary for seven consecutive days. The starting day was randomised. Direct observation was the “gold standard” against which the GPS and diary measures were compared. On the second day, direct observation took place for up to four hours when a research assistant observed participants during their usual routines. Participants were asked not to change their routine and the research assistant only observed participants who intended to go outside the house within the four-scheduled hours. Observation times were randomised to either morning (8 am start) or afternoon (1 pm start). The study was approved by a university ethics committee, and all participants provided written informed consent.
2.2. Participants
Community-dwelling people with stroke were identified from an existing database, contacted by telephone, and invited to participate. Eligibility criteria included a diagnosis of stroke; able to read and write in English; able to walk outdoors alone or with supervision. We purposively recruited people with a range of walking speeds, based on their last known distance walked in 6 minutes; geographic locations across a large metropolitan city with potentially high and low GPS signal reception; age ranges; gender. People were excluded if they had communication or cognitive impairments which affected their participation.
2.3. Procedures
2.3.1. The GPS Device
The GPS device used in this study was supplied by an Australian company Ezy2c Online GPS Monitoring (device EZ 23 Personal Tracker, http://www.Ezy2C.com.au/, see Figure 1(a)). The device is about the same size of a small mobile cell phone. At the initial meeting at each participant’s home, they were taught how to use and recharge the GPS device.
Participants wore the GPS device on a belt loop or around their neck, or stored it in a pocket or bag whenever they went outdoors. They were asked to switch the device on when they woke up each morning, off before they went to bed at night, and to recharge the device every evening. The device was switched on by pressing a button on the side, signalled by a tune. Participants were asked to leave the device switched on and in a visible place (if not worn around their neck) even if they stayed indoors all day, to prevent loss of data should they go out and forget to carry or switch on the device. Like most other GPS devices, this model did not record a signal indoors. Devices were also programmed to “power down” when inactive for eight minutes in order to save battery life.
Each device transmitted real-time geographic information to a password-protected website, for a monthly fee. Data could be accessed as a list of coordinates with two-minute intervals linked to a Google map showing the approximate location of each device. The Google map provided street names but not street numbers. At any time it was possible to select a day of the week and times of the day for analysis (see Figure 1(b)). A research assistant monitored the status of each device in real time through a computer; if a device was not switched on, participants were contacted by telephone. Turning the device off was permitted, but participants were contacted if this occurred to ensure they had turned the device off intentionally. A device that was actively moving for 12 continuous hours a day, over seven days, would produce approximately 2500 maps to be viewed.
2.3.2. Self-Report Diary
Participants also filled in a paper diary for seven consecutive days. The diary contained space for three-time periods: 08:00 to 11:00, 11:00 to 14:00, and 14:00 to 22:00. Participants were asked to record outings taken during these periods and note purpose of outings and mode of transport used (e.g., walking, car, bus, or taxi). An excerpt of a completed diary is shown in Figure 2.
2.3.3. Direct Observation
During the half-day of direct observation, the research assistant accompanied participants and also recorded number and purpose of outings, time of departure, and mode of transport used.
2.4. Outcome Measures
2.4.1. Feasibility
Feasibility was measured in terms of recruitment, compliance, user perception, data recovery, and data extraction. Recruitment was measured as numbers of eligible participants who consented versus declined to take part in the GPS study; reasons for declining were recorded. Compliance was measured as the number of hours participants wore or carried the device, and the percentage of actual hours versus possible hours over seven days.
User perceptions were measured using a questionnaire, completed at the follow-up visit after seven days. Five-closed questions enquired about (1) wearability of the device, (2) ease of switching the device on/off, (3) ease of recharging the device, (4) strategies used to prompt participants to switch the GPS device on, to wear and recharge the device, and (5) activities which were difficult while carrying or wearing the device. One open-ended question asked for other comments.
Data recovery was measured as the amount of data recorded and lost due to either human or device error. Data extraction time included time spent setting up the device at the beginning of the seven days and collecting the device at the end, monitoring the device to make sure it was functioning, downloading the coordinates of participants’ travel, and extracting data about outings from the coordinates.
2.4.2. Validity
Validity was measured as agreement about number and purpose of outings, between a GPS device or diary and direct observation. An outing was defined as an excursion beyond the front gate or perimeter of the person’s property until they returned home. The purpose of each outing was classified according to one of four categories developed for the study: (i) home/personal maintenance, such as outings to the shops, hairdresser, church, post office, or bank; (ii) health-related, such as consultation with health workers in a clinic or hospital; (iii) social, such as outings to visit family and friends, school/university, or entertainment venues such as theatre or restaurant; (iv) an exercise-related, such as going to a park or gym.
Two raters extracted outings from diaries and GPS data. When reliability was checked by comparing counts over a half day, selected randomly from 14 participants, 100% agreement was achieved between the two raters.
2.5. Data Analysis
For feasibility, descriptive statistics including frequencies and proportions were used to analyse questionnaire responses, and data for seven consecutive days from each GPS device. For validity, percent exact agreement between GPS device and direct observation was calculated as well as between diaries and direct observation for outings and their purpose. These measures were compared descriptively.
3. Results
3.1. Participant Characteristics
Of 23 eligible participants contacted, 20 consented to carry the GPS device and keep a diary; of these 20, 18 agreed to be observed. The mean age of the 20 participants was 67 years (SD 12, range 48–85). The majority were men (, 70%). The mean time after stroke was 43 months (SD 20, range 17–85). Mean distance walked during their most recent 6-minute walk test was 288 m (SD 96, range 106–494). The majority (, 60%) used public transport and travelled as a passenger in a car (, 85%). Few participants drove a car (, 5%), used a motorised scooter (, 10%), or taxi/community transport (, 20%) (Table 1).
3.2. Feasibility
3.2.1. Recruitment
Three of the 23 eligible participants who were telephoned declined to participate (13%) because the seven-day data collection period was considered too long (); they did not want to be “tracked” by GPS () or did not find the study topic sufficiently interesting to take part (). The recruitment rate was 87%.
3.2.2. Compliance
Nineteen of the 20 participants carried the device for an average of 85 hours (full 7 days of data collection). One participant carried the device for only one day due to equipment failure (explained later in results), not due to noncompliance.
3.2.3. User Perceptions
The majority of participants (80%) found the GPS device easy to wear, switch on and off, and recharge. The most commonly reported difficulties were using the small on/off button, reading the small screen, the perceived weight of the device, and having to recharge the device more than once a day due to the short battery life. The majority of participants (65%) preferred wearing the GPS device on a lanyard or cord around their neck, while 20% carried it in their pocket, 10% carried the device in their bag, and 5% wore it clipped onto their belt. No one lost or misplaced the device.
3.2.4. Data Recovery
A total of 215 hours of data (of 1680 possible hours) were lost due to human or device error (13%) with an 87% data recovery rate. Eleven participants had difficulty interpreting the power status of the device. When inactivating the device “powered down,” the screen became blank and the device appeared to be “off.” These participants mistakenly switched the device “off” at such times, resulting in 97.3 hours (45%) of lost data. Failure of the device battery charger resulted in another 41.5 hours (19%) of lost data. Furthermore, three of five GPS devices and eight of 15 battery chargers had to be replaced by the supplier due to malfunction. Faulty devices and battery chargers were exchanged on the same day by the research team when possible and returned to the manufacturer. However, often several hours or a whole day of data was lost before a device could be replaced or became operational again, resulting in 60 hours (28%) of lost data. Three participants forgot to switch on and/or recharge the device resulting in 16.5 hours (8%) of lost data.
3.2.5. Data Extraction
On average, five hours were required per participant for setting up and collecting the device at their home, monitoring the device involving checking with the GPS company website that devices were functioning, downloading and extracting data. Two of the five hours on average (SD 0.5) were spent setting up and collecting the device, while one hour (SD 1) was spent monitoring the use of the GPS; two hours (SD 1) were spent viewing, downloading, and extracting data.
3.3. Validity
3.3.1. Number of Outings
The average number of outings recorded over a week by the GPS device and diary is presented in Table 2. Participants went on approximately seven outings per week and travelled about 140 km on foot, by car, bus, taxi, or train. For number of outings, agreement between GPS and observation was 94% and between diary and observation was 89%.
3.3.2. Purpose of Outings
The most common purpose of outings was social, such as going to a restaurant, theatre, family member or friend’s house, or community centre (Table 2). The second most common purpose was home/personal maintenance, such as going shopping, going to the post office, the church or college/school. Few outings were dedicated to exercise and even fewer outings involved healthcare appointments.
For purpose of outings, agreement between GPS and observation was 71% and between diary and observation was 82%. For outings with a home/personal maintenance purpose, exact agreement between GPS and direct observation was 50% and between diary and observation agreement was 72%. For outings with a social purpose, percent exact agreement between GPS and observation was 61% and between diary and observation agreement was 83%. For outings with an exercise-related purpose, percent exact agreement between GPS and observation was 78% and between diary and observation agreement was 83%. Finally, for outings with a health-related purpose, percent exact agreement between GPS and observation was 100% and between diary and observation agreement was 94%.
4. Discussion
There were a number of key findings from this study. First, people with stroke were willing and able to use the GPS device consistently for a week. No one abandoned the device, and compliance was high. Second, the GPS was found to be valid for objectively recording outings but less so for revealing the purpose of outings. Third, while the GPS device was slightly superior to a diary for recording outings, the diary was slightly superior for recording the purpose of outings. Importantly, a diary in this context was not invalid, as we had anticipated. Overall, GPS and diary contributed unique data and should be used together, where possible to triangulate data. Fourth, we identified sources of human and technical errors, which were different for the GPS and diary. More data were lost due to device failure than human error. Some of these problems can be anticipated and managed to minimise data loss.
4.1. Feasibility of Wearing a GPS Device
It was feasible to teach people with stroke to use a GPS device to enable outings to be measured. Unlike people with dementia and their carers [10], this population did not find the procedures overly burdensome, nor the device too large to wear. While most were pensioners aged over 60 years of age, many already carried a mobile or cell telephone. The GPS device had similar characteristics to many mobile telephones and was easily accommodated. Once switched on each morning, the only attention that was required from the participants was a visual check of the screen. Some participants were anxious about the status of the screen, and the “powering down” of the device when it was inactive for extended periods. However, the device should not need to be switched on again during the day unless the battery runs down. Some activities such a kayaking, bowling, and gardening were more “difficult” when using the GPS device. In this situation, the best strategy is probably to leave the GPS device nearby in a bag or jacket, since outings, distances travelled, and time away from home will still be recorded accurately.
The sample did not include people with known communication or cognitive impairments. At least four participants had some residual hand function problems, and three of these people operated the GPS device independently. Only one participant reported having help from a spouse to operate the device.
4.2. Validity of the GPS Device versus a Diary
The GPS device and diary were both accurate, with the GPS device recording 94% of observed outings and the diary recording 89%. These findings are comparable, if slightly better than earlier research; while no previous study has compared GPS and diary against direct observation, Cho and colleagues [17] compared a GPS device with two types of diary. They found that 90% of GPS-recorded walking “trips” were also recorded in diaries. Thus, wearable GPS devices are not 100% accurate, particularly for recording short walking trips. Our GPS device “missed” one of the 18 participants’ outings. When outdoors, a GPS device detects signals from orbiting satellites but can be inconsistent at times, depending on the location of satellites and the device. Presumably the GPS failed to detect a signal on this one occasion. The self-report diary also “missed” one outing (coincidentally, the same outing as noted above), while an extra outing was recorded but not observed.
4.3. Benefits of Triangulating GPS and Diary Data
There are benefits to using GPS technology and diaries together to optimise data accuracy and validity because the purpose of outings could not be classified using GPS data alone. This descriptive information can be helpful for explaining changes in a person’s participation habits, for example, fewer health-related outings and more social outings.
Inspection of Google map data did not provide sufficient information to classify the purpose of outings. Google maps did not show street numbers, only street names. Few businesses are advertised on Google maps, therefore we were unable to correctly classify the type of some locations visited, for example, a doctor’s surgery or local shop. However, diaries can both underreport and overreport outings. On one occasion, a participant did not report a health-related visit in their diary, which was recorded by GPS. The destination was a large hospital and could be identified easily on a Google map.
There are several advantages of using a wearable GPS device to collect objective real-time data about outings into the community. First, reliance on self-reporting by participants is removed, helping to minimise measurement bias. While diary data provide important supplementary information, the primary measure of community participation could be outings measured using GPS.
4.4. Sources of Error and Inaccuracies: Technical versus Human
While GPS technology has improved in recent years, there was still some loss of data. Previous studies have found that technologies such as accelerometers, pedometers and GPS devices miss outings, and lose data. Equipment failure is a common problem for health researchers, with 50% of accelerometers failing in one study due to devices being dropped [5]. In another study, 17% of participants forgot to switch their accelerometer on [18]. In our study, the primary cause of GPS errors and loss of data was equipment failure, resulting in 13% of GPS data being lost (215/1680 hours recorded). Interestingly, failure of the GPS device was a larger problem than participant or human error.
A second reason for GPS inaccuracy was failure of the device to record short outings. Each GPS device was programmed to “power down” when stationery for extended periods to conserve the battery. It took up to one minute on some occasions for the GPS to “power up” and start sending coordinates. If an outing was less than two minutes in duration, such as a quick return trip to a neighbour across the road, the outing was not consistently recorded. While the device can be programmed to transmit coordinates more often, data analysis would become more onerous. Some data would almost certainly still be missed due to delays in the GPS capturing a satellite signal. Cho and colleagues [17] recently developed an algorithm to classify walking trips as separate events, but only if the trips were longer than three to five minutes. They disregarded shorter trips because the GPS device often did not capture a satellite signal and register an outdoor trip of less than three minutes duration. While a device can be programmed to record coordinates at 10 or 20 second intervals and identify shorter trips, very short trips in and out of buildings, or between two adjacent buildings, are likely to remain undetected.
A final possible reason for missed data was poor GPS signal reception and positional accuracy in some suburbs. Rainham and colleagues [8] noted that positional accuracy changed across built environments and forms of transportation. In their study, precision was best in an urban park setting where more satellites are typically used to establish coordinates, compared to an “urban canyon” with dense urban development, narrow streets and close buildings. In dense urban areas, where GPS is most challenged, the same authors found that their GPS device was most accurate (within five metres) when carried or worn in a car (83% accuracy), on a transit bus (60% accuracy), when walking (57% accuracy), or cycling (54% accuracy) [8]. Inaccuracies are greatest when there is potential for signal interference. Most of our participants were resident in and travelled between suburban areas of a large city, not dense urban areas with high buildings.
Many health researchers who use GPS devices recommend collecting concurrent survey or diary data to overcome missing GPS data [8]. Daily diaries add to the burden of data collection, but help fill in the gaps when GPS data are missing. The 13% loss of data reported in our study could possibly be reduced to 10% or less by using strategies such as providing extra opportunities during the participant orientation session to practice “reading” the GPS screen and determine whether the device is switched on, off, or has “powered down” and routinely providing a second battery charger.
4.5. Study Limitations
As with most research, this study had limitations. First, the participants were purposively selected so that all could walk, and none had severe communication or cognitive impairments. This means that findings may be different when nonambulant or more impaired participants use the device. Second, 80% of the sample lived with a spouse, who may have helped people to recharge and remember to use the GPS device. Finally, carrying the GPS device may have resulted in “better than usual” diary use by participants, who knew we were examining both sets of data closely. Routine use of diaries in clinical practice may result in less comprehensive recording.
4.6. Implications for Practice and Research
Researchers are advised to triangulate data using diaries and technology when possible. Clinicians are advised to use diaries to measure the purpose of outings by people with stroke. Diary recordings can be compared at intervals during community mobility and travel training and used as an outcome measure. Technologies such as mobile telephones, Smart Phones, and I-Phones can also be used to record distance travelled, nature of terrain, and speed. If people with stroke own and use these technologies, they can be used to record progress and provide feedback.
Future research could compare the accuracy of GPS technologies which are built into Smart Phones and I-Phones. These devices are common and accessible. Programs may be adapted in the future to record number of outings per day. Researchers and clinicians need to be aware of, and use, new technologies which have the potential to simplify data collection and analysis.
5. Conclusions
The GPS device used in this study was slightly superior to a self-report diary relative to the “gold standard” of outings observed. However, both GPS and diary achieved good levels of agreement. A diary was slightly superior for classifying the purpose of outings. A combination of the two methods is recommended to obtain a full picture of where a person travels to, how far and how often, and the purpose of outings. Although some data were lost because of device and/or human error, we expect that data loss could be reduced to about 10% in future. It was feasible for people with stroke to use a wearable GPS device for a week to record their daily outings. Recruitment and compliance were high and people with stroke reported minor difficulties using the device. People with communication, cognitive and severe motor impairments were not included in the sample, and would require more carer assistance.
Conflict of Interests
The authors declare that they have no conflict of interests.
Suppliers
EZ23 Personal Tracker-Sydney Suite 2402, Westfield Tower 1, 520 Oxford Street, Bondi Junction, NSW 2022, Australia.
Authors Contribution
The first author initiated the study. All authors contributed to the design, analysis and interpretation of the data, and writing of this paper. The fourth author collected the data.
Acknowledgments
The first author was supported by a postdoctoral research fellowship during the conduct of this study, funded by the University of Sydney, the Royal Rehabilitation Centre Sydney, and the National Stroke Foundation, Australia.