Abstract
With the rapid development of mobile computing, wearable wrist worn is becoming more and more popular. But the current vibrotactile feedback patterns of most wrist-worn devices are too simple to enable effective interaction in nonvisual scenarios. In this paper, we propose the wristband system with four vibrating motors placed in different positions in the wristband, providing multiple vibration patterns to transmit multisemantic information for users in eyes-free scenarios. However, we just applied five vibrotactile patterns in experiments (positional up and down, horizontal diagonal, clockwise circular, and total vibration) after contrastive analyzing nine patterns in a pilot experiment. The two experiments with the same 12 participants perform the same experimental process in labs and outdoors. According to the experimental results, users can effectively distinguish the five patterns both in lab and outside, with approximately 90% accuracy (except clockwise circular vibration of outside experiment), proving these five vibration patterns can be used to output multisemantic information. The system can be applied to eyes-free interaction scenarios for wrist-worn devices.
1. Introduction
In the nowadays information society, wearable devices of different forms have come into people’s lives in various aspects, simultaneously posing a challenge to the interactivity design. With the ability of aiding in the remote monitoring of patients, wearables provide real-time access to health records and provide quicker diagnosis and treatment of conditions. Therefore, more studies began to focus on how to effectively exploit wearable technology in the field of e-Healthy [1] or healthcare system.
In contrast to classical graphical user interfaces, wearable devices cannot provide a good visual interaction experience owing to their diverse designs and size limitation. Auditory feedback is a good way to convey semantic information to the users [2]. However, it is difficult for users to receive audio information effectively in mobile environments, where auditory channels are compromised by external noise and social concerns. At the same time many researches have shown that tactile display, without the drawbacks of visual or auditory display, is an ideal interactive mode for distracting situations [3–6].
Vibration is the basic tactile feedback pattern of wrist-worn devices [7]. By this means, users can be notified without the visual load and in private or noisy situations [8]. However, the present vibrotactile displays are too simple, and most of them can output only two kinds of information, i.e., “vibrating” and “not vibrating,” limiting the feedback information received by users. Thus, users tend to take extratime to confirm what kind of information they receive after sensing vibration. At present, there are many studies to investigate the effects of multiple vibrotactile patterns. But these devices array vibration motors on the same side [9–12]. Each kind of vibrotactile patterns always generates on the same plane of the wrist skin. The skin area of the wrist is relatively narrow. Users may be confused to confirm which vibration motor is working during vibration.
In order to study the user’s identification of multiple vibrotactile patterns through a device spatially arraying vibration motors, we present a wearable wristband prototype which spatially arrays vibration motors around the whole wrist. It can produce many kinds of vibrotactile patterns, allowing users to obtain multisemantic information transmitted through the tactile channel. As shown in Figure 1, four vibration motors are embedded into a customized wrist-worn device. Motors vibrating in different positions or in different orders can generate various vibrotactile patterns. The wristband prototype we developed could generate nine vibrotactile patterns (four positional patterns, two diagonal patterns, two circular patterns, and one total vibration pattern). Through preexperiment, we decided to use five patterns to research.
In this paper, we first discuss the relevant researches on tactile display for wearable devices, vibrotactile feedback conveying rich messages for wearable devices, and which body locations more sensitive to vibrations feedback. Next, we present our wristband prototype system and apply five vibrotactile patterns in our study. Then, we describe two experiments that we conducted to examine the accuracy with which users are able to distinguish the five different vibrotactile patterns and the effect of wristband prototype system on the user’s ability to perform in lab and outside eyes-free interaction scenarios. Finally, we show some further applications for the wristband prototype system.
2. Related Work
Wearable devices have limited visual and auditory information output channels owing to their physical characteristics and complex use scenarios. Tactile display is gaining attentions and has been confirmed as desirable feedback for such devices. Roumen et al. [5] made a comparative study of notification channels (light, vibration, sound, poke, and thermal) for wearable interactive rings. They concluded vibration was the most reliable and fastest channel to convey notification. And Hsieh et al. [13] added tactile feedback in the haptic glove which assists to interact with smart glasses enhancing tangibility. Exploring natural vibrotactile interaction has become the main research direction.
A few researchers have shown different vibrotactile parameters (e.g., intensity, frequency, temporal pattern, and spatial pattern) conveying rich messages for wearable devices. Cauchard et al. [6] encoded the vibrations using the duration and rhythm to represent progress. The ActiVibe they produced utilizing the vibrations has been confirmed with up to 88.7% recognition rate through the outdoor experiment and a list of factors that affect the recognition rate was given, such as other vibrations produced during the activity and the materials of device generating uncomfortable feeling. Brewster et al. [9, 14] investigated the perception of Tactons which encode three dimensions of information using three different vibrotactile parameters. The result reveals that spatial patterns are easier to discriminate than frequency and intensity. Van Erp et al. [15] designed a tactile waypoint navigation display that consists of eight tactors around the user’s waist, and they translated distance to vibration rhythm while the direction was translated into vibration location. Their experiments indicated the usefulness of the tactile display on waypoint navigation. In addition, previous studies also propose richer interaction using several vibrations. Yatani and Truong [16] proposed a real-time feedback system, SemFeel, through multiple vibration motors attached to the backside of a mobile device. This system contains 10 vibration patterns that users can distinguish them at approximately 90% accuracy, and it supports accurate eyes-free interactions. Lee and Starner [10] presented wrist-worn wearable tactile displays that provide easy to perceive alerts for on-the-go users. Their system developed with three actuators in a triangular layout on the volar side of the wrist provides 24 vibration patterns with up to 99% accuracy after 40 minutes of training for users. And the comparison test showed users’ perception of incoming alerts for wrist-worn wearable tactile displays does not decrease when visually distracted. All these works can prove that vibration display is an effective way to improve users’ interaction experience.
Studies have also been conducted to explore which body locations are more sensitive to vibrations [17, 18]. The results both indicated that wrists are generally better for feeling vibrations relative to other body parts. According to these analysis and results, some wrist systems based on vibrotactile feedback have been proposed. Bosman et al. [8] developed a dual-wrist system to guide a pedestrian inside an unknown building. The vibrations indicated directions and stops. The result suggests vibration tactile output can greatly help improve the performance of this kind of wearable wristband devices and effectively reduce the disruptiveness of technology. Huxtable et al. [19] presented a tactile interface made up of two wristbands that vibrate to the signals for left and right turns in wayfinding devices designed for cyclists. Dobbelstein et al. [20] presented a bearing-based pedestrian navigation approach that utilizes vibrotactile feedback around the user’s wrist to convey information about the general direction of a target. And these mobile prototypes demonstrate their feasibility in the initial navigation research.
In summary, researches on how to use vibrotactile channels to improve the natural interaction of wearable devices and send complex information have achieved good results. But user interface and interactive design of wrist-worn devices have not been studied systematically and thoroughly. Since previous studies reveal that spatial patterns are easy to discriminate [9]. Our research especially focuses on how much vibration patterns wrists can accurately distinguish.
3. System
To study the acquisition of multisemantic information sent by wrist-worn devices through vibration in eyes-free interactive scenarios, we designed a hardware prototype that generates a variety of vibration patterns. The prototype consists of two components (Figure 2): a PC and development board (Figure 2(b)) to send vibration commands and a wristband device (Figure 2(a)) to receive vibration commands and vibrate accordingly. In our experiment, the wristband device is needed to be worn by users.
(a)
(b)
3.1. Sending End
The physical prototype of the sending end is shown in Figure 2(b). The pattern of the development board that controls the four vibration motors is Arduino UNO R3. It works as an intermediary to receive signals from a computer by connecting to the serial port of the computer, and it can also control the operating power supplied to each motor by using pulse-width modulation (PWM) to change the duty cycle. Therefore, we use this technique to both power on/off motors and to control their vibration intensity.
3.2. Receiving End
Considering that we just focus on vibration patterns, a set of basic vibration motors, 12 mm in diameter and 3.4 mm thick, have been used to output vibrotactile information. Each motor was provided 80 mA at 3 V in the experiment.
Since the wrist-worn devices are in contact with the wrist over a limited area, it is difficult to distinguish much vibration information owing to mutual interference. In addition, referencing the wrist-worn prototype presented by Gupta et al. [21], we selected four locations (Figure 1) to place four vibration motors, which are connected to an Arduino development board to receive the vibration commands received from the PC to the serial port through Bluetooth.
In daily life, there is a certain gap between a wrist-worn device and the wrist, and these small gaps may affect the users’ perception of vibrotactile feedback. Therefore, we focus on elastic sports wristbands. As shown in Figure 2(a), this type of wristband is close to the user’s wrist during usage so that the vibrations of the motors can be accurately perceived. Moreover, since our hardware prototype requires the premise of seamless fitness, the study of vibration patterns for loose wrist-worn devices (such as hand rings) is a subject for future research.
3.3. Interactive Interface
The running interface of the software program is shown in Figure 3, and the program written with the C# programming language runs on a computer with a Windows 10 operating system. Figure 3 presents the five vibrotactile patterns that we used in the two experiments. There are four types of patterns: positional (up and down), horizontal diagonal, clockwise circular, and total. Each icon button of patterns had the same size (90 pixels × 90 pixels). The start button means the tests start and the program starts running. The blue button in the middle requires participants’ press to randomly generate vibrotactile patterns. The interface also displays the number of test and the result of each test whether right or not. When the counting number reaches the setting number, all the results will be saved automatically.
4. Experiment 1: Natural Wristband in Lab
4.1. Participants and Apparatus
12 participants (4 female; 8 male) from the university participated in the study. They are all students, coming from Kunming University of Science and Technology, who are randomly invited to participate in the experiments. The age range was 20 to 28 (mean = 23.3, SD = 1.87). Four participants wore smart wrist devices once previously. The hardware and interactive interface have been introduced in the previous section.
4.2. Vibrotactile Feedback Patterns
The only dependent variable that we controled in this experiment is the vibration patterns. As shown in Figure 4, we designed five patterns for the dancing wristband prototype. These five vibration patterns can be divided into four types: positional vibration (single motor vibrating once); diagonal vibration (two diagonal motors vibrating four times); circular vibration (four motors vibrating four times sequentially in a ring); and total vibration (all motors vibrating simultaneously once). In addition, the time interval from the start of vibration to the end of vibration was 1 second for all patterns, taking into account fairness for the perceived time of vibration.
Before deciding on these five patterns, we set up nine patterns in a pilot experiment to explore recognition rates of different vibration patterns. These nine patterns include four positional vibration patterns (up, down, left, and right), two diagonal vibration patterns (horizontal and vertical directions), two circular vibration patterns (clockwise and anticlockwise), and one total vibration pattern. Each vibration pattern repeatedly presented twelve times. The procedure and equipment of the pilot experiment are the same as experiment 1. Six participants volunteered for the experiment, and the rates of recognition are shown in Table 1.
Although only two recognition rates of positional vibration patterns are over 90%, two main problems which should be solved have been found. (1) Vibration patterns of the same type can very easily interfere with each other. For example, participants have difficulty in distinguishing the horizontal diagonal and vertical vibrotactile patterns. (2) Diagonal vibration patterns and circular vibration patterns have similar vibrotactile feedback and are difficult to distinguish.
To solve these problems, we have taken the following measures. (1) Only one vibration pattern was retained for each type except for the positional vibration type. Through analyzing the results, we found positional vibrations were easily distinguished and two of them were retained. (2) Participants prefer vibration with smoothing over vibration without smoothing [16]. So, we set vibration intensities of the four motors in the circular vibration pattern to be 40%, 60%, 80%, and 100%, respectively. Except for the circular vibration pattern with changing vibration intensity, the vibration intensity was always 100%. This is because we just focused on evaluating how accurately participants can distinguish vibration patterns rather than other vibrotactile parameters.
Finally, we determined the five patterns (positional up and down, horizontal diagonal, clockwise circular, and total vibration) which used in the two experiments. Each of the five patterns had a relatively high recognition rate in their type. There are two advantages. (1) All types were retained to ensure the diversity of the vibrotactile feedback. (2) According the “magical number 7, plus or minus two” rule in human-computer interaction, participants could effectively recognize these five patterns in a short time. And, we can explore more patterns in future.
The generation order of the five vibration patterns in the experiment was completely random. Each vibration pattern was repeated four times in a block of experiments, and the entire experiment contained four blocks. Hence, each participant took part in
4.3. Task
In the experiment, the participants were required to feel the vibration patterns randomly generated by the wrist-worn device. As shown in Figure 3, the system randomly generated a vibration pattern after the participants clicked the blue button. The participants needed to select a pattern from the five representative vibration patterns according to the vibration they felt and then clicked the icon button for the selected pattern. Each time after a participant made a choice, the system displayed the correct vibration pattern in the dialog box in black font for a correct choice and red font for an incorrect choice. The number of completed vibrations was displayed in the upper part of the dialog prompt box. Throughout the experiment, participants were asked to complete the task as quickly and accurately as possible.
4.4. Procedure
Each participant was informed of the use of this system before the experiment system. And participants were needed to wear the wristband part of the system in their nondominant hand and used their dominant hand to operate the mouse and software program for interaction.
At the same time, the participants were asked to wear the headphone which was playing music so as not to notice the sound of vibration motors. And the wristband needs to sense the signals beyond the vision of participants in order to reproduce an eyes-free situation.
Before the formal experiment started, participants were asked to become familiar with the whole process of the formal experiment and practiced until they adapted to the system. The participants took a short break after each block in the formal experiment. Generally speaking, a participant completed the practice period in about 15 minutes. In total, a complete experiment lasted about 30 minutes.
4.5. Results and Discussion for the Lab Study
We tested two kinds of data: reaction time and error rate. The reaction time contains two parts, the user feeling vibrotactile patterns and choosing patterns, and the actual reaction time should be shorter. The error rate for each pattern was calculated per block per participant (i.e., 100 × (the number of the wrong responses)/4). The results are drawn as bar diagrams as shown in Figures 5 and 6. The time variable was measured from the time when motors started to vibrate to the time when participants clicked an icon button to make a choice. After the experiment was completed, the experimental data were collected for calculating statistics and analysis.
Figure 5 shows the average error rates of the five vibration patterns. All the average error rates are below 9%, and the lowest average error rate is positional (down). A one-way analysis of variance (ANOVA) test (the significant level is 0.05) for the average error rates against pattern indicates existence of statistically significant differences (mean = 4.27, SD = 10.73, F4,235 = 3.94, and ). The post hoc Tukey multiple comparison revealed that statistically significant differences exist between positional (down) and circular (clockwise) ().
For the reaction time, all vibrotactile types average reaction time is between 1600 and 2200 ms. Positional vibrotactile type requires shortest reaction time on average. We made a one-way ANOVA test (the significant level is 0.05) between the four vibration types (positional vibration, diagonal vibration, circular vibration, and total vibration). The result shows the existence of statistically significant differences (mean = 1798.73, SD = 861.18, F3,956 = 20.44, and ). Then, we combine the two types of positional vibration into one category, so the sample sizes are unbalanced. To accommodate the unbalance, a Scheffe multiple comparison was used in the post hoc test. It indicated that the reaction time for the circular vibration was significantly different from others ().
Figure 7 shows the average error rates across blocks. Each participant performs an entire experiment containing eighty trials that divide four blocks. The whole results of all participants are classified according to the four blocks. The average error rate of each block is less than 5%. Block 1 has the lowest average error rate. And the average error rate of blocks shows a trend of growth first and then tends to be stable. A one-way ANOVA test (the significant level is 0.05) for the average error rates against block indicates that there is no statistically significant difference (mean = 4.27; SD = 3.57).
Figure 8 shows the average reaction time across blocks. The average reaction time of each block is between 1700 and 1900 ms. Block 2 requires the shortest reaction time on average. The trend fluctuates slightly between blocks. A one-way ANOVA test (the significant level is 0.05) for the reaction time against block indicates that there is no statistically significant difference (mean = 1798.73; SD = 355.40).
The results indicate that participants were able to distinguish the five vibrotactile patterns, and recognition accuracy of all patterns is more than 90%. But the recognition accuracy of the diagonal pattern and circular pattern is lower than average (average recognition accuracy of all patterns is 95.73%). By comparing the change of average error rates between blocks, error rate rises between block 1 and block 2. One possible reason is that participants may feel a little numb after long-time participation. Their ability to respond to the stimulus decreases, so the reaction time increases. We need to give them enough break time to rest after they adapt the wristband prototype and after finishing each block experiment.
5. Experiment 2: Outdoors Natural Wristband
We used the same equipment to run the experiment, and participants of experiment 2 are the same as experiment 1. This time participants needed to go outside, and not just sitting. They took some activities instead, such as walking and jogging. We need to do various activities in daily life, but the laboratory environment does not represent all the real living environments. We chose the outdoor scene in the university, surrounded by many classrooms and a bus station, where many people would come and go.
The participants came outdoors to the experiment, so an experimental assistant was also required to help the participants choose the vibration types (Figure 9). We also made sure the assistant was within the range of hearing the participants’ answers exactly.
5.1. Procedure
Participants wore the wristband part of the system in their nondominant hand to feel vibrotactile patterns. Meanwhile, participants needed to walk within a range rather than standing still. When the participants felt vibration, they were required to speak out which pattern they felt. Then, the assistant chose the answer and performed subsequent interactions through the interactive interface.
Before the formal experiment started, the participants and the assistant were asked to get familiar with the whole process practiced until they cooperated well. Each participant repeated twenty trials in a block of the formal experiment which consists of four blocks. The participants could take a short break after each block. In total, an entire experiment took about 30 minutes.
5.2. Results and Discussion for the Outdoors Study
Although the number of participants involved in the experiment is limited, some conclusions can be drawn from the statistical analysis. The results are shown in Figures 5–8 and compared with the results of the laboratory study.
Figure 5 shows all the average error rates below 8% except the circular vibrotactile pattern (its average error rate is 13.5%). The average error rates are statistically significant different in the dynamic state for the five vibration patterns (mean = 7.08, SD = 13.98, F4,235 = 3.86, and ). And after the post hoc Tukey multiple comparisons, it discovered that the statistical difference exists between positional (down) pattern and circular pattern, and total pattern and circular pattern().
The one-way ANOVA test (the significant level is 0.05) suggests that there is a statistically significant difference between the reaction times under dynamic conditions for the four vibration types (mean = 2009.22, SD = 1162.07, F3,956 = 6.29, and ) outside. The post hoc Scheffe multiple comparison revealed that there are statistically significant differences between the two groups: positional pattern and circular pattern (), total pattern, and circular pattern ().
The result of error rates across blocks is different from the experiment 1. Its tendency changes from high to low and then stabilizes. A one-way ANOVA test (the significant level is 0.05) for the error rates against blocks indicates the existence of statistically significant differences (mean = 7.08, SD = 6.75, F3,44 = 3.59, and ). The post hoc Tukey multiple comparison revealed that statistically significant differences exist between block 1 and block 3 (). However, the one-way ANOVA test results of average reaction time against blocks are the same as those of the experiment. There is no statistically significant difference between blocks (mean = 2008.28; SD = 438.94).
The results show that participants can still recognize the five vibrotactile patterns. The recognition accuracy of all patterns is more than 90% except circular pattern (its recognition accuracy is 86.5%). And compared with the results of experiment 1, all the results of experiment 2 are higher except the average error rate result of blocks. Maybe users’ perceptual ability will be disturbed when the outdoors are noisy and they are in a dynamic state. Additionally, there is still a gap between wristband and skin, and it will affect participants to feel motor vibration. These findings prove that users can recognize multiple vibrotactile patterns and the efficacy of our wristband prototype.
6. Implications
This research can be applied to nonvisual interaction environments because the vibrotactile channel suffers least from interference originating in external environments compared with other output channels.
Example 1. Navigation for the blind and warning of obstacles in different directions. Walking outdoors is inconvenient for the blind because they cannot visually obtain information, but different vibrotactile feedback patterns on the wrist can provide them with navigation direction and obstacle warnings.
Example 2. Different vibration prompts for different application messages on mobile phones. People in a complex environment often cannot receive mobile phone messages in a timely manner. After establishing contact with a mobile phone via Bluetooth, wristband devices can vibrate accordingly when a mobile phone receives application messages so that users do not miss the messages and can determine the types of messages received.
7. Conclusions and Future Work
Existing wristband devices do not support various vibration patterns, thus limiting users’ perceptions of information from wristband devices in nonvisual interaction scenarios. To solve this problem, we have developed a vibration feedback-based dancing wristband system that controls multiple vibration motors embedded in a wristband to generate different vibration patterns for multisemantic information transfer in eyes-free scenarios. The experiment we conducted verified that users can identify five kinds of vibration patterns, and the experimental results show that users can successfully recognize all patterns at a rate over 90% in the static state, meanwhile, the rate over 85% in the dynamic state. The following factors may affect the users’ performance of the system: firstly, the material of the wearable wristband that contains four motors is not thin enough for users to feel vibration; secondly, the contact surfaces of motors (left and right sides of the wrist) and wrist are not large enough, so the users are not sensitive to vibration generating from the two sides.
Although the results in this paper proved the effect of our system, much further research is required. We need to do much further work to enhance users’ experience of the wearable wristband according to these possible influencing factors. We can improve users experience through changing the material of wristband or adding other tactile forms to form a mixed tactile pattern. Since the system requires users’ to distinguish between different vibration patterns, we will explore more vibration patterns that can be accurately identified. The participants of our experiments are young people, and we will explore the availability of this wristband prototype system at different ages.
Data Availability
The data used in all the charts of this article are derived from the analysis of the original data. The original data have been uploaded, and the connection is as follows: http://dataremain.oss-cn-beijing.aliyuncs.com/.
Disclosure
Part of the manuscript was accepted by the 37th Chinese Control Conference, but the manuscript submitted this time has been significantly rewritten.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work is supported by the National Natural Science Foundation of China (NSFC) under Grant 61462053. Sincerely thanks to the project for funding. We also gratefully acknowledge all the participants of our studies for their time and effort.