About this Journal Submit a Manuscript Table of Contents
Journal of Robotics
Volume 2010 (2010), Article ID 542360, 14 pages
http://dx.doi.org/10.1155/2010/542360
Research Article

Human Perception Test of Discontinuous Force and a Trial of Skill Transfer Using a Five-Fingered Haptic Interface

Department of Human and Information Systems, Gifu University, 1-1 Yanagido, Gifu 501-1193, Japan

Received 30 October 2009; Accepted 11 March 2010

Academic Editor: Noriyasu Homma

Copyright © 2010 Takahiro Endo et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

In the transferring of expert skills, it takes a great deal of time and effort for beginners to obtain new skills, and it is difficult to teach the skills by using only words. For those reasons, a skill transfer system that uses virtual reality (VR) and a haptic interface technique is very attractive. In this study, we investigated the human perception of fingertip force with respect to the following changes: (1) the spatial change of the presented force, and (2) the change of the time to present the force. Based on the results of the perception experiments, we considered the skill transfer to a person's five fingers by using a five-fingered haptic interface robot.

1. Introduction

In the medical fields, expert skills such as surgical techniques, palpation techniques, and the like are obtained by long-term training, and the skill is normally acquired by the experience of working with actual patients. However, it is difficult for residents and medical students to train directly with actual patients because of a decrease in volunteers willing to cooperate in the training and the risk of medical error. To transfer expert skills from a trainer (senior doctor) to a trainee (medical student), the trainer has to teach the trainee how to move the hands, and how to exert the exact amount of force with the fingertips. It is difficult to teach the accurate data of position and force by using only words. Because of these challenges, a skill transfer system that uses virtual reality (VR) and a haptic interface has been researched aggressively (e.g., see [19] and the references in the survey papers [1012]), and the results of studies indicate that such a system could contribute to the skill of performing real surgery [7] and to learning of real motor skills [8, 9].

A haptic interface allows a user to communicate with a virtual environment, and the user feels realistic force and tactile sensations when touching virtual objects in a virtual environment. Benefits of a skill transfer system that uses VR and a haptic interface include the following: the movement of the trainer’s hand and the operation of the trainer’s force can be recorded, so that accurate information can be transmitted to the trainee using a screen and the haptic interface, training according to the trainee’s skill level can be selected, and the effect of the training can be presented to the user, and training can occur at a remote site via a network terminal, and several trainees can receive training at the same time.

In most skill transfer systems, a single-point haptic interface, which makes single-point contact between the user and a virtual environment, is used. Thus, the presented force is limited to one point, and the skill transfer for multipoint contacts, which is needed for tasks such as palpation and the like, is not targeted. Another limitation is that only movement in the horizontal plane or the vertical plane is considered, and the skill transfer that requires movement in three-dimensional space is not considered. Multipoint interaction allows a user to perform natural actions such as grasping, manipulation, and exploration of virtual objects, and it will dramatically increase the believability of the haptic experience [1315]. In performing activities in our daily lives, we usually use multiple fingers; so it is important to exert force at multiple fingertips to make a sensation highly realistic. Multipoint interaction has been achieved in some cases by combining two haptic devices in parallel, which confines the user to a small workspace [5], or by having haptic interfaces that allow the user to exert force at multiple fingertips of a human hand, but the presented force is only a one-directional force [6]. For example, when the haptic interface generates force by using a wire, the force presented to the operator is only exerted in the direction that the wire pulls. In actual situations, there are huge numbers of tasks that need multiple fingers of contact (with three-directional force). Thus a skill transfer system for multipoint contacts is necessary and important.

To design and develop a skill transfer system in which the operation of the trainer’s force is presented to the trainee, it is necessary to investigate and clarify the human perception of the force presented by the haptic interface. In particular, it is important to consider the transfer method based on the human perception. The force that the human being exerts using the hands can be expressed by the direction vector and the magnitude. Many studies of the human perception of the force magnitude have been reported (see [1622] and the references therein). Although there have been only a few studies about the perception of the force direction, the subject has been researched aggressively in recent years [2326]. To date, however, there has been no study that evaluated the perception of the spatial fingertip force, that is, the fingertip force in three-dimensional space, and the perception of the fingertip force concerning the time variation. The perception ability is likely to alter based on the spatial change of the presented force and the change of the time used to present the force. If we consider the skill transfer based on human perception, we must evaluate these perception abilities while considering spatial variation and time variation. Although we have examined the time needed for a human being to distinguish a force direction [27], there is no published data regarding the effects on human perception of spatial variation and time variation. In this study, we investigated the perception of the fingertip force concerning the space variation and the presentation time variation, and then we improved the skill transfer method for the multifinger use [27] based on the results of our investigation.

The paper is organized as follows. In the next section, the multifingered haptic interface used here and our previous research are summarized. In Section 3, the human perception of the fingertip force is examined, and the skill transfer to a person’s five fingers is developed in Section 4. Then, we consider the simple skill transfer system by using the transfer method and the five-fingered haptic interface, and we describe experimental tests that were carried out to demonstrate the validity of the method. Section 5 contains our conclusions.

2. Five-Fingered Haptic Interface

2.1. Interface Development

The authors have developed multifingered haptic interface robots that are placed opposite a human hand, including HIRO [28], HIRO [29], and HIRO III [30], which is shown in Figure 1. HIRO III can present three-dimensional forces at a human operator’s five fingertips. The specifications of HIRO III are shown in Table 1. HIRO III can be briefly summarized as follows.

tab1
Table 1: Specifications of HIRO III.
542360.fig.001
Figure 1: Five-fingered haptic interface robot: HIRO III. An operator connects his/her five fingertips to HIRO III through passive spherical permanent magnet joints.

HIRO III consists of an interface arm and a five-fingered haptic hand. The interface arm is a PUMA-type robot arm consisting of an upper arm (humerus), a lower arm (forearm), and a wrist. The interface arm has 3 degrees of freedom (DOF) at the arm joint and 3 DOF at the wrist joint. The interface arm, therefore, has 6 joints allowing 6 DOF. On the other hand, the haptic hand is constructed of five haptic fingers. Each haptic finger has 3 joints, allowing 3 DOF. The total DOF of HIRO III is 21, and its work space covers VR manipulation on the space of a desktop. Furthermore, a 3-axis force sensor is installed at the top of each finger. To manipulate HIRO III, the operator has to wear a finger holder on his/her fingertips. Figure 2 shows the finger holder and its connection to the haptic finger of HIRO III. The finger holder has a sphere which, when attached to the permanent magnet at the force sensor tip, forms a passive spherical joint. Its role is to adjust for differences between the human and haptic finger orientations, which means that it is safe to use and involves no oppressive feeling for the user. HIRO III allows object manipulation in VR with high realistic sensation. For more details, please see [30].

fig2
Figure 2: Finger holder and its connection. The operator wears the finger holder at his/her fingertips and connects to HIRO III as shown in (b).
2.2. Our Previous Skill Transfer System Using Hiro II+

In the skill transfer system considered in this paper, the trainer’s work is recorded, and it is reproduced in the VR space. The trainee is then trained to imitate the trainer’s work. The trainee’s goal is to make his/her fingertip positions and forces track the trainer’s positions and forces, respectively. We have proposed cues to assist so that the trainee may efficiently acquire the position information and the force information [31]. Figure 3 shows the visual cues. In this figure, the grasping of a ball is considered as the task of the trainer. In the VR space, a yellow ball is the grasped object. The fingertip positions of the trainer and trainee are shown as small circles in VR space, and the fingertip forces are shown as a tetrahedron (see the right side of Figure 3), which is called a force gauge. The height of the tetrahedron expresses the magnitude of the force, and its direction expresses the direction of the force. That is, we tried to display the position information, force magnitude information, and force direction information on the VR display. However, it turned out that it was extremely difficult for the trainee to see the trainer’s five-finger position information and force information at the same time, and to control his/her finger positions and forces so that his/her finger positions and forces matched the trainer’s finger positions and forces.

542360.fig.003
Figure 3: Visual cues.

In our earlier research, we tried to transfer the trainer’s force information and position information to the trainee by using only visual information. However, we obtained feedback that it was difficult for the trainee to learn the trainer’s force information and position information on the five fingertips at the same time. So, in this paper, we attempted to transmit the trainer’s force information to the trainee by using visuohaptic information, namely, by using not only visual cues but also haptic cues.

3. Measurement of the Perception of Fingertip Force

It is useful to measure human perception ability when we consider the transmission of force. In particular, it is important to know how human beings accurately perceive fingertip force by the haptic interface. The results of investigating the human perception of fingertip force form the foundation of skill transfer. First, we measured the human perception ability with regard to the direction of force by using HIRO III. Here note that all three experiments in this section can be done with a single-point haptic interface. However, in Section 4, we describe how we developed the skill transfer system by using HIRO III and VR technique. To develop the skill transfer system based on the measurement results of human perception, we needed to use HIRO III in the experiments.

3.1. Measurement of the Human Perception of Force Direction

We examined the perception of the force direction with regard to spatial variation and time variation.

3.1.1. Experimental Setup

Ten people in their twenties (nine males and one female) participated in this measurement. All of the participants were right-handed. The participants connected their index finger to the HIRO III at a bar, as shown in Figure 4(a), and HIRO III presented the force to the participants. Note that we used the hand part of HIRO III in this experiment, and we did not use the arm of HIRO III. In the experiment, the hand part of HIRO III was fixed to the board as shown in Figure 4(a). During the measurement, a cloth covered both HIRO III and the participant's hands; so sight information was not available, as shown in Figure 4(c). The participants responded to the direction of the presented force by using the measuring instrument shown in Figure 4(b). As a measuring instrument, we used a goniometer. One axis of the goniometer was fixed, and the participants responded by using the other axis of the goniometer to indicate the direction of the presented force.

fig4
Figure 4: Measurement environment.

To measure the human perception of the force direction with regard to spatial variation, we considered the following two types of measurement.() We consider the measurement of the human perception of force in the horizontal direction ( plane in Figure 4(d)). In this case, we set the force (N) to show on human index finger as follows: where is the angle in the horizontal plane, as shown in Figure 5(a), and denotes the magnitude of the force.() We consider the measurement of the human perception of force in the vertical direction ( plane in Figure 4(d)). In this case, we set the force (N) to show on human index finger as follows: where is the angle in the vertical plane, as shown in Figure 5(b).

fig5
Figure 5: Presented force.

In measurements (M1) and (M2), the angles and are divided every radians, and 13 kinds of forces are presented to the participants in random order. We set =1.5 (N). In each measurement, we considered the following three conditions for the time required to present the force, to consider the perception of the force direction concerning the time variation (see Figure 6).(a)The force was presented until the participant answered.(b)The cycle in which the force was presented for 0.5 (s) and the force was not presented for 0.5 (s) was repeated until the participant answered. That is, we set  (s),  (s) in Figure 6.(c)The cycle in which the force was presented for 0.2 (s) and the force was not presented for 0.8 (s) was repeated until the participant answered. That is, we set  (s),  (s) in Figure 6.

542360.fig.006
Figure 6: Form of the presented force.

Ten participants carried out the measurement under conditions (a), (b), and (c) for measurements (M1) and (M2). That is, we considered six measurements as shown in Table 2. All participants carried out the six experiments. In particular, to circumvent the effect of the sequence of measurement, the sequence of each participant’s measurement was decided in random order. In each measurement, HIRO III showed the force to the participant after an operator of HIRO III gave the signal to start, and the participant felt the force. After the participant recognized the force direction, he/she signaled the operator of HIRO III and the operator stopped the presentation of the force. The participant then responded to the direction of the presented force by using the measuring instrument. To evaluate the measurement, we noted the angular error between the presented force direction by HIRO III and the answered angle by using a measuring instrument. We note that HIRO III cannot present the accurate force when the presented time of the force is shorter than 0.2 (s). Thus we set the minimum presented time at 0.2 (s) in this experiment. As an example, we show the step response of HIRO III in Figure 7(a). In the measurement of the step response, HIRO III was connected to the wall as shown in Figure 7(b), and we measured the step response when HIROIII pressed the wall straight.

tab2
Table 2: Measurement of the human perception of force direction.
fig7
Figure 7: Performance of HIRO III (Step response).
3.1.2. Experimental Results

Figures 8(a) and 8(b) show the measurement results in the horizontal direction (M1) and the vertical direction (M2), respectively. In each figure, the horizontal axis shows the experimental condition and the vertical axis shows the average value of the angular error between the presented force direction and the answered force direction. That is, we show the value , where is the angle of the force that presented the jth time to the ith participant in the corresponding condition, and is the angle that the ith participant answered on the jth time. The vertical bar shows the standard variation of the corresponding value. From the experimental results, we find that human perception ability regarding the force direction had an average error of 0.23 (rad) in the horizontal direction and 0.25 (rad) in the vertical direction, in the case of condition (a). In other words, there is no big difference between the perception ability of the force in the horizontal plane and in the vertical plane. On the other hand, the average values of conditions (b) and (c) in the horizontal direction were slightly large in contrast with the value of (a), and the average values of (b) and (c) in the vertical direction were slightly small in contrast with the value of (a). However, according to the analysis of variance (ANOVA) with a 5% significance level, there is no significant difference between the conditions (a) (namely, continuous force) and (b)-(c) (namely, discontinuous force).

fig8
Figure 8: Measurement results of the angular error.

Figure 9 shows the average value of the angular error at each presented angle. Figures 9(a) and 9(b) show the results in the horizontal direction (M1) and the vertical direction (M2), respectively. In both figures, the top figure is the result of (a), the middle figure is the result of (b), and the bottom figure is the result of (c). From Figure 9(a), we see that the angular error when the presented force is  (rad) is the smallest, and it grows as the presented angle approaches 0 and  (rad). This is true in all three conditions. Further, from Figure 9(b), in the vertical direction, the same tendency as the horizontal direction can be perceived. Here note that this tendency, anisotropy, was also described for the perception of the human hand [25, 26]. In [25], the participants held a joystick, and the perception ability concerning the direction of the human hand (not the individual finger) was investigated. The anisotropy of the human hand perception for the direction was shown. In [26], the perception ability concerning the force magnitude of the human hand (not the individual finger) was found to be anisotropic. Based on the above results, we concluded the following: the human perception ability of the force direction has 0.23 (rad) error in the horizontal direction and 0.25 (rad) error in the vertical direction, and whether the presented force is continuous or discontinuous has no influence on the human perception.

fig9
Figure 9: Measurement results of the angular error at each presented force direction.

We wondered why the angular error increased around 0 and  (rad) of the presented force direction. To investigate whether this was an influence of the performance of the haptic interface, we connected HIRO III with a wall, as shown in Figure 7(b), and then HIRO III presented force as expressed in (1) and (2) with =1.5 (N) against the wall. Figure 10 shows the angular error. Figures 10(a) and 10(b) show the results in the horizontal direction and the vertical direction, respectively. From these results, we conclude that the angular error was not caused by the presented force direction and that the angular errors occurred in all the presented force directions were very small. Thus it is not easy to conclude that the above-mentioned phenomenon has happened because of the haptic interface.

fig10
Figure 10: Performance of HIRO III (angular error of the presented force).
3.2. Measurement of the Human Perception of Force Magnitude

We considered the perception of force magnitude with regard to spatial variation.

3.2.1. Experimental Setup

Ten people in their twenties (nine males and one female) participated in this measurement. All of the participants were right-handed. The measurement environment was the same as that described in Section 3.1. Here, we measured the point of subjective equality (PSE) of the force magnitude, where the compared forces are presented in more than one direction. As in the experiment of Section 3.1, the participant connected his/her index finger to HIRO III. Then, the following three terms were carried out under conditions (M1) (horizontal direction) and (M2) (vertical direction). (1)By using HIRO III, the standard stimulus force was presented to the participant, where the standard stimulus force was presented in the reference direction.(2)By using HIRO III, the comparison stimulus force was presented to the participant, where the comparison stimulus force was presented in a comparison direction.(3)The participant selected one answer from the following three answers: (i) the comparison stimulus force was larger than the standard stimulus force, (ii) the comparison stimulus force was the same as the standard stimulus force, or was not distinguishable, and (iii) the comparison stimulus force was smaller than the standard stimulus force.

The force was presented until the participant answered. The reference directions of (M1) and (M2) were  (rad), and the comparison direction with respect to the reference direction was in the following five directions: , , , ,  (rad). Further, we set the standard stimulus force at 2.2 (N), and we consider the following 11 comparison stimulus forces: 1.0, 1.2, 1.4, 1.6, 1.8, 2.0, 2.2, 2.4, 2.6, 2.8, 3.0 (N). Before the experiment, we conducted a preliminary experiment several times. From the preliminary experimental result, we found there were comparison directions for which we could not obtain the PSE when the standard stimulus force was set smaller than 2.2 (N). Thus, to obtain the PSE in all comparison directions (, , , ,  (rad)), we set the standard stimulus force to 2.2 (N). For each participant, all the comparison stimulus forces were presented in the comparison directions, where the order of the presentation of the comparison stimulus force was random, and the comparison direction was selected from the above five directions before the measurement. After the measurement, we selected another comparison direction and then conducted measurements –(3). The number of times that the participant answered “the comparison stimulus force is the same as the standard stimulus force, or is not distinguishable” was halved and added to the number of times that the participant answered “the comparison stimulus force is larger than the standard stimulus force” and the number of times that the participant answered “the comparison stimulus force is smaller than the standard stimulus force”. We derived the ratio of the number of times that the participant answered “the comparison stimulus force is larger than the standard stimulus force”. By using the least-squares method, we derived the approximate curve of , and we set the PSE as the force when is 50%.

3.2.2. Experimental Results

Figures 11(a) and 11(b) show the value of the PSE in the horizontal direction and the vertical direction, respectively. In the figure, the horizontal axis is the direction of the presented force, and the vertical axis is the value of the PSE. From the figure, we see that the same tendency was obtained in both directions, indicating that the participant correctly recognized the magnitude of the presented force at all directions of the presented force. Further, we could not obtain the phenomenon of Section 3.1.2: the angular error increased when the presented force tends to 0 and π (rad). For reader’s reference, we show the ratio that the participants answered “the comparison stimulus force is larger than the standard stimulus force” in Figures 11(c) and 11(d). Note that the experiment of Sections 3.1.2 and 3.2.2 considered human perception ability under the condition that the direction of the presented force was unknown or the direction of the presented force (comparison force) differed from the standard stimulus force. Thus, as a difference perception, we are attracted to the perception ability when the direction of the presented force is known (or the direction of the comparison stimulus force is the same as the standard stimulus force). To examine this, we next measured the perception ability when the direction of the presented force was well known.

fig11
Figure 11: Measurement results of the PSE at each direction of the presented force.
3.3. Measurement of the Human Perception of Force Magnitude When the Direction Is Well Known

We measured the human perception ability with a force presented in a well-known direction. In Section 3.2, we considered the PSE under the condition that the direction of the comparison stimulus force differed from the direction of the standard stimulus force. To examine the perception of the force in a well-known direction, we measure the PSE with the condition that the direction of the comparison stimulus force was the same as the direction of the standard stimulus force.

3.3.1. Experimental Setup

Ten people in their twenties (nine males and one female) participated in this measurement. All of the participants were right-handed. The measurement environment was the same as that described in Section 3.1. As in the experiment of Section 3.1, the participant connected his/her index finger to HIRO III. Then, the following three terms were carried out under conditions M1 (horizontal direction) and M2 (vertical direction). (1)By using HIRO III, the standard stimulus force was presented to the participant.(2)By using HIRO III, the comparison stimulus force was presented to the participant.(3)The participant selected one answer from the following three answers: (i) the comparison stimulus force was larger than the standard stimulus force, (ii) the comparison stimulus force was the same as the standard stimulus force, or was not distinguishable, and (iii) the comparison stimulus force was smaller than the standard stimulus force.

The force was presented until the participant answered. In this experiment, for M1 and M2, we measured the PSE in the following five directions: , , , ,  (rad) and , , , ,  (rad), respectively. Further, we set the standard stimulus force to 2.2 (N), and we considered the following 11 comparison stimulus forces: 1.0, 1.2, 1.4, 1.6, 1.8, 2.0, 2.2, 2.4, 2.6, 2.8, 3.0 (N). For each participant, the comparison stimulus forces were presented in random order. The difference between the measurements of Sections 3.2 and 3.3 was that in the measurement of Section 3.2, the direction of the comparison force differed from the direction of the standard stimulus force, while in Section 3.3, the direction of the comparison force was the same as the direction of the standard stimulus force.

3.3.2. Experimental Results

Figures 12(a) and 12(b) show the value of PSE in the horizontal direction and the vertical direction, respectively. In these figures, the horizontal axis shows the direction in which the PSE was measured, and the vertical axis shows the PSE value. From these figures, we see that the same tendency was obtained in both directions. Furthermore, there was no difference between the PSE values with respect to the direction of the presented force. For reader’s reference, we show the ratio that the participants answered “the comparison stimulus force is larger than the standard stimulus force” in Figures 12(c) and 12(d).

fig12
Figure 12: Measurement results of the PSE.

From the results of this section, it seems that human ability to perceive the force direction is anisotropic and the ability to perceive the force magnitude is isotropic. Further, it seems the following: compared with the case that the human answers the force magnitude, he/she makes the ambiguous answer when he/she answers the force direction.

4. Skill Transfer System and Its Experimental Evaluation

4.1. Transfer Method of Force and Position Information

For the skill transfer examined in this study, the aim of the trainee is to make his/her five-fingertip positions and forces track a trainer’s five-fingertip positions and forces, respectively.

For fingertip positions tracking, we used the visual cues shown in Figure 13. The five-fingertip positions of the trainer and trainee are shown as small circle in VR space, and the trainee controlled his/her fingertip positions to track the trainer’s positions.

542360.fig.0013
Figure 13: Visual cues for position tracking.

For the fingertip forces tracking, we considered the transfer method of force based on the measurement of the human perception as described in the previous section. In the expert skill transfer system, there are two kinds of forces transferred to the user, as follows:the reaction force from the virtual object (referred to herein as the reaction force),the force that a trainer exerts on an object. (When we present this force to the user, we consider the force in the opposite direction, i.e., . In the following, the force is called the trainer’s force, .)

As the force transfer method, the reaction force, , and the trainer’s force, , were presented to the user and were switched over time, as shown in Figure 14. In particular, if the time to show the trainer’s force was long, the user could not feel the reaction force; so, it was necessary to shorten the time of presenting as much as possible. Therefore, by considering the results in Section 3, we set HIRO III to show to the user for 0.5 seconds and then to show to the user for 0.2 seconds and then repeat the process. In practice, if the force such as that shown in Figure 14 was presented to the user, the user would feel the pulse force that is the difference between the trainer’s force and the trainee’s force. Thus if the user regulates his/her fingertip forces so that the pulse forces become small, the force transfer is achieved. That is, if the user grasps the virtual object by using the same force as the trainer, the pulse forces disappear. This method has the following advantages: even if the force changes periodically, both force and can be recognized; the number of visual cues for the skill transfer decreases in contrast with the method described in Section 2.2. In the next section, we consider an experiment to evaluate this method.

542360.fig.0014
Figure 14: Presentation of two kinds of forces, and .
4.2. Skill Transfer System and Its Experimental Evaluation

Let us evaluate the skill transfer system based on the results reported in the previous subsection. As the task of a trainer, we consider the grasping of an object in VR space. The goal of the trainee is to assume a fingertip position and exert a grasping force that agree with the fingertip position and grasping force of the trainer, respectively. Figure 15(a) shows the experimental environment. In this figure, the black box is the display system used to present the image at around the fingertips [32]. By using this display, the trainee sees the screen as shown in Figure 15(b).

fig15
Figure 15: Experimental environment of skill transfer.
4.2.1. Experimental Setup

For the comparison, we consider the following two kinds of skill transfer method.(P1) The fingertip positions of the trainer and the trainee are shown as small circles, and the fingertip forces of the trainer and trainee are shown as the force gauge (described in Section 2.2) in VR space.(P2) The fingertip positions of the trainer and the trainee are shown as small circles, and the trainer’s fingertip forces are presented to the trainee by using the method described in Section 4.1.

As the task of the trainer, we considered the following procedure: the trainer approached the virtual object, and the trainer grasped the object. The stiffness and the damping coefficient of the object are 570 (N/m) and 2 (Ns/m), respectively. Before we carried out our experiment, the person who acted as the trainer performed the task. This person was not included among the eight participants in the experiment described below. After we obtained the force and position trajectories of the trainer’s fingertips, we set the trajectories as the trainer’s trajectories. For example, the force trajectory of the trainer’s middle finger and the position trajectory of the trainer’s middle finger are shown in Figures 16(a) and 16(b), respectively. We used the coordinates in Figure 15(c).

fig16
Figure 16: The force and position trajectory of the trainer.

To evaluate the above two methods, we prepared two experiments (E1 and E2). Before we performed the experiments, all participants familiarized themselves with manipulating HIRO III.

() The participant confirmed the trainer’s trajectory. That is, the fingertip positions of the trainer were shown as small circles graphically in VR space, and the participant saw and confirmed the trajectories of the trainer’s fingertips. Then, the fingertip positions of the trainer and the trainee were shown as small circles graphically in VR space, and the trainee carried out the task based on this visual information only, while we measured the initial errors. The trainee carried out the task 20 times continuously under method P1. Finally the trainee carried out the task under the condition that the fingertip positions of the trainer and the trainee were shown as small circles graphically in VR space, and we measured the error after training.() The trainee carried out the same experiment as E1, with P2 in place of P1.

Eight people in their twenties (seven males and one female) participated in this measurement, and we divided the subjects into two groups, group A and group B. So there would not be an effect caused by the sequence of experiments; the subjects in group A carried out experiment E1 first and then carried out experiment E2. The subjects in group B carried out experiment E2 first and then carried out experiment E1. Table 3 shows the sequence of experiments.

tab3
Table 3: The sequence of experiments E1 and E2.
4.2.2. Experimental Results

Figures 17(a), 17(b), and 17(c) show the experimental results. In (a), the vertical axis shows the average value of the position error between the trainer’s position and the trainee’s position. The trainer’s work lasted 15 (s), and the sampling time of the PC was 1 (ms). The horizontal axis shows the experimental condition. Figure 17(b) shows the average value of the error of the force magnitude, and Figure 17(c) shows the average value of the directional error of the force. When we use the polar coordinate, we can express the force by using two variables, and , for example, , , and . As the error of the force direction, we considered the following value: , where and are the trainer’s angle variables at i (ms), and and are the jth participant’s angle variables at i (ms). In each figure, the vertical bar shows the standard variation of the corresponding value.

fig17
Figure 17: Experimental results.

According to the -test with a 1% significance level, there are significant differences between the errors before training P2 and the errors after the training P2. On the other hand, in the error of the force magnitude in P1, there is no difference between the errors before training and the errors after the training (even if we consider the -test with a 5% significance level, there was no difference). Regarding the participants’ opinions, we obtained the following. In P1, there were a lot of visual cues and the users were confused. The users can track his/her one or two positions and one or two forces to the corresponding trainer’s positions and forces, but they were confused when they consider the five positions and forces. In contrast with this, cues were intuitive and comprehensible in P2. Many participants said that even if the position and the magnitude of the force were understood, the difference in the direction of the force was not understood. However, on the result of Figure 17(c), there are significant difference between the errors before training (P1 and P2) and the errors after the training (P1 and P2). We have considered that this is because of the simple task. In this experiment, we considered the simple task that the user just pushes the object, thus eliminating variations in the force direction while the user performs the task. In fact, the variation of z-axis force of the trainer was large, but the variations of x- and y-axis force were small, as shown in Figure 16(a). Therefore, it seems that there were differences between the errors before training and the errors after the training even if we used method P1. We cannot draw any certain conclusions because of the small sample size. However, from Figure 17(b) and the subjects’ opinions, it seems to be more effective to present the trainer’s force periodically rather than using only visual cues. Further, it seems that we need another assistance cue when we consider the training of the direction of the force.

5. Conclusions

In this study, we investigated the human perception of force with respect to the following changes when a user engaged with the five-fingered haptic interface HIRO III: the spatial change of the presented force, and the change of the time to present the force. Since the force can be expressed by the direction vector and the magnitude, we first measured the perception of the force direction, as described in Section 3.1. From Figure 9(a), we see that the angular error when the presented force direction is  (rad) is the smallest and it grows as the presented force direction approaches 0 and  (rad) in the horizontal direction. This is true in the vertical direction as well. However, several parts of Figure 9 show that the angular error when the presented force direction is 0 or  (rad) is smaller than the angular error when the presented force direction is or  (rad). We believe that this was due to the measuring instrument we employed. We used a goniometer as the measuring instrument, and its measuring range is 0~π (rad). Therefore, even if the participant feels an angle larger than  (rad) or an angle smaller than 0 (rad), he/she answered  (rad) or 0 (rad).

From Figure 8, which shows the measurement results of the average angular error, we see that the human perception ability regarding the force direction had an average error of 0.23 (rad) in the horizontal direction and 0.25 (rad) in the vertical direction. In addition, we determined that there is no difference in human perception of force direction between when the presented force is continuous and when the presented force is discontinuous, based on ANOVA with a 5% significance level.

Next, we measured the perception of magnitude considering the force direction. In particular, in Section 3.2 we described the measurement when the direction of the comparison stimulus force differed from the direction of the standard stimulus force (for simplicity, we called the direction “unknown”), and in Section 3.3, we described the measurement when the direction of the comparison stimulus force is the same as the direction of the standard stimulus force (for simplicity, we called the direction “well known”). From Figures 11 and 12, we see that there was no change caused by the presented force direction on the value of PSE. This differed from the case of perception of the force direction. In particular, by using ANOVA with a 5% significant level, we determined that there was no difference between the following four groups: PSE in the horizontal direction when the direction is unknown, PSE in the vertical direction when the direction is unknown, PSE in the horizontal direction when the direction is well known, and PSE in the vertical direction when the direction is well known. Therefore, in both the horizontal and vertical directions, there was no difference in the feeling of magnitude caused by the difference of the presented force direction, and we conclude that humans perceive magnitude, regardless of the presented force direction.

These results of perception measurements showed that the human perception of the fingertip force direction is anisotropic and perception of the fingertip force magnitude is isotropic. Furthermore, regarding the perception of the fingertip force direction, there is no difference between the perception of the continuously presented fingertip force and the discontinuous presentation.

We also considered the transfer method based on our measurement results by using the multi-fingered haptic interface HIRO III. For skill transfer, the reaction force and the trainer’s force were transferred to the user. The proposed skill transfer system consists of the following two parts: for the fingertip position tracking, we used the visual cue shown in Figure 13, and for the fingertip force tracking, HIRO III alternately showed and . This method has the following advantages: even if the force changes periodically, both force and can be recognized, and the number of visual cues decreases, in contrast with the method described in Section 2.2.

We performed tests to demonstrate the validity of the proposed method. According to the -test with a 5% significance level, in the proposed method, there was a difference between the error before training and the error after training. Therefore, we believe that the transfer of the force was achieved more efficiently by presenting the force intermittently, as described in Section 4.1.

The next problem to be tackled is to increase the number of experimental subjects (in particular, the number of female subjects) and to confirm how the results change at that time. We must also extend this research to a more complicated skill transfer system, such as the human body model in VR space. Further, we will attempt to develop an efficient transfer method for transmitting the direction of force.

Acknowledgment

This paper was supported by the Ministry of Internal Affairs and Communication R & D Promotion Programme (SCOPE).

References

  1. Y. Yokokohji, R. L. Hollis, T. Kanade, K. Henmi, and T. Yoshikawa, “Toward machine mediated training of motor skills-skill transfer from human to human via virtual environment,” in Proceedings of the IEEE International Workshop on Robot and Human Communication, pp. 32–37, Tsukub, Japan, 1996. View at Publisher · View at Google Scholar
  2. R. Kikuuwe and T. Yoshikawa, “Haptic display device with fingertip presser for motion/force teaching to human,” in Proceedings of the EEE International Conference on Robotics and Automation, vol. 1, pp. 868–873, Seoul, South Korea, 2001.
  3. S. Saga, N. Kawakami, and S. Tachi, “Haptic teaching using opposite force presentation,” in Proceedings of the 1st Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC '05), vol. 1, Pisa, Italy, 2005, (CD-ROM).
  4. G. Srimathveeravalli and K. Thenkurussi, “Skill training assistance using haptic attributes,” in Proceedings of the 1st Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC '05), vol. 1, pp. 209–214, Pisa, Italy, 2005.
  5. M. Nakao, K. Minato, T. Kuroda, M. Komori, H. Oyama, and T. Takahashi, “Transferring bioelasticity knowledge through haptic interaction,” IEEE Multimedia, vol. 13, no. 3, pp. 50–60, 2006. View at Publisher · View at Google Scholar
  6. Y. Kuroda, M. Hirai, M. Nakao, et al., “Construction of training environment for surgical exclusion with a basic study of multi-finger haptic interaction,” in Proceedings of the 2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC '07), pp. 525–530, Tsukuba, Japan, March 2007. View at Publisher · View at Google Scholar
  7. N. E. Seymour, A. G. Gallagher, S. A. Roman, et al., “Virtual reality training improves operating room performance: results of a randomized, double-blinded study,” Annals of Surgery, vol. 236, no. 4, pp. 458–464, 2002. View at Publisher · View at Google Scholar
  8. P. L. Youngblood, S. Srivastava, M. Curet, W. L. Heinrichs, P. Dev, and S. M. Wren, “Comparison of training on two laparoscopic simulators and assessment of skills transfer to surgical performance,” Journal of the American College of Surgeons, vol. 200, no. 4, pp. 546–551, 2005. View at Publisher · View at Google Scholar · View at PubMed
  9. D. Morris, T. Hong, F. Barbagli, T. Chang, and K. Salisbury, “Haptic feedback enhances force skill learning,” in Proceedings of the 2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC '07), pp. 21–26, Tsukuba, Japan, March 2007. View at Publisher · View at Google Scholar
  10. V. Hayward, O. R. Astley, M. Cruz-Hernandez, D. Grant, and G. Robles-De-La-Torre, “Haptic interfaces and devices,” Sensor Review, vol. 24, no. 1, pp. 16–29, 2004.
  11. A. El Saddik, “The potential of haptics technologies,” IEEE Instrumentation & Measurement Magazine, vol. 10, no. 1, pp. 10–17, 2007. View at Publisher · View at Google Scholar
  12. K. E. Maclean and V. Hayward, “Do it yourself haptics: part II,” IEEE Robotics & Automation Magazine, vol. 15, no. 1, pp. 104–119, 2007.
  13. N. Magnenat-Thalmann and U. Bonanni, “Haptics in virtual reality and multimedia,” IEEE Multimedia, vol. 13, no. 3, pp. 6–11, 2006. View at Publisher · View at Google Scholar
  14. F. Barbagli, K. Salisbury, and R. Devengenzo, “Toward virtual manipulation: from one point of contact to four,” Sensor Review, vol. 24, no. 1, pp. 51–59, 2004.
  15. Z. Najdovski and S. Nahavandi, “Extending haptic device capability for 3D virtual grasping,” in Procedings of the 6th International Conference on Haptics: Perception, Devices and Scenarios (EuroHaptics '08), vol. 5024 of Lecture Notes in Computer Science, pp. 494–503, Madrid, Spain, June 2008. View at Publisher · View at Google Scholar
  16. L. A. Jones, “Perception of force and weight: theory and research,” Psychological Bulletin, vol. 100, no. 1, pp. 29–42, 1986. View at Publisher · View at Google Scholar
  17. X. D. Pang, H. Z. Tan, and N. I. Durlach, “Manual discrimination of force using active finger motion,” Perception & Psychophysics, vol. 49, no. 6, pp. 531–540, 1991.
  18. H. Z. Tan, M. A. Srinivasan, B. Eberman, and B. Cheng, “Human factors for the design of force-reflecting haptic interfaces,” in Proceedings of the 3rd International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, vol. 55–1, pp. 353–359, ASME Dynamic Systems and Control Division, New York, NY, USA, 1994.
  19. S. Allin, Y. Matsuoka, and R. Klatzky, “Measuring just noticeable differences for haptic force feedback: implications for rehabilitation,” in Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment & Teleoperator Systems, pp. 299–303, Orlando, Fla , USA, 2002.
  20. S. Yamakawa, H. Fujimoto, S. Manabe, and Y. Kobayashi, “The necessary conditions of the scaling ratio in master-slave systems based on human difference limen of force sense,” IEEE Transactions on Systems, Man, and Cybernetics, Part A, vol. 35, no. 2, pp. 275–282, 2005. View at Publisher · View at Google Scholar
  21. S. Li, “Perception of individual finger forces during multi-finger force production tasks,” Neuroscience Letters, vol. 4009, no. 3, pp. 239–243, 2006. View at Publisher · View at Google Scholar · View at PubMed
  22. S. A. Cholewiak, H. Z. Tan, and D. S. Ebert, “Haptic identification of stiffness and force magnitude,” in Proceedings of the Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems, pp. 87–91, Reno, Nev, USA, 2008. View at Publisher · View at Google Scholar
  23. F. Barbagli, K. Salisbury, and H. Z. Tan, “Haptic discrimination of force direction and the influence of visual information,” ACM Transactions on Applied Perception, vol. 3, no. 2, pp. 125–135, 2006.
  24. H. Z. Tan, F. Barbagli, K. Salisbury, C. Ho, and C. Spence, “Force-direction is not influenced by reference force direction,” Haptics-e, vol. 4, no. 1, 6 pages, 2006.
  25. I. Elhajj, H. Weerasinghe, A. Dika, and R. Hansen, “Human perception of haptic force direction,” in Proceedings of the IEEE International Conference on Intelligent Robots and Systems, pp. 989–993, Beijing, China, 2006. View at Publisher · View at Google Scholar
  26. E. Dorjgotov, G. R. Bertoline, L. Arns, Z. Pizlo, and S. R. Dunlop, “Force amplitude perception in six orthogonal directions,” in Proceedings of the Symposium on Haptics Interfaces for Virtual Environment & Teleoperator Systems, pp. 121–127, 2008. View at Publisher · View at Google Scholar
  27. T. Endo, H. Kawasaki, K. Kigaku, and T. Mouri, “Transfer method of force information using five-fingered haptic interface robot,” in Proceedings of the 2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC '07), pp. 599–600, Tsukuba, Japan, March 2007. View at Publisher · View at Google Scholar
  28. H. Kawasaki, J. Takai, Y. Tanaka, C. Mrad, and T. Mouri, “Control of multi-fingered haptic interface opposite to human hand,” in Proceedings of the IEEE International Conference on Intelligent Robots and Systems(IROS '03), vol. 3, pp. 2707–2712, Las Vegas, Nev, USA, 2003.
  29. H. Kawasaki and T. Mouri, “Design and control of five-fingered haptic interface opposite to human hand,” IEEE Transactions on Robotics, vol. 23, no. 5, pp. 909–918, 2007. View at Publisher · View at Google Scholar
  30. T. Endo, H. Kawasaki, T. Mouri, et al., “Five-fingered haptic interface robot: HIRO III,” in Proceedingsof the 3rd Joint EuroHaptics conference, and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC '09), pp. 458–463, Salt Lake City, Utah, USA, 2009. View at Publisher · View at Google Scholar
  31. K. Kigaku, H. Kawasaki, M. O. Alhalabi, and T. Mouri, “Research on skill transmission using multi-fingered haptic interface robot,” in Proceedings of the 10th Virtual Reality Society of Japan Annual Conference, Tokyo, Japan, 2005, (CD-ROM).
  32. Y. Yokokohji, R. L. Hollis, and T. Kanade, “WYSIWYF display: a visual/haptic interface to virtual environment,” Presence, vol. 8, no. 4, pp. 412–434, 1999.