Two-Fingered Haptic Device for Robot Hand Teleoperation

Kobayashi, Futoshi; Ikai, George; Fukui, Wataru; Kojima, Fumio

doi:https://doi.org/10.1155/2011/419465

Journal of Robotics

On this page

Abstract Introduction Conclusions References Copyright Related Articles

Special Issue

Rehabilitation Robotics

View this Special Issue

Research Article | Open Access

Volume 2011 | Article ID 419465 | https://doi.org/10.1155/2011/419465

Two-Fingered Haptic Device for Robot Hand Teleoperation

Futoshi Kobayashi,¹George Ikai,¹Wataru Fukui,¹and Fumio Kojima¹

Academic Editor: Tetsuya Mouri

Received31 May 2011

Revised23 Aug 2011

Accepted27 Sept 2011

Published15 Dec 2011

Abstract

A haptic feedback system is required to assist telerehabilitation with robot hand. The system should provide the reaction force measured in the robot hand to an operator. In this paper, we have developed a force feedback device that presents a reaction force to the distal segment of the operator's thumb, middle finger, and basipodite of the middle finger when the robot hand grasps an object. The device uses a shape memory alloy as an actuator, which affords a very compact, lightweight, and accurate device.

1. Introduction

Various humanoid robot hands have been developed so far. The Utah/MIT dexterous hand has four fingers with four joints driven by tendon cables and tactile sensors over the entire surface [1, 2]. The Gifu hand has five fingers and 20 joints with 16 degrees of freedom (DOF) [3], and the KH hand type S has five fingers and 20 joints with 15 DOF [4]. More recently manufactured, robot hands incorporate multiaxis/force torque sensors and tactile sensors with conductive ink and are relatively lightweight. The TWENDY-ONE hand has four fingers and 16 joints with 13 DOF [5]. This robot hand is equipped with the six-axis force sensors and array-type tactile sensors. Honda Motor Co., Ltd., has developed a multifingered robot hand, which has five fingers and 20 joints with 13 DOF [6]. Each DOF is hydraulically actuated, and the robot hand has tactile sensors on the entire surface. AIST also developed a multifingered robot hand with 4 fingers and 17 joints with 13 DOF that are actuated by an electrical servomotor [7]. The AIST robot hand also employed multi-axis force/torque sensors in the fingertips. Many other robot hands have been developed and researched [8, 9]. We have also reported the universal robot hands I [10] and II [11].

A tele-rehabilitation system receives attention for medical care because of a shortage of rehabilitation therapists [12]. The tele-rehabilitation system with a robot hand allows a rehabilitation therapist to care intuitively. However, typically a tele-operation system is unable to give tactile and haptic information to the human operator because conventional teleoperation systems lack a feedback system. It is difficult to complete tasks or control various operations without dexterous tactile and haptic information; thus human operators make errors because there is no tactile feedback. Therefore, many haptic devices were developed to enable the human operator to feel the force. The haptic devices are classified into three types according to their mechanical grounding configuration [13] including the grounded type [14–19], the nongrounded type [20–24], and the body-grounded type [25, 26]. The grounded haptic device easily provides weight sensation or 3-Dimensional (3D) forces, but is limited because it is fixed to a static object like a table or the floor. The nongrounded haptic device includes an internal ground mechanism. The internal ground acts as counteractive base to create linear or angular momentum, and therefore, the device is easily repositioned because it is mechanically ungrounded. However, the ungrounded device cannot generate a large force in multiple directions or maintain continuous stimuli. The body-grounded haptic device is a wearable. These devices are worn and mounted on the human operator, which is used as a counteractive base or a point to impress a force or torque on the human operator. The wearable haptic devices provide a wide range of motion and force. However, for such devices to be usable, they must be small and lightweight. When a human opens a door or picks up an object, the hand and the arm move simultaneously allowing for a wide working space and range of motion. A human also demands a greater force when a solid object like a glass is grasped. Therefore, the body-grounded haptic device is a preferred in applications involving grasping objects while maintaining feeling using tele-operation because it provides a wide workspace and considerable force in many directions.

In this paper, we have developed a two-fingered body-grounded haptic device called ExoPhalanx. The ExoPhalanx provides force to the distal segment of the human operator’s thumb and middle finger, and the basipodite of the middle finger. Here, the ExoPhalanx presents only one-directional force to the human operator because the haptic device becomes small and lightweight in order to mount on the human hand. The performance of the haptic device was measured using a two-finger grasping tele-operation experiment with the Universal Robot Hand II and the haptic device ExoPhalanx.

The remainder of the paper is arranged as follows. The haptic device is introduced in Section 2. The robot hand tele-operation system with the developed haptic device is described in Section 3. The preliminary experiment showing the basic performance of the SMA clutch brake of the ExoPhalanx and the results of the grasping experiment with two fingers are presented in Section 4. Finally, the conclusions of this study are presented in Section 5.

2. Haptic Device

The developed haptic device ExoPhalanx is shown in Figure 1. The ExoPhalanx is used as a passive force feedback device. This device is wearable and mounted above the motion capture data glove CyberGlove using a tightening belt. Here, the body-grounded haptic device can create the illusion of directed forces by adequate positioning and distributing of grounding forces [27]. The ExoPhalanx consists of a thumb exoskeleton, a middle finger exoskeleton, and a manifer exoskeleton. The thumb exoskeleton consists of a distal phalanx and a proximal phalange ring. The middle finger exoskeleton consists of a distal phalanx, a middle phalange ring, and a proximal phalange ring. The proximal phalange ring of the thumb exoskeleton and the middle finger exoskeleton are belted on the proximal phalanx of the operator’s finger. The manifer exoskeleton is belted on the operator’s palm and forearm. This design takes advantage of the fact that fingertip mechanoreceptors are significantly more sensitive than those of the proximal phalanx, the forearm and the palm [28].

2.1. SMA Clutch Brake

An SMA clutch brake is installed in the ExoPhalanx, which is made into a string-like SMA. When the string-like SMA is heated over the transforming temperature, it is shortened. Alternatively, when it is cooled under the transforming temperature, it is lengthened to the original size. The SMA is flexible and the diameter is 150 μm. Here, the SMA is used as a clutch brake as shown in Figure 2. The heated SMA tightens a force transmission wire causing frictional force in the wire. The frictional force is presented as a force to the human operator. The SMA is heated using electricity because the SMA has electrical resistance of ~300 Ω/m. The cooling state of the SMA is based on the natural cooling process.

The driver circuit is simplified for the SMA clutch brake by using the Pulse Width Modulation (PWM) as the heating method. The PWM applies a square wave voltage with the duty ratio . The heat quantity per unit time is as follows, where ,, and are the heat quantity, the applied current, and the applied voltage, respectively. In (1), represents the cycle time of the square wave, is the time of the applied voltage in one wave, and the duty ratio, , is defined as .

The size and weight of the SMA clutch brake are 1 mm³ and 11 g, respectively. Figure 3 shows the clutching force of the clutch brake measured using the force gage FGP-0.2 (Nidec-SHIMPO Corporation) at various duty ratios. According to the graph, the clutching force is almost in proportional relation with the duty ratio.

2.2. Thumb Exoskeleton

The thumb exoskeleton is composed of a distal phalanx, a proximal phalange ring, and a thumb manifer as shown in Figure 4. A CM force transmission wire was used in the exoskeleton. One end of this wire is fixed to the distal phalanx and the other is free. The wire traces the tendon of the operator’s thumb and passes the proximal phalange ring. A CM SMA clutch brake is placed on the thumb manifer. The wire slides normally and bends with IP, MP, CM joints of human thumb, unless it is engaged in which case the wire stops and restrains bending of the thumb.

2.3. Middle Finger Exoskeleton

The middle finger exoskeleton is composed of the distal phalanx, a middle phalanx ring, and a proximal phalange ring as shown in Figure 5. The middle finger exoskeleton consists of two force transmission wires, which were used for restricting the bend in the DIP joint of the human middle finger and for the MP joint or the DIP force transmission wire and MP force transmission wire, respectively. One end of the DIP force transmission wire is fixed to the distal phalanx part and the other is free. Similarly, one end of the MP force transmission wire is fixed to the proximal phalange ring and the other is free end.

The DIP and MP force transmission wires follows the tendons of the human middle finger. The DIP force transmission wire passes the DIP SMA clutch brake on the proximal phalange ring and under the MP SMA clutch brake. The MP force transmission wire passes into the MP SMA clutch brake on the manifer. The DIP wire slides with DIP and PIP joint rotation. The MP wire slides with bend-stretch rotation of the MP joint. When the DIP SMA clutch brake is driven, the operator DIP-PIP joint bending is locked. When the MP SMA clutch brake clutches the MP wire, the operator MP joint bending is locked. A loadcell mounted on the distal phalanx exoskeleton measures fingertip force of the operator’s middle finger. The operator bends their middle finger and pushes the loadcell, which measures the force on the middle fingertip.

3. Robot Hand Teleoperation System with Haptic Device

Figure 6 shows the robot hand tele-operation system with the developed ExoPhalanx device. The tele-operation system consists of the robot hand subsystem with the universal robot hand II, the motion capture subsystem with the CyberGlove which is a well-knows motion capture data glove, and the force feedback subsystem with the developed ExoPhalanx device. In the robot hand tele-operation system, the universal robot hand II is controlled according to the motion of the operator’s hand measured in the motion capture subsystem. Then, the ExoPhalanx gives a reaction force to the operator in the force feedback subsystem.

3.1. The Robot Hand Subsystem with Universal Robot Hand II

This subsystem controls the motion of the universal robot hand by the PID control and acquires the force and the tactile measurements. The universal robot hand II has five movable fingers and a palm as shown in Figure 7. The thumb is 195 mm long and the other fingers are 230 mm long with four joints. Each joint is driven using a miniaturized DC motor with a rotary encoder and reduction gears (manufactured by Harmonic Drive Systems. co.) built into each link. In addition, two fingertip joints, joints 3 and 4, drive at equal ratios as a human finger, so each finger has three degrees of freedom. Each finger has multi-axis force/torque sensors (manufactured by BL AUTOTEC, Ltd.) in the distal link of the finger and array-type tactile sensors on the palmar surface of the fingers as shown in Figure 8. Here, the multi-axis force/torque sensor is used in detection of the distal phalange link contact object and the array-type tactile sensor is used in detection of the proximal phalange link contact object.

3.2. Motion Capture SubSystem with CyberGlove

This subsystem measures the posture of the operator by using the CyberGlove. The measured operator’s posture is sent to the robot hand subsystem through the network. The CyberGlove has three flexion sensors per finger, four abduction sensors, a palm-arch sensor, and sensors to measure wrist flexion and abduction. Here, this subsystem utilizes the sensor values except the DIP joints because the DIP joints and the PIP joints of the universal robot hand synchronize like a human finger.

3.3. Force Feedback Subsystem with ExoPhalanx

A human operator wears the ExoPhalanx, which is mounted on the CyberGlove. The ExoPhalanx works in order to restrain the human fingers according to the contact force measured by the multi-axis force/torque sensors and the array-type tactile sensors in the universal robot hand. Here, the multi-axis force/torque sensor measures the contact force on the fingertips, and the tactile sensor measures the contact force on the palmar surface of the proximal phalange. If the thumb and/or the middle fingertip of the robot hand contacts the object, then the ExoPhalanx control PC drives the CM and/or the DIP SMA clutch brake. Similarly, if the proximal phalange of the middle robot finger contacts the object, then the ExoPhalanx control PC drives the MP SMA clutch brake. Consequently, the human operator cannot bend his thumb and/or the middle finger.

4. Experiments

The performance of the ExoPhalanx is verified through two experiments. The first is the clutch brake experiment used to confirm the performance of the SMA clutch brake. The second is the two finger grasping experiment used to evaluate the haptic feedback performance. Here, the duty ratio for controlling the SMA clutch brake is fixed in these experiments because it is difficult to control the force presented to the operator by changing the duty ratio.

4.1. Clutch Brake Experiment

To confirm the haptic performance of the SMA clutch brake, a preliminary experiment is executed. In this experiment, we confirm whether the SMA clutch brake can restrain the human operator from bending his/her finger.

The human operator wears the CyberGlove and the ExoPhalanx and bends the DIP and PIP joints of their middle finger. The CyberGlove captures the operator’s postures. In this experiment, the operator bends the DIP joint over 20°, theoretically resulting in the ExoPhalanx stopping the corresponding force transmission wire. If the operator feels sufficient force at the specified joints, then the operator stretches the middle finger. The change of the DIP joint angle and force presented to the human operator are shown in Figures 9 and 10, respectively. Figure 10 includes the change of the duty ratio in the experiment. From the clutch brake experiment, the operator recognizes that their DIP joint is locked by the ExoPhalanx and stretches their middle finger. However, if the operator bends the DIP joint over 20°, the SMA clutch brake stops the DIP force transmission wire because the force transmission wire loosens.

4.2. Two-Finger Grasping Experiment

A two-finger grasping experiment is used to evaluate the haptic feedback performance by the ExoPhalanx as shown in Figure 11. The grasped object is a polystyrene ball of 150 mm in diameter similar to the size of a baseball.

The duty ratio is constant for driving the SMA clutch brake. The operator then manipulates the universal robot hand II using the following protocol:(1)The operator stretches all fingers.(2)The ball is set in front of the universal robot hand II.(3)The operator remotely manipulates the robot hand with the thumb and middle finger to grasp the ball with the fingertips of the robot hand.(4)The operator continues bending their fingers until a reaction force on their fingertip from the ExoPhalanx is felt.(5)If the operator feels sufficient force, they release the ball.

This experiment is implemented twice with the experiments being soft and strong grasping.

Figures 12 and 13 show the results of the two-finger grasping experiment. The IP, MP, and CM joint angles of the thumb for the robot hand and the operator are shown in Figures 12(a), 12(b), and 12(c), respectively. The DIP and MP joint angles of the middle finger for the robot and operator are shown in Figures 13(a) and 13(b), respectively. From these figures, the robot hand grasps the ball with the CM joint at 37° of the robot thumb and the DIP joint at 20° of the robot middle finger. Although the robot hand grasps the ball, the operator’s CM joint of the thumb and DIP joint of the middle finger are over the joint angles of the robot hand. This is because the force transmission wire loosens similarly to the clutch brake experiment. Figures 14 and 15 show the force measured in the robot hand and the operator’s hand, respectively. Here, the force of the operator’s hand is measured by the loadcell equipped in ExoPhalanx. In Figures 14 and 15, the maximum forces of the robot hand and the operator’s hand are about 0.2 N and 1.5 N in the soft grasping experiment, and those maximum force are about 0.4 N and 2.0 N in the hard grasping.

(a) IP joint angle

(b) MP joint angle

(c) CM joint angle

(a) DIP Joint Angle

(b) MP joint angle

5. Conclusions and Future Works

5.1. Conclusions

Presented herein was the reaction force using a haptic feedback device allowing for important scientific advancement in the operation of a robot hand remotely and dexterously. We developed the passive force feedback device ExoPhalanx. ExoPhalanx is a small and lightweight device that uses an SMA-fitted clutch brake. The developed ExoPhalanx provides reaction force on the distal phalange and proximal phalange to the remote operator. Moreover, in the tele-operation system, ExoPhalanx was mounted to the motion capture data glove, CyberGlove. The effectiveness of the ExoPhalanx system for tele-operation applications was verified through preliminary experiments.

5.2. Future Works

The future work involves improvement of the SMA clutch brake so that it is proportionally controlled and reduces the backlash between each link of the exoskeletons. Moreover, we will develop a control method of the fingertip force presented to the operator and present the force according to the measured force in the universal robot hand II by the ExoPhalanx. Finally, it is proposed that the ExoPhalanx system will be extended to five fingers.

References

J. M. Hollerbach and S. C. Jacobsen, “Anthropomorphic robots and human interactions,” in Proceedings of the 1st International Symposium on Humanoid Robots, pp. 83–91, 1996.
View at: Google Scholar
D. Johnston, P. Zhang, J. Hollerbach, and S. Jacobsen, “Full tactile sensing suite for dextrous robot hands and use in contact force control,” in Proceedings of the IEEE International Conference on Robotics and Automation, pp. 661–666, 1996.
View at: Google Scholar
T. Mouri, H. Kawasaki, K. Yoshikawa, J. Takai, and S. Ito, “Anthropomorphic robot hand: gifu hand III,” in Proceedings of the International Conference on Control, Automation and Systems, pp. 1288–1293, 2002.
View at: Google Scholar
T. Mouri, H. Kawasaki, and K. Umebayashi, “Developments of new anthropomorphic robot hand and its master slave system,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3474–3479, 2005.
View at: Publisher Site | Google Scholar
H. Iwata and S. Sugano, “Design of anthropomorphic dexterous hand with passive joints and sensitive soft skins,” in Proceedings of the 2nd IEEE International Symposium on System Integration (SII '09), pp. 129–134, 2009.
View at: Publisher Site | Google Scholar
K. Takahashi, H. Waita, M. Kokusho, and M. Hayakawa, “Development of dexterous and powerful multi-fingered hand for humanoid robots,” in Proceedings of the 27th Annual Conference of the Robotic Society of Japan, vol. AC1A1-01, 2009.
View at: Google Scholar
K. Kaneko, K. Harada, and F. Kanehiro, “Development of multi-fingered hand for life-size humanoid robots,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '07), pp. 913–920, Roma, Italy, April 2007.
View at: Publisher Site | Google Scholar
S. Schulz, C. Pylatiuk, and G. Bretthauer, “A new ultralight anthropomorphic hand,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '01), pp. 2437–2441, May 2001.
View at: Publisher Site | Google Scholar
J. Jockusch, J. Walter, and H. Ritter, “A tactile sensor system for a three-fingered robot manipulator,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '97), pp. 3080–3086, April 1997.
View at: Google Scholar
H. Nakamoto, F. Kobayashi, and F. Kojima, “Shape classification using tactile information in rotation manipulation by universal robot hand,” Robotics 2010 Current and Future Challenges, pp. 123–132, 2010.
View at: Google Scholar
W. Fukui, F. Kobayashi, F. Kojima et al., “Development of multi-fingered universal robot Hand with torque limiter mechanism,” in Proceedings of the 35th Annual Conference of the IEEE Industrial Electronics Society, pp. 2225–2230, 2009.
View at: Google Scholar
M. D. Duong, K. Terashima, T. Miyoshi, and T. Okada, “Rehabilitation system using teleoperation with force-feedback-based impedance adjustment and EMG-moment model for arm muscle strength assessment,” Journal of Robotics and Mechatronics, vol. 22, no. 1, pp. 10–20, 2010.
View at: Google Scholar
S. J. Biggs and M. A. Srinivasan, “Haptic interfaces,” in Handbook of Virtual Environments: Design, Implementation, and Applications, Human Factors and Ergonomics, K. Stanney, Ed., chapter 5, pp. 93–115, Lawrence Erlbaum Associates, 2002.
View at: Google Scholar
T. Endo, H. Kawasaki, T. Mouri et al., “Five-fingered haptic interface robot: HIRO III,” in Proceedings of the 3rd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 458–463, Salt Lake City, Utah, USA, March 2009.
View at: Publisher Site | Google Scholar
T. Endo, T. Kanno, M. Kobayashi, and H. Kawasaki, “Human perception test of discontinuous force and a trial of skill transfer using a five-fingered haptic interface,” Journal of Robotics, vol. 2010, Article ID 542360, 14 pages, 2010.
View at: Publisher Site | Google Scholar
P. Berkelman and M. Dzadovsky, “Extending the motion ranges of magnetic levitation for haptic interaction,” in Proceedings of the 3rd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 517–522, Salt Lake City, Utah, USA, March 2009.
View at: Publisher Site | Google Scholar
A. Gupta, M. K. O'Malley, V. Patoglu, and C. Burgar, “Design, control and performance of ricewrist: a force feedback wrist exoskeleton for rehabilitation and training,” International Journal of Robotics Research, vol. 27, no. 2, pp. 233–251, 2008.
View at: Publisher Site | Google Scholar
W. Park and S. Park, “Haptic mouse interface actuated by an electromagnet,” in Proceedings of the International Conference on Complex, Intelligent, and Software Intensive Systems, L. Kim and S. Shin, Eds., pp. 643–646, Seoul, Korea, June 2011.
View at: Google Scholar
Z. Zhang and T. Chen, “Study on the control of 6-DOF manipulators system with force feedback,” in Proceedings of the International Conference on Intelligent Computation Technology and Automation (ICICTA '08), vol. 1, pp. 498–502, Hunan, China, October 2008.
View at: Publisher Site | Google Scholar
T. Amemiya, T. Maeda, and H. Ando, “Location-free haptic interaction for large-area social applications,” Personal and Ubiquitous Computing, vol. 13, no. 5, pp. 379–386, 2009.
View at: Publisher Site | Google Scholar
K. N. Winfree, J. Gewirtz, T. Mather, J. Fiene, and K. J. Kuchenbecker, “A high fidelity ungrounded torque feedback device: the iTorqU 2.0,” in Proceedings of the 3rd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, World Haptics 2009, pp. 261–266, Salt Lake City, Utah, USA, March 2009.
View at: Publisher Site | Google Scholar
H. Gurocak, S. Jayaram, B. Parrish, and U. Jayaram, “Weight sensation in virtual environments using a haptic device with air jets,” Journal of Computing and Information Science in Engineering, vol. 3, no. 2, pp. 130–135, 2003.
View at: Publisher Site | Google Scholar
A. Chang and C. O'Sullivan, “Audio-haptic feedback in mobile phones,” in Proceedings of the International Conference for Human-Computer Interaction (CHI '05), pp. 1264–1267, Portland, Ore, USA, 2005.
View at: Publisher Site | Google Scholar
H. Yano, Y. Miyamoto, and H. Iwata, “Haptic interface for perceiving remote object using a laser range finder,” in Proceedings of the 3rd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, World Haptics 2009, pp. 196–201, Salt Lake City, Utah, USA, March 2009.
View at: Publisher Site | Google Scholar
M. J. Lelieveld, T. Maeno, and T. Tomiyama, “Design and development of two concepts for a 4 DOF portable haptic interface with active and passive multi-point force feedback for the index finger,” in Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information In Engineering Conference (DETC '06), September 2006.
View at: Google Scholar
K. Koyanagi, Y. Fujii, and J. Furusho, “Development of VR-STEF system with force display glove system,” in Proceedings of the International Conference on Augmented Tele-Existence (ICAT '05), pp. 91–97, Christchurch, New Zealand, December 2005.
View at: Publisher Site | Google Scholar
H. Z. Tan, M. A. Srinivasan, B. Eberman, and B. Cheng, “Human factors for the design of force-reflecting haptic interfaces,” American Society of Mechanical Engineers, Dynamic Systems and Control Division, vol. 55, pp. 353–359, 1994.
View at: Google Scholar
R. L. Klatzky and S. J. Lederman, “Touch”, Handbook of Psychology, vol. 4 of Experimental Psychology, John Wiley and Sons, New York, NY, USA, 2003.

Copyright

Copyright © 2011 Futoshi Kobayashi et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

3477

Downloads

1971

Citations