Abstract
This paper presents a new type of finger rehabilitation system using a multifingered haptic interface that is controlled by the patient though a surface electromyogram. We have developed the multifingered haptic interface robot: HIRO III that can give 3-directional forces to 5 fingertips. This robot can also be used as a rehabilitation device that can provide various fingertip exercises and measure various types of information. The sEMG works together with the HIRO III to consider the patient's intent. The proposed system is intended for patients having paralysis in the hand and fingers, and the motions will be provided as biofeedback to the fingertips with the device. In contrast to completely passive rehabilitation, the proposed system can provide active rehabilitation using sEMG. The experiment involved finger opening and closing with this system by ten able-bodied subjects. The results show that almost all subjects felt appropriate motion support from the device.
1. Introduction
Hand finger function is very important for daily life, and finger injury can be a very serious problem. These patients need timely and persistent rehabilitation to recover their lost abilities and to resume their normal daily lives. Long rehabilitation training sessions with therapists, who are in relative shortage, are not always available to patients. One solution to this problem could be a robotic rehabilitation system for hand fingers used by individual patients.
Recently, many robotic systems for rehabilitation have been studied. Many aspects of robotic arm rehabilitation therapy [1–3], including clinical tests [4, 5], have been reported. Arm rehabilitation therapy with the aid of a robot [6], which involves bimanual, mirror-image, patient-controlled therapeutic exercises, is one type of self-controlled rehabilitation. Kong and Jeon [7] have developed a tendon-driven exoskeletal power assistive device that uses an sEMG signal and fuzzy control. Saint-Bauzel et al. [8] have presented a reactive robotized interface for lower limb rehabilitation with clinical results.
Hand rehabilitation, however, is particularly difficult, because the hand possesses many degrees of freedom of motion, and the device must be relatively small in size. Connelly et al. [9] have developed a VR hand rehabilitation system with a pneumatic glove, and Oblak et al. [10] have developed a universal haptic device for arm and wrist rehabilitation. In the former, it is slightly difficult for a patient with hand paralysis to wear the glove, and the latter can provide exercise at the shoulder, arm, and wrist only, not the hand or fingers. Compact devices with a cable system to be used after a stroke [11, 12], and exoskeleton-type robotic devices [13, 14] have been presented. The former is a compact device but provides only limited exercises because of the cable actuate. The latter can assist not only with flexion/extension, but also with abduction/adduction and thumb opposition motions but can assist only with joint motion, not 3-directional fingertip motions.
The surface electromyogram (sEMG) records the electrical activity of muscles underlying the skin and is useful for rehabilitation robotic devices. Rehabilitation control strategies for a gait robot [15], upper limb rehabilitation [16], and a trial of the recovery process for wrist rehabilitation [17] have been reported. However, the application of sEMG to finger rehabilitation has not been widely researched.
In Japan, there are many patients with finger paralysis; for example, carpal tunnel syndrome may lead to finger paralysis, as can long-term dialysis, and the population currently needing dialysis in Japan is approximately 280 thousand. Because hand and finger function is very important for daily life, this paper proposes a new type of finger rehabilitation system for providing more effective rehabilitation. In these cases, the robotic device should be able to provide various exercises including flexion or extension and should be with compact size. A solution to this problem would be a rehabilitation system that could provide 3-directional motions to the fingertips. Usually, it is easier for people to imagine “moving the fingertips” than “moving each finger joint.” The proposed system therefore provides a haptic interface for hand finger rehabilitation. A multifingered haptic interface robot [18] has previously been developed, and this device can provide fingertip motion to the operator. In order to achieve more effective rehabilitation for patients, this system utilizes sEMG control. A finger joint angle estimation system from sEMG has been developed [19], and this system will be helpful for complex exercises. In the opinion of one practitioner, this research could result in a challenging clinical trial, and it is of great significance to integrate the sEMG into a rehabilitation robotics system for hand finger. The above system will be effective for patients with peripheral nerve injury of the arm.
As a pretest for the clinical trial, we present herein the experimental results from this system using able-bodied subjects. The purpose of this pretest was to confirm that the fingertips were moved based on a user’s intentions in the case of a healthy body.
2. System
Figure 1 shows the outline of the proposed system.
This system uses a haptic device for the fingertips and sEMG information from the patient’s forearm or hand. Since this haptic device is so small, it can be used on a desk while the patient is seated. First, this system decides the appreciate fingertips exercises from the sEMG; after that, the device provides exercises at the fingertips. For example, if the sEMG from muscles for finger extension (extensor digitorum muscle) activates and crosses over a certain threshold value, the haptic device exerts 3-axis forces at each fingertip in order to open the fingers. The following sections provide the details of the system.
2.1. Multifingered Haptic Interface HIRO III
Figure 2 shows the multifingered haptic interface robot, the HIRO III, that we have developed [18] (the specifications for the hand part are in Table 1).
(a) The haptic interface: HIRO III overview with arm
(b) Connect to hand part of HIRO III
(c) Installation of finger folder at fingertip
(d) Passive spherical joint at the top of haptic finger
(e) Finger mechanism
(f) Work space and joint axis of a haptic finger
This robot consists of a hand part and an arm part. The hand part has 5 haptic fingers, and each haptic finger has 3 joints and 3 degrees of freedom (DOF). The arm part has 6 joints and 6 DOF. Therefore, in total, the HIRO III has 21 DOF. This robot has the following features.(i)The haptic finger has three DC motors in the frames and three active joints (Figures 2(e) and 2(f)). The first joint is for abduction-adduction of the middle phalanx (MP) joint of the haptic finger, and the second and third are for flexion extension of the MP and proximal interphalangeal (PIP) joint, respectively. Additionally, a haptic finger has a 3-axis force sensor at the tip (Figure 2(d)), and each motor has an encoder.(ii)HIRO III has five haptic fingers and can simultaneously provide 3-directional forces on multiple fingertips.(iii)Also, the haptic finger has a permanent magnet at the tip (Figure 2(d)). The operator puts up the finger folders at the fingertips, and this folder has a metal ball (Figure 2(c)). The folder and the metal ball together form a passive spherical joint (Figure 2(d)) that can change the posture of a user’s finger against the haptic finger.(iv)The HIRO III system with the arm can realize the large work space. Meanwhile, the hand part can be use as a compact haptic device.
In the case that this device is used as a rehabilitation device, there are some advantages. Ordinary robotics devices for finger rehabilitation are generally an exoskeleton type [13, 14], or a tendon-driven type at the fingertip [11, 12]. The HIRO III has the following advantages over these device types.(i)Three-directional motion at each fingertip: while a tendon-driven type device can only provide motion directed along the wire to the user’s finger, the HIRO III allows for 3-directional motion. That is, various types of finger exercises are possible with the HIRO III. Because this robot has a 3-axis force sensor per finger, the system can limit applying force to user. It can realize safety control strategy. The value of maximum output force of one finger (3.6 N) can be considered as sufficient value from medical aspect for the case of biofeedback interface (described later sections). By comparison, an exoskeleton type [13] is designed in such a way as to apply about 0.3 Nm torque at finger joint.(ii)Connecting to the device: an exoskeleton-type device must firmly connect the device’s link to the user’s finger link. In contrast, the user connects to the HIRO III with a finger folder and passive spherical magnet joints, allowing them to connect quickly and easily.(iii)Compact size: a robot system that supports the motion of finger extension with a tendon-driven or exoskeleton type is generally quite large. While gear transfer mechanism has backlash, this system adopts the force depending control method not position control. In this aspect, this demerit of gear mechanism is not large problem. The HIRO III without the arm is comparatively compact in size and can support desktop rehabilitation.(iv)Daily rehabilitation: thanks to the ease of connection and rehabilitation on the table, the HIRO III system provides rehabilitation that is easy to keep doing every day. Daily rehabilitation is very important to prevent joint contracture and provide early recovery.
2.2. The Electromyogram for Finger Rehabilitation
The sEMG measures bioelectrical signals from muscles in response to voluntary contractions. These signals provide various type of information about a person’s intent. It is therefore hoped that the sEMG will function as a new type of human-machine interface, and it has been studied by many researchers [19, 20].
In the rehabilitation area, sEMG is used as a biofeedback tool. For instance, rehabilitation with sEMG is very efficient in the case of “nerve injury,” because it is easy for the patient to identify the muscle that should be focused upon when using sEMG as biofeedback. In rehabilitation applications, the forms of sEMG as biofeedback to the patient are primarily waveforms on display or sound.
As another way of using the sEMG, this signal shows muscle activity and is therefore a barometer of recovery level. In the case of “peripheral” or “central nerve injury,” the degree of recovery can be determined by measuring the sEMG over long period.
The control methods for ordinary rehabilitation robotics systems can be divided into the following types: the simple teaching playback type [11], the symmetry master-slave type [12], and the telerehabilitation system [14]. The teaching playback type systems and telerehabilitation systems cannot reflect the patient’s intent or thinking. It is a strictly passive rehabilitation, while effective rehabilitation should actually involve patient activity. Systems adopting the symmetry master-slave structure can provide exercises along to user’s intent. But this type can be used only in limited injury cases, for example, hemiplegia or one-side paralysis of the brain in which case the patient can move the healthy side of the body. Furthermore, in these systems, patients tend to concentrate on moving only their healthy side, not the paralysis side.
In the present study, the haptic device was controlled by the sEMG. First, the motion intent of the user was extracted from the sEMG, and the fingertip exercises along with the intent were then provided to the user by the HIRO III. This biofeedback is very easy to image for the patient compared with waveforms or sounds. And as opposed to completely passive exercise, the proposed system can give the patient a rehabilitative experience that reflects the patient’s intent.
2.3. Medical Aspects of the Proposed System
This section includes the feedback of 2 medical orthopedic doctors. Feedback from practitioners is clearly very important when carrying out research in the medical or rehabilitation area.
The opinions of doctors A and B are as follows.
Doctor B indicated that from a medical perspective, this research will lead to a challenging clinical trial. It is of great significance to integrate the sEMG into the rehabilitation robotics system for hands and fingers. This proposed system will be able to be applied to general cases of hand finger paralysis.
The above system will be effective for patients with “peripheral nerve injury” of the arm (note that this case is also one type of hand finger paralysis). In the forearm, muscles with similar function run parallel to each other. It will be important in this research to analyze the relationship between the sEMG and the muscle and finger function. According to the degree of paralysis or recovery of the patients, the fixation of the upper limb or elbow might be needed.
As pointed out above, the proposed system can be applied to the case of hand finger paralysis. While this condition has many different causes, two cases related to peripheral nervous are described below.(i)Peripheral nerve injury: the peripheral nervous system resides or extends outside the central nervous system, which consists of the brain and spinal cord. The following three peripheral nerves are related to the fingers and hand: median nerve, ulnar nerve, and radial nerve. Injury of these nerves can be caused by external wounds such as those inflicted with a sharp knife.(ii)Carpal tunnel syndrome: this case is caused by compressing the peripheral nerve running in the carpal tunnel for some reason. In particular, carpal tunnel syndrome may be caused by dialysis over a long term, and in 2010, the number of patients who need dialysis in Japan is approximately 280 thousand.
In this paper, a case of peripheral nerve injury is considered, as suggested by a medical doctor. In such cases, the patient have undergone surgery. After the operation, recovered neuraxons will reach end organs, again. This is the reinnervation. However, in this process, the neuraxons might reach different organs than those to which they were originally connected. If the neuraxons do reach different organs, the patients will not be able to move their muscles in the same way that they did previously. To solve this problem, rehabilitation for muscle reeducation is needed. During this reeducation period, the patients cannot do the desired motion. As such, a patient’s motivation to continue with the rehabilitation will be reduced, and contracture of the joints will begin to occur due to the reduced movement of the muscle.
Rehabilitation exercises will differ according to the injured nerves or the degree of recovery, so the therapist should determine the appropriate exercise for each case. Examples of the rehabilitation fingertip exercise are shown below, and Figure 3 shows the relationship of the muscle and the work in the forearm and hand.(i)Median nerve injury: thumb opposability, lunge motion (MP flexion and PIP, DIP extension) of the index or middle finger, and independent flexion of the thumb, index, and middle fingers.(ii)Ulnar nerve injury: ring finger and pinkie flexion.(iii)Radial nerve injury: finger extension, curl up (MP extension and PIP, DIP flexion) motion of fingers.
In the case of nerve injury, these exercises are important for muscle reeducation for reinnervation.
2.4. Advantage of the Proposed System
As mentioned above, the proposed system can provide “daily” and “active” rehabilitation.
The former is realized due to the compactness of the HIRO III device, the ease of connecting the device to various sensors (for fingertip forces, fingertip positions, and sEMG signals at certain muscles). Daily rehabilitation is very important to prevent joint contracture and early recovery. Furthermore, recording the above information at every rehabilitation time is efficient for evaluating the degree of recovery over the long term.
Active rehabilitation is then achieved by using the sEMG to make fingertip motions. Because the sEMG is closely related to voluntary motion, it helps to reflect the intent of the patient, allowing the patient to do active rehabilitation without assistance. This system requires a positive intent of the patient for successful rehabilitation and recovery.
2.5. The Control Strategy Outline
The general control strategy for the proposed system will be described in this section, while the details of the experiment will be presented in a later section.
First, the sEMG is helpful for understanding the patient’s intent. But raw sEMG data also provide undesired information (e.g., typical biological noise or personal difference, etc.), so signal processing will be needed. As for signal processing of the sEMG, the system adopts an extracting feature value method, determining the mean absolute value (MAV) or root means square (RMS) with segmentation and normalizing [21]. For these processes, it will be useful to use feature values rather than raw sEMG data.
Second, a patient’s motion intent should be extracted from the sEMG information. While the relationship between the sEMG and finger motion is quite complex, by focusing on certain finger functions, many methods are available. For example, if MAV from the sEMG around common digital extensor muscle is over a certain threshold, then the intent of the user is “to open the fingers.” Of course, this rule must be determined depending on the case of injury or the patient. For almost all cases, this rule or extracting method should be simple. However, if it is necessary or effective to utilize a more complex finger exercise, a system to estimate the finger joint angles [21] will be helpful.
After obtaining the desired fingertip exercise by the above method, the system must transform from the exercise into a continuous fingertip trajectory. The haptic device HIRO III provides fingertip forces depending on the desired fingertip trajectory. For rehabilitation, force control is appropriate rather than position control in that the force can be adjusted. As a simple example, one switches the target positions by the intent. It is necessary to measure these positions beforehand. When the joints of a patient do not have contraction, the target positions can be measured with the help of a therapist or by the patient. When the joints have contraction, the length of the patient’s finger link and range of motion are measured, and solving the kinematics with these parameters will identify the target positions. The simplest method to generate the desired fingertip forces is to use the virtual spring model (Figure 4).
This method requires only the target position and the current position. The desired force vector of one finger is calculated with where is the spring coefficient diagonal matrix, is the current fingertip position of a certain finger, and shows the target or the desired fingertip position. These motors inside the haptic device have an encoder, and all kinematical parameters are available so the current fingertip positions can be calculated by forward kinematics. Furthermore, in order to adjust the forces to the fingertip, the desired forces should be applied with a low-pass filter (LPF), limiting to the appropriate range.
Finally, the haptic finger of HIRO III will be controlled by the PI control law for fingertip forces [18]. First, the error of torque vector is defined as follows: where is the kinematical Jacobian matrix of a haptic finger, is the desired fingertip force vector calculated by the virtual spring method, as explained in the previous section, and is the measured fingertip force vector from a 3-axis force sensor located at the top of the haptic finger. The HIRO III system adopts the following PI control law: where is the applied torque vector onto a haptic finger and and are the proposed and integral component gain matrix, respectively. The equation shows the PI control law. The gains or parameters are calibrated in advance. Figure 5 shows the entire block diagram of this system.
3. Experiment
As a pretest for the clinical trial, the proposed system was tested with ten able-bodied persons (ten adult males, ages 22–24). In this pretest, the target exercises were to “open the hand” and “pinch with all fingers”. These are simple but basic hand functions and important exercises for rehabilitation for peripheral nerve injury (the former is for the radial nerve and the latter for the median and radial nerves). To measure how the sEMG relates to these motions, two terminals will be placed around the common digital extensor muscle and the flexor digitorum profundus muscle, respectively. The subject applies five finger folders to each fingertip and connects with the device. In this experiment, five fingers will be used (Figure 6).
The purpose of this pretest was to confirm that fingertips in response to the user’s intention extracted from the sEMG by a simple strategy in the case of a healthy body. Note that in the doctor’s opinion, the sEMG can be measured from the patient with peripheral nerve injury. To make it easier to concentrate, two sEMG terminal are used and the strategy adopt simple switching method for deciding the exercise mode from the sEMG.
3.1. Acquisition and Signal Processing for the sEMG
Sampling was carried out with a gain of 1000, 1 ms, and 12 bits. In this experiment, the system used 2 sEMG electro nodes, one for the common digital extensor muscle and the other for the flexor digitorum profundus muscle. The raw sEMG data were first filtered with a band pass filter (10–350 Hz) and then segmented (length of segment is 256, and the length of the shift is 32). Finally, MAV of the sEMG was extracted from the segmented sEMG. In the following sections, the “MAV1” refers to the value of MAV extracted from the common digital extensor muscle, and the “MAV2” to that from the flexor digitorum profundus muscle. Similarly, “Thre1” refers to the threshold value for MAV1 and “Thre2” to the threshold value for MAV2. When the MAV value crosses over the corresponding threshold, it can be assumed that the muscle is activated.
3.2. Extracting the Motion Intent
In this pretest, the system selects the following mode as the subject’s intention with the if-then rule. This rule will decide the following fingertip exercise mode: the Mode 0 means that the haptic device does not provide any force, and the patient’s hand will maintain a relaxed state. In the case of Mode 1, the desired fingertip exercise is opening the hand. Finally, with Mode 2, the desired exercise is pinching or closing all the fingers. The detail of the if-then rule is shown in Algorithm 1.
| ||||||||||||
Note that when MAV1 > Thre1 and MAV2 > Thre2, then the mode is 2. This rule is determined through a trial-and-error process.
3.3. Generating the Fingertip Motion Corresponding to the Intention
As a simple method to determine the fingertip motion, in this pretest, the fingertip position data measured beforehand is selected according to the mode every 32 milliseconds. It is necessary to determine these data (the fingertip positions at the state of “open the hand” and “pinch with all fingers”) in advance.
3.4. Generating the Desired Forces Sent to the Haptic Device
The virtual spring model generates the desired forces from the selected target fingertip positions (Figure 4). The spring coefficient is 150 N/m, the maximum force limit at one axis is 3 N, and the cutoff frequency of the low-pass (first order) filter is 0.2 Hz (Figure 5). These parameters are determined so that the generated forces may not cause rapid change. The cutoff frequency presents system response character, because this experiment targets able-bodied subject, the value is determined as above by try and error. When this system is applied to people has failure at hand, the value should be started from small value. Similarly, spring coefficient and maximum force limit are also must be determined in a careful way. In the case with user who has spasticity of the fingers, the value of spring coefficient and maximum force limit are should be small.
3.5. Experimental Procedure
The experimental procedure included the following steps.
First, the system and the experiment were explained to the subject. After the subject understood the method and purpose, the experiment was carried out.
Secondary, two sEMG terminals were attached to the subject’s forearm (common digital extensor muscle and flexor digitorum profundus muscle), and the five fingertips were connected to the HIRO III with a finger folder.
Next, the fingertip positions of the opened and closed hand were measured.
After that, it was necessary to calibrate the two MAV thresholds. These were adjusted so that the subject could switch the exercise mode as subject’s thought.
Finally, the experiment and data acquisition were carried out (Figure 6). Subject used the armrest for resting the forearm and wrist and connected each fingertip into the haptic device on the opposite side.
After the experiment, the subject responded to the questionnaire described below.
3.6. Subjective Evaluation
After the experiment, the subject responded to the following questionnaire. The scores corresponding to each question items ranged from 1 to 5. In this questionnaire, 5 corresponds to good/desirable and 1 to bad/undesirable.(i): “Was the considered movement supported by the device?” Point: 1 (Not at all.)–5 (I think so.)(ii):“Was the support natural?” Point: 1 (I think it’s not natural.)–5 (I think very natural.)(iii): “Was there anxiety?” Point: 1 (I felt fear.)–5 (I relaxed.)
4. Results and Discussion
Table 2 shows the responses of the subjects to the questionnaire.
Almost all items have high point totals, indicating that the proposed system may be useful as a rehabilitation device. In particular, items (i) and (iii) have very high points. This point is very important for a rehabilitation device controlled by sEMG, and this fact points out that the proposed system may be effective.
Figure 7 shows the system coordinates and the origin, and example graphs of experimental results corresponding to the axis of thumb and the axis of the middle finger are shown in Figure 8. In this graph case, Thre1 is 0.024 and Thre2 is 0.003.
(a)
(b)
The table and the graph show that the fingertip motions corresponding to the intention of the subject are generated to some extent. However, in the graph that shows mode switching, the switching may have an error at the transition of the mode, for example at approximately 2 or 7 seconds. This type of error sometimes exists at the other subject, and almost all cases are caused at the mode transition from the other mode to mode 0. The reason for this error is thought to be that the sEMG can increase corresponding to the subject’s intentions but that it is difficult for the person to quickly decrease the sEMG. Generally, a small amount of time is necessary to decrease the sEMG through oscillations. It is therefore necessary to modify the system so that the operator can switch the mode corresponding to the operator’s intent. For example, to improve this error, one solution is to use a hysteresis threshold method, that is, using two threshold values (one is for exceeding and the other for falling below). While this should be considered, this experiment shows that an able-bodied subject can control the exercise mode by sEMG and switch from a relaxed to a motion state.
5. Conclusion and Future Works
This paper proposes a new type of finger rehabilitation system with a multifingered haptic interface controlled by sEMG. The system can therefore provide the patient with effective finger rehabilitation. According to the doctor’s opinion, it seems that this research will lead to a challenging clinical trial.
As a pretest for the clinical trial, experimental results obtained with the system for able-bodied subjects are presented. This study targeted two types of movements with two sEMG terminals. Our results show that the system provided 3-directional forces at each fingertip corresponding to the subject’s sEMG, and that the subjects felt natural movement support. The results did show that the system had a switching error at the transition of the mode. This problem must be solved so that the patient can switch the mode corresponding to their intent.
In the future, it will be necessary to modify the system so that the system can provide a greater variety of movements. For example, the pinch exercise with thumb and index finger is very important and is also a barometer for recovery. Paralysis related to this motion can be caused by carpal canal syndrome or injury of the medial nerve. Of particular concern is that Japanese use chopsticks in daily life, and this tool requires a pinch motion. Our intent is to carry out a clinical trial with the proposed system.