Abstract
To improve the control effectiveness and make the prosthetic hand not only controllable but also perceivable, an EMG prosthetic hand control strategy was proposed in this paper. The control strategy consists of EMG self-learning motion recognition, backstepping controller with stiffness fuzzy observation, and force tactile representation. EMG self-learning motion recognition is used to reduce the influence on EMG signals caused by the uncertainty of the contacting position of the EMG sensors. Backstepping controller with stiffness fuzzy observation is used to realize the position control and grasp force control. Velocity proportional control in free space and grasp force tracking control in restricted space can be realized by the same controller. The force tactile representation helps the user perceive the states of the prosthetic hand. Several experiments were implemented to verify the effect of the proposed control strategy. The results indicate that the proposed strategy has effectiveness. During the experiments, the comments of the participants show that the proposed strategy is a better choice for amputees because of the improved controllability and perceptibility.
1. Introduction
Prosthetic hands are of great importance to the upper limbs amputees which can help them to complete some manipulations such as grasping. How to make the prosthetic hand grasp objects by following inclinations of amputees is a meaningful research direction. Scholars in this filed have done a lot of research work and made some important development.
At present, there are some kinds of prosthetic hands: decorative prosthetic hand (for the purpose of decoration, without function), switches controlled prosthetic hand, electromyography (EMG) signal controlled prosthetic hand, and so forth. A typical control scheme of EMG prosthetic hand is shown in Figure 1 [1]. and are EMG signals acquired from a pair of antagonistic muscles, is the scale factor, is the expected grasp force, and is the grasp force measured by force sensors. is the feedback gain and is the proportional gain for the force error. This control mode is popular due to its simple operation and in accordance with the operation habits of natural hand. However, there are still some problems to be solved. Firstly, this control mode does not fully consider the influence of EMG signals on control strategy: the EMG signals measured from different persons are different, and the EMG signals measured from different state of the same individual may be different. Secondly, this control mode lacks perceptibility: the user cannot perceive the grasp force of the prosthetic hand when grasping an object.
EMG motion pattern recognition is a basic technique of EMG prosthetic hand; a great number of recognition methods have been developed, such as time-domain method and frequency-domain method [2]. Parameters of ARMA model and Kalman filter were adopted as character vectors to identify the movement patterns [3]. Khoshaba et al. used the integral absolute value of the EMG signals to recognize the movement patterns [4]. Autoregressive (AR) model, power spectrum, wavelet coefficients, neural network, and some signal processing methods were also used [5]. However, due to the difference of the sticking positions of EMG sensors and the difference of temperature and humidity of the environment, the EMG signals measured from different persons or different states of the same individual may lead to different results. This kind of phenomenon may affect the accuracy of the movement pattern recognition, which may cause descending of the control effect.
The design of the controller is another important part of the EMG prosthetic hand; most of the control strategies for the prosthetic hands aimed at the controllability, such as accuracy force tracking control and speed control of opening and closing. In order to get a satisfying grasping performance, Fassih et al. proposed a control strategy based on defining virtual spring-damper model between two finger tips and damping force at each finger joint [6]. Chen et al. focused on hybrid of soft computing technique of adaptive neuron-fuzzy inference system (ANFIS) and hard computing technique of adaptive control for a two-dimensional movement with thumb and index prosthetic hand [7]. A hybrid sliding mode-backstepping (HSMBS) parallel force-velocity controller was proposed to improve the control effect of powered prosthetic hands by Engeberg and Meek [8]. According to the tactile feedback and visual feedback, when grasping an object, natural hand can adjust the grasp force in time. But for prosthetic hand which has no tactile feedback to the user, it is hard to obtain expected control effect. So a prosthetic hand which is not only controllable but also appreciable is probably a good choice for amputees. The controllability of the prosthetic hand just functionally assists amputees to complete some simple actions such as grasping. However, the perceptibility of the prosthetic hand considers more about the amputee himself. It not only satisfies the functional needs of the hands but also meets the psychological requirements. In a view of the perceptibility of the prosthetic hand, amputees get more information when using it. It shows a faster acceptance for amputees when using the hand for a long time, and this phantom limb feeling may also improve the control effect.
This paper describes a control strategy for the EMG prosthetic hand, which mainly includes EMG self-learning recognition, backstepping controller with stiffness fuzzy observation, and grasp force tactile representation. EMG self-learning recognition aims to reduce the influence on EMG signals caused by the uncertainty of the contacting positions of the EMG sensors. Backstepping controller with stiffness fuzzy observation is used to realize the position control and grasp force control. The grasp force tactile representation aims to improve the proprioception of the prosthetic hand, which can improve the control effect. Finally, experiments were implemented to verify the proposed control strategy.
2. System Components
The MPH-III prosthetic hand, which is designed by Robot Sensor and Control Lab in Southeast University, is used in this paper. The MPH-III consists of three components: prosthetic hand, EMGs acquisition device, and tactile representation device (see Figure 2). The lithium batteries are used as power source for all components.
2.1. Prosthetic Hand
The MPH-III is a one-DOF (degree of freedom) prosthetic hand, which is equipped with three force sensors (to measure the grasp force) and a position sensor (encoder). Wearing silicone glove, the MPH-III looks like a natural hand. The core of the control system is a single chip microcomputer (C8051F320, Silicon Laboratories) [9]. The control board has a USB interface on which data can be exchanged between prosthetic hand’s control board and computer.
2.2. EMGs Acquisition Device
Two EMG sensors are adopted to acquire EMG signals. A 10-bit A/D converter is used to digitize signals (sampling ratio is 1 kHz). A Bluetooth module is utilized to transmit the EMG signals to the controller of prosthetic hand. All the components of the EMGs acquisition device are fixed on a ribbon (see Figure 3) to make it convenient for users to wear, and the positions of the EMG sensors on the ribbon are adjustable because the detecting positions for different users are different.
2.3. Tactile Representation Device
The tactile representation device (TRD) consists of six vibration motors, which are controlled by an electronic system equipped with a Bluetooth module. The vibration motors are fixed on a ribbon (see Figure 4). When the ribbon is worn on the upper arm, the distribution of vibration motors is shown in Figure 5. These vibration motors may generate stimulation on the skin of the amputee. The TRD receives the force information of the prosthetic hand from the control module via Bluetooth module. Then command is generated to control the vibration motors orderly, so that the user can perceive the force states of the prosthetic hand.
3. Design of the Control Strategy
The designed control strategy is based on manipulation patterns of the natural hand. User plays a dominant role in the system. When she/he intends to operate the prosthetic hand, a control command will be generated from the brain and transmitted to motor nerves through spinal cord. The motor nerves control the muscle movement. And the EMG signal will be generated on the surface of skin at that time. These EMG signals can be used to control the prosthetic hand after processing. Processing these EMG signals probably involves amplifying, filtering, acquiring, feature extracting, and motion identifying. The force sensors installed in fingers of the prosthetic hand are used to detect the grasp force. On one hand, the grasp force information feeds back to the controller; on the other hand it feeds back to the user through a certain tactile feedback type. According to the specialty of the object, users can combine visual and tactile message and adjust the control strategy in time. In this way, three closed loops are achieved to control the prosthetic hand: the first one is from user to prosthetic hand then back to the user through tactile feedback device, the second one is from controller to prosthetic hand then back to the controller, and the last one is from the user to prosthetic hand then back to the user through user’s eyes. The functional scheme of this control strategy is shown in the Figure 6.
3.1. EMG Self-Learning Recognition
The accuracy of the pattern recognition of EMG signals is directly related to the control effect of the prosthetic hand. The surface EMG signal can illustrate the activity of skeletal muscles, and its amplitude ranges from less than 50 μV to 30 mV, and the frequency range is from dozens to hundreds Hz, depending on the muscle under observed.
Because the EMG signal strengths of different users are different and due to some other factors, an EMG self-learning recognition method is proposed as shown in Figure 9. Before the pattern recognition, EMG signals are processed as shown in Figure 7.
EMG sensors are attached to the surface of the muscle to acquire the EMG signals. Figures 8(a), 8(b), and 8(c) show the amplified EMG signals, the shaped EMG signals, and the filtered EMG signals, respectively.
(a)
(b)
(c)
In Figure 9, and are the EMG signals which are preprocessed as shown in Figure 7. The values of and are between 0 V and 3.3 V. The EMG learner is designed to record and update the minimum and maximum values of the EMG signals. A moving window is adopted, and the principles of the recording and updating are as follows:
Step 1. Calculate the average value () of the data in moving time window:where is the length of the time window. represents the current data of the EMG signal and represents the previous th data of the EMG signal.
Step 2. Update the data:where and are the maximum and minimum values of the , respectively. The initial value of is set to 0, and the initial value of is set to 3.3.
The maximum and minimum values of the , , and are updated by using the same method.
Adjustable factors (, ) are defined as follows:The output, , is determined by and , which is as follows:where is the scale factor.
In free space, reflects the closing or opening speed of the prosthetic hand, and in restricted space, reflects the grasp force.
With the help of this recognition method, the influence of the diversity of the EMG signals on the accuracy of the pattern recognition is reduced.
3.2. Backstepping Controller with Stiffness Fuzzy Observation
To realize the position control and grasp force control, a backstepping controller with stiffness fuzzy observation (BCSFO) is designed in this paper. The designed controller is shown in Figure 10. The input signal, , is the output of the motion recognizer. is the grasp force measured by force sensor which is attached to the prosthetic hand’s finger, is the scaling factor, is the differential scaling factor, and is the voltage applied to motor. and are, respectively, the position and velocity of the prosthetic hand’s finger. is object’s stiffness and the stiffness is defined aswhere is the original size of the object and it is the position of the prosthetic hand’s finger when the object and finger contact for the first time.
In free space, the output of the planner is as follows:where is the position when prosthetic hand’s finger changes from restricted space to free space.
In restricted space, the stiffness of the object and the deformation of the structure may affect the relationship between the grasp force of the prosthetic hand and the angle of the motor rotation. Since the range of the designed grasp force is relatively small (0~30 N), the influence of the structure deformation is ignored, and the output of the planner in restricted is as follows:The system model of prosthetic hand is selected as follows:where , , and are, respectively, the inertia, damping, and unknown nonlinear damping of the system. is the output of the system, and it is the control voltage of the motor.
Two error subsystems are defined aswhere is a virtual control variable (i.e., the estimate of ):Take , ; thenThe Lyapunov function of the first error subsystem is defined asThe Lyapunov function of the second error subsystem is defined asTake the control law for as follows:ThenBy the Lyapunov stability theory, the designed control system can reach a steady state in a limited time, so the system has the stability.
In order to grasp objects with different stiffness stably, a stiffness fuzzy observer is designed to adjust . The employed fuzzy logic reasoning has double inputs ( and ) and a single output . and are, respectively, the stiffness and the stiffness’ derivative. is adjusted as follows:During fuzzification and defuzzification, the inputs and the output types are defined as several fuzzy sets with trigonometry/trapezoidal membership functions as shown in Figure 11. The fuzzy reasoning rules for are shown in Table 1. Figure 12 shows the overall input–output relationship of the fuzzy logic reasoning.
(a)
(b)
3.3. Tactile Representation System
There are lots of neurons distributed on the skin all over the body. The mechanism is called tactile perception. These neurons can receive the information (temperature, humidity, pain, pressure, vibration, etc.) outside the body. One of the most common phenomena of tactile perception is that when natural hand touches an object, the characteristics of the object such as the shape and surface roughness can be felt. Tactile perception is a way to obtain information. It involves human physiology, tactile physiology, tactile phenomenon, and so forth [10, 11].
When working in this, tactile representation is important. It is important to transmit grasp force and slide information back to the user.
3.3.1. Grasp Force Detection
In order to achieve a more comprehensive and more accurate measurement of the grasp force, several force sensors are developed [12, 13]. Figure 13(a) shows the thumb of the prosthetic hand. A force sensor (FSS SMT Series, Honeywell) is installed on the tip of the finger; the other two half bridges (Wheatstone half bridge) are separately fixed on the middle and the root of the thumb. This kind of distribution is utilized because the force points on the finger may be different when user grasps objects.
(a)
(b)
The sensitivity of the force sensor is 12.2 mV/N. The force, , applied to area 1 is as follows:where is the output of the force sensor.
Four strain gauges (sg1, sg2, sg3, and sg4) which have the same properties (material, size, strain coefficient, etc.) are attached to the thumb of the prosthetic hand as shown in Figure 13. , , , and represent the resistance of these strain gauges, respectively. And , , , and are of the same value.
The thumb of the prosthetic hand is made of aluminum. There is a deformation in the thumb when the force is applied to it. The deformation of the thumb will lead to the corresponding deformation of the strain gauges. And the deformation of the strain gauges will lead to the change of the resistances’ value of these strain gauges.
Take sg1 and sg2; for example, tensile deformation and compressive deformation occurred in sg1 and sg2, respectively, when the force is applied to the thumb. Tensile deformation of sg1 results in the increasing of the resistance’ value of sg1. On the contrary, compressive deformation of sg2 results in the decreasing of the resistance’ value of sg2. The variations of the resistance’ value of sg1 and sg2 are considered to be the same for the reason that the deformation is small and sg1 and sg2 have the same properties.
sg1 and sg2 are connected in the measurement circuit as shown in Figure 13(b). sg1, sg2, and two additional resistances (, ) constitute a Wheatstone bridge. The output of the circuit, , is as follows:where is the power voltage supplied to the circuit, is the change of resistance of sg1 and sg2, and is the value of and .
Because the force, , applied to area 2 (as shown in Figure 13) is proportional to the deformation of the strain gauges, it can be obtained by measuring the output of the circuit (, shown in Figure 13(b)).
The force, , applied to area 3 can be obtained by using the same method.
When the prosthetic hand contacts the object in areas 1, 2, and 3, the grasp forces are measured by force sensor, half bridge 1, and half bridge 2, respectively.
3.3.2. Sliding State Detection
A polyvinylidene fluoride (PVDF) piezoelectric film is often used for tactile sensor design to detect the sliding for its excellent dynamic characteristics and physical characteristics, such as light quality, soft, large contact resistance, and plasticity [14, 15]. In this paper, a PVDF piezoelectric film is attached to the surface of the silicone glove of the prosthetic hand. The signal in Figure 14(a) shows a sliding during a grasp operation. In this figure, indicates the time span of a sliding process.
(a)
(b)
To reduce the influence of noise on sliding detection, the filtering process is implemented. In addition, a small threshold is subtracted. Figure 14(b) shows the sliding signals after these two processes. Compared with the sliding signals, temperature is a slow change variable. The influence of the temperature is ignored when using PVDF to detect the slide state.
According to the signals shown in Figure 14 and the characteristics of the PVDF, the number of zero-crossing per time unit is adopted to indicate the sliding situation.
3.3.3. Tactile Representation
The main tactile representation techniques are pneumatic stimulation, vibration stimulation, functional neuromuscular stimulation, thimble stimulation, thermal stimulation, and so forth [16]. The vibration stimulation is adopted because it is convenient to use and does not cause damage to human body. The vibration coding patterns, including vibration frequency, amplitude, duration, rhythm, and order [17, 18], affect the accuracy of the tactile perception of the user directly. However, due to the existence of the tactile illusion phenomenon, an efficient vibration coding pattern must be established. The tactile illusion is a kind of phenomenon in which tactile perceptions do not match the objective stimulation. Many kinds of tactile illusion phenomena have already been discovered, such as phantom sensation and apparent movement. The reasonable utilization of these phenomena may contribute to realization of the tactile representation.
A coding pattern (vibration coding) of grasp force is designed. As shown in Figure 15, when the grasp force is detected, motor number 2 begins to vibrate, and then motor number 1 and motor number 3 begin to vibrate. is the beginning time of motor number 2, is the beginning time of motors number 1 and number 3, and is the ending time of all these motors. is the time of grasp force being detected as well, and shows the time when the grasp force reduces to zero. The interval between and is 300 ms. The grasp force measured by the sensor is used to modulate the vibration strength according to the principle as shown in formula (19), and the vibration frequency is set to 100 Hz. By this method, the user may feel his/her arm being grasped:where I, II, III, IV, and V are the vibration strength levels; the relationship between vibration strength levels and the voltage applied to the motor is as shown in Table 2. is the ratio of grasp force detected by the force sensors to the maximum allowable value of the grasp force:where is the grasp force detected by the force sensors and is the maximum allowable value of the grasp force.
A sliding tactile representation coding pattern is designed according to the phenomenon of apparent movement (see appendix) [19]. The vibration strength is set to level III, and the vibration frequency is set to 100 Hz. As shown in Figure 16, , , and are, respectively, the beginning time of the motors number 4, number 5, and number 6. The ending times of these three motors, respectively, are , , and . The time intervals of ~, ~, ~, and ~ are 250 ms. is the time of sliding situation being detected as well. By this method, the user may feel something sliding on his/her arm.
4. Experiments and Results
To verify the validity of the control strategy presented in this paper, lab-based experiments were carried out.
4.1. Evaluation Experiments of the EMG Self-Learning Motion Recognition Method
Motion recognition experiment was conducted to verify the effectiveness of the EMG self-learning motion recognition method. In the experiment, four EMG sensors were distributed on the participants’ forearms as shown in Figure 17. The participants executed hand motions (hand grasp and hand open) according to the commands. Figure 18 shows the EMG signals when participants executed the hand motions.
According to the laminating positions of the EMG sensors, EMG signals coming from four sensors can be combined into four groups as shown in Table 3. Two recognition methods, including one shown in Figure 1 and the other EMG self-learning recognition method, have been applied to these four groups EMG signals. Figure 19 shows the recognition results by using the recognition method as shown in Figure 1, and Figure 20 shows the recognition results by using the EMG self-learning recognition method.
To evaluate the consistency of the recognition results, the correlation coefficients between every two recognition results were calculated as shown in Tables 4 and 5. Correlation coefficient () is a measurement of the linear correlation (dependence) between two variables and . It is widely used as a measurement of the degree of linear dependence between two variables. The formula for iswhere cov is the covariance, is the standard deviation of , is the mean of , and is the expectation.
From the results of the EMG motion recognition experiment, recognition results by using the EMG self-learning recognition method have a better consistency. EMG self-learning can reduce the influence on EMG signals caused by the uncertainty of the contacting position of the EMG sensors.
4.2. Evaluation Experiments of the Backstepping Controller
To verify the effectiveness of the BCSFO, velocity tracking and force tracking experiments were implemented. The designed speed was inputted to the controller in free space, while the designed grasp force was inputted to the controller in restricted space. The results are shown in Figures 21 and 22. The objects with different stiffness were grasped by prosthetic hand in the experiment, and the results are shown in Table 6. Table 6 contains the mean value () and mean variance of the force tracking error. The results show that the controller can track the designed velocity and designed grasp force quickly; the tracking error is in the acceptable range.
(a)
(b)
(a)
(b)
4.3. Tactile Representation Coding Experiment and Results
Ten nonamputee volunteers (five males and five females, aged from 22 to 27) were chosen to use the MPH-III. The EMG acquisition device was worn on the forearm, and the EMG sensors were put on a pair of antagonistic muscles. The TRD was worn on the upper arm. Five minutes or more was given to each participant to get familiar with the prosthetic hand. After experiments, they graded these three control strategies.
The TRD outputs the grasp force vibration coding for 25 times. Among them, the sliding state vibration coding occurs for five times randomly. The participants were asked to record the vibration state according to their feelings. Each vibration state occurs 5 times. Each participant accomplished experiments without influence from others. The results of the experiment are shown in Figures 23 and 24.
In Figure 23, -axis represents the actual strength level outputted by the TRD; -axis represents the times the participants recorded the strength level. The different strength levels the participant felt are represented by different colors. That is to say, the strength levels I, II, III, IV, and V the participants felt are represented by red, yellow, blue, green, and black, respectively. Red, yellow, blue, green, and black represent the strength levels the participator felt which are I, II, III, IV, and V, respectively.
Take the experimental results of participant 1; for example, when the actual strength level is “I” (see -axis), the times of the strength levels I, II, III, IV, and V perceived by the participants are 5, 0, 0, 0, and 0, respectively. When the actual strength level is “II,” the times of the strength levels I, II, III, IV, and V perceived by the participants are 0, 5, 0, 0, and 0, respectively. When the actual strength level is “III,” the times of the strength levels I, II, III, IV, and V perceived by the participants are 0, 0, 4, 1, and 0, respectively. When the actual strength level is “IV,” the times of the strength levels I, II, III, IV, and V perceived by the participants are 0, 0, 1, 3, and 1, respectively. When the actual strength level is “V,” the times of the strength levels I, II, III, IV, and V perceived by the participants are 0, 0, 0, 1, and 4, respectively.
Figure 24 shows the average values of each strength level perceived by the participants in each actual strength level.
The results of Figures 23 and 24 show that the participants can distinguish the strength level precisely when the strength level of tests is given in low level (levels 1 and 2), while on the condition of high level (levels 3 to 5), the strength level perceived by the participants in each actual strength level fluctuates around the actual strength level. The errors between false records and correct records are no more than 1 level. The results of all the participants show the same trend. In the view of the experimental results, this kind of vibration coding can help users to perceive the grasp force of the prosthetic hand.
The experimental result of tactile representation of sliding state is as its expected because the coding pattern is simple and there is no relationship between the siding coding pattern and grasp force tactile feedback. Most of the participants can make the judgment whether the sliding occurs or not.
4.4. Grasp Experiments and Results
The force control strategy (M1) shown in Figure 1, the EMG self-learning control strategy (M2), and control strategy shown in Figure 2 (M3) were compared with each other. These three strategies all contain visual feedback. The differences are that M2 has EMG self-learning function and the controller in M2 is BCSFO. Besides the features of M2, M3 has an additional function of tactile feedback. The experiments assume that the functions of visual feedback in these three strategies are the same. Five minutes was given for participants to get familiar with these control strategies. Then, ten participants were asked to grasp and lift a paper cup which was full of water (see Figure 25). In this process, the participant should keep water without overflowing or dropping. All control strategies were tested for ten times. The number of successful lifting was used to quantify the performance of the control strategy. And the participants were asked to evaluate every control strategy after experiments according to their feeling about the performance of the control strategy. And the performance includes flexibility and usability. The best control strategy was marked as highest grade, 10. Table 7 shows the results of the experiment. Figure 26 shows the grasp force curve which was recorded in the process of a participant grasping a paper cup by using the control strategy of M3.
In this experiment, the comments from all participants show that the M3 is the best one among these three strategies. When using prosthetic hand without EMG self-learning, most of the participants have to adjust the positions of the EMG sensors on the skin surface, while it is unnecessary to adjust by using the strategies with EMG self-learning.
5. Conclusions
The control strategy with tactile feedback for EMG prosthetic hand is described in detail. Aiming at reducing the influence on the EMG signals which comes from the attaching positions of EMG sensors, an EMG self-learning recognition method is proposed. A BCSFO is proposed to realize the velocity proportional control in free space and grasp force tracking control in restricted space. A tactile representation system is designed to help the user perceive the state of the prosthetic hand, and the states include grasp force and slid information.
The experiments are implemented to verify the effect of the proposed control strategy. And the results show that the different contacting positions between sensors and arm lead to the variance of the EMG signals, and this kind of influence can be reduced by the proposed EMG self-learning method. The proposed BCSFO can meet the requirements of the prosthetic hand (velocity proportional control in free space and grasp force tracking control in restricted space). And the results of the grasping experiments show that the strategy with EMG self-learning method and tactile feedback (M3) is better than the strategy of the force control (M1) and the strategy of the EMG self-learning control (M2) in the aspect of control effect of the prosthetic hand.
Moreover, all the participants think that the EMG signal self-learning pattern recognition method is much more helpful and convenient in the process of manipulating the prosthetic hand.
For the future work, after lots of experiments we will research a more effective coding pattern for tactile representation, which would be easily accepted by amputees.
Appendix
This phenomenon was described in detail in [19]. Its basic principle is as follows.(1)Point A starts to vibrate.(2)Point B starts to vibrate.(3)Point A stops.By applying vibration stimulation in this order, the participant will get an illusion that point A is moving towards point B (see Figure 27).
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
This paper was supported by the Nature Science foundation of China under Grant no. 61325018.