Abstract
This work gives a biologically inspired control scheme for controlling a robotic system. Novel adaptive behaviors are observed from humans or animals even in unexpected disturbances or environment changes. This is why they have neural oscillator networks in the spinal cord to yield rhythmic-motor primitives robustly under a changing task. Hence, this work focuses on rhythmic arm movements that can be accomplished in terms of employing a control approach based on an artificial neural oscillator model. The main challenge is to determine various parameters for applying a neural feedback to robotic systems with performing a desired behavior and self-maintaining the entrainment effect. Hence, this work proposes a task-oriented parameter tuning algorithm based on the simulated annealing (SA). This work also illustrates how to technically implement the proposed control scheme exploiting a virtual force and neural feedback. With parameters tuned, it is verified in simulations that a 3-DOF planar robotic arm traces a given trajectory precisely, adapting to uneven external disturbances.
1. Introduction
Researches on a robotic control of allowing natural behaviors have been given increasing attention due to the fact that animals are capable of naturally behaving under uneven environmental changes. For instance, animal natural movements, such as bipedal or quadrupedal walking and fluttering wings, are realized by means of interaction between the musculoskeletal system and neural oscillator networks, termed central pattern generators (CPGs), on spinal level in the central nervous system [1, 2]. In humans, specifically, each limb segment system joined with muscles and tendons shows a mechanical spring-like motion connected to a mechanical link by neural signals. This is called the neuromusculoskeletal system of humans. Hence, if a robot is built similarly as the human-like system with CPGs then it may be possible to show natural behavior. Here, the function, known as the entrainment property, of CPGs may be replaced with an artificial neural oscillator network [3]. Thus our attention focuses on the robotic application of the entrainment function to adapt the nervous system to the natural frequency corresponding to a sensory feedback.
The application with an artificial neural oscillator to a robot control was started from Matsuoka’s work [4, 5]. He showed mathematical proof on the steady state rhythmic patterned output including the output property according to various networks. Inspired by his works, his neural model has expanded for the various applications to the fields of robotics such as various walking robots for adaptive locomotion [6–9], crawling, swimming [10, 11], and flying [12]. In addition, employing other models, biologically inspired motions also have been proposed and verified [13].
Although the CPG-inspired control approaches contribute to enhancing adaptive mobility for a locomotive robot irrespective of uneven environments, novel potential from biological inspiration has been limited till now owing to the difficulty of producing a desired motion with many parameters and nonlinearity. In another application, a rhythmic arm motion of neuromechanical system coupled with the network of neural oscillators was performed under unknown collisions successfully [14]. However even in the application to the robot arm control [15], precision control [16, 17] according to a given motion was not exposed clearly. Recently, although parameter optimization methods [18, 19] for satisfying performance requirements were proposed successfully, the dynamic performance of the coupled system interacting with various environments was not considered. Here, many parameters and the nonlinear property of the neural oscillator with the sensory feedbacks from unknown environments make parameter tuning difficult.
From early works, in order to follow a given trajectory including complicated behaviors in dexterous manipulation, meaningful works [20, 21] have been performed with substantial difficulty of selecting the parameters. But such approaches are not clear and much time and efforts are needed in viewpoint of precisely performing an energy-efficient motion. Thus this paper presents a parameter tuning framework based on simulated annealing (SA) according to a priority condition. In the proposed approach, the control gains and parameters of the neural oscillator networks are selected in terms of cost energy function sets prioritized as closely as possible without interfering with higher-priority constraints [22]. This method is incorporated to a three-link robotic arm coupled with the neural oscillators for performing a given task considering task failures. In addition, extensive simulations for the verification explore dynamic performances with respect to the robustness of the tuned parameters and adaptive behaviors against unknown disturbances.
In this paper, the artificial neural oscillator as CPGs employed, at first, is briefly illustrated with the entrainment property. And the control method of coupling a robotic system with the neural oscillator is described. Section 3 gives how to tune the control gains with the oscillator’s parameters based on the proposed optimization process in order for an imposed task. The verifications of the proposed tuning methodology are dealt with in Section 4 and Section 5 gives the properties of novel self-adaptive behavior in the proposed control approach. Finally, Section 6 draws conclusions.
2. Neural Oscillator Based Control
2.1. Neural Model Proposed by Matsuoka
It has been known that the CPG in vertebrates generates rhythmic-motor patterns and allows them to adapt to uneven external disturbances using sensory feedbacks. In order for technically applying such novel characteristics to a robotic system, the neural oscillator model suggested by Matsuoka is employed in this work. The model is composed of two neurons arranged for mutual inhibition as seen in Figure 1. If the neural model is coupled to a robotic system, the entrainment property of the neural oscillator enables the coupled system to show biologically inspired behavior that easily interacts with various environments: where and are the inner state of the th neurons for . Here, the subscripts “” and “” denote the extensor and flexor neurons, respectively. indicates the degree of adaptation. The adaptation constant of the th neuron is represented by . The output of each neuron is taken as the positive part of and the final output of the oscillator is the difference in the outputs between the extensor and flexor neurons. is a connecting weight from the th neuron to the th neuron: are 0 for and 1 for . indicates the total input from the neurons arranged to excite one neuron and to inhibit the other, respectively. and mean the time constants of the inner state and the adaptation effect, respectively, and is an external input with a constant rate. is a weight of the extensor neuron or the flexor neuron and indicates a sensory input from the coupled system which is scaled by the gain .
2.2. Entrainment Property of Neural Oscillator
In Figure 2, the entrainment procedure caused by the sensory feedback of the neural oscillator is well illustrated. There are many unknown parameters related with the entrainment property. From our previous works [16, 17, 20], insight and technique for parameter tuning have been given with various applications. Here, the neural dynamics should satisfy the condition [21] for the stable limit cycle behavior, and then the sensory feedback gain which is most sensitive to entrainment should be properly tuned. In simulation, the gain was set with 0.02, 0.2, and 0.576 as seen in Figures 2(a) to 2(c), respectively. When is set to 0.2 and 0.24, the output of the neural oscillator does not entrain the sensory feedback signal (see Figures 2(a) and 2(b)). In Figure 2(c), when is tuned manually as 0.576, the closely entrained output is generated from the neural oscillator. It can be verified from the result that the sensory feedback to be tuned enables the neural oscillator to follow an unknown input signal, although a nonperiodic input is fed into the neural oscillator [16].
(a)
(b)
(c)
In general, in order to apply the entrainment property to the robotic systems control, a desired torque input to the joint can be expressed aswhere indicates the proportional gain of the joint, is the damping coefficient, is the joint angle, and is the output of the neural oscillator that produces the neural command corresponding to the joint. The neural oscillator senses the sensory signal fed again from the joint and its output may be changed within a range corresponding to the sensory input. This characteristic is called “entrainment” that can be regarded as the tracking of sensory feedback signals. This causes that the coupled system shows the performance on adaptive behaviors interacting with various environments.
In Figure 3, the simulation results indicate the dynamic responses including three characteristics based on the entrainment property with manually tuned parameters of the neural oscillator: (1) motion limitation employing the saturation effect of the output of the neural oscillator caused from a sensory feedback, (2) fully entrained output for interacting with various unknown environments, and (3) motion synchronization with a neural network for a multi-DOF robotic systems control. The first characteristic on the saturation effect is well illustrated between 5 s and 20 s. This property can be applied to the motion limitation using high gains. And the entrained motion for interacting with unknown environments through the sensory feedback is illustrated from 25 s to 30 s. In addition, in the next step, while entraining, we examined whether the synchronized motion by means of the network is possible or not. In ①, (a) and (b) of Figure 3 have the phase difference when the network of the neural oscillators is not connected. If the network between the neural oscillators is connected with a network gain, the phase becomes synchronized as shown in ② of Figure 3. This is the third property (3) of the neural oscillator mentioned above.
(a)
(b)
2.3. Control of the Coupled Mechanical System with Virtual Force and Neural Oscillator
If dynamics with th DOFs is considered for controlling a robot as seen in Figure 4, the governing equation could be expressed as where denotes a vector of joint positions and and denote an inertia matrix and a centrifugal and Coriolis matrix, respectively. is a vector of gravitational forces or torques and is a control input vector at the level of generalized forces described in (3):wherewhere and are the spring stiffness and damping coefficient, respectively, for the virtual components. is the joint damping in which is chosen within . and are the proportional gain and output of the neural oscillator that produces neural commands for entrainment, respectively. is the specified target position vector and is the vector of the current position in Cartesian coordinates.
In (4), the neural oscillator based control is composed of a virtual force and neural sensory feedback. The first term indicates the task space control of producing a desired force in terms of a spring and a damper. In the second and third terms, the desired input is yielded by an entrainment rule of the neural oscillator as described in Section 2.2. This enables a robot coupled with the neural oscillator to show self-reflexive motions against uneven environmental perturbations. And a joint damping, , plays an important role in restraining a certain self-motion under a redundant system.
3. Parameter Tuning Based on Simulated Annealing
Because rhythmic motions of the dynamic system are generated by the neural oscillator, it is usually difficult to obtain the desired motion required in the target task. This is the reason that there are many parameters in the nonlinear neural oscillator required to be tuned, and various responses are produced according to the interoscillator dynamics and the effect of the entrainment property in terms of sensory feedbacks (see Figure 3). Thus, for strengthening the advantageous of the proposed control approach employing the neural oscillator, this section informs a parameter tuning method considering the priority of the cost functions which is based on simulated annealing (SA) [23, 24].
From the optimization problem to minimize cost functions , the parameters () can be adopted by means of the proposed tuning method as seen in Figure 5. The parameters related with the virtual force and the neural oscillators are continuously modified during the cooling schedule with initial temperature , the learning rate , and a random number between −1 and 1. ConsiderNote that if , the state corresponding to the th iteration is accepted and saved to database. If not, the transition probability, , gives the new state by where a random value is set from 0 to 1 uniformly. The temperature cooling schedule is , where is the effective annealing gain and temperature-dependant normalization factor notes . In case that and or , the state is denied. The parameter tuning procedure is terminated when the temperature equals zero.
In addition, the SA algorithm cannot be applied to parameter tuning for the robot task control directly because it is difficult to distinct whether the objective task is attained properly with the selected parameters or not. Hence, the optimization process is modified and upgraded in this work. First, each energy level calculated by the cost function sets is weighed and prioritized with a given objective and total energy is evaluated. Then, the task completion judgment step is newly considered as shown in Figure 5. If the desired task is not satisfied, the algorithm searches for and reselects the parameter set which gives the lowest energy level.
4. Application and Verification
In this section, whether a desired motion for multi-DOFs robotic systems coupled with the neural oscillators can be attained or not is verified. Also it is confirmed that the main characteristics of the neural oscillators illustrated in Figure 3 are helpful to keep the joint repeatability in a redundant system. For this, each joint of a three-link robotic system is coupled with the three neural oscillators, respectively, as shown in Figure 6. And the specilized SA algorithm for the parameter tuning is disclosed. The dynamics parameters and the initial control gains for the dynamic computer simulation are described in Table 1.
4.1. Implementation to a Real Three-Link Robotic Arm
The desired trajectory is set to be of (4). In simulation, the circular trajectory is given to the end-effector of a 3-DOF planner robotic arm (see Figure 6). The SA-based parameter tuning algorithm proposed in Figure 4 is additionally upgraded, since many parameters related with the virtual forces and the neural oscillators effect on attaining an objective task and embodying the entrainment property of the proposed control approach. Hence energetic cost functions for dynamic performance [25] of the robot arm and a tuning process are newly considered as seen in Figure 7. The proposed parameter tuning rule basically follows a hierarchical priority condition illustrated in Figure 7 for the reason that the solutions to consider the entrainment effect in the cost functions of the second step are subjected to the solution sets of satisfying the constraint cost function for a desired motion of the first step. In addition, in each process, the energetic cost functions () are minimized for amplifying dynamic performance such as energy consumption and energy-efficient motion. Here, we define the parameter tuning rule based on the SA incorporating the three steps of Figure 7.
(1) Step 1. Set the initial parameters of the neural oscillator exploiting the following cost functions:subject towhere and is the natural frequency and the amplitude of the neural oscillator for generating a desired motion. And indicates the natural frequency of the output produced in the neural oscillator. and mean the maximum and minimum amplitude constraints, respectively. is the gain for the performance.
(2) Step 2. After the initial procedure of Step 1, the parameter tuning algorithm considering the cost functions corresponding to the first priority is operated as illustrated in Figure 7. Here the priority rule is decided and divided due to the multiple objectives. In this step, the parameters are tuned for attaining the desired motion, considering the control input of the neural oscillator for entrainment. The cost functions are expressed as a weighted linear combination:wherewhere is the constraint cost function for tracing a given trajectory, indicates the energy applied to the 3-DOF robotic arm, represents the dissipative energy from the viscous friction of each joint, and the energy additionally loaded from the neural oscillators can be described as . and denote stiffness and damping values as a weight, respectively.
(3) Step 3. The parameters selected in the procedure of Step 2 are fixed. Then, the third step focuses on enhancing the entrainment effect that contributes to repeatability and adaptability of the joints of a multi-DOF robotic arm. Thus the parameters related with the internal neural dynamics are tuned by means of the following cost functions:where , where indicates a spring constant.
4.2. Verification through Simulation
Incorporating Steps 1 to 3 in sequence of the proposed tuning method, we are able to acquire the appropriate initial and tuned parameters as seen in Tables 1 and 2. Figures 8 and 9 show the tuning procedure of the parameters described in Table 2 corresponding to Steps 2 and 3, respectively. Figures 8(a) and 9(a) indicate the cooling state in terms of the cooling schedule. The cooling or annealing gain was set to 0.98. According to the cooling schedule, control gains and inner parameters of neural dynamics are well tuned (see Figures 8(b)–8(f) and 9(b)–9(f)). In Figures 8(b) and 8(c), the spring stiffness and damping coefficient for the virtual force to guide a desired motion are represented. Control inputs for the entrainment effect are converged to the steady state value in Figures 8(d)–8(f). Figures 9(c)–9(f) illustrate the procedure for selecting parameters (inhibitory connecting weight, rising and adaptation time constant, sensory gain, and tonic input gain) of the neural oscillator coupled to the 1st joint. The steady state values converged in Figures 8(a) and 9(a); the tuning process was performed successfully with minimum energy, as shown by the consumed cost function values.
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
Even though the optimized parameters correspond to the lowest cost energy level needed to execute the required task, completion of the task may fail. Hence, the proposed SA-based approach confirms whether the constraint condition on the position error is satisfied or not through dynamics simulation. From Figures 10 and 11, if the parameters of the proposed controller are properly selected for a desired task such as the circular motion, the given motion could be achieved clearly. In addition, the motion repeatability for a redundant robotic system is compared in Figures 10(b) and 11(b). Differently from Figure 10(b), each joint motion shown in Figure 11(b) sustains the repeatability of the joint motion. This is why each joint is tightly coupled with the neural oscillators. It is verified that each joint motion in Figure 11(b) is fully entrained with the outputs of the neural oscillators. This property enables the arm to sustain the given task against changes in parameters of arm kinematics and dynamics as well as disturbances.
(a)
(b)
(a)
(b)
5. Discussion on Entrainment Property
We, in this section, investigate the dynamic performances of the three-link robotic arm coupled with the neural oscillators. The parameters of the neural oscillators are set for enhancing the entrainment property by the proposed tuning algorithm. Entrainment implicates that their outputs lock onto sensory feedbacks to interact with unknown environments in phase as mentioned in Section 2.2. In simulation, during the circular motion of the end-effector shown in Figures 10 and 11, the robot arm collides with the unknown virtual walls: a vertical wall (Case I) and sphere wall (Case II). Here, it is confirmed that the end-effector of the robotic arm coupled with the neural oscillators traces the changed paths compliantly according to the surface of an unknown wall. The parameters of the neural dynamics were set in Table 2 and remained unchanged for all cases. In particular, Case III finally examines whether the control approach tuned precisely can be applied even in the change of the desired trajectory.(1)Case I. A virtually built-up vertical wall.(2)Case II. A virtually built-up sphere wall.(3)Case III. A variability of the desired motion.
In Figures 12 and 13, unknown environmental contact models such as a vertical and sphere wall for various collision effects make use of the unilateral spring-damper elements in simulation. The end-effector of the robotic arm is collided against the virtually modelled walls from 4.6 s to 7.5 s, respectively, as seen in Figures 12(a), 12(d), 13(a), and 13(d). Note that the neural model-coupled robot arm shows good adaptive motion as illustrated in Figures 12(d)–12(f) and 13(d)–13(f) in terms of the entrainment property comparing with Figures 12(a)–12(c) and 13(a)–13(c), although there happens to be collision with unknown walls irrespective of the shape. The outputs generated from the neural dynamics are helpful in keeping each joint motion. This is confirmed from the dash dotted lines of Figures 12(f) and 13(f). The blue line and red line of Figures 12(b), 12(e), 13(b), and 13(e) indicate the real motion drawn by the end-effector of the robotic arm and the desired motion, respectively. In addition, without coupling with the neural model, the repeatability of each joint motion is not guaranteed even in a rhythmic motion (see Figures 12(c) and 13(c)). Differently from this, comparing (d) of Figures 12 and 13 with Figures 12(a) and 13(a), respectively, the repeatability of the joint position in the 2nd joint is verified from ① and ② in Figures 12(d) and 13(d). ① and ② are the joint motions before and after the collision. In both of them, it is observed that the 2nd joint draws a specific motion repeatedly.
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
The additional novel feature of the tuned coupled system described in Case III is explored in Figure 14. Experimental results of Figure 14 indicate that there are significant improvement and possibility on the variability of the motion in the coupled robotic arm with the neural oscillator. Although the desired motions of the end-effector are given as the vertical and horizontal motions as seen in Figures 14(b) and 14(e), respectively, the coupled robotic arm is able to follow the new desired motions regardless of the parameter tuning as shown in Figures 14(a) to 14(b) and 14(d) to 14(e). It also can be verified from Figures 14(c) and 14(f) that each joint of the robotic arm keeps the motion repeatability.
(a)
(b)
(c)
(d)
(e)
(f)
As concluding remark of this section, reflex function by means of reciprocal action between the flexor and extensor neurons contributes to enhancing the natural behavior of a robot arm. In particular, irrespective of the change of a desired motion, the proposed control approach properly tuned for an imposed motion is possible to exhibit the entrained motion corresponding to a variety of environments adjusting the control outputs in an adaptive way.
6. Conclusion
This work introduces the parameter tuning algorithm for the application of the biologically inspired control approach to the skillful manipulation of a robotic arm. This can be regarded as an example for the embodiment of a human-like behavior property using a robotic arm whose joints were coupled to neural dynamics composed of a network. Differently from previous works that were only capable of rhythmic-motor patterned outputs, the main contribution of this work is to allow the end-effector of a robotic arm to precisely follow a predefined given trajectory with entrainment. The entrainment properties are helpful in keeping the motion repeatability and enhancing the compliance of the robotic arm. For technically accomplishing this, we propose a task-oriented parameter tuning method for properly determining the parameter values of the neural dynamics considering the priority condition of the cost functions based on the simulated annealing method. The novel performances of the proposed control and parameter tuning method were verified with various simulations. In addition, the detailed demonstration that the 3-DOF robotic arm compliantly responds to unknown environmental changes was performed with the parameters tuned. Finally, it could then be confirmed that the neural oscillator based control exhibits the variability of the robotic arm motion with the parameters set for only a certain motion.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2014R1A1A2058473) and also was conducted by the Research Grant of Kwangwoon University in 2014.