Bio-Inspired Learning and Adaptation for Optimization and Control of Complex SystemsView this Special Issue
Research Article | Open Access
Chunyu Yang, Yiming Xu, Linna Zhou, Yongzheng Sun, "Model-Free Composite Control of Flexible Manipulators Based on Adaptive Dynamic Programming", Complexity, vol. 2018, Article ID 9720309, 9 pages, 2018. https://doi.org/10.1155/2018/9720309
Model-Free Composite Control of Flexible Manipulators Based on Adaptive Dynamic Programming
This paper studies the problems of tip position regulation and vibration suppression of flexible manipulators without using the model. Because of the two-timescale characteristics of flexible manipulators, applying the existing model-free control methods may lead to ill-conditioned numerical problems. In this paper, the dynamics of a flexible manipulator is decomposed into two subsystems which are linear and controllable at different timescales by singular perturbation (SP) theory and a model-free composite controller is designed to alleviate the ill-conditioned numerical problems. To do this, a model-free composite control strategy is constructed which facilitates in designing the controller in slow and fast timescales. In the slow timescale, the slow subsystem controller is designed by adaptive dynamic programming (ADP) based on the measurements of the slow inputs and the position, while the vibration in the slow timescale is estimated by the least square method. In the fast timescale, the vibration is reconstructed based on the measurements of vibration and its estimate in the slow timescale, by which the fast controller is designed using ADP. Stability of the closed-loop system is proved by SP theory. Finally, simulations are given to show the feasibility and effectiveness of the proposed methods.
Flexible manipulators are widely applied in aero crafts, construction industries, and other areas because of their many advantages such as fast motion, higher payload-to-robot weight ratio, lower manufacturing consumption, and larger workspace [1, 2]. Taking the physical forces caused by actuation and inertial effect into consideration, the motion of flexible manipulators includes macro rigid-body rotation and micro flexible vibration, which are strongly coupled with each other . Flexible manipulators are known as nonlinear, infinite-order, and uncertain systems . Thus, it is a challenging problem to improve the positioning accuracy and avoid vibration caused by flexibility simultaneously.
Based on the dynamic model of flexible manipulators, researchers have made many studies on the topic of flexible manipulator control. On the one hand, some effective control strategies are investigated based on the coupling system model, such as the traditional PID control [5, 6], optimal control [7, 8], sliding mode control [9, 10], control , robust control [12–14], boundary control , and neural network control [16, 17]. On the other hand, taking the two-timescale characteristics into account, the SP approach is successfully introduced into the modeling and control of the complex flexible manipulator systems [18, 19]. In , a composite controller based on a computed torque control and linear-quadratic control was proposed to suppress the joint and link vibration satisfactorily and achieve a perfect trajectory tracking performance. In , an adaptive boundary control scheme using hyperbolic functions was developed to suppress the vibration and regulate the tip position. In , the dual sliding-mode scheme was employed to track a desired trajectory and stabilize the link vibration. Using output redefinition, a two-performance enhanced controller based on PD control and neural networks was designed for flexible manipulators in . It can be seen that the controller design for the subsystems is more efficient and can achieve a higher performance using various effective controllers based on SP theory.
Though many results have been achieved about the flexible manipulator control, most of the control strategies are based on the dynamic model. However, flexible manipulators are usually subject to uncertainties. So studying the control of flexible manipulators using the measurements of the inputs and states is a hotpot. In [24, 25], neural networks are designed for system uncertainty approximation. In , a nonlinear partial differential equation observer was proposed to estimate the positions and the velocities of a flexible pendulum. And the sliding-mode scheme was designed for the vibration suppression based on SP theory. It can be seen that these studies just talk about the controller design in the case that the dynamics are partially unknown, but model-free composite controller design has not been discussed. In , a fuzzy logic controller by the SP approach for a single flexible arm was proposed. The slow subsystem fuzzy controller realized the trajectory tracking, and the fast subsystem hybrid fuzzy controller was designed to damp out the vibration caused by the elasticity of the system structure. But it is not easy to tune the fuzzy controller parameters to achieve the optimal performance. In [20, 28, 29], the optimal control schemes were used to realize the vibration suppression based on the subsystem models. Experimental results showed that these methods had a good performance, but they required accurate system parameters. Thus, it is of great significance to study the model-free optimal control of flexible manipulators.
In recent years, using ADP theory to solve optimal control problems for unknown systems has received much attention [30–32]. ADP uses a function approximation structure to obtain the approximate optimal control strategy. Thus, the optimal control problem of linear or nonlinear systems can be effectively solved . By employing the ADP theory, the optimal controller can be designed by solving the algebraic Riccati equation based on the measurements of the inputs and states of the system. This learning process greatly simplifies the design of the controller . Based on SP theory [35, 36], the flexible manipulator dynamics can be decomposed into slow and fast subsystems, which are linear and controllable. Inspired of the two-timescale characteristics of flexible manipulators, we will apply the ADP theory to solve the optimal control problem of flexible manipulators without using the system model.
In this paper, a model-free composite controller of flexible manipulators is proposed based on ADP. By employing this method, the dual control targets of position regulation and vibration suppression are achieved. First, the dynamics of a flexible manipulator is decomposed into two subsystems at different timescales by SP theory. Then, a slow subsystem optimal controller is designed by the inputs and the position in the slow timescale using ADP. At the same time, the vibration in the slow timescale is estimated by the least square (LS) method, which lays the foundation of the fast subsystem controller design. A fast subsystem optimal controller is designed by the fast states in the fast timescale using ADP. The contributions of this paper include the following points. (1) This paper proposes a novel controller for flexible manipulators based on ADP without using the model. And simulation results show that the design leads to a better control performance. (2) It is proved that the close-loop system is stable under the model-free composite controller by SP theory. (3) The proposed composite control structure based on dual ADP lays the theory foundation for model-free control of general two-timescale systems.
In Section 2, the dynamic model is established using Lagrange and assumed-mode methods and is decomposed into two subsystems by SP theory. And the problem under consideration is formulated. In Section 3, a model-free composite controller is designed by ADP. In Section 4, the numerical simulations are performed to verify the effectiveness of the proposed methods. Section 5 concludes the paper.
2. Problem Description
Figure 1 gives a mechanical structure diagram of the single manipulator system. As shown in Figure 1, and represent the inertia axis and the local rotating reference axis, respectively. is the control input, is the beam mass, is the payload mass, and is the beam length. The variable represents the rotating angle, and represents the actual vibration which can be measured by sensor .
The flexible manipulators’ dynamic model is established by using Lagrange and assumed-mode methods  as follows: where is the positive definite inertia matrix. and are the nonlinear terms. is the stiffness matrix. is the vector of the joint torque. is the vector of the rotating angle. is the generalized coordinate vector of modes used to describe the actual vibration measured by sensor . When the system model is unknown, it is a challenging problem to realize the position regulation and vibration suppression.
Then, (1) can be written as
Substituting (7) into (5), the slow dynamics can be written as where the superscript “” means the slow dynamic. is the control torque of the slow subsystem (slow controller). is the approximation of in the slow timescale. Considering the two-timescale characteristics of flexible manipulators , we define and . Setting yields
Thus, the slow variables are regarded as constants in the fast timescale. Taking (6), (7), (8), and (9) into account, the fast dynamics can be written as where is the control torque of the fast subsystem (fast controller).
Figure 2 gives the block diagram of the classical composite control. Combining the slow and fast controllers together, the full control of flexible manipulators can be achieved by the following composite controller:
Based on the Tikhonov theorem , the relationship between the subsystems (8) and (10) and the full-order system (1) is as follows: where stands for the infinitesimal of the higher order of . From (13), the flexible mode trajectory includes and . relies on in the slow timescale, and relies on as well as on the flexible mode trajectory in the fast timescale. When the parameters of the dynamics are known, can be obtained by (7). The fast state can be reconstructed for the fast controller design.
Most of the existing controller design methods for flexible manipulators are based on fully known or partially unknown dynamics [24–26]. When the dynamic model is unknown, the abovementioned methods are invalid. This paper will consider the model-free composite control problems for flexible manipulators.
3. ADP-Based Model-Free Composite Control
In this section, ADP is adopted to design a model-free composite controller for flexible manipulators. In the framework of ADP, the controller is designed by using the measurements of the inputs and states where the rotating angle and actual vibration can be measured in engineering. However, the states of the slow subsystem (8) and the fast subsystem (10) cannot be measured directly. From (7), (12), (13), the position information can be used to design an ADP-based slow controller. The vibration in the slow timescale should be estimated to reconstruct the fast state , which can be used to design the ADP-based fast controller. The flow chart is shown in Figure 3.
3.1. ADP-Based Slow Controller Design
Define the trajectory tracking error as where is the desired joint angle of an end-effector. New variables are defined as and . Then, the slow subsystem (8) can be rewritten as where
Choose the performance index  as follows: where , , and is observable.
3.2. Estimation of the Vibration in the Slow Timescale
According to (13), the vibration caused by flexibility includes and . To design the fast controller using ADP, must be estimated first. In the slow timescale, the LS method is a good way to estimate . As shown in (7), the approximate structure of the mathematical model between and is as follows: where and are the parameters to be estimated.
Considering the existence of a random error, (19) can be rewritten as where represents the random error and is the measured data. According to the LS method, the weighted function is defined as
To minimize the weighted function, the method of finding the extremum was used to get
Furthermore, the estimated values of and are derived as
Therefore, we have the estimate of as follows:
3.3. ADP-Based Fast Controller Design
The fast subsystem represents the flexible-mode motion of flexible manipulators as shown in (10). By defining new state variables and , the state space equation of the fast subsystem described in (10) can be expressed as where
In the fast timescale, choose the performance index  as follows: where , , and is observable.
The ADP algorithm  was used to solve optimal control problems for uncertain systems, which is shown as follows: (1)Design an initial controller on the time interval , in which is a positive integer: where is any stabilizing feedback gain matrix and is the exploration noise. Compute , , and until (31) is satisfied. In (31), , , , and are the matrices used to collect state and input information in the learning process. The matrices , , and are defined as follows: where represents the Kronecker product.(2)Solve and from (33), where is the real symmetric positive definite solution of the Riccati equation during the convergence process and is the real feedback gain. where represents the vectorization of matrix , namely, (3)Let , if , where is a small threshold; then, return to step 2.(4)By letting , the approximated optimal control law can be solved as
3.4. Composite Controller Design
As described in (11), the composite controller of the SP system can be achieved as where and .
Theorem 1. Choose , , , and with and being observable. Let and be any stabilizing feedback gain matrix, such that (15) and (25) are asymptotically stable. Then, the obtained composite controller (37) stabilizes the whole system.
Proof 1. Since and are observable and and are stabilizing feedback gain matrices, the obtained and make the subsystems (15) and (25) asymptotically stable [34, 39]. Then, according to the SP theory , the system (1) is asymptotically stable under the obtained composite controller.
Figure 4 shows the model-free composite control algorithm flowchart of flexible manipulators based on ADP.
4. Simulation and Analysis
To verify the effectiveness of the method proposed in this paper, simulation results of flexible manipulators made of aluminum alloy are given. The parameters of a flexible manipulator are shown in Table 1.
In the framework of ADP, a model-free composite controller for flexible manipulators by using the measurements of the inputs and states is designed by the proposed method which does not rely on the system parameters. According to the SP theory, the nonlinear system can be decomposed into two subsystems describing the rid and the flexible motion of flexible manipulators, respectively. For the slow subsystem, is equal to approximately as mentioned in (12); then, it can be directly applied for designing the optimal controller using ADP introduced in Section 3.3, where the initial stabilizing feedback gain is chosen as and the weighted matrices are set as and . After finite iteration, the final optimal feedback gain matrix is obtained as follows:
By solving directly the algebraic Riccati equation, where , , , and , the optimal solution is
It can be seen that is equal to approximately. And the slow controller can be obtained by using ADP with the inputs and the states of the system. Figure 5 gives the convergence of to the optimal value . It is noticed that the feedback gain converges to the optimal values after four iterations.
For the fast subsystem controller design, by the LS method, the estimated value can be obtained as
Thus, according to (27), the fast state variables are obtained, which can be applied for designing the fast subsystem controller based on the algorithm introduced in Section 3.3. We choose as the initial feedback gain, and the weighted matrices are set as and . The final optimal feedback gain matrices are obtained as
By solving directly the algebraic Riccati equation, where , , , and , the optimal solution is
It can be seen that is equal to approximately. Figure 6 gives the convergence of to the optimal values . As shown in Figure 6, after five iterations, converges to the optimal values. Figure 7 gives the control inputs under the ADP-based composite controller.
In order to verify the performance of the model-free composite controller designed in this paper, the comparison experimental results between the ADP-based composite controller and the fuzzy logic composite controller designed in  are given.
Figure 8 shows the trajectory of the flexible manipulator from 0 to 1 rad. As shown in Figure 6, the system achieves the steady state after 5 seconds under the ADP-based composite controller. But the fuzzy logic composite controller takes about 12 seconds to achieve the steady state. The flexible manipulator under the composite controller designed based on ADP can reach the ideal position quickly and accurately.
The performances of the first two modes of the flexible manipulator are shown in Figures 9 and 10, respectively, which show that the controller designed in this paper has a better vibration suppression effect than the fuzzy logic composite controller.
This paper has proposed a novel composite controller of flexible manipulators with completely unknown dynamics. By SP theory, the dynamics can be decoupled into two linear and controllable subsystems. In the slow timescale, the vibration is estimated by the LS method, while the slow subsystem controller is designed by ADP based on the measurements of the information of input and slow states. In the fast timescale, the fast states are reconstructed based on the vibration and its estimate in the slow timescale. Then, the fast subsystem controller is designed by ADP. Finally, a model-free composite controller based on ADP is designed to realize the goals of tip position regulation and vibration suppression. Compared with the existing methods, the proposed composite controller design approach is model-free and can guarantee the stability of the closed-loop system, and the dual-ADP structure gives an example for the model-free control design of general two-timescale systems.
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest.
This work was supported by the Fundamental Research Funds for the Central Universities under Grant 2017XKQY 055.
- X. Yang, S. S. Ge, and W. He, “Dynamic modelling and adaptive robust tracking control of a space robot with two-link flexible manipulators under unknown disturbances,” International Journal of Control, vol. 91, no. 4, pp. 969–988, 2017.
- S. K. Dwivedy and P. Eberhard, “Dynamic analysis of flexible manipulators, a literature review,” Mechanism and Machine Theory, vol. 41, no. 7, pp. 749–777, 2006.
- L. Yu, S. Fei, L. Sun, J. Huang, and G. Yang, “Design of robust adaptive neural switching controller for robotic manipulators with uncertainty and disturbances,” Journal of Intelligent & Robotic Systems, vol. 77, no. 3-4, pp. 571–581, 2015.
- S. S. Ge, T. H. Lee, and G. Zhu, “Improving regulation of a single-link flexible manipulator with strain feedback,” IEEE Transactions on Robotics and Automation, vol. 14, no. 1, pp. 179–185, 1998.
- J. Kim and E. A. Croft, “Full-state tracking control for flexible joint robots with singular perturbation techniques,” IEEE Transactions on Control Systems Technology, vol. PP, no. 99, pp. 1–11, 2017.
- J. Q. Lou and D. Y. Wei, “Modeling and active vibration control of an intelligent flexible manipulator system,” Robot, vol. 36, no. 5, pp. 552–559, 2014.
- R. J. Wai and M. C. Lee, “Intelligent optimal control of single-link flexible robot arm,” IEEE Transactions on Industrial Electronics, vol. 51, no. 1, pp. 201–220, 2004.
- P. Jargeat, D. Rekangalt, and M. C. Verner, “Implementation of artificial intelligent control in single-link flexible robot arm,” IEEE International Symposium on Computational Intelligence in Robotics and Automation, vol. 3, no. 3, pp. 1270–1275, 2003.
- X. Chen and T. Fukuda, “Robust sliding-mode tip position control for flexible arms,” IEEE Transactions on Industrial Electronics, vol. 48, no. 6, pp. 1048–1056, 2001.
- N. U. Dar, M. Farooq, and D. Wang, “Improved hybrid position/force controller design of a flexible robot manipulator using a sliding observer,” Journal of Systems Engineering and Electronics, vol. 20, no. 1, pp. 146–158, 2009.
- M. J. Yazdanpanah, K. Khorasani, and R. V. Patel, “Uncertainty compensation for a flexible-link manipulator using nonlinear control,” International Journal of Control, vol. 69, no. 6, pp. 753–771, 2010.
- I. M. Díaz, E. Pereira, V. Feliu, and J. J. L. Cela, “Concurrent design of multimode input shapers and link dynamics for flexible manipulators,” IEEE/ASME Transactions on Mechatronics, vol. 15, no. 4, pp. 646–651, 2010.
- J. Na, M. N. Mahyuddin, G. Herrmann, X. Ren, and P. Barber, “Robust adaptive finite-time parameter estimation and control for robotic systems,” International Journal of Robust and Nonlinear Control, vol. 25, no. 16, pp. 3045–3071, 2015.
- W. Shang and S. Cong, “Robust nonlinear control of a planar 2-DOF parallel manipulator with redundant actuation,” Robotics & Computer Integrated Manufacturing, vol. 30, no. 6, pp. 597–604, 2014.
- W. He, X. He, M. Zou, and H. Li, “PDE model-based boundary control design for a flexible robotic manipulator with input backlash,” IEEE Transactions on Control Systems Technology, no. 99, pp. 1–8, 2018.
- W. He, Y. Ouyang, and J. Hong, “Vibration control of a flexible robotic manipulator in the presence of input deadzone,” IEEE Transactions on Industrial Informatics, vol. 13, no. 1, pp. 48–59, 2017.
- H. Gao, W. He, C. Zhou, and C. Sun, “Neural network control of a two-link flexible robotic manipulator using assumed mode method,” IEEE Transactions on Industrial Informatics, p. 1, 2018.
- B. Siciliano and W. J. Book, “A singular perturbation approach to control of lightweight flexible manipulators,” The International Journal of Robotics Research, vol. 7, no. 4, pp. 79–90, 1988.
- S. K. Pradhan and B. Subudhi, “Position control of a flexible manipulator using a new nonlinear self tuning PID controller,” IEEE/CAA Journal of Automatica Sinica, pp. 1–14, 2018.
- B. Subudhi and A. S. Morris, “Singular perturbation approach to trajectory tracking of flexible robot with joint elasticity,” International Journal of Systems Science, vol. 34, no. 3, pp. 167–179, 2003.
- Z. Liu, J. Liu, and W. He, “Adaptive boundary control of a flexible manipulator with input saturation,” International Journal of Control, vol. 89, no. 6, pp. 1191–1202, 2015.
- Y. Zhang, T. Yang, and Z. Sun, “Neuro-sliding-mode control of flexible-link manipulators based on singularly perturbed model,” Tsinghua Science and Technology, vol. 14, no. 4, pp. 444–451, 2009.
- B. Xu and Y. Yuan, “Two performance enhanced control of flexible-link manipulator with system uncertainty and disturbances,” Science China Information Sciences, vol. 60, no. 5, pp. 1–11, 2017.
- B. Xu and P. Zhang, “Composite learning sliding mode control of flexible-link manipulator,” Complexity, vol. 2017, Article ID 9430259, 6 pages, 2017.
- B. Xu, “Composite learning control of flexible-link manipulator using NN and DOB,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, no. 99, pp. 1–7, 2017.
- Y. Peng and J. Liu, “Modeling and vibration control for a flexible pendulum inverted system based on a PDE observer,” International Journal of Control, vol. 90, no. 8, pp. 1736–1751, 2016.
- J. Lin and F. L. Lewis, “Two-time scale fuzzy logic controller of flexible link robot arm,” Fuzzy Sets and Systems, vol. 139, no. 1, pp. 125–149, 2003.
- M. W. Mehrez and A. A. El-Badawy, “Effect of the joint inertia on selection of under-actuated control algorithm for flexible-link manipulators,” Mechanism & Machine Theory, vol. 45, no. 7, pp. 967–980, 2010.
- Y. Xu and E. Ritz, “Vision based flexible beam tip point control,” IEEE Transactions on Control Systems Technology, vol. 17, no. 5, pp. 1220–1227, 2009.
- F. Y. Wang, H. Zhang, and D. Liu, “Adaptive dynamic programming: an introduction,” IEEE Computational Intelligence Magazine, vol. 4, no. 2, pp. 39–47, 2009.
- C. Mu, D. Wang, and H. He, “Novel iterative neural dynamic programming for data-based approximate optimal control design,” Automatica, vol. 81, pp. 240–252, 2017.
- J. Na and G. Herrmann, “Online adaptive approximate optimal tracking control with simplified dual approximation structure for continuous-time unknown nonlinear systems,” IEEE/CAA Journal of Automatica Sinica, vol. 1, no. 4, pp. 412–422, 2014.
- Y. Lv, X. Ren, and J. Na, “Online optimal solutions for multi-player nonzero-sum game with completely unknown dynamics,” Neurocomputing, vol. 283, no. 3, pp. 87–97, 2018.
- Y. Jiang and Z. P. Jiang, “Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics,” Automatica, vol. 48, no. 10, pp. 2699–2704, 2012.
- T. Sun, D. Liang, and Y. Song, “Singular-perturbation-based nonlinear hybrid control of redundant parallel robot,” IEEE Transactions on Industrial Electronics, vol. 65, no. 4, pp. 3326–3336, 2018.
- C. Yang, Z. Che, J. Fu, and L. Zhou, “Passivity-based integral sliding mode control and ε-bound estimation for uncertain singularly perturbed systems with disturbances,” IEEE Transactions on Circuits and Systems II: Express Briefs, p. 1, 2018.
- P. V. Kokotovic, J. O’Reilly, and H. K. Khalil, Singular Perturbation Methods in Control: Analysis and Design, SIAM, Philadelphia, PA, USA, 1999.
- F. L. Lewis, D. L. Vrabie, and V. L. Syrmos, Optimal Control, Wiley-Interscience, 1995.
- D. Kleinman, “On an iterative technique for Riccati equation computations,” IEEE Transactions on Automatic Control, vol. 13, no. 1, pp. 114-115, 1968.
Copyright © 2018 Chunyu Yang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.