BioInspired Learning and Adaptation for Optimization and Control of Complex Systems
View this Special IssueResearch Article  Open Access
Chunyu Yang, Yiming Xu, Linna Zhou, Yongzheng Sun, "ModelFree Composite Control of Flexible Manipulators Based on Adaptive Dynamic Programming", Complexity, vol. 2018, Article ID 9720309, 9 pages, 2018. https://doi.org/10.1155/2018/9720309
ModelFree Composite Control of Flexible Manipulators Based on Adaptive Dynamic Programming
Abstract
This paper studies the problems of tip position regulation and vibration suppression of flexible manipulators without using the model. Because of the twotimescale characteristics of flexible manipulators, applying the existing modelfree control methods may lead to illconditioned numerical problems. In this paper, the dynamics of a flexible manipulator is decomposed into two subsystems which are linear and controllable at different timescales by singular perturbation (SP) theory and a modelfree composite controller is designed to alleviate the illconditioned numerical problems. To do this, a modelfree composite control strategy is constructed which facilitates in designing the controller in slow and fast timescales. In the slow timescale, the slow subsystem controller is designed by adaptive dynamic programming (ADP) based on the measurements of the slow inputs and the position, while the vibration in the slow timescale is estimated by the least square method. In the fast timescale, the vibration is reconstructed based on the measurements of vibration and its estimate in the slow timescale, by which the fast controller is designed using ADP. Stability of the closedloop system is proved by SP theory. Finally, simulations are given to show the feasibility and effectiveness of the proposed methods.
1. Introduction
Flexible manipulators are widely applied in aero crafts, construction industries, and other areas because of their many advantages such as fast motion, higher payloadtorobot weight ratio, lower manufacturing consumption, and larger workspace [1, 2]. Taking the physical forces caused by actuation and inertial effect into consideration, the motion of flexible manipulators includes macro rigidbody rotation and micro flexible vibration, which are strongly coupled with each other [3]. Flexible manipulators are known as nonlinear, infiniteorder, and uncertain systems [4]. Thus, it is a challenging problem to improve the positioning accuracy and avoid vibration caused by flexibility simultaneously.
Based on the dynamic model of flexible manipulators, researchers have made many studies on the topic of flexible manipulator control. On the one hand, some effective control strategies are investigated based on the coupling system model, such as the traditional PID control [5, 6], optimal control [7, 8], sliding mode control [9, 10], control [11], robust control [12–14], boundary control [15], and neural network control [16, 17]. On the other hand, taking the twotimescale characteristics into account, the SP approach is successfully introduced into the modeling and control of the complex flexible manipulator systems [18, 19]. In [20], a composite controller based on a computed torque control and linearquadratic control was proposed to suppress the joint and link vibration satisfactorily and achieve a perfect trajectory tracking performance. In [21], an adaptive boundary control scheme using hyperbolic functions was developed to suppress the vibration and regulate the tip position. In [22], the dual slidingmode scheme was employed to track a desired trajectory and stabilize the link vibration. Using output redefinition, a twoperformance enhanced controller based on PD control and neural networks was designed for flexible manipulators in [23]. It can be seen that the controller design for the subsystems is more efficient and can achieve a higher performance using various effective controllers based on SP theory.
Though many results have been achieved about the flexible manipulator control, most of the control strategies are based on the dynamic model. However, flexible manipulators are usually subject to uncertainties. So studying the control of flexible manipulators using the measurements of the inputs and states is a hotpot. In [24, 25], neural networks are designed for system uncertainty approximation. In [26], a nonlinear partial differential equation observer was proposed to estimate the positions and the velocities of a flexible pendulum. And the slidingmode scheme was designed for the vibration suppression based on SP theory. It can be seen that these studies just talk about the controller design in the case that the dynamics are partially unknown, but modelfree composite controller design has not been discussed. In [27], a fuzzy logic controller by the SP approach for a single flexible arm was proposed. The slow subsystem fuzzy controller realized the trajectory tracking, and the fast subsystem hybrid fuzzy controller was designed to damp out the vibration caused by the elasticity of the system structure. But it is not easy to tune the fuzzy controller parameters to achieve the optimal performance. In [20, 28, 29], the optimal control schemes were used to realize the vibration suppression based on the subsystem models. Experimental results showed that these methods had a good performance, but they required accurate system parameters. Thus, it is of great significance to study the modelfree optimal control of flexible manipulators.
In recent years, using ADP theory to solve optimal control problems for unknown systems has received much attention [30–32]. ADP uses a function approximation structure to obtain the approximate optimal control strategy. Thus, the optimal control problem of linear or nonlinear systems can be effectively solved [33]. By employing the ADP theory, the optimal controller can be designed by solving the algebraic Riccati equation based on the measurements of the inputs and states of the system. This learning process greatly simplifies the design of the controller [34]. Based on SP theory [35, 36], the flexible manipulator dynamics can be decomposed into slow and fast subsystems, which are linear and controllable. Inspired of the twotimescale characteristics of flexible manipulators, we will apply the ADP theory to solve the optimal control problem of flexible manipulators without using the system model.
In this paper, a modelfree composite controller of flexible manipulators is proposed based on ADP. By employing this method, the dual control targets of position regulation and vibration suppression are achieved. First, the dynamics of a flexible manipulator is decomposed into two subsystems at different timescales by SP theory. Then, a slow subsystem optimal controller is designed by the inputs and the position in the slow timescale using ADP. At the same time, the vibration in the slow timescale is estimated by the least square (LS) method, which lays the foundation of the fast subsystem controller design. A fast subsystem optimal controller is designed by the fast states in the fast timescale using ADP. The contributions of this paper include the following points. (1) This paper proposes a novel controller for flexible manipulators based on ADP without using the model. And simulation results show that the design leads to a better control performance. (2) It is proved that the closeloop system is stable under the modelfree composite controller by SP theory. (3) The proposed composite control structure based on dual ADP lays the theory foundation for modelfree control of general twotimescale systems.
In Section 2, the dynamic model is established using Lagrange and assumedmode methods and is decomposed into two subsystems by SP theory. And the problem under consideration is formulated. In Section 3, a modelfree composite controller is designed by ADP. In Section 4, the numerical simulations are performed to verify the effectiveness of the proposed methods. Section 5 concludes the paper.
2. Problem Description
Figure 1 gives a mechanical structure diagram of the single manipulator system. As shown in Figure 1, and represent the inertia axis and the local rotating reference axis, respectively. is the control input, is the beam mass, is the payload mass, and is the beam length. The variable represents the rotating angle, and represents the actual vibration which can be measured by sensor .
The flexible manipulators’ dynamic model is established by using Lagrange and assumedmode methods [18] as follows: where is the positive definite inertia matrix. and are the nonlinear terms. is the stiffness matrix. is the vector of the joint torque. is the vector of the rotating angle. is the generalized coordinate vector of modes used to describe the actual vibration measured by sensor . When the system model is unknown, it is a challenging problem to realize the position regulation and vibration suppression.
Define
Then, (1) can be written as
Suppose , , and , where is the minimum eigenvalue of the stiffness . Then, (3) and (4) can be rewritten as
Based on SP theory, (1) can be decomposed into two subsystems at different timescales. Since is small enough [18], by letting in (5) and (6), the state in the slow timescale can be obtained as
Substituting (7) into (5), the slow dynamics can be written as where the superscript “” means the slow dynamic. is the control torque of the slow subsystem (slow controller). is the approximation of in the slow timescale. Considering the twotimescale characteristics of flexible manipulators [35], we define and . Setting yields
Thus, the slow variables are regarded as constants in the fast timescale. Taking (6), (7), (8), and (9) into account, the fast dynamics can be written as where is the control torque of the fast subsystem (fast controller).
Figure 2 gives the block diagram of the classical composite control. Combining the slow and fast controllers together, the full control of flexible manipulators can be achieved by the following composite controller:
Based on the Tikhonov theorem [37], the relationship between the subsystems (8) and (10) and the fullorder system (1) is as follows: where stands for the infinitesimal of the higher order of . From (13), the flexible mode trajectory includes and . relies on in the slow timescale, and relies on as well as on the flexible mode trajectory in the fast timescale. When the parameters of the dynamics are known, can be obtained by (7). The fast state can be reconstructed for the fast controller design.
Most of the existing controller design methods for flexible manipulators are based on fully known or partially unknown dynamics [24–26]. When the dynamic model is unknown, the abovementioned methods are invalid. This paper will consider the modelfree composite control problems for flexible manipulators.
3. ADPBased ModelFree Composite Control
In this section, ADP is adopted to design a modelfree composite controller for flexible manipulators. In the framework of ADP, the controller is designed by using the measurements of the inputs and states where the rotating angle and actual vibration can be measured in engineering. However, the states of the slow subsystem (8) and the fast subsystem (10) cannot be measured directly. From (7), (12), (13), the position information can be used to design an ADPbased slow controller. The vibration in the slow timescale should be estimated to reconstruct the fast state , which can be used to design the ADPbased fast controller. The flow chart is shown in Figure 3.
3.1. ADPBased Slow Controller Design
As shown in (8), the slow subsystem represents the rigid body motion of the flexible manipulator system. From (12), can be approximated by the state variable which is easy to be measured.
Define the trajectory tracking error as where is the desired joint angle of an endeffector. New variables are defined as and . Then, the slow subsystem (8) can be rewritten as where
Choose the performance index [38] as follows: where , , and is observable.
Implement the algorithm mentioned in Section 3.3 with , , and being a stabilizing feedback gain matrix for (15). Then, the slow controller can be obtained as
3.2. Estimation of the Vibration in the Slow Timescale
According to (13), the vibration caused by flexibility includes and . To design the fast controller using ADP, must be estimated first. In the slow timescale, the LS method is a good way to estimate . As shown in (7), the approximate structure of the mathematical model between and is as follows: where and are the parameters to be estimated.
Considering the existence of a random error, (19) can be rewritten as where represents the random error and is the measured data. According to the LS method, the weighted function is defined as
To minimize the weighted function, the method of finding the extremum was used to get
Furthermore, the estimated values of and are derived as
Therefore, we have the estimate of as follows:
3.3. ADPBased Fast Controller Design
The fast subsystem represents the flexiblemode motion of flexible manipulators as shown in (10). By defining new state variables and , the state space equation of the fast subsystem described in (10) can be expressed as where
By combining (13) and (24), , that is , can be obtained as follows:
In the fast timescale, choose the performance index [38] as follows: where , , and is observable.
Implement the algorithm mentioned in Section 3.3 with , , and being a stabilizing feedback gain matrix for (25). Then, the fast controller can be obtained as
The ADP algorithm [34] was used to solve optimal control problems for uncertain systems, which is shown as follows: (1)Design an initial controller on the time interval , in which is a positive integer: where is any stabilizing feedback gain matrix and is the exploration noise. Compute , , and until (31) is satisfied. In (31), , , , and are the matrices used to collect state and input information in the learning process. The matrices , , and are defined as follows: where represents the Kronecker product.(2)Solve and from (33), where is the real symmetric positive definite solution of the Riccati equation during the convergence process and is the real feedback gain. where represents the vectorization of matrix , namely, (3)Let , if , where is a small threshold; then, return to step 2.(4)By letting , the approximated optimal control law can be solved as
3.4. Composite Controller Design
As described in (11), the composite controller of the SP system can be achieved as where and .
Theorem 1. Choose , , , and with and being observable. Let and be any stabilizing feedback gain matrix, such that (15) and (25) are asymptotically stable. Then, the obtained composite controller (37) stabilizes the whole system.
Proof 1. Since and are observable and and are stabilizing feedback gain matrices, the obtained and make the subsystems (15) and (25) asymptotically stable [34, 39]. Then, according to the SP theory [37], the system (1) is asymptotically stable under the obtained composite controller.
Figure 4 shows the modelfree composite control algorithm flowchart of flexible manipulators based on ADP.
4. Simulation and Analysis
To verify the effectiveness of the method proposed in this paper, simulation results of flexible manipulators made of aluminum alloy are given. The parameters of a flexible manipulator are shown in Table 1.

In the framework of ADP, a modelfree composite controller for flexible manipulators by using the measurements of the inputs and states is designed by the proposed method which does not rely on the system parameters. According to the SP theory, the nonlinear system can be decomposed into two subsystems describing the rid and the flexible motion of flexible manipulators, respectively. For the slow subsystem, is equal to approximately as mentioned in (12); then, it can be directly applied for designing the optimal controller using ADP introduced in Section 3.3, where the initial stabilizing feedback gain is chosen as and the weighted matrices are set as and . After finite iteration, the final optimal feedback gain matrix is obtained as follows:
By solving directly the algebraic Riccati equation, where , , , and , the optimal solution is
It can be seen that is equal to approximately. And the slow controller can be obtained by using ADP with the inputs and the states of the system. Figure 5 gives the convergence of to the optimal value . It is noticed that the feedback gain converges to the optimal values after four iterations.
For the fast subsystem controller design, by the LS method, the estimated value can be obtained as
Thus, according to (27), the fast state variables are obtained, which can be applied for designing the fast subsystem controller based on the algorithm introduced in Section 3.3. We choose as the initial feedback gain, and the weighted matrices are set as and . The final optimal feedback gain matrices are obtained as
By solving directly the algebraic Riccati equation, where , , , and , the optimal solution is
It can be seen that is equal to approximately. Figure 6 gives the convergence of to the optimal values . As shown in Figure 6, after five iterations, converges to the optimal values. Figure 7 gives the control inputs under the ADPbased composite controller.
In order to verify the performance of the modelfree composite controller designed in this paper, the comparison experimental results between the ADPbased composite controller and the fuzzy logic composite controller designed in [27] are given.
Figure 8 shows the trajectory of the flexible manipulator from 0 to 1 rad. As shown in Figure 6, the system achieves the steady state after 5 seconds under the ADPbased composite controller. But the fuzzy logic composite controller takes about 12 seconds to achieve the steady state. The flexible manipulator under the composite controller designed based on ADP can reach the ideal position quickly and accurately.
The performances of the first two modes of the flexible manipulator are shown in Figures 9 and 10, respectively, which show that the controller designed in this paper has a better vibration suppression effect than the fuzzy logic composite controller.
5. Conclusion
This paper has proposed a novel composite controller of flexible manipulators with completely unknown dynamics. By SP theory, the dynamics can be decoupled into two linear and controllable subsystems. In the slow timescale, the vibration is estimated by the LS method, while the slow subsystem controller is designed by ADP based on the measurements of the information of input and slow states. In the fast timescale, the fast states are reconstructed based on the vibration and its estimate in the slow timescale. Then, the fast subsystem controller is designed by ADP. Finally, a modelfree composite controller based on ADP is designed to realize the goals of tip position regulation and vibration suppression. Compared with the existing methods, the proposed composite controller design approach is modelfree and can guarantee the stability of the closedloop system, and the dualADP structure gives an example for the modelfree control design of general twotimescale systems.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest.
Acknowledgments
This work was supported by the Fundamental Research Funds for the Central Universities under Grant 2017XKQY 055.
References
 X. Yang, S. S. Ge, and W. He, “Dynamic modelling and adaptive robust tracking control of a space robot with twolink flexible manipulators under unknown disturbances,” International Journal of Control, vol. 91, no. 4, pp. 969–988, 2017. View at: Publisher Site  Google Scholar
 S. K. Dwivedy and P. Eberhard, “Dynamic analysis of flexible manipulators, a literature review,” Mechanism and Machine Theory, vol. 41, no. 7, pp. 749–777, 2006. View at: Publisher Site  Google Scholar
 L. Yu, S. Fei, L. Sun, J. Huang, and G. Yang, “Design of robust adaptive neural switching controller for robotic manipulators with uncertainty and disturbances,” Journal of Intelligent & Robotic Systems, vol. 77, no. 34, pp. 571–581, 2015. View at: Publisher Site  Google Scholar
 S. S. Ge, T. H. Lee, and G. Zhu, “Improving regulation of a singlelink flexible manipulator with strain feedback,” IEEE Transactions on Robotics and Automation, vol. 14, no. 1, pp. 179–185, 1998. View at: Publisher Site  Google Scholar
 J. Kim and E. A. Croft, “Fullstate tracking control for flexible joint robots with singular perturbation techniques,” IEEE Transactions on Control Systems Technology, vol. PP, no. 99, pp. 1–11, 2017. View at: Publisher Site  Google Scholar
 J. Q. Lou and D. Y. Wei, “Modeling and active vibration control of an intelligent flexible manipulator system,” Robot, vol. 36, no. 5, pp. 552–559, 2014. View at: Google Scholar
 R. J. Wai and M. C. Lee, “Intelligent optimal control of singlelink flexible robot arm,” IEEE Transactions on Industrial Electronics, vol. 51, no. 1, pp. 201–220, 2004. View at: Publisher Site  Google Scholar
 P. Jargeat, D. Rekangalt, and M. C. Verner, “Implementation of artificial intelligent control in singlelink flexible robot arm,” IEEE International Symposium on Computational Intelligence in Robotics and Automation, vol. 3, no. 3, pp. 1270–1275, 2003. View at: Google Scholar
 X. Chen and T. Fukuda, “Robust slidingmode tip position control for flexible arms,” IEEE Transactions on Industrial Electronics, vol. 48, no. 6, pp. 1048–1056, 2001. View at: Publisher Site  Google Scholar
 N. U. Dar, M. Farooq, and D. Wang, “Improved hybrid position/force controller design of a flexible robot manipulator using a sliding observer,” Journal of Systems Engineering and Electronics, vol. 20, no. 1, pp. 146–158, 2009. View at: Google Scholar
 M. J. Yazdanpanah, K. Khorasani, and R. V. Patel, “Uncertainty compensation for a flexiblelink manipulator using nonlinear control,” International Journal of Control, vol. 69, no. 6, pp. 753–771, 2010. View at: Publisher Site  Google Scholar
 I. M. Díaz, E. Pereira, V. Feliu, and J. J. L. Cela, “Concurrent design of multimode input shapers and link dynamics for flexible manipulators,” IEEE/ASME Transactions on Mechatronics, vol. 15, no. 4, pp. 646–651, 2010. View at: Publisher Site  Google Scholar
 J. Na, M. N. Mahyuddin, G. Herrmann, X. Ren, and P. Barber, “Robust adaptive finitetime parameter estimation and control for robotic systems,” International Journal of Robust and Nonlinear Control, vol. 25, no. 16, pp. 3045–3071, 2015. View at: Publisher Site  Google Scholar
 W. Shang and S. Cong, “Robust nonlinear control of a planar 2DOF parallel manipulator with redundant actuation,” Robotics & Computer Integrated Manufacturing, vol. 30, no. 6, pp. 597–604, 2014. View at: Publisher Site  Google Scholar
 W. He, X. He, M. Zou, and H. Li, “PDE modelbased boundary control design for a flexible robotic manipulator with input backlash,” IEEE Transactions on Control Systems Technology, no. 99, pp. 1–8, 2018. View at: Publisher Site  Google Scholar
 W. He, Y. Ouyang, and J. Hong, “Vibration control of a flexible robotic manipulator in the presence of input deadzone,” IEEE Transactions on Industrial Informatics, vol. 13, no. 1, pp. 48–59, 2017. View at: Publisher Site  Google Scholar
 H. Gao, W. He, C. Zhou, and C. Sun, “Neural network control of a twolink flexible robotic manipulator using assumed mode method,” IEEE Transactions on Industrial Informatics, p. 1, 2018. View at: Publisher Site  Google Scholar
 B. Siciliano and W. J. Book, “A singular perturbation approach to control of lightweight flexible manipulators,” The International Journal of Robotics Research, vol. 7, no. 4, pp. 79–90, 1988. View at: Publisher Site  Google Scholar
 S. K. Pradhan and B. Subudhi, “Position control of a flexible manipulator using a new nonlinear self tuning PID controller,” IEEE/CAA Journal of Automatica Sinica, pp. 1–14, 2018. View at: Publisher Site  Google Scholar
 B. Subudhi and A. S. Morris, “Singular perturbation approach to trajectory tracking of flexible robot with joint elasticity,” International Journal of Systems Science, vol. 34, no. 3, pp. 167–179, 2003. View at: Publisher Site  Google Scholar
 Z. Liu, J. Liu, and W. He, “Adaptive boundary control of a flexible manipulator with input saturation,” International Journal of Control, vol. 89, no. 6, pp. 1191–1202, 2015. View at: Publisher Site  Google Scholar
 Y. Zhang, T. Yang, and Z. Sun, “Neuroslidingmode control of flexiblelink manipulators based on singularly perturbed model,” Tsinghua Science and Technology, vol. 14, no. 4, pp. 444–451, 2009. View at: Publisher Site  Google Scholar
 B. Xu and Y. Yuan, “Two performance enhanced control of flexiblelink manipulator with system uncertainty and disturbances,” Science China Information Sciences, vol. 60, no. 5, pp. 1–11, 2017. View at: Publisher Site  Google Scholar
 B. Xu and P. Zhang, “Composite learning sliding mode control of flexiblelink manipulator,” Complexity, vol. 2017, Article ID 9430259, 6 pages, 2017. View at: Publisher Site  Google Scholar
 B. Xu, “Composite learning control of flexiblelink manipulator using NN and DOB,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, no. 99, pp. 1–7, 2017. View at: Publisher Site  Google Scholar
 Y. Peng and J. Liu, “Modeling and vibration control for a flexible pendulum inverted system based on a PDE observer,” International Journal of Control, vol. 90, no. 8, pp. 1736–1751, 2016. View at: Publisher Site  Google Scholar
 J. Lin and F. L. Lewis, “Twotime scale fuzzy logic controller of flexible link robot arm,” Fuzzy Sets and Systems, vol. 139, no. 1, pp. 125–149, 2003. View at: Publisher Site  Google Scholar
 M. W. Mehrez and A. A. ElBadawy, “Effect of the joint inertia on selection of underactuated control algorithm for flexiblelink manipulators,” Mechanism & Machine Theory, vol. 45, no. 7, pp. 967–980, 2010. View at: Publisher Site  Google Scholar
 Y. Xu and E. Ritz, “Vision based flexible beam tip point control,” IEEE Transactions on Control Systems Technology, vol. 17, no. 5, pp. 1220–1227, 2009. View at: Publisher Site  Google Scholar
 F. Y. Wang, H. Zhang, and D. Liu, “Adaptive dynamic programming: an introduction,” IEEE Computational Intelligence Magazine, vol. 4, no. 2, pp. 39–47, 2009. View at: Publisher Site  Google Scholar
 C. Mu, D. Wang, and H. He, “Novel iterative neural dynamic programming for databased approximate optimal control design,” Automatica, vol. 81, pp. 240–252, 2017. View at: Publisher Site  Google Scholar
 J. Na and G. Herrmann, “Online adaptive approximate optimal tracking control with simplified dual approximation structure for continuoustime unknown nonlinear systems,” IEEE/CAA Journal of Automatica Sinica, vol. 1, no. 4, pp. 412–422, 2014. View at: Publisher Site  Google Scholar
 Y. Lv, X. Ren, and J. Na, “Online optimal solutions for multiplayer nonzerosum game with completely unknown dynamics,” Neurocomputing, vol. 283, no. 3, pp. 87–97, 2018. View at: Publisher Site  Google Scholar
 Y. Jiang and Z. P. Jiang, “Computational adaptive optimal control for continuoustime linear systems with completely unknown dynamics,” Automatica, vol. 48, no. 10, pp. 2699–2704, 2012. View at: Publisher Site  Google Scholar
 T. Sun, D. Liang, and Y. Song, “Singularperturbationbased nonlinear hybrid control of redundant parallel robot,” IEEE Transactions on Industrial Electronics, vol. 65, no. 4, pp. 3326–3336, 2018. View at: Publisher Site  Google Scholar
 C. Yang, Z. Che, J. Fu, and L. Zhou, “Passivitybased integral sliding mode control and εbound estimation for uncertain singularly perturbed systems with disturbances,” IEEE Transactions on Circuits and Systems II: Express Briefs, p. 1, 2018. View at: Publisher Site  Google Scholar
 P. V. Kokotovic, J. O’Reilly, and H. K. Khalil, Singular Perturbation Methods in Control: Analysis and Design, SIAM, Philadelphia, PA, USA, 1999.
 F. L. Lewis, D. L. Vrabie, and V. L. Syrmos, Optimal Control, WileyInterscience, 1995.
 D. Kleinman, “On an iterative technique for Riccati equation computations,” IEEE Transactions on Automatic Control, vol. 13, no. 1, pp. 114115, 1968. View at: Publisher Site  Google Scholar
Copyright
Copyright © 2018 Chunyu Yang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.