Applications of Methods of Numerical Linear Algebra in EngineeringView this Special Issue
Research Article | Open Access
Optimal Grasping Manipulation for Multifingered Robots Using Semismooth Newton Method
Multifingered robots play an important role in manipulation applications. They can grasp various shaped objects to perform point-to-point movement. It is important to plan the motion path of the object and appropriately control the grasping forces for multifingered robot manipulation. In this paper, we perform the optimal grasping control to find both optimal motion path of the object and minimum grasping forces in the manipulation. The rigid body dynamics of the object and the grasping forces subjected to the second-order cone (SOC) constraints are considered in optimal control problem. The minimum principle is applied to obtain the system equalities and the SOC complementarity problems. The SOC complementarity problems are further recast as the equations with the Fischer-Burmeister (FB) function. Since the FB function is semismooth, the semismooth Newton method with the generalized Jacobian of FB function is used to solve the nonlinear equations. The 2D and 3D simulations of grasping manipulation are performed to demonstrate the effectiveness of the proposed approach.
Multifingered robots have attracted much attention in robotics manipulation applications. They can grasp various shaped objects and dexterously perform point-to-point manipulations. Many researches [1–6] have been proposed for grasping and manipulating objects with multifingered robots. Miller and Allen  proposed a user interface with grasp quality evaluation for the robot hand design. Yokokohji et al.  proposed a measure of dynamic manipulability of multifingered grasping for the systems consisting of a multifingered hand and a grasped object. Xu and Li  proposed a modeling method for the manipulation involving finger gaits. Kawamura et al.  used soft finger tips for stable grasping. Takahashi et al.  proposed robust force and position control with the information of tactile sensor. It is important to appropriately control the grasping forces for multifingered robot manipulation.
Since the grasping manipulation utilizes the contact and friction forces to hold and move an object, the grasping forces should satisfy the point-contact friction constraint and be equal to the dynamic wrench of the grasped object. It is required to find the minimum forces for moving the grasped object in the manipulation. Boyd and Wegbreit  used the semidefinite programming and second-order cone programming to efficiently find the grasping forces. Helmke et al.  proposed quadratically convergent algorithms for optimal dexterous grasping. Han et al.  used the convex optimization involving linear matrix inequalities for grasping forces computation. Liu et al.  presented a unified geometric framework for efficient grasping force optimization. Zheng et al.  developed an algorithm to determine the minimum required friction coefficient and the corresponding reliable minimum contact forces in practice. Ko et al.  proposed a neural network to calculate the optimal grasping forces. Because the external wrench of the object varies with the manipulation path and orientation of the object, it is important to plan a manipulation trajectory  for achieving the minimum grasping forces.
In this paper, we perform the optimal grasping control to find both optimal manipulation path of the object and minimum grasping forces. The rigid body dynamics of the object and the grasping forces subjected to the second-order cone (SOC) constraints are considered in the grasping control problem. The minimum principle  is applied to obtain the system equalities and the SOC complementarity problems. The SOC complementarity problems can be recast as the equations with the Fischer-Burmeister (FB) function. The semismooth Newton method with the generalized Jacobian of FB function is then used to solve the equations. Finally, simulations of optimal grasping manipulation are performed to demonstrate the effectiveness of the proposed approach.
The remainder of this paper is organized as follows: Section 2 describes the optimal grasping control problem. In Section 3, the semismooth Newton method with the generalized Jacobian of Fischer-Burmeister function is addressed. Section 4 presents the simulation results of 2D and 3D grasping manipulations. Finally, concluding remarks are given in Section 5.
2. Optimal Grasping Control
Figure 1 shows the multifingered robot grasping manipulation. The multifingered robot grasps and moves the object from the initial position to the final position. The dynamic equation of the object can be expressed with Newton-Euler equations [15, 16] as where is the position, is the velocity, is the quaternion, is the angular velocity, is the object mass, is the matrix of moment of inertia, is the gravity constant, means the grasping forces which is represented by a matrix, is the contact matrix, is the rotation matrix of the object, and can be expressed as Moreover, the grasping forces are subject to the contact friction constraint, expressed as where is the normal force of the ith finger, and are the friction forces of the th finger, is the -norm, and is the friction coefficient.
To find the path that can be achieved with the minimum grasping forces, the optimal control problem can be recast as where In addition, represents the right hand side of system (1), is the control duration, and are the initial and final states, respectively, is the diagonal matrix with the friction coefficient, and denotes the second-order cone which is given by
The optimal control problem (4) can be solved by using the Pontryagin’s minimum principle, see [14, 17, 18]. In optimization language, it is to write out the KKT conditions for problem (4) which consist of two parts. The first part involves a few equalities about Lagrange multipliers, while the other part is related to complementarity conditions. More specifically, with the Hamiltonian function, the first part can be reformulated as follows: where , , and are the Lagrange multipliers, and . The second part forms a second-order cone complementarity problem (SOCCP) as follows: where .
From [19, 20], we see that the previous SOCCP (8) can be further recast as a system of equations: by employing the so-called complementarity function which is a vector-valued function defined as for . We point out that the square term and square-root term in (10) are calculated via Jordan product In particular, the expressions for and are given by respectively.
3. Semismooth Newton Method with Generalized Jacobian of FB Function
In order to apply the semismooth Newton method [21, 22] to (7) and (9), we need the following three linear equations: Most information in linear equations (14)–(16) is known except the generalized Jacobian , in (16). What do they represent? We provide a brief introduction here. First, we recall the concept of the -subdifferential. Given a mapping , if is locally Lipschitz continuous, then the set is nonempty and is called the -subdifferential of at , where denotes the set of points at which is differentiable. The convex hull is the generalized Jacobian of Clarke . From this definition, we see that the generalized Jacobian of can be obtained by computing . From [24, Proposition 3.1], the -subdifferential of in (16) is exactly expressed as Moreover, by denoting , we have(a) If , then and .(b) If and , then for some satisfying and , where .(c) If , then , for some , with , or for some such that satisfying , and satisfying .
Note that the calculations of and are given by For more details, please refer to . Now, we write down the iterative scheme of semismooth Newton method for solving the optimal grasping control problem.
Step 1. Choose , , , , and set .
Step 2. If convergence criterion is satisfied, stop.
Step 4. Set and go to Step 2.
A few words about the implementations. From (14) and (16), the parameters can be eliminated and the differential equations regarding are obtained. With the boundary conditions (15), the solutions of can be achieved. Finally, the solutions of can be obtained by (16). Once all linear equations are solved by the above procedures, the iterative scheme for the calculation of optimal grasping force is kept going.
To evaluate the performance of the proposed approach, we do simulations for 2D and 3D multifingered robots for grasping manipulations. The 2D grasping simulations are performed with a plane three-fingered robot. The parameter values of the object are , , , and the grasping matrices are
The first 2D simulation is the manipulation of degrees rotation of the object. The start and end points are set to be and , respectively. Figure 2 shows the manipulation path of degrees rotation with the time s and the friction coefficient . The simulation result indicates that the proposed scheme grasps the object to the end point smoothly and accurately. Figure 3 depicts the trajectories of the variables , , , , , and . We observe that the rotation angle varies around which results in a small grasping force. Moreover, the translation speeds are kept within m/s and the turning speed within rad/s. The trajectories of the grasping forces are shown in Figure 4. The simulation results show that the normal forces are all nonnegative and the tangent forces satisfy the friction constraint. To evaluate the effect of the friction, simulation with a different value of friction coefficient is also conducted for degrees rotation simulation. Figure 5 shows the manipulation path with the friction coefficient . We observe that the manipulation path length becomes longer as the friction coefficient decreases. Meanwhile, the value of the objective function is computed to be when was and it becomes when reduces to , leading to the increase in grasping force.
The second 2D simulation is the manipulation of degrees rotation of the object. The start and end points are set to be and , respectively. The friction coefficient is set as . Figures 6-7 depict the manipulation paths of degrees rotation with the time s and s, respectively. We observe that the object moves down initially and reaches to the end point accurately. Moreover, the mean of rotation angle decreases as increases.
The 3D grasping simulation is performed with a five-fingered robot which has not been implemented in the literature. The object is considered as a block and its parameter values were set as The two fingers of the robot grasp the top of the object, while the other three grasp the bottom of the object. The matrices and are The start and end points are set to be and , respectively. Figure 8 shows the 3D manipulation path with the time s and the friction coefficient . As we can see, the object moves and rotates to the end point smoothly and accurately. Figures 9-10 depict the trajectories of the variables , , , , , , and , , , , , , respectively. We observe that the trajectories are smooth. The translation speeds are kept within m/s and the turning speed within /s. The trajectories of the grasping forces are shown in Figure 11. The simulation results indicate that the grasping forces satisfy the friction constraint. The 3D grasping simulations are also performed with a four-fingered robot and a six-fingered robot, respectively. The four-fingered and six-fingered robots place one finger and three fingers on the top center of the object, respectively, while their other three fingers grasping the bottom. Figures 12-13 show the 3D manipulation paths with the four-fingered and six-fingered robots, respectively. From Figures 8, 12, and 13, we observe that increasing the number of the robot fingers can reduce manipulation path length and enhance the maneuverability. Consequently, the simulation results show that the proposed scheme can achieve the accurate manipulation for multifingered robots.
In this paper, we have proposed an effective method for multifingered robot path planning and grasping forces computation. The optimal grasping control problem was formulated with the rigid body dynamics of the object and the second-order cone constraints of grasping forces. The SOC complementarity problem was recast as the equations with the Fischer-Burmeister (FB) function, and the semismooth Newton method with the generalized Jacobian of FB function was used to solve the system equations. The simulation results show that the optimal grasping forces can accurately move the object to a goal, demonstrating the effectiveness of the proposed method.
This work was supported in part by the National Science Council of Taiwan, ROC under the Grant NSC 100-2221-E-214-016. The author's work is supported by National Science Council of Taiwan.
- A. Kawamura, K. Tahara, R. Kurazume, and T. Hasegawa, “Dynamic grasping for an arbitrary polyhedral object by a multi-fingered hand-arm system,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '09), pp. 2264–2270, St. Louis, Mo., USA, October 2009.
- A. T. Miller and P. K. Allen, “Graspit: a versatile simulator for robotic grasping,” IEEE Robotics and Automation Magazine, vol. 11, no. 4, pp. 110–122, 2004.
- R. M. Murray, Z. X. Li, and S. S. Sastry, A Mathematical Introduction to Robotic Manipulation, CRC Press, Boca Raton, Fla, USA, 1994.
- T. Takahashi, T. Tsuboi, T. Kishida et al., “Adaptive grasping by multi fingered hand with tactile sensor based on robust force and position control,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '08), pp. 264–271, Pasadena, Calif, USA, May 2008.
- J. Xu and Z. Li, “A kinematic model of finger gaits by multifingered hand as hybrid automaton,” IEEE Transactions on Automation Science and Engineering, vol. 5, no. 3, pp. 467–479, 2008.
- Y. Yokokohji, J. S. Martin, and M. Fujiwara, “Dynamic manipulability of multifingered grasping,” IEEE Transactions on Robotics, vol. 25, no. 4, pp. 947–954, 2009.
- S. P. Boyd and B. Wegbreit, “Fast computation of optimal contact forces,” IEEE Transactions on Robotics, vol. 23, no. 6, pp. 1117–1132, 2007.
- U. Helmke, K. Hüper, and J. B. Moore, “Quadratically convergent algorithms for optimal dextrous hand grasping,” IEEE Transactions on Robotics and Automation, vol. 18, no. 2, pp. 138–146, 2002.
- L. Han, J. C. Trinkle, and Z. X. Li, “Grasp analysis as linear matrix inequality problems,” IEEE Transactions on Robotics and Automation, vol. 16, no. 6, pp. 663–674, 2000.
- G. F. Liu, J. J. Xu, and Z. X. Li, “On geometric algorithms for real-time grasping force optimization,” IEEE Transactions on Control Systems Technology, vol. 12, no. 6, pp. 843–859, 2004.
- Y. Zheng, M. C. Lin, and D. Manocha, “On computing reliable optimal grasping forces,” IEEE Transactions on Robotics, vol. 28, no. 3, pp. 619–633, 2012.
- C. H. Ko, J. S. Chen, and C. Y. Yang, “Recurrent neural networks for solving second-order cone programs,” Neurocomputing, vol. 74, no. 17, pp. 3646–3653, 2011.
- D. Prattichizzo, M. Malvezzi, M. Aggravi, and T. Wimbock, “Object motion-decoupled internal force control for a compliant multifingered hand,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '12), pp. 1508–1513, St Paul, Minn, USA, May, 2012.
- F. L. Lewis and V. L. Syrmos, Optimal Control, John Wiley & Sons, New York, NY, USA, 1995.
- W. Ham and H. Kwon, “Sliding mode control for the hovering of helicopter by using quaternion dynamics,” in Proceedings of the 50th Annual Conference on Society of Instrument and Control Engineers (SICE '11), pp. 1024–1028, Tokyo, Japan, September 2011.
- C. G. Mayhew, R. G. Sanfelice, and A. R. Teel, “On quaternion-based attitude control and the unwinding phenomenon,” in Proceedings of the American Control Conference (ACC '09), pp. 299–304, San Francisco, Calif, USA, July 2011.
- H. P. Geering, Optimal Control With Engineering Applications, Springer, Berlin, Germany, 2007.
- D. G. Hull, Optimal Control Theory for Applications, Mechanical Engineering Series, Springer, Berlin, Germany, 2003.
- J.-S. Chen and P. Tseng, “An unconstrained smooth minimization reformulation of the second-order cone complementarity problem,” Mathematical Programming, vol. 104, no. 2-3, pp. 293–327, 2005.
- M. Fukushima, Z. Q. Luo, and P. Tseng, “Smoothing functions for second-order-cone complementarity problems,” SIAM Journal on Optimization, vol. 12, no. 2, pp. 436–460, 2002.
- M. Gerdts, “A nonsmooth Newton's method for control-state constrained optimal control problems,” Mathematics and Computers in Simulation, vol. 79, no. 4, pp. 925–936, 2008.
- L. Q. Qi and J. Sun, “A nonsmooth version of Newton's method,” Mathematical Programming, vol. 58, no. 3, pp. 353–367, 1993.
- F. H. Clarke, Optimization and Nonsmooth Analysis, vol. 5 of Classics in Applied Mathematics, SIAM, Philadelphia, Pa, 2nd edition, 1990.
- S. H. Pan and J.-S. Chen, “A damped Gauss-Newton method for the second-order cone complementarity problem,” Applied Mathematics and Optimization, vol. 59, no. 3, pp. 293–318, 2009.
Copyright © 2013 Chun-Hsu Ko and Jein-Shan Chen. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.