Research Article  Open Access
Suguru Arimoto, Masahiro Sekimoto, Kenji Tahara, "Iterative Learning without Reinforcement or Reward for Multijoint Movements: A Revisit of Bernstein's DOF Problem on Dexterity", Journal of Robotics, vol. 2010, Article ID 217867, 15 pages, 2010. https://doi.org/10.1155/2010/217867
Iterative Learning without Reinforcement or Reward for Multijoint Movements: A Revisit of Bernstein's DOF Problem on Dexterity
Abstract
A robot designed to mimic a human becomes kinematically redundant and its total degrees of freedom becomes larger than the number of physical variables required for describing a given task. Kinematic redundancy may contribute to enhancement of dexterity and versatility but it incurs a problem of illposedness of inverse kinematics from the task space to the joint space. This illposedness was originally found by Bernstein, who tried to unveil the secret of the central nervous system and how nicely it coordinates a skeletomotor system with many DOFs interacting in complex ways. In the history of robotics research, such illposedness has not yet been resolved directly but circumvented by introducing an artificial performance index and determining uniquely an inverse kinematics solution by minimization. This paper tackles such Bernstein's problem and proposes a new method for resolving the illposedness in a natural way without invoking any artificial index. First, given a curve on a horizontal plane for a redundant robot arm whose endpoint is imposed to trace the curve, the existence of a unique ideal joint trajectory is proved. Second, such a uniquely determined motion can be acquired eventually as a joint control signal through iterative learning without reinforcement or reward.
1. Introduction
Almost a quarter century ago, “robotics” was defined by Professor Brady at the first International Conference of Robotics Research [1] as “the intelligent connection of perception to action.” After a great deal of researches on developments of industrial robots and their applications, a variety of research projects on “humanoid” have attracted many roboticists during the past decade and nowadays robots that can walk with a bipedal mode are not peculiar. Nevertheless, the present state of the art of humanoid still lacks dexterity in fulfillment of ordinary tasks that human encounter in their everyday life. More than a half century ago preceding the birth of “humanoid,” Bernstein [2, 3] noted that dexterity of human body movements resides in involvement of surplus degrees of freedom of limb joints but this incurs the illposedness of inverse kinematics. This was introduced to the robotics community through the famous textbook [4] in page 303 in such a statement as
“The study of human biological motor control mechanisms led the Russian psychologist Bernstein to question how the brain could control a system with so many different degrees of freedom interacting in such a complex fashion. Many of these same complexities are also present in robotic systems and limit our ability to use multifingered hands and other robotic systems to their full advantage.”
Actually, this was originally quoted from Hinton's article [5] in which he summarized what Bernstein challenged in the following way.
(a) What can we infer about the code that the brain uses to communicate with the periphery, and what does that tell us about how the computation is organized?, (b) If the brain knew just what movements it wanted the body to make, could it figure out what to tell the muscles in order to make it happen? (c) How is it possible to coordinate a system with so many degrees of freedom that interact in such complex ways? (d) How does the brain make sensible choices among the myriad possibilities for movement that the body offers?
This paper discusses what are the fundamentals of biomimetic control by focusing Bernstein's DOF problem and shows one way for solving such difficult problems from the standpoint of Riemannian geometry. It is shown in the case of multijoint reaching with DOF redundancy that, given a starting posture in the robot configuration space (or a Riemannian manifold as a set of all postures) and a target endpoint in the task space, there exists a unique orbit of joint motion, provided that the gravity term is compensated in a feedforward manner. Then, it is shown that such an ideal joint motion can be acquired through the iterative learning control without introducing any kind of performance index or reinforcement. In the second illustrative example, a handwriting motion with DOF redundancy is analyzed from the viewpoint of Riemannian geometry under the situation that writing with a ball pencil is imposed to trace an arbitrary smooth curve of class on an arbitrary smooth surface in the threedimensional Euclidean space. Even in this case there exists a unique joint motion in the base Riemannian manifold with DOF redundancy and it can be acquired through repeated exercises of handwriting motion, that is, an ILC scheme without introducing any artificial performance index. In conclusion, an ideal multijoint motion can be acquired through repeated exercises of motion even under the existence of redundancy in DOF, irrelevantly to any kind of reinforcement with the aid of some sort of reward [6, 7].
2. Riemannian Manifold and Euler's Equation
It is widely known among roboticists that kinematics and planning of multijoint robots are treataed in the configuration space regarded as an dimensional numerical space [8, 9]. On the other hand, Arnold [10] pointed out the importance of Riemannian geometry in the analysis of mechanical systems and shown that the dynamics of motion of a double pendulum can be described by an orbit on a twodimensional torus that is regarded as , where denotes a unit circle. In line with this notion, an DOF robot arm can be treated on an dimensional Rimeannian manifold like an dimensional torus , and the stability problems of PD feedback with damping shaping [11] were retreated in a Riemanniangeometric manner [12, 13]. More recently, the author and his group showed that, given a robot arm, the set of all possible postures can be regarded as a Riemannian manifold with the Riemannian metric that constitutes the inertia matrix [14, 15] (see Figure 1). Thus, an orbit of motion as a geodesic solution to the Euler equation can be regarded as an inertiainduced motion without affection of damping and gravity forces [16].
It is well known as in a text book that motion of a robot manipulator as a serially connected rigidbody system is governed by an EulerLagrange equation shown in the form (see [17]) where denotes the vector of joint angles, does the inertia matrix, a control toque vector, with a scalar function called the potential, and a skewsymmetric matrix defined as If we consider a control torque that can exactly compensate the gravity term, that is, then substitution of (3) into (1) yields which is considered to be an ideal equation of motion without affection of gravity and joint damping forces like a robot arm on an artificial satellite in space. It is pointed out in the recent papers [14, 16] that (4) is equivalently written in the form where denotes the Christoffel's symbol of the second kind and the symbol of summation with respect to and in (5) is omitted by obeying the Einstein's rule in differential geometry [18, 19]. Equation (5) is also expressed equivalently in the form which is called the Euler equation. In (5) and (6), and (i.e., called the Christoffel's symbol of the first kind) are defined as follows: where denotes the inverse of . The equivalence relation of (5) to (4) is shown in the previous paper [15].
3. Multijoint Movements with DOF Redundancy
Let us now consider motion of a redundant planar robot arm whose endpoint is free to move in the horizontal plane as shown in Figure 2. The Lagrange equation of motion of the whole armhand system depicted in Figure 2 is expressed as where and each rotational axis of the four joints (shoulder, elbow, wrist, and index finger MP (metacarpophalangeal) joint) is in the direction perpendicular to the plane. Given a robot arm posture in the configuration space or in the 4dim. base manifold , the endpoint position x can be determined by the forward kinematics. A vectorvalued function of class. However, given an endpoint position in , there arises an infinite number of inverses that realize and thereby the problem for obtaining inverse kinematics from the 2dimensional Euclidean space to the 4dimansional configuration space becomes illposed.
A variety of ideas for solving such illposedness of inverse kinematics for redundant robotic systems with excess DOFs has been proposed in the area of robotics, based upon the use of the form where should be computed so as to optimize a certain performance index related to joint position variables (for example, manipulability index [20], obstacle avoidance [21], etc.). In equation (9), stands for the Jacobian matrix of x in , that is, x, and denotes the pseudoinverse of . The original idea of use of the pseudoinverse is due to Whitney [22]. Once a desired joint velocity is planned, it is claimed that the computed torque method can be applied for determining the control input that must generate the whole joint motion of the robot. This is called “inverse kinematics approach”. Another idea of direct generation of a control signal called “inverse dynamics approach” is based upon a form of control input where v is computed so as to optimize a certain performance index related to velocity variables (e.g., kinetic energy [23], torque [24], energy dissipation [25], etc.). In (10), means compensation for the remaining nonlinear function including centrifugal and Coriolis forces and the gravity effect. In the physiological literature, main concerns are focussed on the question why human skilled multijoint reaching movements exhibit typical characteristics that endpoint trajectory becomes a quasistraight line and less variable throughout repetitions, velocity profiles of the endpoint velocity becomes bellshaped, though joint trajectories are rather variable trialsbytrials [26]. Then, a variety of cost functions for derivation of such properties of pointtopoint reaching movements has been proposed, among which a quadratic function of endpoint jerk (rate of acceleration) was the first [27] and successively a cost function based on joint torques was introduced [28] for planning not only an endpoint trajectory but also joint trajectories. However, in the physiological literature, there is a dearth of papers that attempted to directly deal with reaching movements with redundant joints, though the importance of Bernstein's DOF problem [2] has been widely known among physiologists.
Differently from the traditional approaches, a simpler control method for multijoint reaching movements was proposed very recently in our previous papers [29–31] and shown to be effective in both cases of human and robotic arms with redundant DOFs. In those papers, only planar motions confined to a horizontal plane are treated and therefore the control signal is free from gravity with a simple form (see Figure 3) where denotes a diagonal positive definite matrix as follows: Notwithstanding this simpler form than (10), once damping factors and single stiffness parameter are chosen carefully, it is shown that it generates smooth reaching movements by realizing a quasistraight endpoint trajectory, bellshaped velocity profiles, and doublepeaked acceleration profiles typically seen in case of human skilled multijoint movements [26, 32]. Therefore, the method can get rid of undesirable fluctuations of the endpoint trajectory tracking as pointed out in a recent elaborate work [33]. Such fluctuations in task space tracking caused by using the computed torque under uncertainty in link parameters become more noticeable in cases of robots with redundant DOFs. In contrast, in the use of control defined in (11), there is no need of planning any desired endpoint trajectory. However, all these treatments are restricted to planar motions as well as in most of the previous papers on multijoint reaching movements. In addition, another disadvantage is that choice of damping factors recommended in [29] is not fit to the scale of coefficients of viscosity of human muscles [34, 35].
To reduce damping factors in general, another control method based upon “Virtual SpringDamper Hypothesis” was suggested in [31], which in the case of planar motions without affection of the gravity is expressed by the form where is chosen as follows: together with positive constant . The effectiveness of this control signal particularly in the case of middlerange reaching was demonstrated through computer simulation and its performance was compared with that of the control of (11). Theoretical verification of the effectiveness of this springdamper hypothesis was also presented, on the basis of an energy conservation law like a Lyapunovlike relation obtained by substituting (13) into (8) and taking the inner product of this resulted closedloop dynamics and as follows: These results suggest that skilled multijoint movements can be generated even in the case of robot arms with redundant DOFs without construction of any inverse dynamics through “errorfeedback learning” as claimed in a physiological journal [36] for modifying EquilibriumPoint hypothesis [35], EndPoint hypothesis [37], and Virtual Trajectory hypothesis [37].
4. Existence of Desired JointMotion
More recently in the paper [16], an interesting result is found that the endpoint trajectory of a solution to the closedloop dynamics for a given starting posture resembles considerably the endpoint trajectory of a geodesic solution to the Euler equation of (5) or (6) starting from the same given posture to a certain different posture with the prescribed endpoint . The existence of such a geodesic solution to (6) is ascertained by considering a twodimensional Riemannian submanifold that is defined by the set of all postures satisfying , that is (see Figure 4), Similarly, define another submanifold For a given endpoint position in , constitutes a twodimensional submanifold of the base manifold , that is called the equilibriumpoint manifold or simply the EPmanifold in this paper. Denote by any smooth orbit of motion of the robot starting at from the same posture and reaching the submanifold at so that it satisfies . Then, consider the infimum over the set of all such possible orbits of robot motion connecting and . If is not so distant from , it is reasonable from the text books of Riemannian geometry [18, 19] that there exists a unique optimal orbit , , that minimizes the righthand side of (18). Then, the quantity is entitled to be called the Riemannian distance from to the submanifold in the base manifold . It is well known that the optimal orbit must satisfy the Euler equation (6) for and, moreover, it must satisfy the following equation: where , the Jacobian matrix of with respect to . In other words, must belong to the image space of at any instant in . That is, does not have any component in the kernel space of .
Consider now an endpoint trajectory tracking problem for a redundant multijoint arm of Figure 2 in the case that a desired endpoint motion is given as a curve . The control task is to maneuver the robot to let its endpoint trace the given trajectory in through a task space control signal provided that the robot dynamics is governed by the Lagrange equation where is a positive definite damping coefficient matrix. We assume that the initial posture of the robot at is given by and motion of the robot starts from the still state, that is, . The first problem is to find an adequate control signal , , so that the solution to the Lagrange equation of (21) starting from and satisfies for . In order to find a solution to this problem, we decompose any solution trajectory of joint velocity in such a way that where , , is a vector and is the matrix defined by and is a matrix whose column vectors and are orthogonal to ( belongs to the kernel space of ) and satisfy and . Then, if we define then, by substituting (22) into (21) and multiplying the resultant equation by the transpose of from the left, we have Note that is again skewsymmetric. For convenience let us define and decompose and in such a way that where all and are of matrix. Then, it follows from (25) that This equation means that if is set as and is also given then can be determined uniquely from solving the differential equation of (28) as an initialvalue problem by setting . It should be remarked at this stage that the given curve is of class described in terms of time parameter in , that is, it is an valued function of with the initial value , and it has the continuous time derivative with the initial value . Now, multiplying (28) by and accompanying this with (22) by setting and , we have This couple implies a set of six simultaneous differential equations of 1st order concerning six variables though the right hand side of (30) contains and hence it is an implicit function expression. Fortunately, it is possible to obtain an explicit expression of the six simultaneous differential equation by putting Hence, the right hand sides of (29) and (30) are nonlinear in and , but they are Lipschitz continuous in and locally. Therefore, for given and there exists a unique solution for an interval with some satisfying and , where signifies an initial posture satisfying . This fact was already discussed in our previous paper [38]. In this paper, we now prove the unique existence of the solution to the pair of 1storder differential equations of (29) and (30) over the time interval , provided that is not so small in comprison with the scale of and both the quantities of and are within an adequate physical scale.
First, we analyze (30) or (28) by rewriting them in a more detailed manner as follows: where and denote the submatrix of for or 4, and plus constitutes . Note that is linear and homogeneous in but are irrelevant to . In this paper, we restrict our consideration to the case that overall movements of the robot is confined to a single chart of the base manifold in which at any posture in the chart the Jacobian matrix is nondegenerate and therefore there exists a positive constant such that inside the chart. Under this condition, we analyze the following relation obtained by taking the inner product of (32) and : The second term of the right hand side is quadratic in and therefore there exists a constant such that and depend on the maximum magnitude of and . The third term of the right hand side is bounded from the above in the following way As discussed previously in [39], the damping matrix can be chosen to be of the order of and further so as to satisfy Then, (34) is reduced to the inequality relation where we put with some positive constant Clearly, since , (38) implies which concludes that is uniformly bounded in .
Once and are obtained for the given and , the desired input signal in the image space of is obtained by setting , , in (25).
5. Iterative Learning Control in the Task Space
Given a desired trajectory of the endpoint in the task space for a redundant robot arm, there exists a unique trajectory of robot motion in the joint space for a specified initial posture. In particular, it is shown in the previous section that there is uniquely a control signal in the task space that maneuvers the robot through the transpose of the Jacobian matrix to realize the endpoint tracking. However, such a control signal can not be obtained in any analytical form. Nevertheless, it is possible for us to acquire such a desired control signal by using a simple iterative learning control scheme, provided that the endpoint trajectory in the task space can be measured by visual sensing.
At the th trial of iterative learning, the control signal for the dynamics of (21) is designed in the form where means at , and The first term of the right hand side of (41) signifies the inner taskspace PD feedback, denotes the previous control signal at the th trial, and is an adequate positive definite constant matrix. At the first trial, usually we set for . At the second trial must contain erroneous terms. Fortunately, without knowing the desired ideal control , it is possible to expect that and for as . We give an illustrative example of numerical simulation conducted for the 4DOF robot arm shown in Figure 2 with physical parameters given in Table 1. The values for length, mass, and inertia moment of the first link correspond to those of an upper arm of average human adult (male), and the values for the second link do to those of a lower arm. The third link corresponds to a human palm and the fourth an index finger. The desired task is to write a handwritten character “” on the plane. More explicitly, the endpoint trajectory is given by the equation where [s] and The initial posture of the arm is given in Table 2. Based on these data, the system of differential equations is numerically solved by using the RungeKutta method. In Figure 5 we show endpoint trajectories at 1st, 3rd, 5th, and 10th trials and their corresponding postures at and . On the other side, we are able to obtain the desired control signal numerically by numerically solving a couple of 1storder differential equations of (29) and (30). Based upon knowing physical data given in Table 1, the initial posture of given in Table 2, and the specified endpoint trajectory of (43), we obtain the desired ideal control signal shown in Figure 6. It is quite interesting to know that, through simulations of the iterative learning, calculated control signals , , approach the desired one as the trial number increases as shown in Figure 6. When , the trajectory of control signal almost coincides with the ideal one , that uniquely exists just in the image space of the Jacobian matrix with for all .

(a)  
 
(b)  

(a) 1st trial
(b) 3rd trial
(c) 5th trial
(d) 10th trial
(a) vd1
(b) vd2
(a) vk1
(b) vk2
It should be remarked that the desired endpoint trajectory in the case of multijoint reaching for the robot arm of Figure 2 is obtained by solving the Euler equation of (4) or (5) as a twopoint boundaryvalue problem when the initial posture at is given and the terminal condition at is partially specified so as to move the endpoint of the arm to meet at and pass it away. This is the problem to find a curved orbit of the endpoint connecting two given points and by selecting an adequate initial joint velocity that is nonzero. However, the orbit of the endpoint can be represented by a curve on with the aid of length parameters. In the case of middlerange reaching, the profile of this endpoint geodesic curve quite resembles those of humanlike multijoint reaching characterized typically by a quasistraight line movement of the endpoint starting from a fixed still state.
6. Extension to the Case of Existence of Effect of Gravity
Most of the previous results in Sections 4 and 5 can be extended to the case that robot dynamics is subject to the effect of gravity. In such a robot with redundancy in DOF, robot dynamics is expressed by the Lagrange equation: where stands for the gravity term that can be regarded as a gradient vector of a potential function with respect to , that is, . Let us denote again the endpoint position by in the dimensional Euclidean space . Then, we split into That is, is a component of in the image space of and is that of in the kernel space of , that is orthogonally complement to the image space. In accordance with the split of the term , let us choose an matrix with dimensional unit column vectors that are mutually orthogonal and satisfy (i.e., ). Then, in a similar way to (22), we define which leads to On the other hand, for a given desired endpoint trajectory for , we consider the control signal with the task space PD feedback where . Substituting (50) into (46) yields the closedloop dynamics where expresses a feedforward task space control signal. Then, consider the transform of to with an dimensional vector in such a form that In order that the equation of (51) with a desired control signal has a unique solution and so that its endpoint trajectory is coincident with , and , the following equations must be satisfied: where and with . The latter dimensional component of (51) can be expressed by where and are decomposed into and and are of matrix, and and are of matrix. The simultaneous differential equations (53) of 1st order together with determine a unique solution and for given , , and with the initial conditions , , and .
7. ILC for Handwriting
All the considerations in the previous sections can be extended to the case of a handwriting robot whose last link is a ballpoint pen constrained on a hypersurface , where in and is a scalar function of class (see Figure 8). First, we consider a pure mathematical problem of finding a geodesic curve by ignoring the effect of gravity and any joint damping. In this case, let us consider an open connected area on the hypersurface as shown in Figure 8, on which any point satisfies the equality . Then, we denote by a local coordinate chart defined by and assume that is connected and at any the Jacobian matrix is nondegenerate. Then, given a point on with the cartesian coordinates , the set of all such that and constitutes a singledimensional submanifold (see Figure 9). Hence, for another given point on , it is possible to consider an orbit in the submanifold (equivalently, in the configuration space) starting from , lying on , and reaching some point lying on that is another singledimensional submanifold defined by the set of all satisfying and . Thus, it is reasonable to suppose that there exists an optimal orbit that gives minimization of the Riemannian distance from to such that where the infimum is taken over all the orbits lying on (see Figure 9), starting from and reaching , where is rewritten by . It is reasonable to conclude that the optimal orbit in (58) is a solution to the Euler equation under the constraint : where denotes a Lagrange multiplier. In other words, the path of from to some point on can be called the geodesic on the Riemannian submanifold induced naturally from the constraint .
Next, consider the full dynamics of the handwriting robot with 4 DOFs shown in Figure 8 by taking into account the effect of gravity forces and damping torques at joints (see [38]). The dynamics is described by where , denotes the gravity potential, a positive definite damping matrix, and . It should be remarked that stands for a vector that originates at the position in and is normal to the surface which the ball pen contacts with. Now we introduce a length parameter in by the quantity and define the unit normal at the contact point as follows: Then, by rewriting , (60) can be rewritten into Note that the inner product of and the righthand side of (63) vanish. In other words, is orthogonal to at the contact point in . Under the assumption that both the endpoint position and the velocity are measured in real time by visual sensing and the Jacobian matrix is calculated from the measurement data of and in real time, suppose that the control signal must be constructed through the Jacobian transpose in the form For a given desired endpoint trajectory , , together with and and a given desired pressing force , we are concerned with the problem to find a desired control signal of (64) that maneuvers the robot to make the endpoint of the last link (ball pen) trace on the hypersurface with the pressing force in the direction normal to the surface. To show the existence of a control signal in the image space of , we express it in a decomposed form such that where signifies the unit vector tangent to the surface in the direction of and . Note that substituting (65) into (63) yields where Note that is an orthogonal matrix belonging to , is a vector in , and a component of to where is the unit vector satisfying , and are a scalar and express a velocity vector such that at the contact point between the tip of the ball pen and the surface. Since that implies that the velocity of the tip of the pen in the direction normal to the surface is zero, substituting (68) into (66) and multiplying (66) by yield where is of , is of , and In (70), is a function of class with respect to, and . Thus, for a given desired trajectory of the tip of the ball pen, and , it is possible to prove the existence of and that satisfies where . The two equations (74) are the set of five simultaneous differential equations of the 1st order in five variables of and . Once the desired trajectory of joint motion with its first and second derivatives is determined together with the motion of excessDOF, the desired control signal is given by (72) by putting , , and . Moreover, if a desired pressing force is specified, the desired control signal in the direction normal to the surface can be constructed through (70) such that
8. What Should the Central Processing Unit Learn?
Given a smooth curve on a hypersurface over time interval , it is shown in the previous section that there exists uniquely a control signal lying in the image space that makes the endpoint of the robot trace the curve on the hypersurface. Irrespective of the redundancy of DOF, once a starting posture of the robot is specified in the Euclidean space with its endpoint position on the hypersurface as shown in Figure 10, the orbit that realizes itself is determined uniquely correspondingly to the uniquely existing control signal (see Figure 10). Then, there arises a problem whether it is possible to acquire such a desired control signal without knowing the details of the robot dynamics. In order to discuss the problem, let us consider a learning control scheme that is expressed by the following form of the control signal (see Figure 11): where stands for a constant expressing the stiffness parameter of position feedback and a constant lying in interval of . The first term of the righthand side expresses the task space PD feedback for a given set of position and velocity and the second term stands for the learning control signal at the th operation that is constructed as follows: where is an appropriate constant such that and is a positivedefinite constant matrix. In the case of ordinary handwriting when the constraint surface is a horizontal plane, it is shown in the previous paper in [38] that the learning update law for the control signal based on (76) makes the motion of the endpoint trace the given curve on the plane and at the same time the joint motions and converge to the ideal trajectories and as . The proof of convergence of the control signals to the desired one as is also given in [39] on the basis of the “passivity” of the input and the output of the system dynamics of (63) provided that is given through (64) and the gravity effect is carefully compensated. Some numerical simulation results are also presented in [39].
Returning to Bernstein's problem discussed in the introductory section, we are fortunately able to quote Latash's commentary [40] to the book originally written by Bernstein in Russian cited as [2]. Latash says,
Bernstein's definition of degrees of freedom relied heavily on the analysis of kinematic degrees of freedom of various systems. In particular, when analyzing the human upper limb, Bernstein considered all possible orthogonal axes of rotation in a joint as independent degrees of freedom, which were later summed over the joints. The apparent redundancy of the joints of the human arm in comparison to the threedimensional space where we happen to live and where movements take place led Bernstein to his famous formulation that the essence of motor control is the elimination of the redundant degrees of freedom. The beauty and brevity of this formulation is stunning. A skeptic may want to decide whether control can always be reduced to the elimination of redundant degrees of freedom. In other words, I am suggesting that Bernstein's famous definition may not always be correct (blasphemy!!!)
Even in the case of simple humanlike multijoint pointtopoint reaching, Riemannian geometry suggests a good reason to think that the central nervous system is functioning according to a natural law of Newton's mechanics, that is, the law of inertia for multijoint mechanisms. Once a geodesic curve connecting the given initial posture and the equilibrium submanifold (see (18)) is determined with some initial velocity , then the threedimensional orbit for is obtained. This orbit can be rewritten into a threedimensional curve with the aid of length parameter for a corresponding interval . Then this curve can be spelled out by a desired trajectory given as a movement of the tip of the arm in in the form of for with the aid of a scale change of “time” through a monotonously increasing function . Once a desired orbit of motion of a tip of the arm is given in and at the same time an initial posture of the arm is chosen, the trajectory of motion in the joint space is uniquely determined without eliminating any excess of the system's DOF. However, it is important to note that, even if there arises a small change of the initial posture, say , it is possible to obtain the same desired orbit in but the trajectory of joints differes slightly from that obtained when the initial posture is set as and also the desired control signal may differ. Another noteworthy characteristic of humanlike multijoint movements is called “variability”, which was first pointed out by Bernstein [2]. Latash [32] observed that in the case of human skilled motion the grade of variability of each endpoint trajectory is quite low relative to variable profiles of joint responses at each trial of reaching. The mathematical arguments in the previous section suggest that the main part of variability in joint space is caused by small fluctuations of choice of an initial posture trial by trial and the others may be noise.
We do not discuss the importance of redundancy of the muscles involved in a specific motion of multijoint reaching. The central nervous system surely uses its own means of communication with the muscles, which is not analogous to the language of joint kinematics that specifies the trajectories in individual joint. Instead of overcoming the problem of redundancy of the muscles, we implicitly assume in this paper that a desired torque of individual joint is finally generated by a total of forces of all the muscles involved in movement of the joint. This premise may not be justified by any reason, but it is important to quote Jackson's observation as a neurophysiologist [41] (see Figure 12):
“To speak figuratively, the central nervous system knows nothing of muscles, it only knows movements.”
“The highest centres represent all parts of the body, literally all parts supplied by nerves.”
9. Conclusions
This paper discusses difficult problems of control for humanlike robots with redundant DOFs from the standpoint of Riemannian geometry. Irrespective of joint redundancy, we have shown that there is no necessity to resolve the inverse kinematics problem by introducing any artificial performance index and optimizing it in both the cases of multijoint reaching movement with excess joints, and handwriting through iterative learning control. The most important conclusion is that Riemannian geometry does not care about the redundancy of joints through which rigid links are connected in series but it is powerful in establishing a correspondence between workingpoint trajectories in the external threedimensional space and movement trajectories in a space of joint angles (a configuration space or a Riemannian manifold as a set of whole possible postures).
References
 M. Brady, Robotics Research: The First International Symposium, MIT Press, Cambridge, Mass, USA, 1948.
 N. A. Bernstein, “On dexterity and its development,” in Dexterity and Its Development, M. L. Latash and M. T. Turvey, Eds., pp. 3–275, Lawrence Erlbaum Associates, Mahway, NJ, USA, 1996. View at: Google Scholar
 N. A. Bernstein, The Coordination and Regulation of Movements, Pergamon, London, UK, 1967.
 R. Murray, Z. Li, and S. S. Sastry, A Mathematical Introduction to Robotic Manipulation, CRC Press, Boca Raton, Fla, USA, 1994.
 G. Hinton, “Some computational solutions to Bernstein's problems,” in Human Motor Actions: Bernstein Reassessed, H. T. A. Whiting, Ed., pp. 413–418, Elsevier, Amsterdam, The Netherlands, 1984. View at: Google Scholar
 A. G. Barto, R. S. Sutton, and C. W. Anderson, “Neuronlike adaptive elements that can solve difficult learning control problems,” IEEE Transactions on Systems, Man and Cybernetics, vol. 13, no. 5, pp. 834–846, 1983. View at: Google Scholar
 J. Morimoto and K. Doya, “Reinforcement learning of dynamic motor sequence: learning to stand up,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1721–1726, Victoria, Canada, October 1998. View at: Google Scholar
 B. Sicilliano and O. Khatib, Eds., Springer Handbook of Robotics, Springer, New York, NY, USA, 2008.
 J. C. Latombe, Ed., Robot Motion Planning, Kluwer Academic Publishers, Norwell, Mass, USA, 1991.
 V. I. Arnold, Ed., Methematical Methods of Classical Mechanics, Springer, New York, NY, USA, 2nd edition, 1989, translated by K. Vogtmann and A. Weinstein.
 M. Takegaki and S. Arimoto, “A new feedback method for dynamic control of manipulators,” Journal of Dynamic Systems, Measurement and Control, vol. 103, no. 2, pp. 119–125, 1981. View at: Google Scholar
 F. Bullo and A. D. Lewis, Eds., Geometric Control of Mechanical Systems: Modeling, Analysis, and Design for Simple Mechanical Control Systems, Springer, New York, NY, USA, 2000.
 W. M. Oliva, Ed., Geometric Mechanics, vol. 1798 of Lecture Notes in Mathematics, Springer, Berlin, Germany, 2002.
 S. Arimoto, M. Yoshida, M. Sekimoto, and K. Tahara, “A riemanniangeometry approach for dynamics and control of object manipulation under constraints,” in Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1683–1690, Kobe, Japan, May 2009. View at: Publisher Site  Google Scholar
 S. Arimoto, “Modeling and control of multibody mechanical systems part 1: a Riemannian geometry approach,” International Journal of Factory Automation, Robotics and Soft Computing, no. 2, pp. 108–122, 2009. View at: Google Scholar
 M. Sekimoto, S. Arimoto, S. Kawamura, and J.H. Bae, “Skilledmotion plannings of multibody systems based upon Riemannian distance,” in Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1233–1238, Pasadena, Calif, USA, May 2008. View at: Google Scholar
 S. Arimoto, Control Theory of Nonlinear Mechanical Systems: A PassivityBased and CircuitTheoretic Approach, Oxford University Press, Oxford, UK, 1996.
 J. Jost, Riemannian Geometry and Geometric Analysis, Springer, Berlin, Germany, 2002.
 S. Gallot, D. Hulin, and J. Lafontaine, Rimennian Geometry, Springer, Berlin, Germany, 2004.
 T. Yoshikawa, “Manipulability of robotic mechanisms,” International Journal of Robotics Research, vol. 4, no. 2, pp. 3–9, 1985. View at: Google Scholar
 Y. Nakamura, Advanced Robotics: Redundancy and Optimization, AddisonWesley, Reading, Mass, USA, 1991.
 D. E. Whitney, “Resolved motion rate control of manipulators and human protheses,” IEEE Transactions on ManMachine Systems, vol. 10, no. 2, pp. 47–53, 1969. View at: Google Scholar
 O. Khatib, “A unified approach for motion and force control of robot manipulators: the operational space formulation,” IEEE Journal of Robotics and Automation, vol. 3, no. 1, pp. 43–53, 1987. View at: Google Scholar
 J. M. Hollerbach and K. C. Suh, “Redundancy resolution of manipulators through torque optimization,” IEEE Journal of Robotics and Automation, vol. 3, no. 4, pp. 308–316, 1987. View at: Google Scholar
 V. Potkonjak, S. Tzafestas, D. Kostic, G. Djordjevic, and M. Rasic, “The handwriting problem,” IEEE Robotics and Automation Magazine, vol. 10, no. 1, pp. 35–46, 2003. View at: Publisher Site  Google Scholar
 P. Morasso, “Spatial control of arm movements,” Experimental Brain Research, vol. 42, no. 2, pp. 223–227, 1981. View at: Google Scholar
 T. Flash and N. Hogan, “The coordination of arm movements: an experimentally confirmed mathematical model,” Journal of Neuroscience, vol. 5, no. 7, pp. 1688–1703, 1985. View at: Google Scholar
 Y. Uno, M. Kawato, and R. Suzuki, “Formation and control of optimal trajectory in human multijoint arm movement: minimum torquechange model,” Biological Cybernetics, vol. 61, no. 2, pp. 89–101, 1989. View at: Google Scholar
 S. Arimoto, M. Sekimoto, H. Hashiguchi, and R. Ozawa, “Natural resolution of illposedness of inverse kinematics for redundant robots: a challenge to Bernstein's degreesoffreedom problem,” Advanced Robotics, vol. 19, no. 4, pp. 401–434, 2005. View at: Publisher Site  Google Scholar
 S. Arimoto, H. Hashiguchi, M. Sekimoto, and R. Ozawa, “Generation of natural motions for redundant multijoint system: a differentialgeometric approach based upon the principle of least actions,” Journal of Robotic Systems, vol. 22, no. 11, pp. 583–605, 2005. View at: Publisher Site  Google Scholar
 S. Arimoto and M. Sekimoto, “Humanlike movements of robotic arms with redundant DOFs: virtual springdamper hypothesis to tackle the Bernstein problem,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '06), vol. May 2006, pp. 1860–1866, Orlando, Fla, USA, 2006. View at: Publisher Site  Google Scholar
 M. L. Latash, Neurophysiological Basis of Movement, Human Kinetics, New York, NY, USA, 1998.
 J. Nakanishi, R. Cory, M. Mistry, J. Peters, and S. Schaal, “Comparative experiments on task space control with redundancy resolution,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1575–1582, Edmonton, Canada, August 2005. View at: Google Scholar
 J. M. Wintess and L. Stark, “Muscle models: what is gained and what is lost by varying model complexty,” Biological Cybernetics, vol. 55, no. 6, pp. 403–420, 1987. View at: Google Scholar
 E. Bizzi, N. Hogan, F. A. MussaIvaldi, and S. Giszter, “Does the nervous systems use equilibriumpoint control to guide single and multiple joint movements,” Behavioral and Brain Sciences, vol. 15, no. 4, pp. 603–613, 1992. View at: Google Scholar
 M. Kawato and H. Gomi, “A computational model of four regions of the cerebellum based on feedbackerror learning,” Biological Cybernetics, vol. 68, no. 2, pp. 95–103, 1992. View at: Publisher Site  Google Scholar
 N. Hogan, “The mechanics of multijoint posture and movement control,” Biological Cybernetics, vol. 52, no. 5, pp. 315–331, 1985. View at: Google Scholar
 S. Arimoto, H. Hashiguchi, and R. Ozawa, “Simple control method coping with a kinematically illposed inverse problem of redundant robots: analysis in case of a handwriting robot,” Asian Journal of Control, vol. 7, no. 2, pp. 112–123, 2005. View at: Google Scholar
 S. Arimoto, M. Sekimoto, and S. Kawamura, “Taskspace iterative learning for redundant robotic systems: existence of a taskspace control and convergence of learning,” SICE Journal of Control, Measurement, and System Integration, vol. 1, no. 4, pp. 312–319, 2008. View at: Google Scholar
 M. L. Latash, “The Bernstein problem: how does the central nervous system make its choices,” in Dexterity and Its Development, pp. 227–303, Lawrence Erlbaum Associates, Mahway, NJ, USA, 1996. View at: Google Scholar
 M. L. Jackson, “Observations on the physiology and pathology of hemichorea,” in John Hughlings Jackson: Selected Writings, J. Taylor, G. Holmes, and F. M. R. Walshe, Eds., vol. 2, p. 400, Hodder and Stoughton, London, UK, 1932. View at: Google Scholar
Copyright
Copyright © 2010 Suguru Arimoto et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.