Iterative Learning without Reinforcement or Reward for Multijoint Movements: A Revisit of Bernstein's DOF Problem on Dexterity

Abstract

A robot designed to mimic a human becomes kinematically redundant and its total degrees of freedom becomes larger than the number of physical variables required for describing a given task. Kinematic redundancy may contribute to enhancement of dexterity and versatility but it incurs a problem of ill-posedness of inverse kinematics from the task space to the joint space. This ill-posedness was originally found by Bernstein, who tried to unveil the secret of the central nervous system and how nicely it coordinates a skeletomotor system with many DOFs interacting in complex ways. In the history of robotics research, such ill-posedness has not yet been resolved directly but circumvented by introducing an artificial performance index and determining uniquely an inverse kinematics solution by minimization. This paper tackles such Bernstein's problem and proposes a new method for resolving the ill-posedness in a natural way without invoking any artificial index. First, given a curve on a horizontal plane for a redundant robot arm whose endpoint is imposed to trace the curve, the existence of a unique ideal joint trajectory is proved. Second, such a uniquely determined motion can be acquired eventually as a joint control signal through iterative learning without reinforcement or reward.

1. Introduction

Almost a quarter century ago, “robotics” was defined by Professor Brady at the first International Conference of Robotics Research [1] as “the intelligent connection of perception to action.” After a great deal of researches on developments of industrial robots and their applications, a variety of research projects on “humanoid” have attracted many roboticists during the past decade and nowadays robots that can walk with a bipedal mode are not peculiar. Nevertheless, the present state of the art of humanoid still lacks dexterity in fulfillment of ordinary tasks that human encounter in their everyday life. More than a half century ago preceding the birth of “humanoid,” Bernstein [2, 3] noted that dexterity of human body movements resides in involvement of surplus degrees of freedom of limb joints but this incurs the ill-posedness of inverse kinematics. This was introduced to the robotics community through the famous textbook [4] in page 303 in such a statement as

“The study of human biological motor control mechanisms led the Russian psychologist Bernstein to question how the brain could control a system with so many different degrees of freedom interacting in such a complex fashion. Many of these same complexities are also present in robotic systems and limit our ability to use multifingered hands and other robotic systems to their full advantage.”

Actually, this was originally quoted from Hinton's article [5] in which he summarized what Bernstein challenged in the following way.

(a) What can we infer about the code that the brain uses to communicate with the periphery, and what does that tell us about how the computation is organized?, (b) If the brain knew just what movements it wanted the body to make, could it figure out what to tell the muscles in order to make it happen? (c) How is it possible to coordinate a system with so many degrees of freedom that interact in such complex ways? (d) How does the brain make sensible choices among the myriad possibilities for movement that the body offers?

This paper discusses what are the fundamentals of biomimetic control by focusing Bernstein's DOF problem and shows one way for solving such difficult problems from the standpoint of Riemannian geometry. It is shown in the case of multijoint reaching with DOF redundancy that, given a starting posture in the robot configuration space (or a Riemannian manifold as a set of all postures) and a target endpoint in the task space, there exists a unique orbit of joint motion, provided that the gravity term is compensated in a feedforward manner. Then, it is shown that such an ideal joint motion can be acquired through the iterative learning control without introducing any kind of performance index or reinforcement. In the second illustrative example, a handwriting motion with DOF redundancy is analyzed from the viewpoint of Riemannian geometry under the situation that writing with a ball pencil is imposed to trace an arbitrary smooth curve of 𝐶^∞-class on an arbitrary smooth surface in the three-dimensional Euclidean space. Even in this case there exists a unique joint motion in the base Riemannian manifold with DOF redundancy and it can be acquired through repeated exercises of handwriting motion, that is, an ILC scheme without introducing any artificial performance index. In conclusion, an ideal multijoint motion can be acquired through repeated exercises of motion even under the existence of redundancy in DOF, irrelevantly to any kind of reinforcement with the aid of some sort of reward [6, 7].

2. Riemannian Manifold and Euler's Equation

It is widely known among roboticists that kinematics and planning of multijoint robots are treataed in the configuration space regarded as an 𝑛-dimensional numerical space 𝐑^𝑛 [8, 9]. On the other hand, Arnold [10] pointed out the importance of Riemannian geometry in the analysis of mechanical systems and shown that the dynamics of motion of a double pendulum can be described by an orbit on a two-dimensional torus 𝑇² that is regarded as 𝑇²=𝑆¹×𝑆¹, where 𝑆¹ denotes a unit circle. In line with this notion, an 𝑛-DOF robot arm can be treated on an 𝑛-dimensional Rimeannian manifold like an 𝑛-dimensional torus 𝑇^𝑛, and the stability problems of PD feedback with damping shaping [11] were retreated in a Riemannian-geometric manner [12, 13]. More recently, the author and his group showed that, given a robot arm, the set of all possible postures can be regarded as a Riemannian manifold with the Riemannian metric that constitutes the inertia matrix [14, 15] (see Figure 1). Thus, an orbit of motion as a geodesic solution to the Euler equation can be regarded as an inertia-induced motion without affection of damping and gravity forces [16].

Figure 1 

The set of all possible postures of a planar robot arm with three joints can be regarded as a Riemannian manifold with Riemannian metric $g_{ij}(q)$ that constitutes the inertia matrix.

It is well known as in a text book that motion of a robot manipulator as a serially connected rigid-body system is governed by an Euler-Lagrange equation shown in the form (see [17])̈1𝐺(𝑞)𝑞+{2𝐺̇(𝑞)+𝑆(𝑞,𝑞)̇}𝑞̇+𝑔(𝑞)=𝑢, (1) where 𝑞=(𝑞₁,…,𝑞_𝑛)^T denotes the vector of joint angles, 𝐺(𝑞)=(𝑔_𝑖𝑗) does the 𝑛×𝑛 inertia matrix, 𝑢 a control toque vector, 𝑔(𝑞)=𝜕𝑃(𝑞)/𝜕𝑞 with a scalar function 𝑃(𝑞) called the potential, and 𝑆(𝑞,𝑞)̇ a skew-symmetric matrix 𝑆=(𝑆_𝑖𝑗) defined as𝑆_𝑖𝑗=12{𝜕𝜕𝑞_𝑗(^𝑛_𝑘=1𝑞̇_𝑘𝑔_𝑖𝑘𝜕)−𝜕𝑞_𝑖(^𝑛_𝑘=1𝑞̇_𝑘𝑔_𝑗𝑘)}. (2) If we consider a control torque that can exactly compensate the gravity term, that is,𝑢=𝑔(𝑞), (3) then substitution of (3) into (1) yields̈1𝐺(𝑞)𝑞+{2𝐺̇(𝑞)+𝑆(𝑞,𝑞)̇}𝑞̇=0, (4) which is considered to be an ideal equation of motion without affection of gravity and joint damping forces like a robot arm on an artificial satellite in space. It is pointed out in the recent papers [14, 16] that (4) is equivalently written in the form𝑔_𝑖𝑘(𝑞)𝑞̈_𝑖+Γ_𝑖𝑘𝑗(𝑞)𝑞̇_𝑗𝑞̇_𝑖=0,𝑘=1,…,𝑛, (5) where Γ_𝑖𝑘𝑗 denotes the Christoffel's symbol of the second kind and the symbol of summation with respect to 𝑖 and 𝑗 in (5) is omitted by obeying the Einstein's rule in differential geometry [18, 19]. Equation (5) is also expressed equivalently in the form𝑞̈_𝑘+Γ^𝑘_𝑖𝑗𝑞̇_𝑖𝑞̇_𝑗=0,𝑘=1,…,𝑛, (6) which is called the Euler equation. In (5) and (6), Γ_𝑖𝑘𝑗 and Γ^𝑘_𝑖𝑗 (i.e., called the Christoffel's symbol of the first kind) are defined as follows:Γ_𝑖𝑘𝑗=12(𝜕𝑔_𝑗𝑘𝜕𝑞_𝑖+𝜕𝑔_𝑖𝑘𝜕𝑞_𝑗−𝜕𝑔_𝑖𝑗𝜕𝑞_𝑘Γ),_𝑘𝑖𝑗=12^𝑛_𝑙=1𝑔^𝑙𝑘(𝜕𝑔_𝑗𝑙𝜕𝑞_𝑖+𝜕𝑔_𝑖𝑙𝜕𝑞_𝑗−𝜕𝑔_𝑖𝑗𝜕𝑞_𝑙1)=2^𝑛_𝑙=1𝑔^𝑙𝑘Γ_𝑖𝑙𝑗, (7) where (𝑔^𝑙𝑘) denotes the inverse of 𝐺(=(𝑔_𝑖𝑗)). The equivalence relation of (5) to (4) is shown in the previous paper [15].

3. Multijoint Movements with DOF Redundancy

Let us now consider motion of a redundant planar robot arm whose endpoint is free to move in the horizontal plane as shown in Figure 2. The Lagrange equation of motion of the whole arm-hand system depicted in Figure 2 is expressed as̈1𝐺(𝑞)𝑞+{2𝐺̇(𝑞)+𝑆(𝑞,𝑞)̇}𝑞̇=𝑢, (8) where 𝑞=(𝑞₁,…,𝑞₄)^T and each rotational axis of the four joints (shoulder, elbow, wrist, and index finger MP (metacarpophalangeal) joint) is in the direction perpendicular to the 𝑥𝑦-plane. Given a robot arm posture 𝑞 in the configuration space 𝐑⁴ or in the 4-dim. base manifold {𝑀,𝑔_𝑖𝑗}, the endpoint position x can be determined by the forward kinematics. A vector-valued function 𝐱(𝑞)=(𝑥(𝑞),𝑦(𝑞)) of 𝐶^∞-class. However, given an endpoint position 𝐱_𝑑 in 𝐑⁴, there arises an infinite number of inverses that realize 𝐱(𝑞_𝑑)=𝐱_𝑑 and thereby the problem for obtaining inverse kinematics from the 2-dimensional Euclidean space 𝐄² to the 4-dimansional configuration space 𝐑⁴ becomes ill-posed.

Figure 2 

“Reaching” by means of a surplus DOF system of hand-arm dynamics.

A variety of ideas for solving such ill-posedness of inverse kinematics for redundant robotic systems with excess DOFs has been proposed in the area of robotics, based upon the use of the form𝑞̇_𝑑(𝑡)=𝐽⁺(𝑞(𝑡))𝐱̇_𝑑+(𝐼−𝐽⁺(𝑞)𝐽(𝑞))𝐯, (9) where 𝐯 should be computed so as to optimize a certain performance index related to joint position variables (for example, manipulability index [20], obstacle avoidance [21], etc.). In equation (9), 𝐽(𝑞) stands for the Jacobian matrix of x in 𝑞, that is, 𝐽(𝑞)=𝜕x/𝜕𝑞^T, and 𝐽⁺(𝑞) denotes the pseudo-inverse of 𝐽(𝑞). The original idea of use of the pseudo-inverse 𝐽⁺(𝑞) is due to Whitney [22]. Once a desired joint velocity 𝑞̇_𝑑(𝑡) is planned, it is claimed that the computed torque method can be applied for determining the control input that must generate the whole joint motion of the robot. This is called “inverse kinematics approach”. Another idea of direct generation of a control signal called “inverse dynamics approach” is based upon a form of control input𝑢=𝐺(𝑞)𝐽⁺(𝑞){𝐱̈_𝑑−𝐽(̇𝑞)𝑞}̇+𝑔(𝑞,𝑞)̇+(𝐼−𝐽⁺(𝑞)𝐽(𝑞))𝐯, (10) where v is computed so as to optimize a certain performance index related to velocity variables (e.g., kinetic energy [23], torque [24], energy dissipation [25], etc.). In (10), 𝑔(𝑞,𝑞)̇ means compensation for the remaining nonlinear function including centrifugal and Coriolis forces and the gravity effect. In the physiological literature, main concerns are focussed on the question why human skilled multijoint reaching movements exhibit typical characteristics that (1) endpoint trajectory becomes a quasistraight line and less variable throughout repetitions, (2) velocity profiles of the endpoint velocity becomes bell-shaped, though (3) joint trajectories are rather variable trials-by-trials [26]. Then, a variety of cost functions for derivation of such properties of point-to-point reaching movements has been proposed, among which a quadratic function of endpoint jerk (rate of acceleration) was the first [27] and successively a cost function based on joint torques was introduced [28] for planning not only an endpoint trajectory but also joint trajectories. However, in the physiological literature, there is a dearth of papers that attempted to directly deal with reaching movements with redundant joints, though the importance of Bernstein's DOF problem [2] has been widely known among physiologists.

Differently from the traditional approaches, a simpler control method for multijoint reaching movements was proposed very recently in our previous papers [29–31] and shown to be effective in both cases of human and robotic arms with redundant DOFs. In those papers, only planar motions confined to a horizontal plane are treated and therefore the control signal is free from gravity with a simple form (see Figure 3)𝑢=−𝐶𝑞̇−𝐽^T(𝑞)𝑘Δ𝐱, (11) where 𝐶 denotes a diagonal positive definite matrix as follows:𝐶=diag(𝑐₁,…,𝑐_𝑛). (12) Notwithstanding this simpler form than (10), once damping factors 𝐶 and single stiffness parameter 𝑘 are chosen carefully, it is shown that it generates smooth reaching movements by realizing a quasistraight endpoint trajectory, bell-shaped velocity profiles, and double-peaked acceleration profiles typically seen in case of human skilled multijoint movements [26, 32]. Therefore, the method can get rid of undesirable fluctuations of the endpoint trajectory tracking as pointed out in a recent elaborate work [33]. Such fluctuations in task space tracking caused by using the computed torque under uncertainty in link parameters become more noticeable in cases of robots with redundant DOFs. In contrast, in the use of control defined in (11), there is no need of planning any desired endpoint trajectory. However, all these treatments are restricted to planar motions as well as in most of the previous papers on multijoint reaching movements. In addition, another disadvantage is that choice of damping factors recommended in [29] is not fit to the scale of coefficients of viscosity of human muscles [34, 35].

Figure 3 

Virtual spring-damper hypothesis.

To reduce damping factors in general, another control method based upon “Virtual Spring-Damper Hypothesis” was suggested in [31], which in the case of planar motions without affection of the gravity is expressed by the form𝑢=−𝐶₀𝑞̇−𝐽^T(𝑞)(𝜁√𝑘𝐱̇+𝑘Δ𝐱)=−𝐶₀𝑞̇−𝜁√𝑘𝐽^T(𝑞)𝐽(𝑞)𝑞̇−𝐽^T(𝑞)𝑘Δ𝐱, (13) where 𝐶₀ is chosen as follows:𝐶₀=𝜁₀diag(𝑐₁,…,𝑐_𝑛), (14) together with positive constant 0<𝜁₀<1.0. The effectiveness of this control signal particularly in the case of middle-range reaching was demonstrated through computer simulation and its performance was compared with that of the control of (11). Theoretical verification of the effectiveness of this spring-damper hypothesis was also presented, on the basis of an energy conservation law like a Lyapunov-like relation obtained by substituting (13) into (8) and taking the inner product of this resulted closed-loop dynamics and 𝑞̇ as follows:̈1𝐺(𝑞)𝑞+{2𝐺̇+𝑆+𝐶₀}𝑞̇+𝐽^T(𝑞){𝜁√𝑘̇d𝐱+𝑘Δ𝐱}=0,{1d𝑡2𝑞̇^Ṫ𝑘𝐺(𝑞)𝑞+2‖Δ𝐱‖²}=−𝑞̇^T𝐶₀𝑞̇−𝜁√𝑘‖𝐱̇‖². (15) These results suggest that skilled multijoint movements can be generated even in the case of robot arms with redundant DOFs without construction of any inverse dynamics through “error-feedback learning” as claimed in a physiological journal [36] for modifying Equilibrium-Point hypothesis [35], End-Point hypothesis [37], and Virtual Trajectory hypothesis [37].

4. Existence of Desired Joint-Motion

More recently in the paper [16], an interesting result is found that the endpoint trajectory of a solution to the closed-loop dynamics for a given starting posture resembles considerably the endpoint trajectory of a geodesic solution to the Euler equation of (5) or (6) starting from the same given posture to a certain different posture with the prescribed endpoint 𝐱_𝑑. The existence of such a geodesic solution to (6) is ascertained by considering a two-dimensional Riemannian submanifold that is defined by the set of all postures 𝑞 satisfying 𝐱(𝑞)=𝐱_𝑑, that is (see Figure 4), 𝑁_𝑃′={𝑞∣𝐱(𝑞)=𝑃^′(=𝐱_𝑑)}. (16) Similarly, define another submanifold 𝑁_𝑃={𝑞∣𝐱(𝑞)=𝑃(=𝐱(0))}. (17) For a given endpoint position 𝐱_𝑑 in 𝐄², 𝑁_𝑃′ constitutes a two-dimensional submanifold of the base manifold {𝑀,𝑔_𝑖𝑗}, that is called the equilibrium-point manifold or simply the EP-manifold in this paper. Denote by 𝑞(𝑡) any smooth orbit of motion of the robot starting at 𝑡=𝑎 from the same posture 𝑞(𝑡=𝑎)=𝑞⁰ and reaching the submanifold 𝑁_𝑃′ at 𝑡=𝑏 so that it satisfies 𝐱(𝑞(𝑏))=𝐱_𝑑. Then, consider the infimum𝑑(𝑞⁰,𝑁_𝑃′)=inf_𝑞(𝑡)^𝑏_{𝑎𝑖,𝑗}𝑔_𝑖𝑗(𝑞)𝑞̇_𝑖(𝑡)𝑞̇_𝑗(𝑡)d𝑡 (18) over the set of all such possible orbits of robot motion connecting 𝑞⁰ and 𝑁_𝑃′. If 𝐱_𝑑 is not so distant from 𝐱(0)(=𝐱(𝑞⁰)), it is reasonable from the text books of Riemannian geometry [18, 19] that there exists a unique optimal orbit 𝑞^∗(𝑡), 𝑡∈[𝑎,𝑏], that minimizes the right-hand side of (18). Then, the quantity 𝑑(𝑞⁰,𝑁_𝑃′) is entitled to be called the Riemannian distance from 𝑞⁰ to the submanifold 𝑁_𝑃′ in the base manifold {𝑀,𝑔_𝑖𝑗}. It is well known that the optimal orbit 𝑞^∗(𝑡) must satisfy the Euler equation (6) for 𝑡∈[𝑎,𝑏] and, moreover, it must satisfy the following equation:{𝐼₄−𝐽⁺(𝑞^∗(𝑡))𝐽(𝑞^∗(𝑡))}𝑞̇^∗(𝑡)=0, (19) where 𝐽(𝑞)=𝜕𝐱(𝑞)/𝜕𝑞^T, the Jacobian matrix of 𝐱(𝑞) with respect to 𝑞. In other words, 𝑞̇^∗(𝑡) must belong to the image space of 𝐽^T(𝑞^∗(𝑡)) at any instant 𝑡 in [𝑎,𝑏]. That is, 𝑞̇^∗(𝑡) does not have any component in the kernel space of 𝐽^T.

Figure 4 

A set of all postures that have the same endpoint position ${\mathbf{x}}_d$ constitutes a two-dimensional Riemannian submanifold.

Consider now an endpoint trajectory tracking problem for a redundant multijoint arm of Figure 2 in the case that a desired endpoint motion is given as a curve 𝐱_𝑑(𝑡)∶𝐼=[0,𝑇]→𝐄². The control task is to maneuver the robot to let its endpoint trace the given trajectory 𝐱_𝑑(𝑡) in 𝐄² through a task space control signal𝑢=𝐽^T(𝑞)𝑣 (20) provided that the robot dynamics is governed by the Lagrange equation̈1𝐺(𝑞)𝑞+{2𝐺(𝑞)𝑞̇+𝑆(𝑞,𝑞)̇}𝑞̇+𝐶₀𝑞̇=𝐽^T(𝑞)𝑣, (21) where 𝐶₀ is a 4×4 positive definite damping coefficient matrix. We assume that the initial posture of the robot at 𝑡=0 is given by 𝑞(0)=𝑞⁰ and motion of the robot starts from the still state, that is, 𝑞(̇0)=0. The first problem is to find an adequate control signal 𝑣(𝑡), 𝑡∈[0,𝑇], so that the solution to the Lagrange equation of (21) starting from 𝑞(0)=𝑞⁰ and 𝑞(̇0)=0 satisfies 𝐱(𝑞(𝑡))=𝐱_𝑑(𝑡) for 𝑡∈[0,𝑇]. In order to find a solution to this problem, we decompose any solution trajectory of joint velocity 𝑞(̇𝑡) in such a way thaṫ𝐱̇𝜂̇𝑞=(𝑃,𝑊), (22) where 𝐱=𝐱(𝑞), 𝐱̇=𝐽(𝑞)𝑞̇, 𝜂̇ is a 2×1 vector and 𝑃 is the 4×2-matrix defined by𝑃=𝐽^T(𝑞)(𝐽(𝑞)𝐽^T(𝑞))⁻¹=𝐽⁺(𝑞) (23) and 𝑊 is a 4×2-matrix whose column vectors 𝑤₁ and 𝑤₂ are orthogonal to 𝐽^T(𝑞) (𝑤_𝑖 belongs to the kernel space of 𝐽^T(𝑞)) and satisfy ‖𝑤₁‖=‖𝑤₂‖=1 and 𝑤^T₁𝑤₂=0. Then, if we define𝑄=(𝑃,𝑊),𝑄⁻¹=𝑃^T𝑊^T=𝑄^T,𝐺₁=𝑄^T𝐺𝑄,𝐶₁=𝑄^T𝐶₀𝑆𝑄,₁=𝑄^T1𝑆𝑄−2𝑄̇^T𝐺𝑄+12𝑄^T𝐺𝑄̇, (24) then, by substituting (22) into (21) and multiplying the resultant equation by the transpose of 𝑄 from the left, we have𝐺₁𝐱̈𝜂̈+12{𝐺̇₁+𝑆₁+𝐶₁}𝐱̇𝜂̇=𝐯0. (25) Note that 𝑆₁ is again skew-symmetric. For convenience let us define𝐵₁=12𝐺̇₁+𝑆₁+𝐶₁ (26) and decompose 𝐺₁ and 𝐵₁ in such a way that𝐺₁=𝐺₁₁𝐺₁₂𝐺^T₁₂𝐺₂₂,𝐵₁=𝐵₁₁𝐵₁₂𝐵₂₁𝐵₂₂, (27) where all 𝐺_𝑖𝑗 and 𝐵_𝑖𝑗 are of 2×2-matrix. Then, it follows from (25) that𝐺₂₂𝜂̈+𝐵₂₂𝜂̇=−𝐺^T₁₂𝐱̈−𝐵₂₁𝐱̇. (28) This equation means that if 𝐱̇(𝑡) is set as 𝐱̇(𝑡)=𝐱̇_𝑑(𝑡) and 𝑞_𝑑(𝑡) is also given then 𝜂̇(𝑡) can be determined uniquely from solving the differential equation of (28) as an initial-value problem by setting 𝜂̇(0)=0. It should be remarked at this stage that the given curve 𝐱_𝑑(𝑡) is of 𝐶^∞-class described in terms of time parameter 𝑡 in 𝐄², that is, it is an 𝐄²-valued function of 𝑡 with the initial value 𝐱_𝑑(0)=𝐱(𝑞⁰), and it has the continuous time derivative 𝐱̇_𝑑(𝑡) with the initial value 𝐱̇_𝑑(0)=0. Now, multiplying (28) by 𝐺⁻¹₂₂ and accompanying this with (22) by setting 𝐱̇=𝐱̇_𝑑 and 𝐱̈=𝐱̈_𝑑, we have𝑞̇_𝑑=𝑃(𝑞_𝑑)𝐱̇_𝑑+𝑊(𝑞_𝑑)̇̈𝜂,𝜂=−𝐺⁻¹₂₂(𝑞_𝑑){𝐵₂₂(𝑞_𝑑,𝑞̇_𝑑)𝜂̇+𝐺₁₂(𝑞_𝑑)𝐱̈_𝑑}+𝐺⁻¹₂₂(𝑞_𝑑)𝐵₂₁(𝑞_𝑑,𝑞̇_𝑑)𝐱̇_𝑑. (29)(30) This couple implies a set of six simultaneous differential equations of 1st order concerning six variables {𝑞_𝑑,𝜂̇} though the right hand side of (30) contains 𝑞̇_𝑑 and hence it is an implicit function expression. Fortunately, it is possible to obtain an explicit expression of the six simultaneous differential equation by putting𝐵₂₂=𝐵₂₂(𝑞_𝑑,𝑃𝐱̇_𝑑̇𝐵+𝑊𝜂),₂₁=𝐵₂₁(𝑞_𝑑,𝑃𝐱̇_𝑑+𝑊𝜂̇). (31) Hence, the right hand sides of (29) and (30) are nonlinear in 𝑞_𝑑 and 𝜂̇, but they are Lipschitz continuous in 𝑞_𝑑 and 𝜂̇ locally. Therefore, for given 𝐱̇_𝑑 and 𝐱̈_𝑑 there exists a unique solution {𝑞_𝑑(𝑡),𝜂̇(𝑡)} for an interval 𝑡∈[0,𝑎] with some 𝑎>0 satisfying 𝑞_𝑑(0)=𝑞⁰ and 𝜂̇(0)=0, where 𝑞⁰ signifies an initial posture satisfying 𝐱(𝑞⁰)=𝐱_𝑑(0). This fact was already discussed in our previous paper [38]. In this paper, we now prove the unique existence of the solution to the pair of 1st-order differential equations of (29) and (30) over the time interval [0,𝑇], provided that 𝐶₀ is not so small in comprison with the scale of 𝐺(𝑞) and both the quantities of 𝐱̇_𝑑 and 𝐱̈_𝑑 are within an adequate physical scale.

First, we analyze (30) or (28) by rewriting them in a more detailed manner as follows:𝐺₂₂̈1𝜂+{2𝐺̇₂₂+𝑆₂₂}𝜂̇+𝐶₂₂𝜂̇=−𝐺^T₁₂𝐱̈_𝑑−𝐵₂₁𝐱̇_𝑑−𝐶₂₁𝐱̇_𝑑, (32) where𝐺₂₂=𝑊^T𝐺𝑊,𝐶₂₂=𝑊^T𝐶₁𝑊,𝐺^T₁₂=𝑊^T𝐺𝑃,𝐵₂₁=12𝐺̇₂₁+𝑆₂₁,𝐶₂₁=𝑊^T𝐶₁𝑃 (33) and 𝑆₂₂ denote the submatrix (𝑠_𝑖𝑗) of 𝑆₁ for 𝑖,𝑗=3 or 4, and 𝐵₂₁ plus 𝐶₂₁ constitutes 𝐵₂₁. Note that 𝐵₂₁ is linear and homogeneous in 𝜂̇ but 𝐺₁₂ are irrelevant to 𝜂̇. In this paper, we restrict our consideration to the case that overall movements of the robot is confined to a single chart of the base manifold in which at any posture in the chart the Jacobian matrix is nondegenerate and therefore there exists a positive constant 𝜎₀ such that 𝐽(𝑞)𝐽^T(𝑞)≥𝜎₀𝐼₂ inside the chart. Under this condition, we analyze the following relation obtained by taking the inner product of (32) and 𝜂̈:d(1d𝑡2𝜂̇^T𝐺₂₂𝜂̇)=−𝜂̇^T𝐶₂₂𝜂̇−𝜂̇^T𝐵₂₁𝐱̇_𝑑−𝜂̇^T{𝐺^T₁₂𝐱̈_𝑑+𝐶₂₁𝐱̇_𝑑}. (34) The second term of the right hand side is quadratic in 𝜂̇ and therefore there exists a constant 𝛽₀ such that‖𝜂̇^T𝐵₂₁𝐱̇_𝑑‖≤𝛽₀‖𝜂̇‖² (35) and 𝛽₀ depend on the maximum magnitude of 𝐱̇_𝑑 and 1/𝜎₀. The third term of the right hand side is bounded from the above in the following way‖−𝜂̇^T{𝐺^T₁₂𝐱̈_𝑑+𝐶₂₁𝐱̇_𝑑}‖≤𝜂̇^T𝑊^Ṫ1𝐺𝑊𝜂+2𝐱̈_𝑑𝑃^T𝐺𝑃𝐱̈_𝑑+12𝐱̇^T_𝑑𝑃^T𝐶₀𝐺⁻¹𝐶₀𝑃𝐱̇_𝑑. (36) As discussed previously in [39], the damping matrix 𝐶₁ can be chosen to be of the order of 𝐺^1/2 and further so as to satisfy𝐶₀>2𝛽₀𝐼₄,𝐶₀>2𝐺. (37) Then, (34) is reduced to the inequality relationd𝜂d𝑡^𝑑(𝑡)≤−𝛾₀𝜂_𝑑(𝑡)+𝜉_𝑑(𝑡), (38) where we put with some positive constant 𝛾₀𝜂_𝑑1(𝑡)=2𝜂̇^T(𝑡)𝐺₂₂(𝑡)𝜂̇^T𝜉(𝑡),_𝑑1(𝑡)=2𝐱̈^T_𝑑(𝑡)𝑃^T𝐺𝑃𝐱̈_𝑑+1(𝑡)2𝐱̇^T_𝑑(𝑡)𝑃^T𝐶₀𝐺⁻¹𝐶₀𝑃𝐱̇_𝑑(𝑡). (39) Clearly, since 𝜂_𝑑(0)=0, (38) implies𝜂_𝑑(𝑡)≤^𝑡₀𝑒^{−𝛾0(𝑡−𝜏)}𝜉_𝑑(𝜏)d𝜏 (40) which concludes that 𝜂̇(𝑡) is uniformly bounded in 𝑡∈[0,𝑇].

Once 𝑞̇_𝑑(𝑡) and 𝜂(𝑡) are obtained for the given 𝐱̇_𝑑(𝑡) and 𝐱̈_𝑑(𝑡), the desired input signal 𝐯_𝑑(𝑡) in the image space of 𝐽(𝑞_𝑑) is obtained by setting 𝑞(𝑡)=𝑞_𝑑(𝑡), 𝐱̇(𝑡)=𝐱̇_𝑑(𝑡), 𝐱̈(𝑡)=𝐱̈_𝑑(𝑡) in (25).

5. Iterative Learning Control in the Task Space

Given a desired trajectory of the endpoint in the task space 𝐄² for a redundant robot arm, there exists a unique trajectory of robot motion in the joint space for a specified initial posture. In particular, it is shown in the previous section that there is uniquely a control signal in the task space that maneuvers the robot through the transpose of the Jacobian matrix to realize the endpoint tracking. However, such a control signal can not be obtained in any analytical form. Nevertheless, it is possible for us to acquire such a desired control signal by using a simple iterative learning control scheme, provided that the endpoint trajectory in the task space can be measured by visual sensing.

At the 𝑘th trial of iterative learning, the control signal for the dynamics of (21) is designed in the form𝑣_𝑘=−𝐽^T(𝑞_𝑘){𝜅Δ𝐱_𝑘(𝑡)+𝜁₁√𝜅Δ𝐱̇_𝑘(𝑡)}+𝑣_𝑘−1−ΦΔ𝐱̇_𝑘−1(𝑡), (41) where 𝐽(𝑞_𝑘) means 𝜕𝐱/𝜕𝑞 at 𝐱=𝐱(𝑞_𝑘), andΔ𝐱_𝑘(𝑡)=𝐱(𝑞_𝑘(𝑡))−𝐱_𝑑Δ𝐱̇(𝑡),_𝑘(𝑡)=𝐱̇(𝑞_𝑘(𝑡))−𝐱̇_𝑑(𝑡). (42) The first term of the right hand side of (41) signifies the inner task-space PD feedback, 𝑣_𝑘−1 denotes the previous control signal at the (𝑘−1)th trial, and Φ is an adequate positive definite constant matrix. At the first trial, usually we set 𝑣₀(𝑡)=0 for 𝑡∈[0,𝑇]. At the second trial 𝑣₁(𝑡) must contain erroneous terms. Fortunately, without knowing the desired ideal control 𝑣_𝑑(𝑡), it is possible to expect that Δ𝐱_𝑖(𝑡)→0 and Δ𝐱̇_𝑘→0 for 𝑡∈[0,𝑇] as 𝑘→∞. We give an illustrative example of numerical simulation conducted for the 4-DOF robot arm shown in Figure 2 with physical parameters given in Table 1. The values for length, mass, and inertia moment of the first link correspond to those of an upper arm of average human adult (male), and the values for the second link do to those of a lower arm. The third link corresponds to a human palm and the fourth an index finger. The desired task is to write a handwritten character “𝛼” on the 𝑥𝑦-plane. More explicitly, the endpoint trajectory is given by the equation𝐱_𝑑(𝑡)=+0.000.300.075cos𝜔(𝑡)0.100cos1.5𝜔(𝑡), (43) where 𝑇=2.0 [s] and𝑡𝜔(𝑡)=2.0𝜋{−15(𝑇)⁴𝑡+6(𝑇)⁵𝑡+10(𝑇)³}. (44) The initial posture of the arm is given in Table 2. Based on these data, the system of differential equations𝐺(𝑞_𝑘)𝑞̈_𝑘1+{2𝐺̇(𝑞_𝑘)+𝑆(𝑞_𝑘,𝑞̇_𝑘)+𝐶₀}𝑞̇_𝑘+𝐽^T(𝑞_𝑘){𝜅Δ𝐱_𝑘+𝜁₁√𝜅Δ𝐱̇_𝑘}=𝐽^T(𝑞_𝑘)𝐯_𝑘 (45) is numerically solved by using the Runge-Kutta method. In Figure 5 we show endpoint trajectories at 1st, 3rd, 5th, and 10th trials and their corresponding postures at 𝑡=0 and 𝑡=𝑇. On the other side, we are able to obtain the desired control signal numerically by numerically solving a couple of 1st-order differential equations of (29) and (30). Based upon knowing physical data given in Table 1, the initial posture of 𝑞(0) given in Table 2, and the specified endpoint trajectory of (43), we obtain the desired ideal control signal shown in Figure 6. It is quite interesting to know that, through simulations of the iterative learning, calculated control signals 𝑣_𝑘(𝑡), 𝑡∈[0,𝑇], approach the desired one as the trial number 𝑘 increases as shown in Figure 6. When 𝑘=10, the trajectory of control signal 𝐯₁₀(𝑡) almost coincides with the ideal one 𝐯_𝑑(𝑡), that uniquely exists just in the image space of the Jacobian matrix 𝐽(𝑞_𝑑) with 𝐱(𝑞_𝑑)=𝐱_𝑑 for all 𝑡∈[0,𝑇].

Table 1 

Physical parameters of the 4-DOF robot arm.

Table 2 

Initial values and gain settings in the case of the 4-DOF planar robot arm.

(a) 1st trial

(b) 3rd trial

(c) 5th trial

(d) 10th trial

Figure 5 

Endpoint trajectories and the initial and final postures of the arm.

(a) vd1

(b) vd2

Figure 6 

Transient responses of the ILC term ${\mathbf{v}}_d=(v_{d1},v_{d2}{)}^{\text{T}}$.

(a) vk1

(b) vk2

Figure 7 

Transient responses of the ILC term ${\mathbf{v}}_k=(v_{k1},v_{k2}{)}^{\text{T}}$.

It should be remarked that the desired endpoint trajectory 𝐱(𝑡) in the case of multijoint reaching for the robot arm of Figure 2 is obtained by solving the Euler equation of (4) or (5) as a two-point boundary-value problem when the initial posture 𝑞(0)=𝑞⁰ at 𝑡=0 is given and the terminal condition at 𝑡=𝑇 is partially specified so as to move the endpoint of the arm to meet 𝐱(𝑞)=𝐱_𝑑 at 𝑡=𝑇 and pass it away. This is the problem to find a curved orbit of the endpoint connecting two given points 𝐱(𝑞(0))=𝐱⁰ and 𝐱(𝑞(𝑇))=𝐱_𝑑 by selecting an adequate initial joint velocity 𝑞(̇0) that is nonzero. However, the orbit 𝐱(𝑡) of the endpoint can be represented by a curve 𝑐(𝑠) on 𝐄² with the aid of length parameters. In the case of middle-range reaching, the profile of this endpoint geodesic curve 𝑐(𝑠) quite resembles those of human-like multijoint reaching characterized typically by a quasistraight line movement of the endpoint starting from a fixed still state.

6. Extension to the Case of Existence of Effect of Gravity

Most of the previous results in Sections 4 and 5 can be extended to the case that robot dynamics is subject to the effect of gravity. In such a robot with redundancy in DOF, robot dynamics is expressed by the Lagrange equation:̈1𝐺(𝑞)𝑞+{2𝐺̇(𝑞)+𝑆(𝑞,𝑞)̇+𝐶₀}𝑞̇+𝑔(𝑞)=𝐽^T(𝑞)𝐮, (46) where 𝑔(𝑞) stands for the gravity term that can be regarded as a gradient vector of a potential function 𝑈(𝑞) with respect to 𝑞, that is, 𝑔(𝑞)=𝜕𝑈(𝑞)/𝜕𝑞. Let us denote again the endpoint position by 𝐱(𝑞) in the 𝑚-dimensional Euclidean space 𝐄^𝑚. Then, we split 𝑔(𝑞) into𝑔(𝑞)=𝐽⁺(𝑞)𝐽(𝑞)𝑔(𝑞)+(𝐼_𝑛−𝐽⁺(𝑞)𝐽(𝑞))𝑔(𝑞)=𝑔₁(𝑞)+𝑔₂(𝑞). (47) That is, 𝑔₁(𝑞) is a component of 𝑔(𝑞) in the image space of 𝐽(𝑞) and 𝑔₂(𝑞) is that of 𝑔(𝑞) in the kernel space of 𝐽(𝑞), that is orthogonally complement to the image space. In accordance with the split of the term 𝑔(𝑞), let us choose an 𝑛×(𝑛−𝑚)-matrix 𝑊(𝑞)=(𝑤₁,…,𝑤_𝑛−𝑚) with 𝑛-dimensional unit column vectors 𝑤_𝑖(𝑖=1,…,𝑛−𝑚) that are mutually orthogonal and satisfy 𝑊(𝑞)𝐽^T(𝑞)=0 (i.e., 𝑊(𝑞)𝐽⁺(𝑞)=0). Then, in a similar way to (22), we define𝑄(𝑞)=(𝐽⁺(𝑞),𝑊(𝑞)) (48) which leads to𝑄⁻¹𝑊(𝑞)=𝐽(𝑞)_T(𝑞). (49) On the other hand, for a given desired endpoint trajectory 𝐱_𝑑(𝑡) for 𝑡∈[0,𝑇], we consider the control signal with the task space PD feedback𝐮=−{𝜅Δ𝐱+𝜁₁√𝜅Δ𝐱̇}+𝐯, (50) where Δ𝐱=𝐱−𝐱_𝑑. Substituting (50) into (46) yields the closed-loop dynamics̈1𝐺(𝑞)𝑞+{2𝐺̇(𝑞)+𝑆(𝑞,𝑞)̇+𝐶₀}𝑞̇+𝑔₁(𝑞)+𝑔₂(𝑞)+𝐽^T(𝑞){𝜅Δ𝐱+𝜁₁√𝜅Δ𝐱̇}=𝐽^T(𝑞)𝐯, (51) where 𝐯 expresses a feedforward task space control signal. Then, consider the transform of 𝑞̇ to (𝐱̇^T,𝜂̇^T)^T with an (𝑛−𝑚)-dimensional vector in such a form that𝑞̇=(𝐽⁺𝐱̇𝜂̇𝐱̇𝜂̇(𝑞),𝑊(𝑞))=𝑄(𝑞). (52) In order that the equation of (51) with a desired control signal 𝐯_𝑑(𝑡) has a unique solution 𝑞_𝑑(𝑡) and 𝑞̇_𝑑(𝑡) so that its endpoint trajectory 𝐱(𝑡) is coincident with 𝐱_𝑑(𝑡), and 𝐱̇(𝑡)=𝐱̇_𝑑(𝑡), the following equations must be satisfied:𝐺_𝑑𝐱̈_𝑑𝜂̈1+{2𝐺̇_𝑑+𝑆_𝑑+𝐶_𝑑}𝐱̇_𝑑𝜂̇+(𝐽⁺(𝑞_𝑑))^T𝑔_𝑑𝑊^T(𝑞_𝑑)𝑔_𝑑=𝐯_𝑑0, (53) where 𝑔_𝑑=𝑔(𝑞_𝑑) and𝐺_𝑑=𝑄^T_𝑑𝐺(𝑞_𝑑)𝑄_𝑑,𝐶_𝑑=𝑄^T_𝑑𝐶₀𝑄_𝑑,𝑆_𝑑=𝑄^T_𝑑𝑆𝑄_𝑑−12𝑄̇^T_𝑑𝐺(𝑞_𝑑)𝑄_𝑑+12𝑄^T_𝑑𝐺(𝑞_𝑑)𝑄̇_𝑑 (54) with 𝑄_𝑑=𝑄(𝑞_𝑑). The latter (𝑛−𝑚)-dimensional component of (51) can be expressed by𝐺₂₂𝜂̈_𝑑1+{2𝐺̇₂₂+𝑆₂₂+𝐶₂₂}𝜂̇+𝐺^T₁₂𝐱̈_𝑑+𝐵₂₁𝐱̇_𝑑+𝑊^T_𝑑𝑔_𝑑=0, (55) where 𝐺_𝑑 and 𝐵_d=(1/2)𝐺̇_𝑑+𝑆_𝑑+𝐶_𝑑 are decomposed into𝐺_𝑑=𝐺₁₁𝐺₁₂𝐺^T₁₂𝐺₂₂,𝐵_𝑑=𝐵₁₁𝐵₁₂𝐵₂₁𝐵₂₂, (56) and 𝐺₂₂ and 𝐵₂₂ are of (𝑛−𝑚)×(𝑛−𝑚)-matrix, and 𝐺₁₁ and 𝐵₁₁ are of 𝑚×𝑚-matrix. The simultaneous differential equations (53) of 1st order together with𝑞̇_𝑑=𝐽⁺(𝑞_𝑑)𝐱̇_𝑑+𝑊(𝑞_𝑑)𝜂̇ (57) determine a unique solution 𝑞_𝑑(𝑡) and 𝜂̇(𝑡) for given 𝐱_𝑑(𝑡), 𝐱̇_𝑑(𝑡), and 𝐱̈_𝑑(𝑡) with the initial conditions 𝑞_𝑑(0)=𝑞⁰, 𝐱(𝑞⁰)=𝐱_𝑑(0), and 𝜼̇(0)=0.

7. ILC for Handwriting

All the considerations in the previous sections can be extended to the case of a handwriting robot whose last link is a ball-point pen constrained on a hypersurface 𝜑(𝐱)=0, where 𝐱=(𝑥,𝑦,𝑧)^T in 𝐄³ and 𝜑(𝐱) is a scalar function of 𝐶^∞-class (see Figure 8). First, we consider a pure mathematical problem of finding a geodesic curve by ignoring the effect of gravity and any joint damping. In this case, let us consider an open connected area 𝑆 on the hypersurface as shown in Figure 8, on which any point 𝑃(=𝐱) satisfies the equality 𝜑(𝐱)=0. Then, we denote by 𝐹 a local coordinate chart defined by 𝐹={𝑞∈(𝑀,𝑔_𝑖𝑗)∣𝐱(𝑞)∈𝑆} and assume that 𝐹 is connected and at any 𝑞∈𝐹 the Jacobian matrix 𝐽(𝐱(𝑞))=𝜕𝐱(𝑞)/𝜕𝑞^T is nondegenerate. Then, given a point 𝑃 on 𝑆 with the cartesian coordinates 𝐱_𝑃, the set of all 𝑞 such that 𝐱(𝑞)=𝐱_𝑃 and 𝑞∈𝐹 constitutes a single-dimensional submanifold 𝑁_𝑃 (see Figure 9). Hence, for another given point 𝑃^′ on 𝑆, it is possible to consider an orbit in the submanifold 𝐹 (equivalently, 𝜙(𝐹) in the configuration space) starting from 𝑞⁰, lying on 𝑁(𝑆), and reaching some point lying on 𝑁_𝑃′ that is another single-dimensional submanifold defined by the set of all 𝑞 satisfying 𝐱(𝑞)=𝐱_𝑃′ and 𝑞∈𝐹. Thus, it is reasonable to suppose that there exists an optimal orbit 𝑞^∗(𝑡) that gives minimization of the Riemannian distance from 𝑞⁰ to 𝑁_𝑃′ such that𝑑(𝑞⁰,𝑁_𝑃′)=inf_𝑞(𝑡)¹₀𝑔_𝑖𝑗(𝑞(𝑡))𝑞̇_𝑖(𝑡)𝑞̇_𝑗=(𝑡)d𝑡₁₀𝑔_𝑖𝑗(𝑐(𝑡))𝑐_𝑖̇(𝑡)𝑐_𝑗̇(𝑡)d𝑡, (58) where the infimum is taken over all the orbits lying on 𝑁(𝑆) (see Figure 9), starting from 𝑞⁰ and reaching 𝑁_𝑃′, where 𝑞^∗(𝑡) is rewritten by 𝑐(𝑡). It is reasonable to conclude that the optimal orbit 𝑐(𝑡) in (58) is a solution to the Euler equation under the constraint 𝜑(𝐱(𝑐))=0:1𝐺(𝑐(𝑡))+{2𝐺̇(𝑐)+𝑆(𝑐,𝑐)̇}𝑐(̇𝑡)=−𝜆𝐽^T_𝑐(𝐱(𝑐(𝑡)))𝜕𝜑𝜕𝐱^T, (59) where 𝜆 denotes a Lagrange multiplier. In other words, the path of 𝑐(𝑡) from 𝑞⁰ to some point on 𝑁_𝑃′ can be called the geodesic on the Riemannian submanifold 𝑁(𝑆) induced naturally from the constraint 𝜑(𝐱(𝑞))=0.

Figure 8 

A handwriting robot with four DOFs whose endpoint $P$$(=(x,y,z))$ is constrained on a hypersurface $\phi (\mathbf{x})=0$, where $\phi (\mathbf{x})$ is a scalar function of $C^{\infty }$-class.

Figure 9 

A geodesic curve starting from $q^0$ lying on $N_P$ and reaching some point on $N_{P\text{'}}$. Here, $\varphi$ is a homomorphism mapping $F$ in $M$ to $\varphi (F)$ in $R^4$.

Next, consider the full dynamics of the handwriting robot with 4 DOFs shown in Figure 8 by taking into account the effect of gravity forces and damping torques at joints (see [38]). The dynamics is described bÿ1𝐺(𝑞)𝑞+{2𝐺̇(𝑞)+𝑆(𝑞,𝑞)̇}𝑞̇+𝐶₀𝑞̇+𝑔(𝑞)=−𝜆𝐽^T(𝑞)𝜕𝜑(𝐱)𝜕𝐱^T+𝑢, (60) where 𝑔(𝑞)=𝜕𝑈(𝑞)/𝜕𝑞, 𝑈(𝑞) denotes the gravity potential, 𝐶₀ a positive definite damping matrix, and 𝐽(𝑞)=𝜕𝐱(𝑞)/𝜕𝑞^T. It should be remarked that 𝜕𝜑/𝜕𝐱 stands for a vector that originates at the position 𝐱(𝑞) in 𝐄³ and is normal to the surface 𝜑(𝐱)=0 which the ball pen contacts with. Now we introduce a length parameter 𝑠 in 𝐄³ by the quantity𝑠(𝑡)=^𝑡₀‖̇=𝐱(𝜏)‖d𝜏_𝑡0‖𝑞̇^T(𝜏)𝐽(𝑞(𝜏))𝐽^T(𝑞(𝜏))𝑞(̇𝜏)‖d𝜏 (61) and define the unit normal at the contact point as follows:𝐧(𝑠)=𝜕𝜑‖𝜕𝐱𝜕𝜑‖𝜕𝐱⁻¹. (62) Then, by rewriting 𝑓=𝜆‖𝜕𝜑/𝜕𝐱‖, (60) can be rewritten intö1𝐺(𝑞)𝑞+{2𝐺̇(𝑞)+𝑆(𝑞,𝑞)̇}𝑞̇+𝐶₀𝑞̇+𝑔(𝑞)=−𝑓𝐽^T(𝑞)𝐧(𝑠)+𝑢. (63) Note that the inner product of 𝑞̇ and the right-hand side of (63) vanish. In other words, 𝐱̇(=𝐽(𝑞)𝑞)̇ is orthogonal to 𝐧(𝑠) at the contact point in 𝐄³. Under the assumption that both the endpoint position 𝐱(𝑡) and the velocity 𝐱̇(𝑡) are measured in real time by visual sensing and the Jacobian matrix 𝐽(𝑞(𝑡)) is calculated from the measurement data of 𝐱(𝑡) and 𝑞(𝑡) in real time, suppose that the control signal must be constructed through the Jacobian transpose in the form𝑢=𝐽^T(𝑞)𝑣. (64) For a given desired endpoint trajectory 𝐱_𝑑(𝑡), 𝑡∈[0,𝑇], together with 𝐱̇_𝑑(𝑡) and 𝐱̈_𝑑(𝑡) and a given desired pressing force 𝑓_𝑑(𝑡), we are concerned with the problem to find a desired control signal 𝑣_𝑑(𝑡) of (64) that maneuvers the robot to make the endpoint of the last link (ball pen) trace 𝐱(𝑞(𝑡))=𝐱_𝑑(𝑡) on the hypersurface with the pressing force 𝑓_𝑑 in the direction normal to the surface. To show the existence of a control signal in the image space of 𝐽(𝑞), we express it in a decomposed form such that𝑢=𝐽^T(𝑞){𝑣_𝑛𝐧(𝑠)+𝑣_𝑏𝐛(𝑠)+𝑣_𝑒𝐞(𝑠)}, (65) where 𝐛(𝑠) signifies the unit vector tangent to the surface in the direction of −𝐱̇(𝑡) and 𝐞(𝑠)=𝐧(𝑠)×𝐛(𝑠). Note that substituting (65) into (63) yields̈1𝐺(𝑞)𝑞+{2𝐺̇+𝑆}𝑞̇+𝐶₀𝑞̇+(𝐼₄−𝐽⁺𝐽)𝑔(𝑞)=−𝐽^T𝑣Π(𝑠){^𝑛𝑣−𝑓_𝑏𝑣_𝑛+𝑔_𝑛𝑔_𝑏𝑔_𝑒}, (66) where𝑔Π(𝑠)=(𝐧(𝑠),𝐛(𝑠),𝐞(𝑠)),_𝑛𝑔_𝑏𝑔_𝑒=𝑔^T(𝑞)𝐽⁺𝑔𝐧(𝑠)_T(𝑞)𝐽⁺𝑔𝐛(𝑠)_T(𝑞)𝐽⁺𝐞(𝑠)=Π^T(𝑠)(𝐽⁺)^T𝑔(𝑞). (67) Note that Π(𝑠) is an orthogonal matrix belonging to 𝑆𝑂(3), 𝐠=(𝑔_𝑛,𝑔_𝑏,𝑔_𝑒)^T is a vector in 𝐄³, and a component of 𝑔(𝑞) tȯ𝜉̇𝜂̇𝜸̇𝑞=𝑄(𝑞),𝑄(𝑔)=(𝐽⁺(𝑞)𝐧(𝑠),𝑊(𝑞),𝐽⁺(𝑞)(𝐛(𝑠),𝐞(𝑠))), (68)(69) where 𝑊(𝑞) is the 4×1 unit vector satisfying 𝐽(𝑞)𝑊(𝑞)=0, 𝜉̇ and 𝜂̇ are a scalar and express a 2×1 velocity vector such that (𝐛(𝑠),𝐞(𝑠))𝜸̇(𝑠)=𝐱̇(𝑡) at the contact point between the tip of the ball pen and the surface. Since 𝜉̇=0 that implies that the velocity of the tip of the pen in the direction 𝐧(𝑠) normal to the surface is zero, substituting (68) into (66) and multiplying (66) by 𝑄^T(𝑞) yield𝛼(𝜂̇,𝜂̈,𝜸̇,𝜸̈)=−(𝑣_𝑛−𝑓+𝑔_𝑛𝐺),₁₁̈1𝜂+{2𝐺̇₁₁+𝑆₁₁}𝜂̇+𝑊^T𝐶₀𝑊𝜂̇+𝑊^T𝑔(𝑞)+𝐺₁₂𝜸̈+𝐵₁₂̇𝐺𝜸=0,₂₂̈1𝜸+{2𝐺̇₂₂+𝑆₂₂}𝜸̇+𝐶₂₂𝜸̇+𝐺^T₁₂𝜂̈+𝐵₂₁̇𝑔𝜂=−𝑣−^𝑏𝑔_𝑒, (70)(71)(72) where 𝐺₁₁ is of 1×1, 𝐺₂₂ is of 2×2, and𝐺=𝑃^T𝐺𝑃,𝑃=(𝑊,𝐽⁺(𝐛,𝐞)),𝐺𝐺=¹¹𝐺₁₂𝐺^T₁₂𝐺₂₂,𝐵𝐵=¹¹𝐵₁₂𝐵₂₁𝐵₂₂,1𝐵=2𝐺̇+𝑆+𝑃^T𝐶₀𝑃,𝑆=𝑃^T1𝑆𝑃−2𝑃̇^T𝐺𝑃+12𝑃^T𝐺𝑃̇. (73) In (70), 𝛼 is a function of 𝐶^∞-class with respect to, 𝜂̇,𝜂̈,𝜸̇,𝜸̈, and 𝑞. Thus, for a given desired trajectory of the tip of the ball pen, 𝜸̇(𝑡)=𝐱̇_𝑑(𝑡) and 𝜸̈(𝑡)=𝐱̈_𝑑(𝑡), it is possible to prove the existence of 𝑞_𝑑 and 𝜂̇(=𝜂̇_𝑑) that satisfies 𝑞̇_𝑑=𝑃(𝑞_𝑑)𝜂̇_𝑑𝐱̇_𝑑,𝐺₁₁𝜂̈_𝑑1+{2𝐺̇₁₁+𝑆₁₁}𝜂̇_𝑑+𝐶₁₁𝜂̇_𝑑+𝑊^T𝑔(𝑞_𝑑)+𝐺₁₂𝐱̈_𝑑+𝐵₁₂𝐱̇_𝑑=0, (74) where 𝐶₁₁=𝑊^T𝐶₀𝑊. The two equations (74) are the set of five simultaneous differential equations of the 1st order in five variables of 𝜂̇_𝑑 and 𝑞_𝑑. Once the desired trajectory of joint motion 𝑞_𝑑(𝑡) with its first and second derivatives is determined together with the motion 𝜂̇_𝑑(𝑡) of excess-DOF, the desired control signal 𝑣_𝑑 is given by (72) by putting 𝜸̇=𝐱̇_𝑑, 𝑞=𝑞_𝑑, and 𝜂̇=𝜂̇_𝑑. Moreover, if a desired pressing force 𝑓_𝑑 is specified, the desired control signal in the direction normal to the surface can be constructed through (70) such that𝑣_𝑛=𝑓_𝑑−𝑔_𝑛(𝑞_𝑑)−𝛼(𝜂̇_𝑑,𝜂̈_𝑑,𝐱̇_𝑑,𝐱̈_𝑑). (75)

8. What Should the Central Processing Unit Learn?

Given a smooth curve 𝛾(𝑠(𝑡)) on a hypersurface 𝜑(𝐱)=0 over time interval 𝑡∈[0,𝑇], it is shown in the previous section that there exists uniquely a control signal 𝑣(𝑡) lying in the image space that makes the endpoint of the robot trace the curve 𝛾(𝑠(𝑡)) on the hypersurface. Irrespective of the redundancy of DOF, once a starting posture of the robot is specified in the Euclidean space 𝐄³ with its endpoint position 𝑃=𝐱(0) on the hypersurface 𝜑(𝐱)=0 as shown in Figure 10, the orbit 𝑞_𝑑(𝑡) that realizes 𝐱(𝑞_𝑑(𝑡))=𝐱_𝑑(𝑡) itself is determined uniquely correspondingly to the uniquely existing control signal 𝑣_𝑑(𝑡) (see Figure 10). Then, there arises a problem whether it is possible to acquire such a desired control signal without knowing the details of the robot dynamics. In order to discuss the problem, let us consider a learning control scheme that is expressed by the following form of the control signal (see Figure 11):𝑢_𝑘=−𝐽^T(𝑞){𝜅(𝐱−𝐱_𝑑)+𝜁√𝜅(𝐱̇−𝐱̇_𝑑)}−𝐽^T(𝑞){𝐯_𝑘+𝜎_𝑘𝐧(𝑠)}, (76) where 𝜅>0 stands for a constant expressing the stiffness parameter of position feedback and 𝜁>0 a constant lying in interval [0.5,2.0] of 𝐑¹. The first term of the right-hand side expresses the task space (𝐄³) PD feedback for a given set of position and velocity (𝐱_𝑑,𝐱̇_𝑑) and the second term stands for the learning control signal at the 𝑘th operation that is constructed as follows:𝐯_𝑘+1=𝐯_𝑘−ΦΔ𝐱̇_𝑘,𝜎_𝑘+1=𝜎_𝑘+𝜓Δ𝑓_𝑘, (77) where 𝜓 is an appropriate constant such that 0<𝜓<1 and Φ is a positive-definite 3×3 constant matrix. In the case of ordinary handwriting when the constraint surface is a horizontal plane, it is shown in the previous paper in [38] that the learning update law for the control signal based on (76) makes the motion of the endpoint trace the given curve 𝐱_𝑑(𝑡) on the plane and at the same time the joint motions 𝑞_𝑘(𝑡) and 𝑞̇_𝑘(𝑡) converge to the ideal trajectories 𝑞_𝑑(𝑡) and 𝑞̇_𝑑(𝑡) as 𝑘→∞. The proof of convergence of the control signals 𝐯_𝑘 to the desired one 𝐯_𝑑 as 𝑘→∞ is also given in [39] on the basis of the “passivity” of the input 𝐯 and the output 𝐱̇ of the system dynamics of (63) provided that 𝑢 is given through (64) and the gravity effect 𝑔(𝑞) is carefully compensated. Some numerical simulation results are also presented in [39].

Figure 10 

Handwriting on a hypersurface $\phi (\mathbf{x})=0$.

Figure 11 

Iterative learning control scheme for robotic handwriting.

Returning to Bernstein's problem discussed in the introductory section, we are fortunately able to quote Latash's commentary [40] to the book originally written by Bernstein in Russian cited as [2]. Latash says,

Bernstein's definition of degrees of freedom relied heavily on the analysis of kinematic degrees of freedom of various systems. In particular, when analyzing the human upper limb, Bernstein considered all possible orthogonal axes of rotation in a joint as independent degrees of freedom, which were later summed over the joints. The apparent redundancy of the joints of the human arm in comparison to the three-dimensional space where we happen to live and where movements take place led Bernstein to his famous formulation that the essence of motor control is the elimination of the redundant degrees of freedom. The beauty and brevity of this formulation is stunning. A skeptic may want to decide whether control can always be reduced to the elimination of redundant degrees of freedom. In other words, I am suggesting that Bernstein's famous definition may not always be correct (blasphemy!!!)

Even in the case of simple humanlike multijoint point-to-point reaching, Riemannian geometry suggests a good reason to think that the central nervous system is functioning according to a natural law of Newton's mechanics, that is, the law of inertia for multijoint mechanisms. Once a geodesic curve 𝑞^∗(𝑡) connecting the given initial posture 𝑞⁰ and the equilibrium submanifold 𝑁_𝑃′ (see (18)) is determined with some initial velocity 𝑞̇^∗(0), then the three-dimensional orbit 𝐱(𝑞^∗(𝑡)) for 𝑡∈[0,𝑇] is obtained. This orbit can be rewritten into a three-dimensional curve 𝛾(𝑠) with the aid of length parameter 𝑠 for a corresponding interval 𝑠∈[𝑎,𝑏]. Then this curve can be spelled out by a desired trajectory given as a movement of the tip of the arm in 𝐄³ in the form of 𝐱_𝑑(𝑡)=𝛾(𝑠(𝑡)) for 𝑡∈[0,𝑇] with the aid of a scale change of “time” through a monotonously increasing function 𝑠(𝑡). Once a desired orbit 𝐱_𝑑(𝑡) of motion of a tip of the arm is given in 𝐄³ and at the same time an initial posture of the arm is chosen, the trajectory of motion in the joint space is uniquely determined without eliminating any excess of the system's DOF. However, it is important to note that, even if there arises a small change of the initial posture, say 𝑞⁰+𝛿𝑞, it is possible to obtain the same desired orbit 𝐱_𝑑(𝑡) in 𝐄³ but the trajectory 𝑞_𝑑(𝑡) of joints differes slightly from that obtained when the initial posture is set as 𝑞(0)=𝑞⁰ and also the desired control signal 𝑣_𝑑(𝑡) may differ. Another noteworthy characteristic of humanlike multijoint movements is called “variability”, which was first pointed out by Bernstein [2]. Latash [32] observed that in the case of human skilled motion the grade of variability of each endpoint trajectory is quite low relative to variable profiles of joint responses at each trial of reaching. The mathematical arguments in the previous section suggest that the main part of variability in joint space is caused by small fluctuations of choice of an initial posture trial by trial and the others may be noise.

We do not discuss the importance of redundancy of the muscles involved in a specific motion of multijoint reaching. The central nervous system surely uses its own means of communication with the muscles, which is not analogous to the language of joint kinematics that specifies the trajectories in individual joint. Instead of overcoming the problem of redundancy of the muscles, we implicitly assume in this paper that a desired torque of individual joint is finally generated by a total of forces of all the muscles involved in movement of the joint. This premise may not be justified by any reason, but it is important to quote Jackson's observation as a neurophysiologist [41] (see Figure 12):

Figure 12 

Sensory-motor coordination.

“To speak figuratively, the central nervous system knows nothing of muscles, it only knows movements.”

“The highest centres represent all parts of the body, literally all parts supplied by nerves.”

9. Conclusions

This paper discusses difficult problems of control for human-like robots with redundant DOFs from the standpoint of Riemannian geometry. Irrespective of joint redundancy, we have shown that there is no necessity to resolve the inverse kinematics problem by introducing any artificial performance index and optimizing it in both the cases of (1) multijoint reaching movement with excess joints, and (2) handwriting through iterative learning control. The most important conclusion is that Riemannian geometry does not care about the redundancy of joints through which rigid links are connected in series but it is powerful in establishing a correspondence between working-point trajectories in the external three-dimensional space (𝐄³) and movement trajectories in a space of joint angles (a configuration space 𝐑^𝑛 or a Riemannian manifold {𝑀,𝑔_𝑖𝑗} as a set of whole possible postures).