Abstract
This paper analyzes the stability of a trilateral teleoperation system of a mobile robot. This type of system is nonlinear, time-varying, and delayed and includes a master-slave kinematic dissimilarity. To close the control loop, three controllers are used under a position master/slave velocity strategy. The stability analysis is based on Lyapunov-Krasovskii theory where a functional is proposed and analyzed to get conditions for the control parameters that assure a stable behavior, keeping the synchronism errors bounded. Finally, the theoretical result is verified in practice by means of a simple test, where two human operators both collaboratively and simultaneously drive a 3D simulator of a mobile robot to achieve an established task on a remote shared environment.
1. Introduction
Robot teleoperation allows the execution of a task in a remote environment; generally dangerous and harmful jobs for the human operator are addressed necessarily by these systems to improve the safety level. They are formed by a human operator, a device called master, a robot called slave acting on a remote environment, and a communication channel that connect user and environment. Besides, the term bilateral indicates that there is a physical coupled via force feedback additional to the common multimedia information such as video and audio. That is, the slave follows the motions carried out by the master while simultaneously the human operator receives multimodal information [1]. This feature about tact at distance increases the application field to varied areas such as telemedicine, health assistance, exploration, entertainment, teleservices, and telemanufacturing, among others. Besides, the use of the Internet as a low-cost communication channel allows easily connecting the master and slave devices. However, all Internet-like communication adds nonsymmetric time-varying delays, which if become larger could cause oscillations, low performance [2, 3], and poor transparency [4, 5]. In general, the teleoperation systems are represented by delayed time-varying nonlinear models. So, one of the research topics in delayed teleoperation systems is related to the analysis of different mathematical tools to get delay-dependent conditions for the controller parameters that allow assuring the stability of the delayed closed-loop system.
In the field about robot teleoperation, most control schemes are designed and analyzed considering two manipulators called master and slave [3–6] (and references therein). In general, energy is removed to assure stability or passivity [7]. A classic strategy is the use of scattering signals or wave transformations [8, 9], which adds an apparent damping to get passivity analyzing the system as a two- or four-channel port. In the last years, many researches were focused on the use of schemes to get a stable operation in the sense of Lyapunov by using a sufficiently large damping injected into master and slave [10–12]. Besides, many strategies designed for teleoperation of manipulators were tested also into bilateral teleoperation of WMR (wheeled mobile robots), for example, control based on impedance [13], control based on -passivity [14, 15], schemes including prediction and augmented reality [16], haptic teledriving of a WMR coupled with slippage, where an acceleration-level control law is proposed to assure the passivity of the bilateral system [17, 18], and also control schemes [19, 20]. It is important to remark that the stability analysis of a master position/slave velocity coupling is different mathematically to the common position/position used in a teleoperation between two manipulators due to the master-slave kinematic dissimilarity: master device (e.g., joystick) has a bounded workspace, while slave mobile robot can navigate by an unbounded workspace.
On the other hand, several applications, such as rehabilitation, surgical training, and safety [21, 22], require multiple users instead of a single operator due to the fact that a multilateral teleoperation provides various useful capabilities such as increased dexterity, improved loading, redundancy, and handling capacity, among others. For example, these systems can be addressed to training of human operators to learn the driving of industrial machines. However, the analysis about these systems is even more complex. With respect to the control of them, there are strategies for trilateral systems considering all devices as robot manipulators, for example, control based on passivity [23], where observers and controllers of passivity are added to retain the system energy bounded; adaptive fuzzy control scheme [24]; and controllers [25], where the stability of a dual-master/single slave teleoperation system is evaluated. On the other hand, few papers have been proposed to trilateral systems of WMR. In [26], a WMR trilateral teleoperation scheme is proposed but the time delays are not considered. Also, recently, many papers have been focused on teleoperation networks [27, 28]. This kind of system is an extension of the traditional bilateral teleoperation but includes multiple users working collaboratively and simultaneously from different locations on a shared remote environment, through robots. However, to the best of the authors’ knowledge, the -like strategies using a master position/WMR velocity mapping present in the current literature cannot be applied in a direct way to the delayed trilateral teleoperation of a mobile robot because both manipulator-manipulator and mobile-manipulator connections simultaneously occur, making difficult the energy analysis inside the system. That is, the application of controllers to a delayed trilateral teleoperation of a mobile robot requires a new analysis to get delay-dependent conditions that can be held from an adequate calibration of control parameters, to assure a stable behavior for the errors between master 1 position, master 2 position, and mobile robot velocity.
This paper presents a stability analysis based on a Lyapunov-Krasovskii functional applied to a delayed trilateral teleoperation of a mobile robot. The main contribution of this work involves a new functional which is analyzed along the system trajectories, allowing getting information about the evolution of velocity and position of both master 1 and master 2, velocity and acceleration of mobile robot, and synchronism errors. As a result of the mathematical analysis being exposed, depending-delay conditions for the calibration of the controller are achieved. Furthermore, a test about trilateral teleoperation using a 3D simulator is shown in order to verify that the evolution of the delayed trilateral system state is in agreement with the theoretical result achieved.
This work is organized as follows: Section 2 presents some preliminary aspects such as the employed dynamic models and the assumptions used. In Section 3, a control scheme applied to delayed trilateral teleoperation of a mobile robot is proposed and its stability analysis based on a Lyapunov-Krasovskii functional (LKF) is performed. Section 5 shows an experimental test, where two users drive simultaneously a wheeled robot simulator. Finally, in Section 6, the conclusions of this work are given.
2. Preliminary
This paper will analyze teleoperation systems in which two human operators generally called trainer and trainee, use each one a master device (called master 1 and master 2, resp.) to drive a mobile robot slave in a trilateral way, as it is illustrated in Figure 1.
Master Model. Each human operator drives a manipulator robot to generate velocity commands and also receive force feedback. The standard nonlinear dynamic model of a manipulator robot to represent each master is used [10, 11], that is,where index indicates master 1 or master 2, is the joint position of the master; is the joint velocity; is the inertia matrix; is the matrix representing centripetal and Coriolis torques; is the gravitational torque; is the torque caused by the human operator force; and is the torque applied to the master by the controller. Commonly, master 1 is associated with the trainer and master 2 with the trainee.
Mobile Robot Model. For the case of teleoperation of a mobile robot, the dynamic model of a mobile robot with differential traction is considered [14]. It has two independently actuated rear wheels and is represented bywhere is the robot velocity vector with and representing the linear and angular velocity of the mobile robot, is the force caused by the elements of the environment on the robot as well as other nonmodeled external forces such as static and dynamic frictions, is the inertia matrix and is the Coriolis matrix where is the mass of the robot, is the rotational inertia, and is the distance between the mass center and the geometric center. In addition, the control action involves a force and torque is applied to the robot.
The devices are numbered as follows: index 1 represents master 1; index 2 indicates master 2; and index 3 represents the mobile robot. Besides, to represent the communication channel of a trilateral system, a forward time delay (from device to device ) and a backward time delay (from device to device ) for each pair master 1-master 2, master 1-slave, and master 2-slave are taken as time-varying and nonpredictable.
Next, common properties, assumptions, and lemmas will be used in this paper [7, 10, 11].
Property 1. The inertia matrices and are symmetric positive definite.
Property 2. Matrix is skew-symmetric.
Property 3. There exists such that for all time .
Property 4. The vector is bounded if is bounded.
Assumption 5. All time delays and are bounded. Therefore, there exist positive scalars such that for all .
Assumption 6. The human operator has finite energy [7] and the environment is represented by a damping-like model plus a finite-energy perturbation [12]. Such models are mathematically represented as follows:where is the human force, is a finite value that bounds the energy of the human operator, is the environment’s damping, and is assumed to be bounded with finite energy, that is , with a positive constant and .
Lemma 7 (see [11]). For real vector functions and and a time-varying scalar with , the following inequality holds:
Relation 8. The time delayed variable can be written as follows:
3. Trilateral Teleoperation of a Mobile Robot
The teleoperation trilateral system is used to control the velocity of a mobile robot, where the trainer and trainee (or in general two human operators) permanently send commands and perceive a force feedback. Based on the control schemes applied to bilateral teleoperation of manipulator robots [10–12] and mobile robots [19, 20], and its recent use in delayed trilateral teleoperation of manipulator robots [25], the stability of a dual-master single mobile robot system under a master position/slave velocity strategy will be analyzed in this work.
The set of control actions is established as follows:where the parameters and are positive constant and they represent the proportional gain and acceleration-dependent damping added by the velocity controller placed on the remote site, and are the damping injected in the masters, and and represent a relative spring depending on the mismatch between the master position and a reference feedback. Besides, the parameter linearly maps the master position to a velocity reference, and represents the available information about the mobile robot acceleration but an infinitesimal before the current time instant. It is assumed that this signal can be represented by the real mobile robot acceleration plus an infinitesimal error [20, 29]:
Furthermore, two dominance factors are used. The factor determines the authority of trainer over trainee and implies the supremacy of trainee over trainer. Moreover, the dominance factors and indicate the supremacy of trainer and trainee over the mobile robot, respectively. The relation between them is defined in [25] as follows:with . As is closer to 1, the following occurs. (i)On master device 1, the force feedback from the mobile robot is bigger than the feedback of master device 2; then the trainer has greater perception about the state of the mobile robot than about the command provided by the trainee.(ii)On master device 2, the force feedback from master device 1 is bigger than the force feedback from the mobile robot; thus the trainee has greater perception about the command generated by the trainer than about the mobile robot state.(iii)On mobile robot, increases with (Figure 2); thus the command of the trainer has higher priority than the command of the trainee.
Relation between and can be seen on Figure 2. To the best of the authors’ knowledge, there is not a conceptual justification about the above definition, but from it the stability analysis can be completed. The study about different definitions of that allow keeping the system stability is an open subject.
4. Stability of the Delayed Closed-Loop System
The stability analysis of the control scheme is based on the Lyapunov-Krasovskii theory applied to trilateral teleoperation of a mobile robot in presence of time delays.
Theorem 9. Consider a teleoperation system wherein two human operators (3) use two master devices (1) to drive a slave mobile robot (2) which interacts with an environment described by (4). If the control actions (7) are applied to such system, and if the controller gains hold withthen the vector is bounded, and the variables vectors .
Proof. A Lyapunov-Krasovskii functional is proposed, in order to analyze its evolution along the system trajectories and then infer the behavior of which includes the kinetic energy of the two masters and mobile robot as well as the potential energy of the synchronism errors of the trilateral system. Next each is represented mathematically as follows:where are arbitrary positive constants which will be determined later.
To get , the time derivative of (11)–(18) will be computed along the system trajectories. First, is obtained from (11) as follows:Next, the master dynamics (1) are inserted into (19) considering also Properties 1 and 2 of the master model:Now, the control action from (7) is included in (20) and Relation 8 (6) is applied over the delayed terms: Following a similar procedure, the time derivative of (12) is given as follows:Next, the dynamics (1) of master 2 are introduced into (22) while Properties 1 and 2 of the master model are used, too:If the control action from (7) is replaced into (23) and Relation 8 is applied again over delayed terms, the following expression is achieved:The functions and are easily reached from (13) and (14), respectively:Including the dynamics of the mobile robot (2) into (26) and then adding the control action from (7) and the environment force of (4) to the resulting equation, we getIf relations (6) and (8) and from (9) are considered, the terms of (27) can be reorganized as follows:Next, the derivative of the synchronism errors of the trilateral system are obtained from (15), (16), and (17):Finally, is deduced from (18) (see [11, 20] for further details) as follows:Now, observing the expressions obtained for , there are terms with integrals that difficult the analysis. To solve this, the terms with integrals can be linked by means of Lemma 7 (5) to get quadratic expressions which are convenient to compare the contribution of each factor on the stability.
If each integral term of (21), (24), and (28) is associated with one integral term of (30) through the structure of (5), the following relations can be stated: Joining the functions and applying relations (31)–(37), the terms including integrals are replaced with expressions more adequate to comparison:where the constants are given byAs , (32) can be rewritten asSince (Assumption 6) and , if the damping coefficients are sufficiently high such that in (34) then it is possible to state that .
Then, if (34) is integrated from 0 to , we haveFurthermore, as is considered infinitesimal, the terms containing in (34) and (35) can be neglected. Besides, the integral term is bounded by Therefore, (35) can be rewritten asAs is radially unbounded and from (37) also it is bounded for all , then . Thus, the synchronism errors are bounded. Even more, considering this result in (35), we getIn this way, relation (38) allows asserting that the variables . The proof is complete.
Remark 10. Taking the result achieved in (34), as the parameters are larger, then (both master velocities and slave acceleration) will tend to a smaller convergence ball.
Remark 11. From the vector boundness and employing Properties 2 and 3, Proving that the control actions (7) are bounded is direct.
Remark 12. The following procedure to calibrate the control parameters of a delayed trilateral teleoperation system formed by a single mobile robot and dual-master is proposed:(1)Calibrate the gain parameters and map for the nondelayed trilateral teleoperation system considering and .(2)Set depending on some training process.(3)According to (), () and considering the maximum magnitude (estimated or predicted based on previous connections) of all time delays added by the trilateral communication channel (), the damping level is calibrated to hold Theorem 9.
5. Experimental Results
In this section, the proposed control scheme is tested. Two human operators teleoperate a mobile robot, receiving visual and force feedback. Both master devices employed are a low-cost manipulator, Novint Falcon model http://www.novint.com/ which has 3DOF and force feedback. Two DOFs of each master device are used to generate velocity commands for the mobile robot: one for the linear velocity and the other to set the angular velocity reference. To implement the teleoperation system, the following tools are used: MATLAB/Simulink from https://www.mathworks.com/ running with the real-time module and the SAS library from https://drive.google.com/drive/folders/0B2jklwyyOJqPNVA2SWFSaGFSNnc?usp=sharing compatible with V-REP environment (http://www.coppeliarobotics.com/). The methodology used allows easily doing tests of a delayed trilateral teleoperation system using a simulated robot but considering the two human operators inside the system, which commonly is called human-in-the-loop simulations, as it is shown in Figure 3. The communication between the master devices, Simulink, and V-REP is made through shared memory.
For the test, all time delays are configured randomly with values from 0.35 seconds to 0.65 seconds while the control parameters are calibrated following the procedure described in Remark 12, obtaining the following setting:
The objective of the experiment is to navigate avoiding the walls for reaching the yellow cube. Then the yellow cube must be pushed from its initial position to the green painted floor, as it is shown in Figure 4. Both operators feel not only the motion exerted by the other operator but also the dynamics of the mobile robot and the box weight.
Figure 5 shows the evolution on the remote site of each component (linear and angular velocity references) of the references and as well as the mobile robot velocity (where is the linear velocity and is the angular velocity). It can be seen that the mobile robot velocity is closer to master 1 reference, due to and therefore from (8) we get . Last value establishes the priority level over the mobile robot. Figure 6 shows an image sequence captured from the test, in which it is possible to observe that the task is done adequately in spite of the delays added by the trilateral communication channel.
Finally, in Figures 7 and 8, the main signals of the system about its stability are shown, such that it was proven in the theoretical analysis (Theorem 9 and Remarks 10 and 11), the synchronism errors between: master 1 and master 2 , master 1 and mobile robot , and master 2 and mobile robot , as well as the mobile robot acceleration , master 1 velocity , and master 2 velocity , remain bounded throughout the test. It is important to point out that about 53 seconds (subplot named T5 on Figure 5) the mobile robot, driven by the two human operators, starts to push the box. In the error signals and (Figure 7) as well as the mobile robot acceleration (Figure 6), it is possible to appreciate such action. That is, two human operators both collaboratively and simultaneously drive a mobile robot, fulfilling the required task on the remote environment.
Discussion. The scheme control developed is addressed to keep the motion synchronism errors bounded, that is, , , and . On the other, the controller is not designed to achieve force tracking but the control actions are bounded (Remark 11) and therefore the force errors given by , , and . On the other hand, the human operator performance generally is quantified with indirect measures such as time to complete the task and position errors of the mobile robot with respect to a goal path (navigation task). These indexes could be used to allocate a training process where the expert (trainer) gradually will have less control ( and are decreased) as the performance of the trainee will be better.
6. Conclusions
In this paper, the stability of a delayed trilateral teleoperation system of a single mobile robot-dual manipulator-like master is carried out. The kinematics and dynamics mismatch between the mobile robot and the type-manipulator masters makes the analysis presented different with respect to the one applied for the case where all robots are manipulator robots. To get a result about stability, a new Lyapunov-Krasovskii functional has been proposed, which is analyzed along the system trajectories. The result of the analysis exposed is formalized through Theorem 9, which establishes sufficient conditions to hold bounded the position and velocity of each master, velocity and acceleration of the mobile robot, motion synchronism errors, and all the control actions of the delayed trilateral system. Besides, a procedure suitable in practice for the parameters calibration is proposed considering the six values of time delays present on the trilateral system and some training process. A common training methodology requires that the trainer gradually sets the convenient level of depending on the learning stage of the trainee, which could be evaluated from indexes such as time to complete the task. Furthermore, a simple test about trilateral teleoperation using a 3D simulator is shown, whose result agrees with the result achieved from the theoretical analysis proposed. Finally, it is important to remark that the main current application field is the remote training of operators to drive different types of mobile machines or robotic systems such as collaborative mobile manipulators.
Conflicts of Interest
The authors declare that they have no conflicts of interest.