Abstract
This paper proposes a control scheme applied to the delayed bilateral teleoperation of wheeled robots with force feedback, considering the performance of the operator’s command execution. In addition, the stability of the system is analyzed taking into account the dynamic model of the master as well as the remote mobile robot under asymmetric and time-varying delays of the communication channel. Besides, the performance of the teleoperation system, where a human operator drives a 3D simulator of a wheeled dynamic robot, is evaluated. In addition, we present an experiment where a robot Pioneer is teleoperated, based on the system architecture proposed.
1. Introduction
Robot teleoperation allows the execution of different tasks in remote environments including possibly dangerous and harmful jobs for the human operator [1]. In the teleoperation systems of robots with force feedback, a user completes some task and physically interacts with the environment through a master-slave system. Thus, several control schemes and strategies for the teleoperation of mobile robots have been developed in order to solve tasks such as land surveying in inaccessible or remote sites, transportation and storage of hazardous material, inspection of high-voltage power lines, deactivation of explosive devices, high-risk fire control, pesticide and fertilizer crop spraying and dusting, mining exploration, and various other tasks [2–8]. Additionally, the use of Internet as a communication channel increases the application of the teleoperation systems. However, it is known that the presence of time delay(s) can induce instability or poor performance in a delayed teleoperation system [9–11] as well as a poor transparency [12].
Some strategies involve compensation based on a human operator model [13] in which only visual feedback is considered, ordinary structures such as control based on impedance [14], event-based control [15], signals fusion [16], and others based on kinematic models like [17–20]. Finally, [21, 22] consider a dynamic model and analyze the r-passivity of the system, while [23] presents a stability analysis of teleoperation systems using Linear Matrix Inequalities (LMIs) approach based on Lyapunov-Krasovskii functionals.
In addition, researchers have considered human factors in the design or evaluation of the teleoperated systems as it is shown in [24], where about 150 papers are described, covering human performance issues and mitigation solutions for teleoperated systems. However [25] emphasizes that human factor (HF) potential is underexploited and [26] expresses that, despite advances in autonomy, there will always be a need for human involvement in vehicle teleoperation. In addition, [27] states that many serious accidents, in which human error has been involved, can be attributed to faulty operator decision making. In this context, an operator can correctly select the actions but executes them poorly by overcorrecting, losing control, or flipping the commands.
This work presents a novel method to assist operators of a remote mobile robot in driving task. The method consists of weighting the operator commands by the risk associated to the current command activating also some safety action in the remote robot but only if the operator command increases the risk of collision. A control scheme based on impedance [28] is proposed to avoid obstacles, and the gain is variable according to the risk-based metric of the user commands (which is also presented in this work). The benefits of this scheme were analyzed in a previous work presented by the authors [29], where the channel delay was not considered. Besides, the stability of the system is analyzed considering the dynamics of master and slave robots as well as time-varying delays. Furthermore, the controller is tested from experiences over a virtual environment (Human in the Loop tests), in which a user drives a mobile robot teleoperated in presence of time delay, avoiding obstacles and feeling the force executed by the robot. In addition, an experiment of driving a real robot through a local network is presented.
This work is organized as follows. First, in Section 2, the statement of the problem is presented, while in Section 3 the control scheme and a method for a dynamic risk-based command metric are exposed. Then, in Section 4, the stability of the system is analyzed. Then, an explanation of the implementation of the method developed is given and experiments are shown in Section 5. Finally, conclusions are exposed (Section 7).
2. Preliminary
This paper proposes and analyses teleoperation systems in which a human operator drives a wheeled robot while he perceives the environment near the robot through visual feedback and a force feedback related to the force executed by the mobile robot. In addition, the scheme computes the command performance in order to help the controller, as it is shown in Figure 1.
The master or local device is represented by the typical modelwhere is the joint position of the master (in this work , ); is the joint velocity (in this work ); is the inertia matrix; is the matrix representing centripetal and Coriolis torques; is the gravitational torque; is the torque caused by the human operator force; and is the control torque applied to the master.
For the case of teleoperation of wheeled robots, the dynamic model of a unicycle-type mobile robot is considered [21]. It has two independently actuated rear wheels and is represented bywhere is the robot velocity vector with and representing the linear and angular velocity of the mobile robot, is the force caused by the elements of the environment on the robot, is the inertia matrix, and is the Coriolis matrix, where is the mass of the robot, is the rotational inertia, and is the distance between the mass center and the geometric center. In addition, involves a control force and a control torque , with and where is the radius of the wheels, is the half-width of the cart, and and are the torques of the left and right rear wheels, respectively. Furthermore, the communication channel adds a forward time delay and a backward time delay . Generally, these delays are time-varying and different between them (asymmetric delays).
On the other hand, the following ordinary properties, assumptions, and lemmas will be used in this paper.
Property 1. The inertia matrices and are symmetric positive definite.
Property 2. The matrix is skew-symmetric.
Assumption 1. The time delays and are bounded. Therefore, there exist positive scalars and such that and for all .
Lemma 2 (see [30]). For the real vector functions and and a time-varying scalar with , the following inequality holds:where is a positive definite matrix.
3. Control Scheme Based on Human’s Commands
The scheme proposed includes a PD-like velocity control, feedback of the force executed by the mobile robot, and an impedance loop to avoid obstacles depending on the performance of the delayed user’s actions. The control structure is well known in teleoperation systems of manipulator robots [30, 31].
Considering the control actions, we have for the local siteand for the remote sitewhere the controller is formed by (torque applied to the master), (torque applied to the robot), and (velocity impedance). The parameters and are positive constant (diagonal matrices) and they represent the proportional gain and damping added by a remote velocity controller; is the damping injected in the master, represents the scaling of force applied to the master, relative to the real force exerted by the mobile robot, converts position to a velocity command compatible with the restrictions about maximum speed of the mobile robot (all of them are also diagonal matrices), is a fictitious force, and is defined bywith . Signal represents the mobile robot acceleration at an infinitesimal time instant before so it is assumed considering ż without discontinuities.
In addition, is a parameter based on the risk-based command metric and is a user impedance tuning parameter.
3.1. Impedance Subsystem
In the control loop of the teleoperated robot, an impedance control law is applied based on a fictitious force, which depends on the nearest of the obstacles around the robot, in order to prevent collisions [32]. Despite operator commands, the robot automatically reduces its mobility when some object is near it. Therefore, the reference commands sent by the human could differ from the real actions executed by the controller when the impedance is active.
Figure 2 shows a graphics representation of the fictitious force vector, generated from the robot-obstacle interaction. is the distance between the robot and the obstacle, and a minimum and a maximum distance are set, in order to restrict the values of the fictitious force vector. It is decomposed in two components that affect the linear and angular speed impedance signal. Equation for computing the vector mentioned is described in [33], where , being , with and as positive constant such that and . Then, and , where is the angle between and .
The impedance signal () is calculated like (5), where speed references will keep the sense and direction of the user commands.
Variation of is proportional to the risk-based command metric proposed (). That is, .
3.2. Proposed Definition of the Command Metric
In this section, we propose a performance index to quantify the Operator command execution in front of risk. In order to define , the following terms are established.
(i) Real Risk (; ). It is the level of risk produced by each obstacle near to the robot, in the current time.
(ii) Command Risk (; ). It is the level of risk that would appear if the robot controller directly applies the user commands (considering ), in the current time.
(iii) Graduation of Command’s Performance (; ). It is the level of improvement or worsening of Real risk as an effect of the Command risk, taking into account the ideal commands that are possible to be applied.
We differentiate when the effect of the operator’s command (, command risk) potentially improves, preservers, or worsens the level of risk . For example, if the current is valuated in and , it means that the command applied by the user would decrease the real risk.
Then, in order to valuate the user skill required for the current situation, the more the high-risk level is, the more strongly should the metric weigh the improvement or worsening. For example, for the same numeric difference between and or and at a time instant , should throw a higher value for the first case due to the improvement demanding wiser and quicker commands.
Therefore, in order to quantify as a function of and , a surface is proposed and showed in Figure 3. is bounded between −1 and 1 and includes 3 ranges: preservation, improvements, and worsening. Figure 3(b) shows the nonlinear quantification of the signal. Figure 3(a) pictures the contour of the surface. Green line corresponds to .
(a) Contour of the surface
(b) Surface of the , , and
Now, we define as “The degree of goodness of the command that the user executes before risk situations.”
is computed as the Distance between the Ideal Action and the Real Action, where the Ideal Action () is the best possible user’s command. That is,where with calculated like but considering andwith calculated as but taking into account .
Besides, corresponds to the identification subindex for each obstacle and the variable is the amount of obstacles in the current time .
Figure 4 shows the vectorial model proposed in order to compute the metric definition (), considering only two obstacles for simplicity. A value of implies a perfect operator decision making in command execution, and limits the worst decision for the present obstacles. is the worst value of the range Preservation. The upper limit of is nonlinear and it is altered by the increment of obstacles in the environment.
3.3. Implementation of the Metric in This Work
A function calculates the collision probability based on the robot and obstacle position information, while signal is calculated from the robot’s kinematic model and the direct operator’s commands. Once the calculation of and is made, is computed through a neural network.
In order to create many points to train the net, a four-plane surface is defined as it is shown in Figure 5. The net consists in one hidden layer with 10 neurons, and a linear function output layer. Figure 6 shows the approximation reached with the neural network used. After computing for each obstacle, is calculated as in (7).
4. System Stability Analysis
It is important to highlight the common states variables that should be analyzed in a teleoperation system. They are the master-slave tracking error, the master velocity ( should be bounded or tend to ), and the acceleration of the master ( should tend to ).
The stability analysis of the scheme proposed is based on the theory of Lyapunov-Krasovskii. Therefore, a positive definite functional is proposed. It is important to remark that there is not an equilibrium point but a Krasovskii-like equilibrium solution that depends on the state in the time interval [34, 35]. The functional is formed by four parts. The first, second, and third subfunctional are proposed in the following manner:
The time derivative of , along master dynamics (1) and taking into account Properties 1 and 2, is the following:
Now, if the control action (5) is included in (13), it yields
Next, along the dynamics of wheeled robot (2) is obtained as
Later, from including (6) into the derivative of (12), and considering (2) and (5), can be written in the following way:
It can be appreciated from (14) to (16) that there are delayed variables that make the stability analysis difficult. For solving the inconvenience of integral terms in the time derivative form of the Lyapunov’s candidate, is proposed (17) as a mathematic resource:where are positive definite matrices.
Considering the assumption of Preliminary section, can be formed as
On the other hand, each term of that includes a delayed variable can be conveniently joined using Lemma 2 (3) with one term of . Consider
In (22), the integral function is applied in a closed interval to (assumed without discontinuities); therefore the function has a maximum real value that bounds such function and then .
Next, joining terms from (15) and (16), the following inequality can be deduced:with being a positive definite matrix. Finally, can be built linking from (14) to (23) in the following way:where is the identity matrix and . The controller can be set to guarantee that the quadratic terms of (24) are negative definite and therefore the signals , , and will tend to a ball centered in the origin and whose size depends on the maximum magnitude of , , , and .
5. Experimental Test and Results
In our test, a human drives a mobile robot as quick as possible (Figure 7), from an initial point to a final one, in an environment with multiple fixed and mobile obstacles. Operator drives the robot by using a Novint joystick while he receives the image captured by the camera onboard the robot, in presence of time delays and . The accomplishment of the task was evaluated based on some performance metrics proposed in Table 1.
The experimental test consisted in two parts ( and ). In the first one, many participant were asked to drive over a simulated robot (running under Matlab-Simulink), where also a simulated channel time delay was considered. Simulated environment and channel allows performing the same task under similar conditions, which makes possible a comparison between differences in human aspects. The second test was made using a real robot Pioneer through a local network.
5.1. Experiment
For in this work, we simulate three levels of channel time delays:(1)very low (),(2)medium (; ),(3)high (; , quite big considering the robot dynamic.
The parameters used for the mobile robot are kg, kg, and .
In the stability analysis of the proposed scheme, parameters , , , , and were considered as scalar due to simplicity, but the analysis and results are also valid for positive definite diagonal matrices. Therefore, for the PD-like controller in the local site, the parameters set areand, for the controller of the master (local side),
Finally, in order to transform the position of the master to a speed reference, is established as follows:
The range of the master positions in both axes is from to ; therefore, the maximum speed linear and angular references are and . The parameter is set to 0.8.
In order to analyze the control scheme proposed against classical methods based on velocity impedance or other methods that do not incorporate an impedance law, different are used: a constant high value (maximum), null, and variable value (from to a maximum, depending on the metric proposed (). In Table 1, cases are for null velocity impedance , cases for constant velocity impedance (classical method based on impedance), and cases for impedance varying according to the metric .
About the fictitious force for the impedance subsystem, the parameters taken are and ; therefore and (see Section 3.1).
Different drivers were taken to perform the required task (age range from 20 to 60, students and workers from the Universidad Nacional de San Juan).
Table 1 resumes the average values of evaluation index for the system (computed for experiment , 10 trials for each case row) in order to test the control system proposed and compare different cases listed above. represents the percentage of experiments in which the robot crashes against any obstacle. In this case the experiment is stopped. Next, measures the time to complete the task.
Then, HCP is an index about high collision probability. On one hand, the main task is required to be done as quickly as possible (evaluated by ), while, on the other hand, obstacles over the path represent restrictions to the movement of the robot. Hence, this index is about the risk the robot faces. We describe this evaluation metric aswhere is the quantity of obstacles, is the real risk , is the time of the experiment, and is the collisions probability of the robot against obstacle . Designers can use any method for computing the collision probability. We use a method based on a linear integration of the velocities (velocity in and coordinates of the plane) of the robot and obstacle. The cubic function of the collision probability is chosen, due to the fact that it enhances high values of risk (high collision probabilities attempt mission’s safety) and reduces the low values (not important for evaluating risk). Standard deviation is also computed for each case ( and ).
In order to enhance the understanding of the readers, Figure 8 introduces a graphic of the index of the table.
(a)
(b)
(c)
A short part of one trial of experiment was extracted and the main signals were plotted. Figure 9 shows the trajectories followed by the robot and the obstacles, where the darkness of each point represents the experiment’s time for each sample (more dark is addressed to a time increasing). The separation of the points gives an idea of the speed reached by the robot and obstacles. One obstacle moves around a point and the human must take the short path through the other two fixed obstacles, trying to keep a low risk. The situation is present seven times along the path between the initial and final point, in order to stimulate the performance of the user before risk.
The torque applied to the master for both speed references (linear and angular) is shown in Figure 10.
(a)
(b)
The impedance reduces the intensity of the commands applied in some moments (like at time s), depending on the value of the metric proposed (Figure 13). As a result of this, the robot dynamics and the channel delays, velocities (linear and angular) reached by the robot, are different with respect to the human’s commands references (Figure 11).
(a)
(b)
Figure 12 shows that is bound to all time. Figure 13 exposes the real and command collision probabilities of the robot against each obstacle (), as well as the computation of the command metric proposed. It can be observed for that the metric is high because the command tends to worsen the collision probability situation against obstacle 1.
(a)
(b)
Additionally, a cost function to minimize was introduced in Table 1 in order to summarize the information of the index introduced. For each row of the table, was computed aswhere variables in bold represent the normalized value of the index (e.g., ), and the weight gains were chosen according to the relevance of each index.
Next, in order to achieve a direct performance comparison between schemes, cost functions of each one ( is the first row of , corresponding to the cost functional of the scheme under low delays) can be averaged as in
It can be noted that scheme (which represents the main proposal of this work) is the one that achieves the minimum cost function.
5.2. Experiment
In order to test the system in a real robot, a Pioneer was teleoperated through a Local Network, based on the video feedback. The video of the experiment is available in Supplementary Material available online at http://dx.doi.org/10.1155/2015/460476.
The channel delay was approximately about 300 ms and despite being lower than the one used in the experiments over a simulated robot, the visual feedback is affected by video limitations like field of view and scope and quality and size of the image. The video was transmitted using the free software YawCam (http://www.yawcam.com). In Figure 14, a resume of the local and remote sides is shown. The robot must go from the starting point to the final one, going between the two obstacles present in the environment.
The results obtained are in agreement with the results achieved with the 3D simulator.
6. Discussions
6.1. Fix Velocity Impedance (Cases , , Classical Method)
When the remote control applies a fixed gain-impedance, despite good or wrong operator commands, the robot moves very slowly when he navigates in the vicinity of the obstacle. The navigation of the robot is always safe for any kind of commands, but it offers bad performance in path and time to complete task for all occasions (about 35% more than the cases with the same time delay and using the metric; see of Table 1). In this case, the Transparency is unsatisfactory because the user’s command is highly modified by the controller, and the operator perceives that the robot does not respond to his orders. This impedance law does not consider human driver performance. The increment in the time delay does not affect significantly the (less than ). The indexes of collisions probabilities measurements are nice, concluding that a safe robot navigation can be assured by using a fix impedance but with a bad (time to complete the task), regardless of operator performance. Therefore, good commands are wasted by the local controller.
6.2. Null Velocity Impedance (Cases , )
Here, the safety is low and only depends on the delayed operator’s command performance. When bad commands occur, a high risk situation appears, and collisions could happen. This was demonstrated by the high percentage of collisions as a result of the bad commands. All participants express that they felt a good sensation when commanding the robot which exactly follows the joystick (nice Transparency) for low delays but, when time delay rises, the driving becomes quite difficult.
Some good drivers achieve good executions, but this condition is difficult to maintain during a long task, or when secondary demanding tasks appear, which overload the attention or cognition of the driver, and therefore, poor decision and execution could be taken.
6.3. Variable Velocity Impedance (Cases , )
In this case, conditions of safety driving do not depend strongly on commands performance. If bad commands occur, takes high values and the risk situation is bounded. Thus, the collisions are mitigated at a cost of losing transparency.
The impedance reduces the received commands only if the potential risk worsens due to the user executions. is lower with this scheme than in the cases of scheme , showing a safer navigation. Furthermore, standard deviation of is approximately the same as the deviation for cases , indicating a more homogeneous driving despite the performance of the commands executed. In addition, despite the time delay increment, standard deviation of each index of cases remains similar, which proves that the homogeneous performance in front of delays increases.
Time to complete the task depends somewhat on the commands performance, but it is smaller than case and closer to the best one reached.
7. Conclusions
The work has presented a bilateral teleoperation controller for wheeled robots under time delays, in which the master controller is a PD-like scheme with gravity compensation and the slave controller is a scheme that incorporates an impedance compensation based on fictitious force and a variable gain, employed for obstacle avoidance. The variable gain is a function of the human operator performance when facing risk situations. The metric gives an online value that qualifies the command performance under risk situations, taking into account multiple obstacles environment.
With multiple experiments (Human in the Loop simulations and an experiment with a real robot), it has been proved that the proposed scheme has a better trade-off between safety, transparency, and time to completion, without a hard dependence on the operators expertise and differences in time delays. It also has been demonstrated that the system is stable even when incorporating the variable impedance compensation, based on a Lyapunov analysis, which also gives a guideline to the calibration of the control parameters.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work was supported by the Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET), the National University of San Juan, and the Automatic Institute of the National University of San Juan.
Supplementary Materials
The video presents a free-collision teleoperation of a mobile robot through a Local Network supported on a smartphone. The operator has a poor visual feedback from a webcam mounted on the robot. The obstacles are fixed and the operation should be completed as quickly as possible.