Abstract
This paper proposes the use of simple proportional plus damping injection (P+d) controllers for delayed bilateral teleoperation of a rotorcraft UAV. The proposed control scheme involves P+d remote and local controllers, considers master and slave dynamics, and takes into account asymmetric time-varying delays. The stability of the proposed teleoperation system is analyzed using Lyapunov-Krasovskii functionals and delay-dependent stability criteria are obtained under linear-matrix-inequalities conditions. The performance of the teleoperation scheme is tested driving a virtual nonlinear 6DOF dynamic model of a minihelicopter in a human-in-the-loop simulation.
1. Introduction
Over the last two decades, unmanned aerial vehicles (UAVs) have become an important scientific research topic. The reason for this is based on three main aspects: UAVs avoid the human risk inherent to manned aerial vehicles in many situations (e.g., hostile environment and adverse weather condition); they are generally far less expensive and smaller than their manned counterparts; and they can reach places and/or overfly specific areas that other vehicles cannot.
Within UAVs, rotorcraft-based UAVs (RUAVs) have unique flight capabilities such as hover, vertical take-off and landing, and side-slip, which cannot be achieved by conventional fixed-wing aircraft. The hovering mode and flying at low speed make RUAVs ideal for reconnaissance missions and very useful in rescue missions [1]. These singular features called the attention of many academic research groups in the last years. Although there has been much research in the area of RUAVs, the majority of published works have covered the design and implementation of control systems for autonomous navigation only (see [2] and references therein). However, a lot of inspection, surveillance, and monitoring tasks cannot be easily performed with a fully autonomous vehicle; the need for high-level reasoning and specific knowledge inherent in the task make RUAV teleoperation more suitable for these missions [3].
In general, UAV teleoperation systems have three elements: a ground station (local site), where a human operator receives information and drives a hand-controller device (master device); a UAV (slave vehicle or device) that receives control signals and flies through an environment (remote site); and a communication channel that links both sites. The master device at the local site and the slave vehicle at the remote site are endowed with local controllers as seen in Figure 1. The information provided in the local site is usually visual and the operator lacks other sensory information that a pilot of a manned aircraft would normally perceive, such as sound, vibration, and motion [4]. Additionally, the communication channel adds delays in the range of 100 to 1670 milliseconds (and even more) [5] that can produce instability or poor performance [6, 7].
In recent years, several authors have developed teleoperation control systems for RUAVs; even so, only a few have focused on delayed bilateral teleoperation schemes. Bilateral teleoperation provides the human operator with a haptic feedback; this signal can complement the visual information to improve the telepresence and enhance safety; as a drawback, it can destabilize the closed-loop teleoperation system [8, 9].
In [10] the influence of different force-feedback settings on a human operator is studied and the wave variables conventional teleoperation technique [11, 12] is used to reduce the negative effects of time delays. In [13] a bilateral teleoperation of multiple mobile slave agents coupled to a single master robot under constant communication delays is achieved using a passifying PD-based controller; nevertheless, it addresses only the master-slave position-position teleoperation and thus it is not suitable for UAVs with unbounded workspace. In [14–17] a bilateral teleoperation of multiple UAVs is performed by a single human operator and the stability is analyzed employing passivity theory. In the first two researches, the passivity of the master side is obtained applying feedback r-passivity notion [18]; then, it is claimed that the passivity of the teleoperation system can be easily enforced using any of the techniques developed in conventional teleoperation settings. In the latter two investigations, passive set-position modulation (PSPM) framework [19] is utilized to theoretically guarantee master-passivity/VPs-stability of the closed-loop teleoperation system (where VPs refer to the virtual points followed by each UAV). In [20] the port-based modeling network, in particular, port-Hamiltonian systems and bond graphs, is used in the bilateral teleoperation of aerial robots, where a variable impedance master controller and a virtual slave system allow the design of a passive teleoperation control architecture.
Contribution. In bilateral teleoperation of manipulator robots, proportional plus damping injection (P+d) controllers have gained momentum in the last few years [21–23]. Due to its simplicity and robustness, this work proposes the use of P+d controllers for bilateral teleoperation of a RUAV with time-varying delay considering the master device and slave vehicle dynamics. The main contribution of the paper involves combining in a simple way P+d controllers, a trajectory tracking controller, an impedance control based on fictitious force, an elastic coupling, and a cascade PID architecture, to achieve a stable bilateral teleoperation of an underactuated 6DOF minihelicopter in presence of time-varying delay. The stability analysis carried out gives as a result a delay-dependent LMI condition, which can be held if the control parameters are set from such condition, assuring thus a stable performance. In the state of the art in the field of delayed bilateral teleoperation of aerial vehicles, most controllers are based on passivity theory and they generally include a difficult calibration procedure. Different from this, the proposed scheme involves a simple parameters setting where the control blocks (P+d control, trajectory tracking controller, impedance control, etc.) can be set independently in practice.
To test the performance of the teleoperation scheme a human-in-the-loop (HITL) simulation is carried out; there, a human operator drives a RUAV through an outdoor environment and has visual and force feedback under a communication channel with asymmetric time-varying delay. Two master devices are used increasing the maneuverability of the RUAV and the visual feedback is only generated by an onboard camera providing a field of view like the one achievable in practice.
The rest of the paper is organized as follows. Sections 2 and 3 introduce the models of the master device and slave vehicle, respectively. Section 4 presents some mathematical properties and assumptions. The P+d controllers are proposed and explained in Section 5. In Section 6 the stability analysis is outlined. The complete teleoperation system is presented in Section 7. The HITL simulation framework and the simulation results are provided in Section 8. Finally, the conclusions are given in Section 9.
2. Master Device
The master device at the local site is composed of two parts: the translational master and the rotational master. The first is a 3-degree-of-freedom (DOF) device with force feedback which controls the translation rates of the UAV and the second is a 1DOF device (a two-pedal configuration) which commands the rotation rate of the UAV (see Figure 2).
2.1. Translational Master Device
The typical nonlinear dynamic model in Cartesian coordinates is used to represent the 3DOF device: where is the end-effector position, is the inertia matrix, is the matrix representing centripetal and Coriolis effects, is the gravitational force, and and are the control and human operator forces applied to the translational master, respectively.
The gravitational force is precompensated by the controller ; that is, . The final model for the translational master is the following:
2.2. Rotational Master Device
The 1DOF device is modeled as two forced mass-spring-damper systems:where is a vector indicating the joint angular displacement of the right and left pedals; , , and are the pedals inertia, viscosity, and stiffness, respectively; and is a vector that indicates the human operator’s right and left feet torques.
The output signal of the rotational master is the difference between the joint angular displacements of the pedals; that is, .
3. Slave Vehicle
RUAVs belong to the class of underactuated mechanical systems, which have fewer control inputs than degrees of freedom. The 6DOF dynamic model of a RUAV is very complex, and it is certainly difficult to obtain the mathematical equations and parameters that follow its physical model [24]. One practical solution to this problem could be to use a more data-driven/model-free design framework of the control scheme (see [25] and references therein). Another solution, applied in this work, is to use a simplified fully actuated 4DOF model in the controller design stage and then couple it to a full RUAV model. In Section 8, the interaction between the 4DOF model and a more realistic underactuated 6DOF RUAV model of a minihelicopter is explained.
In Figure 3, the reference point on the surface of the earth is the origin of a right handed orthogonal system of axes (), points to the north, points to the east, and points vertically “down” along the gravity vector (NED Cartesian system). The kinematics and dynamics of the 4DOF vehicle are equivalent to the ones of a rigid body whose roll and pitch movements are constrained; hence, its body axes () are tantamount to a Cartesian coordinate system “tied” to the earth by means of gravity vector with the difference being that the origin coincides with the center of gravity of the aircraft and axis points forward along the vehicle.
The fully actuated 4DOF dynamic model is the following:where is the RUAV velocity vector with , , and representing the forward-backward, lateral, and downward-upward linear velocity, respectively, and is the angular velocity as seen in Figure 3. is the force/torque caused by the interaction between the environment and the RUAV and is the control force/torque applied to the RUAV.
The inertia matrix , the Coriolis matrix , and the gravity vector are the following: where , , and are positive constants denoting the mass, the rotational inertia, and the gravitational acceleration of the RUAV, respectively.
To devise the translational and rotational control actions independently the RUAV dynamic model is decoupled. For this purpose, the control force/torque in (4) is established as follows:
Including (6) in (4) we obtain , being the translational slave dynamicsand the rotational slave dynamicswhere , , and .
4. Preliminaries
The following properties, assumptions, and lemma will be used in this paper.
Property 1. The inertia matrix is a symmetric positive-definite function and there exist positive constants and such that .
Property 2. The matrix is skew symmetric.
Property 3. There exists a positive constant such that .
Assumption 1. The human operator and the environment behave in a nonpassive way and they are represented by the following models:where , , and with are the inertia, damping, and exogenous force of the human operator and environment, respectively. The exogenous forces are subject to , and is a positive constant.
Note. Models (9) and (10) are modified versions of the second-order LTI models successfully used by other researchers [26–28]. In (9) and (10), the stiffness of the human operator and environment is ignored considering the operator does not want to maintain a certain hand position and the vehicle makes no physical contact with the environment. Also, and in (9) and (10) were taken as scalars but they can be diagonal matrices or, more generally, positive-definite matrices.
Assumption 2. The communication channel is represented by a forward time-varying delay (local site remote site) and a backward time-varying delay (remote site local site). These time delays are considered bounded; that is, there exist positive scalars and such that and for all .
Lemma 3 (see [22]). For vector functions and , a positive-definite matrix and a time-varying scalar with , the following inequality holds:
5. P+d Controllers for Bilateral Teleoperation
The teleoperation system has two controllers (Figure 1) that interact with the master device and consequently the human operator on the one hand and with the slave vehicle and its environment on the other hand. Both controllers have a simple P+d structure; thus they possess few parameters that can be quickly calibrated.
5.1. Remote Controller
The aim of the remote controller is to synchronize the slave vehicle’s linear and angular velocities () with the position and angular displacement of the master device, respectively. The control actions are the following:where , , , and are positive constant parameters representing the proportional gain and acceleration-dependent damping added by the P+d controllers. and are positive constant parameters that map master position and angular displacement to velocity commands and is an auxiliary vector defined aswhere and . That is, represents the RUAV acceleration at an infinitesimal time instant before .
The end-effector position vector of the translational master and the angular displacement of the rotational master generate linear and angular velocity setpoints for the remote controller. This relation between master device and slave vehicle helps overcome the master-slave kinematic dissimilarity typical of mobile robot teleoperation; that is, the master device has bounded workspace and the slave vehicle has unbounded workspace.
5.2. Local Controller
The objective of the local controller is to provide the human operator with a haptic cue that represents the mismatch between the commanded velocity (specified by the end-effector position vector) and the velocity of the slave. The mismatch gives different kind of information: dynamics of the slave vehicle, external disturbances on the RUAV motion, and quality of velocity tracking in the remote site. The local controller only affects the translational master device as the rotational one is unactuated. The control action is the following:where and are positive constant parameters representing a relative spring depending on the mismatch between the master reference and the RUAV linear velocity and damping added by the P+d controller, respectively. maps slave vehicle’s velocity to master position.
6. Stability of the Bilateral Teleoperation System
The rotational part of the teleoperation system is unilateral; thus the complexity of its control is much reduced; therefore, only the stability of the translational part is analyzed using the theory of Lyapunov-Krasovskii [29]. It is important to remark that there is not an equilibrium point but a Krasovskii-like equilibrium solution that depends on the state in the time interval .
Theorem 4. Consider a delayed bilateral teleoperation system where a human operator (9) using a master device (2) drives a RUAV (7) interacting with an environment (10), and control laws (12), (13), and (15) are included. For positive constant parameters , , , , and , slave vehicle mass , and maximum time delays and , considering Assumptions 1 and 2 and Properties 1, 2, and 3, if there exist positive-definite matrices and such that the following LMI holds,where and represents the transpose of corresponding matrix, then .
Proof. First, a positive-definite functional is proposed in order to analyze its evolution along the system trajectories. The first four subfunctionals depend on and are defined in the following manner:The time derivative of along the master dynamics (2), taking into account Properties 1 and 2 and considering (9), is the following one: Now, if the control action (15) is included in (22), it yields Next, is obtained from (19) considering (14): On the other hand, is computed in a similar way from (20), taking into account (14):Moreover, can be written including (12) in (7) and considering (10) and (14), in the following way: It is possible to appreciate in (23) and (26) that there are terms like which make stability analysis difficult; to solve this, is proposed as follows: where and are positive-definite matrices.
From (27) and considering Assumption 2, can be computed by On the other hand, each term of (23) and (26) that includes a delayed variable can be conveniently joined using Lemma 3 with one term of . The second term of (23) and the second term of (28) can be linked, considering (14): In the same manner, the second term of (26) and the fourth term of (28) can be combined to establish the following: Finally, can be written associating (23)–(26) and (28)–(30), disregarding the terms with the infinitesimal and neglecting and , in the following way: where is the identity matrix.
LMI conditions (16) are equivalent to the following two inequalities:Given and maximum delays and slave vehicle mass , the controllers parameters (, , , , and ) can be selected according to LMI conditions (16) to ensure that the two inequalities (32) are satisfied; that is, the first two terms of (31) are negative definite and therefore the variables , , , and are bounded (). The proof is completed.
Remark 5. If the exogenous force of the human operator and environment are null (), then the system is stable. For this particular case, Barbalat’s lemma can be used in (31), taking into account Assumptions 1 and 2, Property 3, and the fact that (and ); it is possible to deduce that and are bounded; then, is bounded too. Finally, and will tend to zero as .
7. Complete Bilateral Teleoperation System
To test the performance of the teleoperation system a HITL simulation will be executed; there, a human operator will drive a very realistic 6DOF model of a minihelicopter (extracted from [24]) through an obstructed environment. The helicopter model considers nonideal dynamics, for example, flapping, drag, and actuators’ dynamic, and precisely describes the vehicle’s motion for both hovering and low-speed translational flight.
The minihelicopter needs to be outside the closed-loop bilateral teleoperation system for the stability analysis of previous section to remain valid; hence, a coupling between the 4DOF and 6DOF RUAVs is proposed. The complete bilateral teleoperation system can be observed in Figure 4.
The coupling controller has two main functions: allow the 6DOF RUAV to track the trajectory of the 4DOF RUAV and transmit the interactions between the minihelicopter and its environment to the 4DOF RUAV. In addition, obstacle avoidance is incorporated to the minihelicopter. Next, each part of the coupling controller is explained.
7.1. Trajectory Tracking Controller
Inspired by [30], the 4DOF RUAV is treated as a moving reference frame (axes ) whose trajectory is the reference for the helicopter frame (axes ) attached to the 6DOF as seen in Figure 5. The axes () of the frame are not the actual body axes of the minihelicopter but 4DOF RUAV-like axes. The kinematics of the helicopter frame is equal to the kinematics of the reference frame :being for the reference frame and for the helicopter frame.
Considering the RUAVs’ kinematics model (33), the trajectory tracking problem corresponds to the design of a control law that allows the helicopter frame to reach and automatically follow a path described by the reference frame according to the 4DOF RUAV velocities. The controller must generate the flight commands , , , and such thatwhere and .
First, the following errors are defined:
The states are errors , , , and expressed in the helicopter frame . The time derivative of is the following:being
Now, the following Lyapunov candidate is considered:
The time derivative along the system states (35) is given by
If the flight commands in (37) are selected as followswith , and then (37) is replaced in (39), the final expression for iswhich is a negative definite function. This means that system errors asymptotically converge to zero.
7.2. Impedance Controller
To add obstacle avoidance in the coupling controller the following correction to the flight commands (40) is made:
The fictitious force is calculated using a simulated 3D laser sensor which measures distance between the minihelicopter and the obstacles. The force is obtained as follows:, , and being the longitudinal, lateral, and vertical forces, respectively; , , and normalize forces to for and for and . indicates the magnitude of the force for each laser beam (), with such that and ). Angles and express the direction of each laser beam in the horizontal and vertical plane, respectively.
7.3. Velocity Controller
To let the minihelicopter track the flight commands (42), the velocity controller depicted in Figure 6 is incorporated. It has a velocity transformation matrix that maps the 6DOF actual body axes kinematics ( in Figure 5) to the helicopter frame and a cascade PID architecture to generate the servo signal inputs (see details in [31]).
7.4. Elastic Controller
The interaction between the helicopter and its environment is reflected to the human operator, improving the telepresence, thanks to the elastic controller; moreover, the position and orientation errors between RUAVs are minimized. To achieve both goals, the exogenous environment force and an exogenous environment torque acting on the 4DOF RUAV are considered to be the following: and being the position and orientation errors previously defined in (35). Equation (44) must satisfy to fulfill the stability analysis of Section 6.
8. HITL Simulation
The simulation environment has been developed in , where the human operator’s view simulates an onboard camera. The aim of the simulation was to reach eight successive target points (see Figure 7). The calculations of the whole teleoperation system (including RUAVs dynamics) and its connection with the master devices and are accomplished by a program made in C++.
The teleoperation system’s parameters are the following: (obtained using LMI toolbox for each of the four different cases of time delay as seen in Table 1), , , , , , , , , , , and . The 4DOF RUAV has mass = 7.4 [kg] and rotational inertia , and the 6DOF RUAV has the minihelicopter’s nominal parameters (which refer to MIT’s X-Cell .60).
8.1. Simulation Results
In Table 1 the time to complete the task (TCT) and mean square translational synchronization error (MSE) for four different time delays are presented. The values correspond to the average of five simulations for each case of time delay. When time delay increases, the TCT gets bigger, as expected. The behaviour of the MSE is much more complicated: when time delays are zero (case 1), all the MSE is due to the dynamics of the RUAV models and the coupling controller, specially the impedance controller because the obstacle avoidance directly modifies the flight commands; when time delay increases (cases 2, 3, and 4), the MSE gets bigger as the synchronism gets worse; however, when the local controller damping rises considerably because of time delay ( for case 4 so that condition (16) is satisfied), the movement of the master device is slowed down causing the MSE to decrease. The magnitude difference between , , and is promoted by the different velocity mappings , and the invariance in appears as a result of better low level PID control (improved immunity to desynchronization) and the fact that there is only one obstacle that seriously affects the vertical axis (see Figure 8).
(a)
(b)
Figures 8 and 9 present some results obtained in a case 4 HITL simulation ( and ). Figure 8 shows 3D and 2D views of the trajectory made by the minihelicopter in the simulation task, every 5 seconds the 6DOF RUAV is drawn. In Figure 9, it can be observed that all the teleoperation signals are bounded, and when the environment exogenous force decreases, the synchronization error and force feedback diminish too. It is necessary to highlight the complexity of the performed task, mainly due to the big asymmetric time-varying delay but also by the number of obstacles and the speed of the minihelicopter in the simulation.
9. Conclusions
In this paper, a bilateral teleoperation scheme for a RUAV has been proposed. The complete teleoperation system incorporates P+d controllers, a trajectory tracking control, an impedance control with obstacle avoidance, an elastic coupling controller, and a cascade PID control architecture. The stability of the system has been analyzed and delay-dependent stability criteria have been obtained under specific LMI conditions. The teleoperation scheme has performed well in the presence of different time-varying delays as shown in the HITL simulation results. The inclusion of simple controllers in the teleoperation system makes their parameters easy to set, allowing accurately calibrating how the human operator perceives the environment and dynamics of the RUAV through force feedback.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The authors would like to thank the AC3E, Universidad Técnica Federico Santa María, Grant FB0008, CONICYT, and the INAUT, Universidad Nacional de San Juan, CONICET, for their support.