Abstract

Aerial manipulation of objects has a number of advantages as it is not limited by the morphology of the terrain. One of the main problems of the aerial payload process is the lack of real-time prediction of the interaction between the gripper of the aerial robot and the payload. This paper introduces a digital twin (DT) approach based on impedance control of the aerial payload transmission process. The impedance control technique is implemented to develop the target impedance based on emerging the mass of the payload and the model of the gripper fingers. Tracking the position of the interactional point between the fingers of gripper and payload, inside the impedance control, is achieved using model predictive control (MPD) approach. The developed on-board DT offered a model where interaction with the unknown payload and aerial robot dynamics is informed. Beside this, the results showed the ability of the introduced DT to foretell the conditions of the forces acting on the payload which helped to predict the situation of aerial manipulation process. Additionally, the results showed that the DT model could detect real-time errors in the physical asset.

1. Introduction

Digital twin (DT) is a modern application that provides advanced interconnection between real systems and their corresponding virtual representation. Compared to digital model and digital shadow, the data linkage in DT can be transferred from the real system and its corresponding virtual representation in both directions [1]. Hence, DT can detect the abnormality of the system either as on-board or off-board manners. In the on-board diagnostics process, the operation of detecting the abnormality of the system is implemented by sensors and actuators which are equipped in the system itself. The DT compares the outputs of the sensors and actuators with the virtual model to diagnose any unexpected outcome. Regarding the off-board diagnostics process, the operation of detecting the abnormality is remotely implemented from the system by cloud [2]. The performance of DT to predict abnormality of systems in real time can be improved by applying new technologies such as three-dimensional laser scanners [3]. As presented in Table 1, DT is implemented for various purposes such as manufacturing processes, industries, smart cities, healthcare, constructions, and robotics [46].

DT is implemented to enhance the manufacturing process of products in terms of time and amount of wastage where a model setting for injection molding machine aids modelling cases of production steps [14]. Merging the DT with abnormality detecting in machines of industries showed its high ability to diagnose abnormality in rolling bearing [15]. In Industry 4.0 of modern drivers [16], DT has become the major technology which is implemented to integrate the data linkage between the real and virtual machines [17]. The application of artificial intelligence algorithms has a significant effect on the DT systems regarding autonomous driving, production workshops, and smart city traffic [18]. Recently, since 2018, DT systems have started to be integrated within the robotic platforms in many situations in order to enhance the performance of robotics in their specific applications. In [19], by Malik and Bilberg, the process of controlling the cooperation between the human and robot is improved by applying DT. The cooperation task of assembly is simulated by computer to provide a digital representation of the system where the digital representation continuously existed during the production process of the human-robot system. The production of the system has been improved by the real-time monitoring of the physical components. The situation of a manufacturing firm which has a team of human and robotics is introduced for establishing and approving the DT structure. In [20], by Joordens and Jamshidi, DT is applied to help the design process of swarm robotics which was not easy to be tested during setup and control such as robotic fish. In the beginning, the robotic fish was practically produced with the ability of primary swimming. A DT of the robotic fish is developed and transformed to virtual reality which provided the simulations to be implemented and fixed during robotic fish swimming. In [21], by Vassiliev et al., DT is applied to develop a new robotic system that can be used in mechanisms of walking robotics. The designed DT robotic platform showed its behaviour to enhance the control process and algorithms. In [22], by Sørensen et al., a special industrial assembly system is introduced where digitalized assembly orders are developed by an operator. The orders of the operators are visually developed in blocks which provide the feature of programming the robotic system visually. DT allowed us to monitor the execution of the system in real time. In [23], by Kaigom and Roßmann, a robotic DT is presented which assisted and improved the implementation of cyberphysics in robotics. The results showed that the developed robotic DT system can be applied in various situations. In [24], by Douthwaite et al., DT is developed to support the safety of cooperation between multirobotics in manufacturing plants. The introduced framework of the DT supported flexible performance of operator-identified cyberphysical circumferences and utilities for safety test and control. However, in aerial robotics, a remarkable proportion of applications are directed towards aerial payload transportations (APTs) [25]. Since the payload tasks are generally associated with handling objects of various unmeasurable physical properties, they need to be controlled by advanced techniques [26, 27]. With advanced control techniques [28], the term of the interactional control process has been integrated with the model of the robot platform from one side and the unknown payload from another side [29]. Hence, the APT processes are affected by many unknown parameters and disturbances. Unknown parameters represent the unknown weight, shape, size, and so on of the payloads. The disturbances represent the external effects during the flight of the aerial robotics that could disturb the transportation process such as wind. Consequently, it is essential to introduce an approach to compare the ideal physical parameters of the aerial robot system with the real physical parameters in real time for diagnostic purposes. In this paper, a physical digital twin (PDT) approach is developed to introduce a model for real-time diagnosis in order to foretell the conditions of the forces acting on the payload which helped to predict the situation of the ATP process. The impedance control technique is implemented to control the interaction process within the payload where the target impedance is based on emerging the mass of the payload and the model of the gripper fingers. Tracking the position of the interactional point between the fingers of gripper and payload, inside the impedance control, is achieved using model predictive control (MPD) approach.

The rest of the paper is organized as follows: Section 2 presents the interactional model of the aerial robot. Section 3 introduces the implemented impedance control technique to track the interactional force between the payload object and gripper. Section 4 explains the implemented MPD approach which is included to track the position inside the impedance control algorithm of Section 3. In section 5, the designed DT of the aerial robot is developed. In Section 6, simulation results of the developed DT model are introduced. In Section 7, the conclusions of this study are presented. In Section 8, recommendations of future work of this research are explained.

2. System Model

This section presents the interactional model of the aerial robot of this study. Generally, aerial robotics consists of UAV and robotic arms. The UAV has various mechanical configurations [30] of potential sensor systems [31]. The aerial robotic arm grasped payloads via a gripper which is attached at the end of the third link of the aerial arm. Interactional modelling of the payload grasping process is considered in this part. Assuming the gripper shown in Figure 1, it can grasp a specific payload by moving its fingers simultaneously via one actuator. The fingers are assumed to be designed based on the principle of 4-bar linkage mechanism where the rotational motion of the actuator is translated to linear movement.

The schematic gripper/payload model during situations of none-contact, instance-contact, and interactional-contact is shown in Figures 2(a)2(c), respectively.

It is assumed that a potential payload of mass is to be held by closing up the fingers of the gripper. The interactional force , , between the two fingers and the payload is induced once the gripper contacts the latter. Each finger is modelled as second-order system of mass, stiffness, and damping coefficient , , and , respectively. The coordinates of the spring and damper of each finger, at point 1, relative to the origin of the gripper and to the payload mass are denoted by and , respectively. The current position of each finger is . This position, i.e., , is the command of the gripper/payload system while is the force that is intended to be controlled. Referring to the schematic diagram of the interactional model shown in Figure 2(c), each finger exerts a force of equation (1) on the payload: while the force that the payload applies on the finger is presented in where . Regarding the aerial robot platform, it has UAV of type quadcopter while its aerial arm has serial three links of rotational joints as shown in Figure 3. The gripper, which represents the end effector, is attached to the end of the aerial arm. The position of the gripper is achieved by rotation of the propellers of the quadcopter and the joints of the aerial arm.

The schematic representation of the aerial robot is shown in Figure 4 where the subscripts , , and correspond to UAV quadcopter, aerial arm, and aerial robot, respectively. Assume that and are the position and orientation of the UAV quadcopter relative to the quadcopter frame , respectively. The orientation , where , , and represent the rotation angle around , , and axes, respectively. The position vector is while all the joints of the aerial arm are included in the coordinate vector . Consequently, the overall state of the aerial robot is represented in

Referring to Figure 4, the coordinate frames of the inertia and the end effector are denoted by and , respectively. The end effector represents the contact point position of the gripper fingers with the payload which can be calculated with respect to according to where , , and represent the position of the UAV quadcopter relative to , rotation of the UAV quadcopter relative to , and the position of the end effector relative to . On the other hand, both the linear and angular velocities of the end effector can be obtained from where and represent the angular and relative angular velocity of the UAV quadcopter in , respectively. Dynamics of the aerial robot is obtained by applying following Euler-Lagrange approach in where , explained in equation (8), represents the difference between the total kinetic and potential energies of the aerial robot and , respectively: while represents the total torque applied on the aerial robot. In more detail, the total torque includes three main parts: the thrust forces of the UAV quadcopter rotors , the forces applied by the rotors on the aerial robot body , and the forces generated by the aerial arm . The total kinetic energy of the aerial robot can be obtained from where the matrices and represent the mass and moment of inertia of the UAV quadcopter, respectively, the same for , and which represent the mass and the moment of inertia of the corresponding link of the aerial arm. Regarding the total potential energy, it is obtained from where . By inserting equations (9) and (10) into equation (8), the Lagrange function of the aerial robot is obtained which is applied in equation (7) to find the final dynamic model of the aerial robot in where the matrices , , and denotes the aerial robot inertia, Coriolis, and gravity, respectively. In the next section, impedance principle will be designed to control the interactional contact forces by moving the position of the finger .

3. Impedance Control

The control algorithm is implemented using impedance techniques as shown in Figure 5 to track the interactional force between the payload object and gripper. MPD approach is applied to track the position. The aim of this control algorithm is to provide the required impedance between the position of each finger and the interactional force.

The target impedance is dynamically controlled by the stiffness , damping coefficient , and mass . Hence, equation (12) represents the reference impedance where and are the reference displacement and interactional force, respectively.

In a potential application of impedance approach, a tracking position task as an effect of dynamics of an aerial robot manipulator is the result between the current and the target position of the finger. Hence, in term of the target position , the equation of trajectory tracking in equation (13) can be obtained from equation (12):

The error of position is determined from

In a steady-state situation, the tracking error of the interactional force is calculated in

In the case of reaching the required position via the MPC, the reference position in equation (16) can be calculated from equation (15):

In this case, the reference position can be generated from existing the values of the finger stiffness and payload location according to the target international contact force value. In the next section, the MPD approach is applied to track the position.

4. Model Predictive Control

In the presented MPD, the constrained aerial robot system is solved in optimization approach at each step of time to reduce the position tracking error in the impedance control algorithm of Section 3. The states of the aerial robot can be controlled by torque inputs . It is assumed that the solution of optimization is obtained at time , where is a positive number. The value of ranges from “1” to the steps of the estimation horizon “.” Consequently, the optimization algorithm of the MPC is written in

The input torque vector is the control inputs that cause the motion of the aerial robot which is represented as ; denotes the model of the aerial robot. The generic constraint in equation (17) is denoted by while the generic cost function is represented by the general form in where and denote the weights of the input control and cost of the terminal, respectively. The , which is equal to , represents the target cost function that wants to be minimized. This cost function, i.e., , is designed to execute the task of position control during interaction with the payload in

In equation (19), the error of interaction position between each finger and the payload is denoted by which is obtained from

Tracking the position error is affected by the desired location of the point of interaction between the finger and the payload over the estimated horizon. The task and terminal weigh matrices in equation (19) are denoted by and , respectively. Since the manoeuvres are not our aim of this model, the tilt of the aerial robot is not considered in the suggested MPC. Consequently, the control input in equation (21) is directly obtained from derivation equation (3):

Generating the control input in equation (21) aims to get zero . Thus, the function in equation (17) is only the integration of .

5. Physical Digital Twin

In this section, on-board PDT model is developed to detect a potential future payload. The detection of payload via PDT is mentioned as on-board predicting where the derived dynamic model and the developed control algorithm in the previous sections are implemented to mimic the process of payload grasping by the aerial manipulator. As shown in Figure 6, the PDT parts that are developed in this section includes predicting the interactional forces between the gripper and the payload, block diagram of the aerial robot, PDT modelling, and the algorithm of diagnosing the existing of payload.

The gripper of the aerial robot, as explained in Section 2, has one actuator to simultaneously control the movement of the two fingers. The interactional force is calculated according to equation (2). Block diagram of the gripper system is shown in Figure 7. The system has one input which represents the angle of servomotor of the gripper. The outputs of the system are the interactional forces explained in Section 3 which are represented in equation (12).

Detecting payload depends on the orientation of the aerial robot, furthermore the angle of the servomotor. The main idea depends on measuring the interactional forces. If the measured international forces deviate from the interactional forces of the PDT model, an error message will appear to indicate that there was a fault. The implemented diagnostic algorithm is shown in Figure 8.

The developed diagnostic algorithm takes a specific decision according to the results of comparing the measured interactional forces and the DT interactional forces. This will help in diagnosing faults that can be accorded during payload transportation.

6. Simulation Results

Simulation tests, using MATLAB, are implemented in this section to examine the performance of the developed DT model. The DT model is created using SimMechanics™ where the input and the outputs were the angle of the motor gripper and the interactional forces, respectively. The dynamics of the aerial robot system in equation (11), MPD in equation (17), are implemented in MATLAB script file programs. On the other hand, the gripper shown in Figure 1 is modeled using SimMechanics™. The impedance controller is integrated within the DT model to calculate the interactional forces between the fingers of the gripper and the payload. The DT model of the aerial robot shown in Figure 8 is assumed to grasp the payload by the appropriate forces by rotating the motor of the gripper. Due to the fact that the payload is not directly connected by joints with the aerial robot, in addition to the SimSpace™ blocks, the DT model included programming equations (1) and (2) which relate the motion of the gripper and the payload in MATLAB script files. The DT model calculates the interactional forces to monitor the process of grasping a potential payload. The comparison is implemented to diagnose the failure of supporting the required interactional forces. In the first test, the aerial robot physical asset response is examined against impulse function. The position and velocity of each finger of the gripper is shown in Figure 9. The first test showed the functionality of the developed model to represent aerial robot physical asset part in Figure 8. The velocity and the position of each finger of the gripper is obtained as a reaction for input impulse at time 1 sec.

Next, detecting errors in the aerial robot system using the developed DT model of this study is verified in the second test. The response of the system shown in Figure 10 when the fingers of the gripper touch the payload.

The results showed the ability of the developed DT model to diagnose problem. Hence, the process of aerial payload transforming can be monitored online, and any problem in the aerial robot platform can be immediately detected. The simulation failure results showed the ability of the developed DT model to diagnose error of international forces online. When the real model does not support the appropriate force, the DT model can estimate this, for instance, when the fingers could not support the appropriate contact forces due to the problem in its modelled stiffness and damping components. Hence, the failed aerial robot system is introduced into the DT model and simulate the reaction of the failed aerial robot system with the same motor of gripper input.

7. Conclusions

Aerial techniques are recently implemented in payload transportation instead of traditional ground platforms. Practical examples of aerial payload are generally implemented by aerial robotics. PDT supports monitoring in the real-time process of aerial payload transportation. Hence, faults can be immediately diagnosed during aerial robot flight. The required impedance between the fingers of the gripper can be obtained using an impedance controller based on the emerging mass of the payload and the model of the gripper fingers. In the impedance controller, MPD approach tracks the position of the interactional point between the fingers of gripper and payload. Tracking the position of the interactional point between the fingers of gripper and payload, inside the impedance control, is achieved using MPD approach. The developed PDT offered a model for on-board predicting where interaction with the unknown payload and aerial robot dynamics can be informed. Beside this, the results showed the ability of the introduced PDT to foretell the conditions of the forces acting on the payload which helped to predict the situation of aerial manipulation process.

8. Future Work

This study provides a way of diagnosing failure in aerial payload transportation by DT; however, there are limitations that recommendations for future work can be addressed. The study focused on on-board diagnostic process and off-board diagnostic process was not considered. Including cloud can provide further insights on applying DT within the Internet of Things in aerial robotics. While the diagnostic process was considered from the interactional forces between the fingers of the gripper and the payload, the effect of path of flight and orientation of the aerial robot is not considered in this study and can be recommended to be taken in consideration in future work. Furthermore, additional study is required to explore the benefits of DT in aerial transportation.

Data Availability

No data were used to support this study.

Conflicts of Interest

The authors declare that there is no conflict of interest regarding the publication of this paper.

Acknowledgments

This work was supported in part by the Deanship of Scientific Research, King Khalid University, Saudi Arabia, under Grant “RGP. 1/209/43.”