Abstract
The three-arm surgical robot system assisted mandible reconstruction surgery (TMR-MRS) is developed to repair the mandible defects caused by congenital defect, trauma, or acquired disease. The surgical robot system is divided into four parts, including 3D reconstructed image subsystem, robotic subsystem, optical measurement subsystem, and patient subsystem. The spatial registration based on quaternion is proposed to obtain the transformation relationship between four surgical subsystems. A method of hand-eye coordination is presented to control the end-effector of the robot arm to target position according to surgical planning. A least square error criterion is developed to optimize and compensate the hand-eye coordination method. In addition, the position of fibular implant in the robot space is calculated when it is grasped by end-effector of robot middle arm. Finally, the positioning accuracy tests and skull model experiments were conducted to evaluate the function of the surgical robot system. The results validate its feasibility and are consistent with the preoperative surgical planning.
1. Introduction
Mandible, which is a pivotal part of oral and maxillofacial region, plays an important role in the maintenance of facial feature, chewing, speaking, swallowing, facial expressions, and other functions [1]. Mandible defects not only influence the appearance of patients but also result in structural damage or loss of the surrounding important tissues and organs, which has serious effects on facial feature. In general, the complex surgical procedures of mandible reconstruction surgery, including drilling, bone positioning and orienting, bone graft, and fixing bones with screws and plates, have been already implemented manually by surgeons [2–4].
In recent years, due to the expanding application of computer in medicine, computer assisted surgery is widely used as supportive tools for diagnosis, operation planning, and treatment in surgical intervention [5–8]. In clinical routine, the surgeon has been already supported by various computer aided devices such as surgical planning systems, intraoperative navigation systems, and stereolithographic modeling of the patient’s skull, especially the navigation system, which has largely improved the surgical result. The use of navigation system in oral and maxillofacial surgery was first achieved in the 1990s for the removal of skull base tumors, foreign body extractions, and the transfer of osteotomy lines. For example, the Ruprecht-Karls-University (Germany) described the intraoperative realization with navigation systems in maxillofacial and craniofacial surgery in 1995 [9, 10], and they achieved a reduction in operational risk and duration as a result. The commercialized surgery navigation systems in oral and maxillofacial surgery had not become available until the past decade. Actually, four surgical navigation systems, namely, Stryker (Stryker-Leibinger, Kalamazoo, MI, USA), StealthStation (Medtronic-Xomed, Jacksonville, FL, USA), VectorVision (BrainLab, Munich, Germany), and Voxim (IVS Solutions, Chemnitz, Germany) have been frequently reported regarding applications in recent years [11–15], along with the AccuNavi integrated 3D medical modeling, preoperative surgical planning, and intraoperative tracking [6]. Nowadays the navigation system has been already applied in the maxillofacial surgery and other surgical domains, such as orthopedics, neurosurgical, and ENT surgery. Computer assistance through surgical navigation systems has increased surgical precision through accurate reproduction of planned intervention in the operating room [16].
Furthermore, the robot technology is widely known as an important instrument (facility) in the maxillofacial surgery because of its increasing advantages, such as high accuracy, stability, and flexibility of control. In 1998, Lueth et al. proposed the first active surgical robot system (OTTO) in a clinical environment for maxillofacial surgery [17]. The robot was used for inserting nonflexible catheters and for implanting bone fixtures. In 2000, Brief et al. put forward a robot system assisted insertion of dental implants [18, 19]. In 2003, the RoboPoint robot system was successfully developed [20], including positioning unit and a drive unit. The robot adjusted the appropriate position and orientation of drill and then performed the operation manually by a surgeon. The RobaCKa robot system was developed by University of Karlsruhe (TH) and University of Heidelberg for craniofacial surgical osteotomy [21]. In other surgery, many robot systems have been built, such as RoboDoc, CASPAR, CRIGOS, and MAKO.
Even though the anatomic structure of maxillofacial surgery is complicated and the shape is extremely irregular, some experienced surgeon performing the procedure could reduce most of the potential risks and have many patients with functional and esthetic results by using commercialized surgery navigation system. For surgeons, maintaining the intraoperative position of head and bone implant allows them to reduce the surgical duration and risks and alleviate the fatigue. The target of these projects is to present a three-arm robot system as a tool for mandible reconstruction surgery, which integrates visualization data obtained from CT scanner, preoperative surgical planning, robot, and surgical navigation system. The visualization data derives 3D image model of patient, which provides an optimal connection between preoperative planning and intraoperative execution. The preoperative planning, which helps to provide the optimal surgery strategy, is always performed preoperatively and does not prolong the surgical procedure. The robot could assist the surgeons to fix the patient’s head and locate fibular implant in accordance with the planned spatial position and orientation in the mandible construction surgery, so as to reduce the risk and time of operation. This paper aims to relate the components of entire robot system and to realize the interaction among the robot, navigation system, and the patient, namely, the surgical spatial registration for a three-arm robot system. Then the robot could be controlled to move along with the designed trajectory so that it can assist surgeons to complete mandible reconstruction surgery.
2. Methods and Materials
Based on the analysis of mandible reconstruction surgery process, the robotic system mainly includes five components: preoperative 3D image reconstruction and surgery planning, intraoperative three-arm surgical robot assisted mandible reconstruction surgery (TMR-MRS), postoperative evaluation, Virtual Reality (VR) training [22], and PC-based workstation. As can be seen in Figure 1, the details of every component are illustrated as follows.(1)Preoperative 3D image reconstruction and surgery planning, which are performed in an image workstation, provide surgical trajectory for mandible reconstruction surgery.(2)The three-arm robot system, which includes an optical tracking device and a three-arm surgical robot, is used to operate mandible reconstruction surgery for mandible defects.(3)Postoperative evaluation has great effect on improvement and optimization of the robotic system.(4)A PC-based workstation allows for overall application control, which performs a suite of software including robot control, optical tracking, and surgeon interfaces.(5)VR training provides surgeons with virtual surgery for guiding to use the robotic system.
According to the function, the three-arm robot system (Figure 2) assisted mandible reconstruction surgery can be divided into four parts, including 3D reconstructed image subsystem, robotic subsystem, optical measurement subsystem, and patient subsystem.
In this paper, the optical measurement subsystem (NDI Inc., Canada) is focused, which is consisted of an optical tracker, one passive four-marker probe, and two passive four-marker rigid bodies. One of the passive rigid bodies is fixed on the surgical robot, and the other is fixed on the patient skull. In the optical measurement subsystem, passive probe and passive rigid bodies are treated as measurement tools for coordinate measurement in the optical tracking system provided by the optical tracker. The paper also describes the integration of the four subsystems by using optical tracker for planning the movement of the robotic arm.
The spatial registration for the three-arm robot assisted mandible reconstruction surgery completes firstly the transformation of coordinate spaces between four subsystems in order to establish the relationships of them. Secondly, allowing for controlling the end-effect of surgical robot to certain position in the robot work space, through which the hand-eye coordination registration would be researched; thus the robot holds the position of the patient’s head and locates the fibular implant. In addition, the positions of free bones, including fibular implant and mandible ramus, need to be calculated in the robot space to complete surgery [23].
2.1. Navigation Spatial Registration Based on Improved ICP Algorithm
The spatial registration essentially performed the transformation matrix among the 3D Image coordinate system, patient coordinate system, robotic coordinate system, and optical coordinate system (Figure 3).
When the number of point set is small, classical ICP algorithm [24, 25] could not get the accurate and stable results, which needs to iteratively calculate the corresponding relation between two point sets [24]. In order to reduce the invasiveness to patient caused by placing the titanium screws, the registration process could not provide many marker points to derive the complete registration relationship. So the classical ICP algorithm needs to be improved to calculate transformation matrixes. In the improved ICP algorithm, the corresponding relations between two given point settings would be regarded as the initial condition to avoid the iterative solution process between two unknown point settings. Then the mapping relationship would be solved by the least squares method and calculate iteratively the transformation matrixes based on the result error, so as to make sure the accuracy satisfies the certain safety threshold.
The navigation spatial registration based on the improved ICP algorithm is illustrated as follows.(1), which is the transformation matrix from patient space to optical measurement space, can be obtained by optical tracker and passive rigid body on patient skull.(2) is the transformation matrix from patient space to 3D image space. Firstly, the coordinates of some medical marker points (titanium screws) which fixed the patient skull should be obtained by using optical tracker and passive probe. Through the , the above coordinates could be transformed into the patient space. So the coordinates of medical marker points in the patient space could be measured. Based on the coordinates of the medical marker points in 3D image space and patient space, the transformation matrix could be calculated by the improved ICP algorithm.(3), which is the transformation matrix from 3D image space to optical measurement space, could be computed from the following formula: .(4), which is the transformation matrix from robot space to optical measurement space, can be obtained by using optical tracker and passive rigid body on the robot. The robot coordinate system should be established by the passive rigid body of the robot.
However, the kinematics and control of the robot should be realized in the robot base coordinate system which is generated in the robot body. So the transformation matrix from optical measurement space to the robot base coordinate system should be solved.
First, more than 3 points are marked on the fixed position of robot body to establish robot local coordinate system ; thus the transformation matrix from the robot local coordinate system to robot base coordinate system could be got. Next, the coordinate of the 3 points in optical measurement space is measured by using the passive probe to palate the 3 points on the robot, and the transformation matrix from the robot local coordinate system to optical measurement space could be calculated. So the mapping relationship between optical measurement space and robot base coordinate system is measured as follows:
Furthermore, transformation matrix from the robot base coordinate system to the robot space could be calculated by the following formula: .
As a result, controlling the robot to a certain position could be realized, as long as the corresponding parameters are described in the robot space.(5), which is the transformation matrix from patient space to robot space, is calculated by and as follows:(6), which is the transformation matrix from 3D image space to robot space, is calculated by and as follows:
In robot assisted surgery, the target position and orientation matrix , which is planned by surgery planning system in preoperation, can be obtained in 3D image space. The target matrix in robotic space which corresponds to the target matrix can be obtained. According to the above matrix transformation, the equation could be got.
Therefore, can be calculated after getting ; then the robot could move accordingly to target position with right orientation.
2.2. Hand-Eye Coordination Based on Optical Positioning
To control the end-effectors of robot arm from the current position to target position, the target position and orientation of end-effectors in the robot space must be described. After establishing the transformation relationship between the robot space and optical measurement space, transform the position and orientation of the end-effectors into the robot space in order to control the end-effectors to target positions and orientation. This procedure is called the robot hand-eye coordination [26].
In the traditional hand-eye coordination procedure, the passive rigid bodies are installed on the end-effectors of the robot’s arms and bones, respectively, representing the current and target position. The hand-eye coordination procedure means the relative position and posture between the end-effectors and passive rigid bodies would be calculated. Through the passive rigid body, the optical tracker can get the current and target positions and orientation of end-effectors in the optical measurement space. Because of the three arms of surgical robot and the limitation of workspace, it is inevitable for the arms to block the light from the optical tracker to the passive rigid bodies, which will cause positioning errors. Compared with the traditional method, the robot chooses to install the passive rigid body in the robot body to avoid that factor. So the hand-eye coordination aims to get the transformation relationships, including from the base coordination system to passive rigid body and from the passive rigid body to the end-effectors of the robot arm. The hand-eye coordination based on optical positioning is described (Figure 4).
Establish the robot base coordinate system . Because the passive rigid body is fixed on the robot body, the robot coordinate system is built and the transformation relationship is constant matrices. The transformation relationship between the robot base coordinate system and the end-effector of the robot arm could be calculated by using robot kinematics as follows:
The posture matrix of passive rigid body in the optical measurement space is obtained from the navigation software. So the current end-effector posture matrixes in the optical space could be got as follows:
The target bone coordinate systems are , , and , which are attached to the rigid body, namely, representing the target positions. The target posture matrixes of bone coordinate system in the optical measurement space are , , and . The current end-effector position and target position in the optical space could be obtained. So the joint angles form the current position to the target position will be calculated.
Even though the transformation relationships of hand-eye coordination are got, the transformation error is more than 2.5 mm from the experiments. Therefore, the least square error criterion is used to optimize the transformation relationships. A dual number quaternion based algorithm is employed to estimate the transformation matrix in the optical tracker system, which incorporates both orientation and translation information. However, inaccuracy in robot forward kinematics and transformation matrix affects the validity of navigation system. Robot calibration is required to reduce the registration error and inaccuracies in robot parameters of links and joint angles. The transformation matrixes are shown as follows:
Solving the optical transformation typically requires minimizing a least square error criterion given by where corresponded pose pairs, , are recorded at different configurations of robot angle setting.
The forward kinematics of the robot is constructed by the transformations between consecutive joint frames based on the modified D-H parameters. After sufficient number of measurements of the corresponded marker positions, the least square estimation of robot parameter deviation is calculated as follows:where is the identification Jocobian matrix and is the corresponded marker positions error.
So the transformation matrixes ( from the base coordination system to passive rigid body, from the passive rigid body to the end-effectors of the robot arm) are optimized to improve the accuracy of the hand-eye coordination control. Finally, the coordinate control scheme is outlined as in Figure 5.
2.3. The Space Registration of Fibular Implant
In the surgery planning, the target position of fibular implant is provided to the robot. The fibular implant will be grasped by the middle arm of the robot and moved to target position. So the current position of fibular implant represented in the robot space needs to be got in order to control the robot arm to the target position. However, the fibular implant is too small to fix the passive rigid body. The operation would be affected by the fixed passive rigid body because the passive rigid body blocks the infrared light path from the optical tracker. So the position of fibular implant is represented by the local coordinate system which is established by placing some titanium screws on fibular implant. The space registration of fibular implant aims to get the current position of fibular implant in the robot coordinate system.
Before the space registration, the three-dimensional model of fibular implant in the medical workstation is reconstructed by computed tomography scanning data. The fibular implant has been fixed with the medical titanium screws which could be used for registration markers. In the surgery, the fibular implant is grasped by the robot middle arm. The passive probe of the optical navigation system touches the markers (titanium screws) to establish the local coordinate system as the position of fibular implant. If the three points () represent the medical markers, the local coordinate system could be shown as follows:where , , and are, respectively, the axis of the local coordinate system.
The transformation matrix between the optical measurement space and local coordinate system could be calculated. The current position of fibular implant (local coordinate system ) in the optical measurement space is represented as follows:
The target position of fibular implant in the 3D image space is denoted by . Because of the navigation space registration, the target position of fibular implant in the optical measurement space could be developed as . Consider
Once the square root of the measured difference between the current and target position of fibular implant in the optical measurement space is less than a threshold, the current position of fibular implant could be considered to be the ideal position. In addition, both positions ( and ) convert to the robot space in order to be convenient for computational kinematics. Both positions are shown in the robot space as follows:
By the inverse kinematics of robot parameters, the corresponding joint angles for current and target position are solved. When the differences of corresponding joint angles of both positions are given the joint motors, the fibular implant would be placed to the target position.
3. Result
3.1. Positioning Accuracy Test
3.1.1. Experimental Setup
As shown in Figure 6, the optical tracking system (NDI Polaris, Canada), which contains the optical tracker, passive probe, and passive rigid body, is used as a measurement tool in this experiment. The space positioning accuracy is less than 0.25 mm. The passive rigid body of optical tracking system is fixed on the end-effector. The spatial position of passive rigid body in the optical measurement space could represent the end point position of the robot end-effector.
Based on the function of each arm, the middle arm aims to grasp the fibular implant to the planning position with certain orientation in order to complete the surgery successfully. So the high positioning accuracy of middle arm should be provided. And the left and right arms of surgical robot aim to hold mandibular ramus, the repeated positioning accuracy of left and right arms should be provided.
3.1.2. Experimental Procedures
(1)Firstly, the passive rigid body for location fixed on the end-effector of the robot arm is treated as a measurement point. Complete the registration between robotic space and optical measurement space in order to obtain the mapping relationship. The transformation matrix from optical measurement space to robotic space is determined.(2)Secondly, give randomly the coordinate value of a predetermined point in the robot coordinate system. The passive rigid body is controlled to the position of the predetermined point. Record the coordinate value of the passive rigid body point in the optical coordinate system and convert the value into the robot space. The coordinate value of the passive rigid body in the robot space could be got as the actual point.(3)Thirdly, the positioning accuracy is illustrated by the difference between the predetermined point and the actual point as follows:(4)Finally, choose the different predetermined point in turn and obtain the results of multiply measurements, and then process the data.
3.1.3. Experimental Results
The experiment is repeated with 45 different preset points. Figure 7 shows the positioning accuracy test results of each arm.
The results show that the maximum error of positioning accuracy is 1.3262 mm and the minimum error is 0.7188 mm. The accuracy meets the requirement of the mandible reconstruction surgery so that the end-effector of the robot accurately can reach the designated position.
3.2. Skull Model Experiment
As experimental subject using skull model, the three-arm surgical robot assists surgeons to complete positioning of the fibular implant and validate its feasibility (Figure 8). The work flow is shown in Figure 8.
The experiment is conducted to assist surgeons in positioning and holding fibular reconstructed mandible defect implant. During the experiment, surgeons operate the robot and optical navigator with PC-based workstation, with which robot control software and reconstructed 3D model are shown on double displays. The work flow of skull model experiment is shown as follows.(1)According to the actual patient’s condition, manufacture the mandibular defect skull model (Figure 8(a)).(2)Titanium screws for registration between skull model and reconstructed 3D image were fixed on skull model as marker points.(3)High-precision CT scan of skull model (Figure 8(b)) was performed.(4)Based on the CT data, complete 3D skull model reconstruction and surgery design of fibular reconstruction mandibular defect (Figure 8(c)).(5)Design and print the fibular implant. And titanium screws are fixed on fibular implant in order to do its registration in robotic space (Figure 8(d)).(6)Initialize navigation systems and robot. The rigid body was fixed on the skull as patient’s coordinate system (Figure 8(e)).(7)By using passive probe to touch the titanium screws markers on the skull model, the registration between patient coordinate system and 3D image coordinate system was completed (Figure 8(f)).(8)The fibular implant was grasped by end-effector of middle arm. Likewise by using passive probe to touch the titanium screws markers on the fibular implant, complete the registration of fibular implant in the robot coordinate system.(9)Based on the preoperative planning, the data of target position of fibular implant in 3D image coordinate system were transferred to robot coordinate system.(10)According to surgery trajectory planning, control the end-effector of middle arm with fibular implant to target position with certain orientation.(11)At last, fibular implant was firmly fixed at the skull model with screws.
Figure 9 shows the result of skull model experiment. During the experiment, with the help of navigation system, the fibular implant is adjusted to an appropriate position by the middle arm of the robot according to the surgery trajectory planning. The robot works stably. The positioning error in this model experiment is acceptable in clinical application. The error mainly comes from manufacturing and assembly error, 3D reconstruction error, registration error, robot initialization error, coordinate system mapping error, and so on. In further improvement, more model experiments should be conducted to inspect the error of the whole robotic system.
4. Conclusions and Discussions
This paper presents a three-arm surgical robot system assisted mandible reconstruction surgery through the analysis of surgical requirements. The surgical robot system is divided into four parts, including 3D reconstructed image subsystem, robotic subsystem, optical measurement subsystem, and patient subsystem. The spatial registration for three-arm robot assisted mandible reconstruction surgery aims to complete the transformation relationships of coordinate spaces between four surgical subsystems. The hand-eye coordination is proposed to control the end-effector of the surgical robot arm to target position. A least square error criterion is developed to optimize and compensate the hand-eye coordination method. In addition, the position of fibular implant in the robot space is calculated when it is grasped by the end-effector of robot middle arm. Finally, the positioning accuracy tests and skull model experiments are conducted to evaluate the function of the surgical robot system. The results validate its feasibility and are consistent with the preoperative surgical planning.
In the previous robot systems, such as Robodoc, Arobot, the registration was completed without optical navigation system. The robot was controlled to palpate the extremity of titanium screws for the registration between the CT reconstructed image space and robot space during the intraoperative phase [27–29]. So the passive rigid body and passive probe (with four-markers) of optical navigation system are introduced to be a tool for registration. In the optical measurement space, the registration between the patient and 3D reconstructed image was performed by passive probe and titanium screws. In the process, due to the head movement, the relative position between patient and optical tracker is changed to bring the registration error. The passive rigid body as the patient space is introduced to be fixed with the patient skull. Rigid connection between the rigid body and patient skull allows for avoiding the relative position change error of patient skull. In a similar way, the other rigid body is fixed on the robot to eliminate the relative position change error of robot. The method, in which the relative position between fixed rigid body and patient skull or robot could not be changed, allowed for the registration only for one time during whole surgery. As long as the coordinate systems of rigid bodies are obtained through intraoperative tracking in time, the intraoperative position of patient or robot could be acquired. Furthermore, good visibility of patient, robot, and passive probe should be guaranteed by placing the optical navigation tracker. The calibrations of passive probe and rigid body were performed by using the navigation toolset in the preoperative operation.
With the development of navigation system, optical tracking system is commonly carried out with the high accuracy (less than 0.25 mm error). For the surgical robot system, many works have been done in the navigation system to improve the surgical accuracy in recent years. But the robot system is limited to accomplish the special planned movements with one or two arms. This project aims to design a useful tool assisted mandible reconstruction surgery, which could not only hold the head in the planned position, but also control the end-effector to the planned position. This is an exploration to the robot system assisted maxillofacial surgery, an attempt to the spatial registration for multiarm robot assisted mandible reconstruction surgery.
For the multiarm robot for mandible reconstruction surgery with three arms, its workspace is adjusted to the size of patient’s head. Due to the overlap of the workspaces of each arm, the interference is prone to happen, causing a collision [30]. Based on the LSS (Linearly Swept Sphere) linear scan ball model of the rod, the collision avoidance motion control was researched by C-space model, the C-space mapping of obstacle, and improved heuristic search algorithm in order to guarantee the surgery safety.
Although results of these studies show the feasibility of robotic system, there are still a number of shortcomings and designs needed to be overcome, improved, and optimized. The method implanting the titanium screw as the marker is invasive to patient, so that should be replaced. In optical tracking system, the end-effectors of robot cannot be displayed in surgical design system in real-time, which is adverse, to carry out the real-time navigation of robot for surgeons. In the future, some functions need to be optimized in order to improve the system performance. More experiments will be done to measure the error of whole robotic system and evaluate performance of the robotic system. The safety of robotic system should be improved and optimized by installing the sensors on every joint. More flexible robot wrist and more easy-to-use end-effectors need to be designed according to clinical requirements.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work is supported by the National Technology Research of CHINA (863 Project) (Grant no. 2012AA041606), Ph.D. Programs Foundation of Education of China (Grant no. 20111101110004), and Beijing Municipal Natural Science Foundation (Grant no. 7132132).