Recent Applications of Scanning Microscopy in Surface Engineering 2021View this Special Issue
Bias Evaluation of the Accuracy of Two Extraoral Scanners and an Intraoral Scanner Based on ADA Standards
The spread and application of computer-aided design/computer-aided manufacturing (CAD/CAM) technology have contributed to the rapid development of digitalization in dentistry. The accuracy of scan results is closely related to the devising subsequent treatment plans and outcomes. Professional standards for evaluating scanners are specified in the American National Standard/American Dental Association Standard 132 (ANSI/ADA No. 132). The aims of this study were to use the three samples mentioned in ANSI/ADA No. 132 and evaluate the accuracy and reproducibility of two extraoral scanners and an intraoral scanner based on the inspection standards recommended by ANSI/ADA No. 132. In this study, two trained operators used two extraoral scanners (E4, 3Shape, Denmark & SHINING DS100+, Shining, China) and an intraoral scanner (TRIOS SERIES3, 3Shape, Denmark) to perform 30 scans of each of the three samples at a temperature of and export standard tessellation language files and used reverse engineering software to perform measurements and iterative nearest point matching experiments. The measured values obtained were compared with the reference values measured by a coordinate measuring machine (NC8107, Leader Metrology, USA). We performed a normal distribution test (Shapiro-Wilk test), the nonparametric Kruskal-Wallis test, and an independent-samples -test to analyze the reproducibility of each scan for different models. The experimental results indicate that the trueness and precision of the two extraoral scanners and the intraoral scanner had a slight mean deviation. The trueness and precision of the three scanners on the curved surface and groove areas are poor. The accuracy and reproducibility of E4 outperformed SHINING and TRIOS. The iterative closest point matching experiment also showed good matching results. The two extraoral scanners and the intraoral scanner in this study can meet the basic clinical requirements in terms of accuracy, and we hope that digital technology will be more widely used in dentistry in the future.
In dentistry, the use of digital methods such as computer-aided design/computer-aided manufacturing (CAD/CAM) is rapidly increasing [1–4]. It improves diagnostic ability and facilitates contact between doctors and patients by allowing the rapid acquisition of 3D diagnostic information and transmission of digital data, and along with the rapid development of CAD/CAM technology in the dental field, their use in clinical diagnosis and prosthodontics is becoming more widespread [5–15]. Studies have shown that intraoral scanners are more comfortable for the patient and reduce processing time, and their accuracy and precision are within clinically acceptable limits [10, 11, 16–24]. With the progress of digital technology in the dental field, the accuracy rating of oral scanners has become increasingly important. Extraoral scanners still have higher accuracy than intraoral scanners, and the trueness and precision of intraoral scanners have become an issue of concern.
The current study shows that differences in scanning systems and methods affect experimental results to an uncertain extent . Oh et al. found that different scanning strategies affect the accuracy of the results [26, 27], and a study by Giménez et al. found that operator proficiency also affects the accuracy of digital impression testing . Furthermore, the accuracy of intraoral scanners changes with the length and distribution of the dental arch: the larger the scan area, the lower the accuracy with the maximum deviation of the scan in the posterior part of the dental arch . The impact of external factors on the results of oral scanners has been the focus of attention, but the objective of evaluating the accuracy of oral scanners has been neglected. Therefore, it is important to objectively grade oral scanners using a more consistent scanning strategy for a standard sample.
In this study, we designed samples based on the American National Standard/American Dental Association Standard 132 (ANSI/ADA NO.132)  and measured the height of the crown and the radius of its top circle, the height of the inlay, and the radius of the top circle, and the distance between the reference points of the sphere. The aim of this study was to evaluate the accuracy of two extraoral scanners and an intraoral scanner by analyzing and comparing various data based on the inspection standards recommended by ANSI/ADA No. 132. Furthermore, we introduced CAD/CAM technology because it is a proven technique for manufacturing samples and computer-designed stereo images based on absolute reference measurements in the form of a coordinate measuring machine (CMM) to perform iterative closest point matching experiments to further evaluate their accuracy . Meanwhile, we conducted experiments to evaluate the reproducibility of the two extraoral scanners and an intraoral scanner.
2. Materials and Methods
2.1. Fabrication of Models
According to ANSI/ADA No. 132, this study designed three reference models. The theoretical values of the crown model radial of the top surface and height were set to 3.5 mm and 6.0 mm, respectively, the radial of the top surface and height of the inlay model were set to 4.0 and 6.0 mm, respectively, and the center distance of a sphere with a long-distance specimen model diameter of 8.0 mm is set as , , , , , and . The 3D views and optical images are shown in Figure 1. Models of the samples were first drawn using CAD software (AutoCAD 2018, Autodesk, USA) and then exported in standard tessellation language (STL) format for computer numerical control milling. Samples were fabricated from stainless steel according to the STL file and washed three times in an ultrasonic bath at 30°C for 5 min each time. Finally, all the samples were sandblasted with a powder size of 80 μm.
2.2. 3D Scanner
The oral scanners used in this experiment were an intraoral scanner (TRIOS SERIES3, 3Shape, Denmark) and two extraoral scanners (E4, 3Shape, Denmark & SHINING DS100+, Shining, China). The SHINING DS100+, whose field of view is , uses blue light and point cloud to capture and form the corresponding image. Similarly, the other extraoral scanner, E4, has the same light source and imaging type as SHINING DS 100+. By contrast, the light source of the intraoral scanner was white light, and the STL file was obtained by splicing the image. More details regarding these scanners are presented in Table 1. Because it is unlikely that the fabricated samples exactly match the software plotted results, CMM was used in this study to measure the relevant indexes of the samples, and the results obtained were used as reference values.
2.3. Sample Scanning and Data Acquisition
The fully trained operator performed 30 scans at 30 s intervals in strict accordance with the instructions for use under the same conditions to obtain a set named A (). The other operator performed 30 scans in a different environment to obtain a set named B (). Reverse engineering software (Geomagic Control X 2018; 3D SYSTEMS, USA) was used to measure a variety of indexes to complete the experiments (Figure 2).
Indexes marked in Figure 1 were measured, and then, data were compared and analyzed.
Using the principle of outlier elimination, if the difference between the data and the average value exceeds 1.96, standard deviations (i.e., outlier data) and the data are eliminated, and the scan and measurement are performed again. If there are more than two outliers, the experimental results will be canceled, and the experiment will be executed again.
2.4. Calculation of Trueness and Precision
Trueness and precision were quantified in terms of the relative error. represents the trueness of the test results. The smaller the value of , the higher the trueness of the scanner. represents the precision of the test results. The smaller the , the higher is the precision of the scanner.
represents the reference value of the sample, and represents the measured value (including length, depth, height, and distance from the center).
represents the standard deviation and represents the reference value of the sample.
The ANSI/ADA No. 132 professional standard considers the relative error of the indexes in samples 1 and 2. Less than 0.01 mm is the acceptable range in the dental requirements category, and a relative error of less than 0.0025 mm represents the distance between sphere reference points to ensure that the acceptable threshold value in the dental requirements category is used as the threshold value.
2.5. Reproducibility Analysis
The reproducibility is reflected by comparing the values of indexes measured from sets A and B. The closer the results of 30 scans of experimental group B to that of experimental group A, the greater the reproducibility.
2.6. Statistical Analysis
Each set of data used the Shapiro-Wilk test for normality, which uses SPSS v.24.0 (IBM, USA) to determine the correlation between the scans from the different scanners in the experiment. The nonparametric Kruskal-Wallis test was used to analyze the differences in parameters. We then performed an independent samples -test to analyze the reproducibility of each scan for different models.
2.7. Iterative Closest Point Matching Experiments
The values calculated from the scan images of the experiment were used to chart the accuracy and repeatability of the oral scanner. However, in addition to the overall detection indexes of the scanning image, the scanning effect of different scanners for the same part of the sample is also different, which cannot be reflected by the detection indexes. Therefore, it is necessary to perform matching experiments to intuitively compare the scanning differences among the three scanners for the same sample.
Each scanned STL file from one scanner was imported into Geomagic Control X to perform iterative closest point matching experiments with corresponding STL file obtained with another scanner. Before the matching experiments, excess parts of the STL file were removed for better results. Thereafter, the initial alignment is carried out, and then, the best-fit alignment between scan data is obtained by the least-squares method. Finally, 3D comparisons were conducted to obtain color images for the visual observation of the difference between the scanners. The average deviation of all points on the surface between two scanners scans is also calculated.
Because the scanned results were well distributed, the results of the iterative closest point matching experiments were also similar. A representative image was selected to represent the experimental results.
The results of scanning the indexes of each sample with two extraoral scanners and an intraoral scanner are shown in Table 2, and the relative errors () of the samples were compared with the ANSI/ADA No. 132 specified value. Figure 3 is a box plot of trueness values. Figures 4–6 show the measurements and reference values for sample 1 (crown), sample 2 (inlay), and sample 3 scanned by SHINING, E4, and TRIOS scanners.
TRIOS had a large deviation in measuring R3, which exceeded the acceptable range in the dental requirement category. For sample 2, the SHINING scans showed larger relative errors than E4 and TRIOS, while for sample 1, the relative errors for E4 and TRIOS were larger than those for SHINING. Although the scanning data deviated from the reference value, the deviation was controlled between 0.001 and 0.184 mm. The E4 scan value was closest to the reference value.
There were significant differences in SHINING, E4, and TRIOS. The statistical differences between the three scanners are summarized in Table 3.
The results of scanning the indexes of each sample with two extraoral scanners and an intraoral scanner are shown in Table 4, and the relative errors () of the samples were compared with the ANSI/ADA No. 132 specified value. The results of SHINING, E4, and TRIOS scanning crowns and inlays were accepted by ANSI/ADA No. 132. The of E4 was the lowest of all three samples, showing the highest precision. SHINING, E4, and TRIOS have a large deviation from the reference value when scanning the inlay radius. The difference between repeated measurements of TRIOS is the largest, especially the distance between the datum points of the spheres.
3.3. Repeatability and Reproducibility of Samples
Regarding repeatability and reproducibility, most of the results tested by the two operators were statistically different (Table 5).
In the SHINING group, there was no statistically significant difference between groups A and B in the radius of the repeatedly scanned crowns and inlays. In the E4 group, there was no statistically significant difference between groups A and B in the radius of the scanned inlays. In the Trios group, there was no statistically significant difference between groups A and B in the radius of the repeatedly scanned crowns.
3.4. Iterative Closest Point Matching of Samples
The color bar represents qualitative information analysis. These differences between scanners are shown in the color bar . While the color bar depicts deviations between -1 and 1 mm, acceptable errors between -0.01 and 0.01 mm are marked in green, better reflecting the differences between two scanners while scanning the same sample. A positive difference from yellow to red indicates a higher deviation from the reference model, and a negative difference from azure to dark blue indicates a lower deviation from the reference model.
Figures 7 and 8 show that the local deviation between the two scanners occurs on the curved surface and deep areas. Table 6 shows the RMS values between the two scanners, and the RMS values between the scanners is within 1.0 mm.
The oral scanner is evaluated based on the “trueness” and “precision” specified in ISO 5725-1 , where trueness is defined as the consistency between the test result and the acceptable reference value. Precision is defined as the closeness of the independent test results obtained under the specified conditions. The accuracy of the scanner was verified by a combined evaluation of trueness and precision.
This study is aimed at evaluating the trueness and precision of three oral scanners based on three samples provided by ANSI/ADA No. 132. The CMM was used to measure the data obtained from the original sample as a reference value, and the results were compared with those of the three scanners. Experiments were conducted to further analyze the accuracy and reproducibility of the scanners [28, 34–38]. At the same time, some pits generated in the scanning process are introduced into the interactive closest point matching to reflect the differences among the three scanners.
The calculated and indicate that although there were a few large numerical errors (e.g., the average value of the relative error of TRIOS for the radius of sample 1 is 0.0260 mm, and the average value of the relative error of SHINING for the radius of sample 2 is 0.0448), most of the trueness and precision were acceptable, and the accuracy was within the acceptable range. The accuracy of E4 was better than that of SHINING and TRIOS. In terms of reproducibility, most of the results tested by the two operators were statistically different.
As in previous studies [28, 34–39], the CMM measurement sample was used to obtain more accurate standard values. It should be noted that there is always an error between the results of the mechanical scan and the actual values of the sample. However, the CMM used in this study has high accuracy, and using CMM measurement samples as the reference value is also allowed in ANSI/ADA No. 132 [20, 40–42]. Inevitably, many errors were still present during the experiment.
The results of the repeated measures of TRIOS have a large fluctuation range. The high relative error of TRIOS may be related to the image acquisition method of the scanner and the operation of the operator. The extraoral scanner can construct the shape of the object effectively based on the point cloud obtained in the 3D point coordinate system, while the intraoral scanner uses the best-fitting algorithm to stitch the scanned images together. When the surface shape of the scanned object is complex, image alignment is easier, but when the scanned object is flat and smooth (such as edentulous jaws), images are more prone to errors, causing distortion of the STL file . The farther away from the start scanning point and the greater the splicing times, the lower the accuracy of the data. Therefore, some studies suggested placing complex geometry near the edentulous area to improve scanning accuracy when using an intraoral scanner [43–45]. At the same time, previous studies have shown that the posterior dental arch exhibits greater errors during intraoral scanning [10, 44, 46]. Moreover, the handheld intraoral scanner oscillates during scanning and needs to constantly change the coordinates . Therefore, each time the image of the scanner is stitched, the processing and fitting errors increase, resulting in inaccurate measurement and image distortion . Therefore, as the complexity and geometry of tooth preparation increases, intraoral scans become inaccurate [25, 38, 39, 48]. Michaeli et al. suggested alleviating the experimental error by increasing the scanning angle and scanning times . The extraoral scanner automatically scans the fixed model at different angles to reduce the influence of the operator on the scanning process. Therefore, the operator’s impact on the extraoral scanner is much less than that on the intraoral scanner . However, the two extraoral scanners and the intraoral scanner in this study showed poor reproducibility. The authors speculate that this may be related to scanning in different environments.
The best match alignments between scanners were conducted using the best-fit algorithm of Geomagic Control X. Each point in the source point cloud was best aligned with the closest point in the reference point cloud. The offset between digital models obtained from every two scanners was visualized by color mapping. Geomagic Control X can efficiently detect the differences between irregular shapes, which is especially suitable for analyzing oral models. However, because the scanning offset reflected by Geomagic Control X is not applicable to the evaluation of sphere spacing, iterative closest point matching is not considered for the sphere sample in this study.
In Tables 2 and 4, the relative errors of E4, SHINING, and TRIOS in the scan model radius exceeded the acceptable range in the dental requirement category. In Figure 7, scanners showed relatively large differences when scanning curved surfaces and deep area. This may be caused by the energy lost by the light source because of the reflection when the surface of the scanned sample moves, which affects scanned by the scanner, and the curved surface is more likely to cause light reflection . In Figure 8, comparing between each other, three scanners showed obvious deviation when scanning the bottom of the inlay, which may also be related to the fact that the scanner light source cannot perform detailed scanning at the bottom. Different light sources also affect the accuracy of the scanner [10, 51]. Araki et al. scanned under different light sources and evaluated the results. They found that the most suitable lighting conditions for digital impression scanning were 3900 K and 500 lux . However, in the iterative closest point matching experiments, the deviations produced by the best matching process are all less than 0.01 mm.
In previous studies, different methods have been used to evaluate the accuracy of diverse oral scanners. Intraoral scanner systems were found to be less accurate than extraoral scanners, and the ability of extraoral scanners to scan the edges of the crown is also better than that of intraoral scanners [25, 39]. This is consistent with the results of our experiments. Cai et al. evaluated the accuracy of SHINING, CEREC, and TRIOS by scanning an international standard sphere model, and the results showed that the accuracy of the intraoral scanners was better than that of the extraoral scanner . However, in this study, the accuracy of the extraoral scanner E4 was significantly better than that of the intraoral scanner TRIOS.
This study has some limitations, as it was conducted in vitro and did not simulate actual clinical conditions. The effects of temperature and humidity of the oral environment, saliva and blood, soft tissues, patient movement, oral cavity, and scanning laser angle of incidence were not considered [10, 54, 55]. At the same time, many items in the oral cavity, such as implants (ceramics, metals, and composite resins), dentin, enamel, oral soft tissue, and different materials and geometric shapes, also affect the accuracy of scanning. The scanned samples presented in this study have different light effects from the soft tissue around the oral cavity [26, 56]; therefore, there are some limitations. In future studies, we will simulate a clinical setting and increase the number of scanning groups to further evaluate the accuracy of different scanners.
This study evaluated the accuracy of the three scanners by measuring ANSI/ADA No. 132 provided reference models. (1)There was significant difference between the scanning results of the three scanners, with E4 showing the best reproducibility(2)Whether scanning crown, inlay, or arch, the extraoral scanner showed fewer distortions than the intraoral scanner, with E4 showing the least error(3)Most of the scanning errors of the extraoral and intraoral scanners were within the acceptable range of the ADA standard. However, the effect of scanning the sample radius is poor(4)Relatively large differences between scanners occurred when scanning curved surfaces and the deep areas of the inlay
The data used to support the findings of this study are included in this article.
Conflicts of Interest
We declare that we have no financial or personal relationships with other people or organizations that can inappropriately influence our work.
Naiyu Cui, Jiayin Wang, and Xingyu Hou contributed equally to this work. Naiyu Cui and Jiayin Wang are responsible for the conceptualization, methodology, formal analysis, investigation, and writing—original draft; Xingyu Hou for the methodology, validation, writing—review and editing, and visualization; ShiXun Sun and Qixuan Huang for the validation, data curation, and writing—review and editing; Ho-Kyung Lim for the writing—review and editing, and visualization; HongXin Cai and Qi Jia for the validation, resources, writing—review and editing, and supervision; and Eui-Seok Lee and Heng Bo Jiang for the validation, resources, supervision, project administration, and funding acquisition.
O. de Waard, F. A. Rangel, P. S. Fudalej, E. M. Bronkhorst, A. M. Kuijpers-Jagtman, and K. H. Breuning, “Reproducibility and accuracy of linear measurements on dental models derived from cone-beam computed tomography compared with digital dental casts,” American Journal of Orthodontics and Dentofacial Orthopedics, vol. 146, no. 3, pp. 328–336, 2014.View at: Publisher Site | Google Scholar
F. Mangano, C. Mangano, B. Margiani, and O. Admakin, “Combining intraoral and face scans for the design and fabrication of computer-assisted design/computer-assisted manufacturing (CAD/CAM) polyether-ether-ketone (PEEK) implant-supported bars for maxillary overdentures,” Scanning, vol. 2019, 4274714 pages, 2019.View at: Publisher Site | Google Scholar
U. Schepke, H. J. Meijer, W. Kerdijk, G. M. Raghoebar, and M. Cune, “Stock versus CAD/CAM customized zirconia implant abutments - clinical and patient-based outcomes in a randomized controlled clinical trial,” Clinical Implant Dentistry and Related Research, vol. 19, no. 1, pp. 74–84, 2017.View at: Publisher Site | Google Scholar
E. Davidovich, B. Shay, E. Nuni, and E. Mijiritsky, “An innovative treatment approach using digital workflow and CAD-CAM part 1: the restoration of endodontically treated molars in children,” International Journal of Environmental Research and Public Health, vol. 17, no. 4, p. 1364, 2020.View at: Publisher Site | Google Scholar
A. M. R. Cuperus, M. C. Harms, F. A. Rangel, E. M. Bronkhorst, J. G. J. H. Schols, and K. H. Breuning, “Dental models made with an intraoral scanner: a validation study,” American Journal of Orthodontics and Dentofacial Orthopedics, vol. 142, no. 3, pp. 308–313, 2012.View at: Publisher Site | Google Scholar
M. G. Wiranto, W. P. Engelbrecht, H. E. Tutein Nolthenius, W. J. van der Meer, and Y. Ren, “Validity, reliability, and reproducibility of linear measurements on digital models obtained from intraoral and cone-beam computed tomography scans of alginate impressions,” American Journal of Orthodontics and Dentofacial Orthopedics, vol. 143, no. 1, pp. 140–147, 2013.View at: Publisher Site | Google Scholar
M. F. Sfondrini, P. Gandini, M. Malfatto, F. di Corato, F. Trovati, and A. Scribante, “Computerized casts for orthodontic purpose using powder-free intraoral scanners: accuracy, execution time, and patient feedback,” BioMed Research International, vol. 2018, 8 pages, 2018.View at: Publisher Site | Google Scholar
C. Motel, E. Kirchner, W. Adler, M. Wichmann, and R. E. Matta, “Impact of different scan bodies and scan strategies on the accuracy of digital implant impressions assessed with an intraoral scanner: an in vitro study,” Journal of Prosthodontics, vol. 29, pp. 309–314, 2020.View at: Publisher Site | Google Scholar
B. Giménez, M. Özcan, F. Martínez-Rus, and G. Pradíes, “Accuracy of a digital impression system based on active wavefront sampling technology for implants considering operator experience, implant angulation, and depth,” Clinical Implant Dentistry and Related Research, vol. 17, pp. e54–e64, 2015.View at: Publisher Site | Google Scholar
“Scanning Accuracy of Dental Chairside and Laboratory CAD/CAM System,” ADA/ANSI Standard No. 132, American Dental Association, Chicago, IL, USA, 2015.View at: Google Scholar
“Accuracy (trueness and precision) of measurement methods and results — Part 1: General principles and definitions,” International Standard (ISO) 5725-1, International Organization for Standardization, Geneva, Switzerland, 1994.View at: Google Scholar
B. Gimenez-Gonzalez, B. Hassan, M. Ozcan, and G. Pradies, “An in vitro study of factors influencing the performance of digital intraoral impressions operating on active wavefront sampling technology with multiple implants in the edentulous maxilla,” Journal of Prosthodontics, vol. 26, no. 8, pp. 650–655, 2017.View at: Publisher Site | Google Scholar
M. Bratos, J. M. Bergin, J. E. Rubenstein, and J. A. Sorensen, “Effect of simulated intraoral variables on the accuracy of a photogrammetric imaging technique for complete-arch implant prostheses,” The Journal of Prosthetic Dentistry, vol. 120, no. 2, pp. 232–241, 2018.View at: Publisher Site | Google Scholar
M. Revilla-Leon, S. G. Subramanian, W. Att, and V. R. Krishnamurthy, “Analysis of different illuminance of the room lighting condition on the accuracy (trueness and precision) of an intraoral scanner,” Journal of Prosthodontics, vol. 30, no. 2, pp. 157–162, 2021.View at: Publisher Site | Google Scholar
F. Ferrini, G. Sannino, C. Chiola, P. Capparé, G. Gastaldi, and E. Gherlone, “Influence of intra-oral scanner (I.O.S.) on the marginal accuracy of CAD/CAM single crowns,” International Journal of Environmental Research and Public Health, vol. 16, no. 4, p. 544, 2019.View at: Publisher Site | Google Scholar