Table of Contents
Journal of Anthropology
Volume 2016 (2016), Article ID 6218659, 6 pages
Research Article

Reliability and Accuracy of Angular Measurements on Laser Scanning Created 3D Models of Dry Skulls

1Institute of Experimental Morphology, Pathology and Anthropology with Museum, Bulgarian Academy of Sciences, 1113 Sofia, Bulgaria
2Institute of Information and Communication Technologies, Bulgarian Academy of Sciences, 1113 Sofia, Bulgaria
3Institute of Mathematics and Informatics, Bulgarian Academy of Sciences, 1113 Sofia, Bulgaria

Received 30 April 2016; Accepted 4 August 2016

Academic Editor: Santos Alonso

Copyright © 2016 Diana Toneva et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


The aim of the study was to assess the accuracy and reliability of skull angles measured on 3D models created by laser scanning. Five skulls were measured through both conventional and digital measuring methods. The 3D models were created using a hand-held laser scanner Creaform VIUscan. Seven angular characteristics were measured three times by two observers. The intra- and interobserver reliability were assessed using intraclass correlation coefficient. The differences between the digital and direct measurements were assessed using the measurement and relative errors. The intraclass correlation coefficients for digitally taken angles indicated almost perfect intra- and interobserver reliability, except for the alveolar profile angle showing moderate interobserver agreement. The overall measurement error based on the differences between digitally and directly measured angles was 0.61° and the relative error was 0.68%.

1. Introduction

Craniometry is a main part of many anthropological studies giving information about the size and proportions of the skull and its parts. It is mainly used by the paleoanthropologists and forensic anthropologists to estimate sex, race, and ancestry. Besides the numerous linear craniometric measurements, additional quantitative information on the skull is obtained by a variety of angular characteristics. The skull angles are useful in the assessment of skull shape and degree of prognathism [1].

The face is the region of the body that human beings are most identifiable with [2]. The skull morphology determines the features of the face, so there are different angles utilized to characterize the separate facial parts. The various degrees of projection or flattening of the different facial bones, such as zygomatic bones, alveolar part of the maxilla, nasal bones, mandible, and so forth, give the basic outline of the facial profile and play an important part in the determination of racial affiliation [35] as well as the process of facial reconstruction [1, 3, 6]. Although, as a part of the neurocranium, the frontal bone characterized by its slope and the prominence of the supraorbital ridge also contributes to the overall perception of the facial appearance. The frontal bone shows significant variations due to ancestry, which makes its metric characteristics a key object of investigation in human evolutionary studies [7, 8]. However, the middle part of the viscerocranium is most used for ancestry determination [9]. Norman [2] has summarized that neurocranium more probably represents phylogenetically acquired traits, while the morphology of middle and lower face reflects to a great extent adaptive responses to climatic circumstances and dietary specialization, as the upper face features are less affected by such influences. Hence, the angular characteristics could contribute to identifying the most discriminant regions between hominid fossils. Furthermore, some angles show differences in both sexes, such as the angle of the frontal bone [10], the mandibular angle [11], and the zygomatic angle [12]. Therefore, the skull angles appear to be very important quantitative indicators in various anthropological aspects.

Most of the angles are taken between a line joining definite craniometric landmarks and the Frankfurt horizontal plane (FH), determined by both landmarks of porion and the left orbitale. Other angles are constructed between two lines passing through definite anthropometric landmarks. The measurements of most angles require a set of craniometric instruments or craniograms, necessitating a craniograph. Skull angles could also be measured on profile X-rays. However, this option should be applied with discretion in work with bone remains from archaeological excavations or forensic cases, since the exposure of bone to clinical levels of radiation reduces the amount of amplifiable DNA [13]. Besides the abovementioned measuring methods, there are angles, which are calculated through a nomogram based on corresponding linear measurements. All this makes the measuring of angles a more complicated task compared to the common linear measurements. However, the 3D models of human skulls created by different imaging technologies facilitate the measuring of angles with the help of computer software. In this connection, because of the increasing application of hand-held laser scanners in paleoanthropology and forensic anthropology, the assessment of the reliability and accuracy of the digitally taken skull angles is an important topic for the future anthropometric investigations based on 3D models.

2. Materials and Methods

The study was conducted on five well-preserved adult skulls randomly selected from the osteological collection at the Institute of Experimental Morphology, Pathology, and Anthropology with Museum, Bulgarian Academy of Sciences.

Two measuring methods were applied: (1) direct measuring method, based on the conventional craniometry, and (2) digital measuring method, accomplished on 3D models created by laser scanning, with a software program. Seven angular characteristics were measured on both dry skulls and 3D models (Table 1). According to the conventional method, five of the angles were taken between definite craniometric measurements and the FH using a standard sliding caliper, a goniometer, and a Mollison craniophor (after Martin and Saller [14]). The nasomalar and zygomaxillary angles were calculated on the basis of a nomogram as a trigonometric function of two linear measurements (after Alekseev and Debets [15]). The first angle was determined based on the frontomalar orbital breadth and the distance from nasion to the line of the breadth; and the second angle was calculated between the zygomaxillary breadth and the distance from subspinale to the line of the breadth. The measurements for these two angles were taken with a coordinate caliper.

Table 1: Description of the measured cranial angles after Martin and Saller [14] and Alekseev and Debets [15]. The numbers in brackets of the first five angles correspond to the number of the measurements by Martin and Saller [14].

The 3D models were created using a hand-held laser scanner Creaform VIUscan. The skulls were scanned without mandibles. The scanning was performed at a resolution of 0.70 mm and a texture resolution of 150 DPI. The accuracy of the laser scanner was up to 0.050 mm. The surface image data collected by the laser scanning were postprocessed in the scanner software platform VXelements. The measurements on the 3D models (STL format) were taken using the free software Geomagic Verify Viewer (3D Systems Inc.). The five angles with the FH were measured as angles between a vector (the linear measurement between definite craniometric landmarks) and a plane (FH) (Figure 1(a)). The frontomalar and zygomaxillary angles were measured as angles between 3 points with a vertex at nasion and subspinale, respectively (Figure 1(b)).

Figure 1: Digital measurement of (a) facial profile angle (angle between a vector and a plane) and (b) nasomalar profile angle (angle constructed by 3 points).

All of the 3D models were measured three times by two observers to test the intra- and interobserver reliability. The dry skulls were measured three times by one of the observers to assess the agreement between the directly and digitally obtained measurements. Each set of measurements was taken on a separate day in a random order.

2.1. Statistical Analysis

The intra- and interobserver reliability were assessed using intraclass correlation coefficient (ICC). The intraobserver reliability was computed for each angle based on the triple measurements of each observer. The interobserver reliability was calculated using the repeated measurements taken by both observers.

The differences between digital and direct measurements of all 7 angles were assessed using the formulas for measurement error and relative error:

The positive values indicated higher values for the digital measurements.

3. Results

Almost perfect intraobserver reliability was found for the digital and direct measurements (). The ICCs for the digital measurements ranged from 0.884 to 0.982 for the first observer and from 0.803 to 0.989 for the second one. The ICCs for the direct measurements of the first observer ranged from 0.933 to 0.989. The interobserver correlation coefficients also indicated almost perfect reliability for all measurements, except for the alveolar profile angle showing moderate agreement (Table 2).

Table 2: Intra- and interobserver reliability (ICC).

The means of the digitally and directly measured angles as well as the measurement and relative errors are presented in Table 3. All angles were measured with higher values on the 3D models compared to the dry skulls. The measurement errors were below 1° in six of the angles, except for the nasomalar angle. A relative error exceeding 1% was observed in the facial and alveolar profile angles. The overall measurement error between digitally and directly measured angles was 0.61° with a standard deviation of 0.99°. The overall relative error was 0.68% with a standard deviation of 1.17%.

Table 3: Means of the digitally and directly taken angles based on the two trials of both observers. Measurement and relative errors calculated on the basis of the digital and direct measurements of the first observer.

4. Discussion

Cranial landmarks are defined as biologically meaningful loci that can be easily identified and repeatedly located with a high degree of accuracy and precision [17]. According to Bookstein [16], the landmarks have been classified into 3 types: type I landmarks are defined by biological criteria, representing juxtaposition of tissues, such as suture intersections; type II are determined as those located at geometric maxima of bony protrusions or depressions; type III are defined by geometric criteria and are determined by other landmarks or plains. This classification has been widely used in craniometric studies, which assess the precision in the location of the landmarks depending on used technologies, such as digitizing [1821], laser scanning [20], and CT [19, 21, 22]. Type I landmarks have been more precisely identified using a 3D digitizer, while type III landmarks have been lowly reproducible with this technology [1820]. Identifying type III landmarks on CT reconstructions has been found to be problematic as well [22]. However, on 3D laser scanner models, landmarks of this type have yielded the most precise coordinate data [20] and the measurements between them have been found to be very consistent [23]. Type II landmarks have shown the most consistent measurement precision using different technologies [20]. Since the landmarks of different types have produced different coordinate measurement errors, affecting the precision and accuracy of the linear measurements taken between them, we aimed to check if angles including definite types of landmarks would show some common effect on their reliability and accuracy.

Most of the studies devoted to the accuracy of metric characteristics obtained on 3D models of human bones have been focused on the linear cranial measurements. The laser scanning created 3D models have been reported to give reliable linear measurements comparable with the conventional ones [23, 24]. Furthermore, precise results have been established for the cranial volume and surface area based on 3D laser scanning technology [25]. The 3D volumetric representations obtained by traditional CT [23, 26, 27] and cone beam CT (CBCT) [2832] scans have also provided accurate and reliable digital measurements. However, there have been few comparative studies including angular characteristics. Moreira et al. [31] have established reliable and accurate maxillofacial angular measurements on 3D CBCT images. In other studies, the angle measurements on conventional cephalometric radiographs have been compared to their measuring on traditional CT [33] and CBCT 3D models [34] or with radiographs synthesized from CBCT volume data [35]. However, results on the reliability and accuracy of skull angular characteristics obtained on 3D laser scanner models have not been reported, in spite of their importance for a variety of anthropological studies. Our results demonstrated excellent intra- and interobserver reliability, similar to the results of the other authors for the linear 3D measurements. The alveolar profile angle was the only one in our study showing excellent intraobserver repeatability but moderate interobserver agreement. Thus, the lowest result on reliability was observed for an angle constructed from type II (linear measurement) and tType III (FH) landmarks.

Six of the seven angles in our study were measured with a difference within 1°. The lowest measurement error of 0.04° was observed for the nasal profile angle and the highest one of 1.12° was established for the nasomalar angle. The SD of the measurement errors of four of the angles exceeded 1°, as the largest SD of 1.26° was observed for the alveolar profile angle. The other angles with SD above 1° were the nasal and frontal profile angles as well as the zygomaxillary angle. The landmarks in the alveolar profile angle and zygomaxillary angle represented mainly type II, except these of the FH in the alveolar profile angle. However, type II landmarks have not been reported as highly variable on laser scanning created 3D models [20]. There is a possibility that the deviations of the angles, including FH, could be prompted by the placement of the FH landmarks, since the plane was constructed independently for each measurement. However, Daboul et al. [36] have stated that the FH is a sufficiently stable landmark-based reference plane, revealing excellent intra- and interobserver reliability of the plane identification in magnetic resonance imaging, with the FH angulation differing from 1.0° to 1.5° among observers. Taking into account the mean measurement error of ° for the angles including the FH and the fact that the differences between both measuring methods were determined not only from the FH but also by the share of the related linear measurement, it is likely that the plane did not show high variance in its placement. Moreover, the FH is constructed by type III landmarks, which were suggested as most suitable reference points for plane-based morphometric analysis based on 3D laser scanner models [20]. Therefore, the larger deviation in the angles with the FH could be due to the identification of the landmarks included in the linear measurements. It should be mentioned that these angles had larger deviation of the relative error (SD ≥ 1%) than the three-point angles. This result could be relevant to the study of Hale et al. [21] reporting greater variation in the midline landmarks (mainly type I and type II). However, according to the observed relative errors, it could not be summarized that angles including definite type landmarks gave better results than others, since the angles with the smallest relative error were the nasal profile angle with types I and II landmarks for the linear measurement and type III for the FH; the frontal profile angle including landmarks types I and III; and the zygomaxillary angle including only type II landmarks. The highest SD obtained for the relative errors of the alveolar profile angle and frontal slope angle could be explained with the lower true values of these two angles.

Human imprecision is an inevitable source of error in the method comparison studies. Unlike the linear measurements requiring the placement of two points, in the cases of angles, there is a placement of a minimum of 3 points, as the common case refers to an angular characteristic including a linear measurement (2 points) and a plane (3 points). This suggests an increase of the human error. However, it was not observed in our study that angles constructed by 3 points provided a smaller measurement error than the ones built by the placement of more points. In addition, it should be noticed that the observed intermethod differences could result from the different precision of the conventionally acquired data, constrained up to a degree, and the digital measurements obtained with the software (with precision to the fourth decimal place).

The acceptable range of measurement error for linear measurements has been reported to be within 2 mm for anthropological purposes [26]. If an equivalent range was admitted for the angle characteristics (up to 2°), our results could be reported as completely acceptable considering the magnitude of the measurements. Nevertheless, the observed measurement errors should be taken into account especially in cases when values are very close to the borderlines of angles’ categories, since it could lead to wrong conclusion if the skulls are examined by their categorizations. It should be also noticed that all angles were predominantly measured with higher values on the 3D models, compared to the directly taken ones.

5. Conclusions

As a whole, our results showed that the angular characteristics obtained on 3D laser scanner models were highly reliable and comparable to the direct measurements. Thus, the 3D models provide an easier measuring of a variety of angles. Moreover, there is a possibility to be taken angles, different from the conventional ones, which are impossible to be measured directly but would be of importance for different anthropological studies.

Competing Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.


This study was supported by the “Program for Support for Young Scientists in Bulgarian Academy of Sciences,” Research Grant DFNP-75/27.04.2016.


  1. G. Quatrehomme and G. Subsol, “Classical non computer-assisted craniofacial reconstruction,” in Computer Graphic Facial Reconstruction, J. G. Clement and M. K. Marks, Eds., pp. 15–32, Elsevier Academic Press, New York, NY, USA, 2005. View at Google Scholar
  2. B. Norman, “Facial prognathism in the hominid and human species,” 1999,
  3. M. M. Gerasimov, Facial Reconstruction from the Cranium, Works of the Institute of Ethnography, 28, Akademiia Nauk SSSR, Moscow, Russia, 1955.
  4. Y. Y. Roginskii and M. G. Levin, Antropologiia, Vysshaya Shkola, Moscow, Russia, 1978.
  5. D. R. Johnson, P. O'Higgins, W. J. Moore, and T. J. McAndrew, “Determination of race and sex of the human skull by discriminant function analysis of linear and angular dimensions,” Forensic Science International, vol. 41, no. 1-2, pp. 41–53, 1989. View at Publisher · View at Google Scholar · View at Scopus
  6. J. Jordanov, Head Reconstruction by the Skull, Marin Drinov Academic Publishing House, Sofia, Bulgaria, 2003.
  7. X. Wu and M. Wu, “Early Homo sapiens in China,” in Paleoanthropology and Paleolithic Archaeology in the People's Republic of China, R. Wu and J. Olsen, Eds., pp. 91–105, Academic Press, London, UK, 1985. View at Google Scholar
  8. S. Athreya, “The frontal bone in the genus Homo: a survey of functional and phylogenetic sources of variation,” Journal of Anthropological Sciences, vol. 90, pp. 1–22, 2012. View at Google Scholar
  9. J. Wiegand, Ancestry determination from the cranial base angle [Ph.D. thesis], The University of Hawaii at Manoa, Honolulu, Hawaii, USA, 2014.
  10. M. D. Russell, T. Brown, S. M. Garn et al., “The supraorbital torus: a most remarkable peculiarity,” Current Anthropology, vol. 26, pp. 337–360, 1985. View at Publisher · View at Google Scholar
  11. B. A. Williams and T. L. Rogers, “Evaluating the accuracy and precision of cranial morphological traits for sex determination,” Journal of Forensic Sciences, vol. 51, no. 4, pp. 729–735, 2006. View at Publisher · View at Google Scholar · View at Scopus
  12. T. Ikeda, M. Nakamura, and M. Itoh, “Sex differences in the zygomatic angle in Japanese patients analyzed by MRI with reference to moire fringe patterns,” Aesthetic Plastic Surgery, vol. 23, no. 5, pp. 349–353, 1999. View at Publisher · View at Google Scholar · View at Scopus
  13. B. M. Grieshaber, D. L. Osborne, A. F. Doubleday, and F. A. Kaestle, “A pilot study into the effects of X-ray and computed tomography exposure on the amplification of DNA from bone,” Journal of Archaeological Science, vol. 35, no. 3, pp. 681–687, 2008. View at Publisher · View at Google Scholar · View at Scopus
  14. R. Martin and K. Saller, Lehrbuch der Anthropologie in Systematicsher Darstellung, Band I, Gustav Fischer, Stuttgart, Germany, 1957.
  15. V. P. Alekseev and G. F. Debets, Craniometry: Technique of Anthropological Researches, Nauka, Moscow, Russia, 1964.
  16. F. Bookstein, Morphometric Tools for Landmark Data: Geometry and Biology, Cambridge University Press, Cambridge, UK, 1991.
  17. J. T. Richtsmeier, C. H. Paik, P. C. Elfert, T. M. Cole III, and H. R. Dahlman, “Precision, repeatability, and validation of the localization of cranial landmarks using computed tomography scans,” Cleft Palate-Craniofacial Journal, vol. 32, no. 3, pp. 217–227, 1995. View at Google Scholar · View at Scopus
  18. A. H. Ross and S. Williams, “Testing repeatability and error of coordinate landmark data acquired from crania,” Journal of Forensic Sciences, vol. 53, no. 4, pp. 782–785, 2008. View at Publisher · View at Google Scholar · View at Scopus
  19. A. H. Ross, D. E. Slice, and S. E. Williams, “Geometric morphometric tools for the classification of human skulls,” Document 231195, Department of Justice, 2010. View at Google Scholar
  20. S. B. Sholts, L. Flores, P. L. Walker, and S. K. T. S. Wärmländer, “Comparison of coordinate measurement precision of different landmark types on human crania using a 3D laser scanner and a 3D digitiser: implications for applications of digital morphometrics,” International Journal of Osteoarchaeology, vol. 21, no. 5, pp. 535–543, 2011. View at Publisher · View at Google Scholar · View at Scopus
  21. A. R. Hale, K. K. Honeycutt, and A. H. Ross, “A geometric morphometric validation study of computed tomography-extracted craniofacial landmarks,” The Journal of Craniofacial Surgery, vol. 25, no. 1, pp. 231–237, 2014. View at Publisher · View at Google Scholar · View at Scopus
  22. J. Barbeito-Andrés, M. Anzelmo, F. Ventrice, and M. L. Sardi, “Measurement error of 3D cranial landmarks of an ontogenetic sample using Computed Tomography,” Journal of Oral Biology and Craniofacial Research, vol. 2, no. 2, pp. 77–82, 2012. View at Publisher · View at Google Scholar
  23. A. H. Richard, C. L. Parks, and K. L. Monson, “Accuracy of standard craniometric measurements using multiple data formats,” Forensic Science International, vol. 242, pp. 177–185, 2014. View at Publisher · View at Google Scholar · View at Scopus
  24. H.-K. Park, J.-W. Chung, and H.-S. Kho, “Use of hand-held laser scanning in the assessment of craniometry,” Forensic Science International, vol. 160, no. 2-3, pp. 200–206, 2006. View at Publisher · View at Google Scholar · View at Scopus
  25. S. B. Sholts, S. K. T. S. Wärmländer, L. M. Flores, K. W. P. Miller, and P. L. Walker, “Variation in the measurement of cranial volume and surface area using 3D laser scanning technology,” Journal of Forensic Sciences, vol. 55, no. 4, pp. 871–876, 2010. View at Publisher · View at Google Scholar · View at Scopus
  26. K. E. Stull, M. L. Tise, Z. Ali, and D. R. Fowler, “Accuracy and reliability of measurements obtained from computed tomography 3D volume rendered images,” Forensic Science International, vol. 238, pp. 133–140, 2014. View at Publisher · View at Google Scholar · View at Scopus
  27. D. Lorkiewicz-Muszyńska, W. Kociemba, A. Sroka et al., “Accuracy of the anthropometric measurements of skeletonized skulls with corresponding measurements of their 3D reconstructions obtained by CT scanning,” Anthropologischer Anzeiger, vol. 72, no. 3, pp. 293–301, 2015. View at Publisher · View at Google Scholar
  28. S. Baumgaertel, J. M. Palomo, L. Palomo, and M. G. Hans, “Reliability and accuracy of cone-beam computed tomography dental measurements,” American Journal of Orthodontics and Dentofacial Orthopedics, vol. 136, no. 1, pp. 19–25, 2009. View at Publisher · View at Google Scholar · View at Scopus
  29. M. Berco, P. H. Rigali Jr., R. M. Miner, S. DeLuca, N. K. Anderson, and L. A. Will, “Accuracy and reliability of linear cephalometric measurements from cone-beam computed tomography scans of a dry human skull,” American Journal of Orthodontics and Dentofacial Orthopedics, vol. 136, no. 1, pp. 17.e1–17.e9, 2009. View at Publisher · View at Google Scholar · View at Scopus
  30. B. Hassan, P. van der Stelt, and G. Sanderink, “Accuracy of three-dimensional measurements obtained from cone beam computed tomography surface-rendered images for cephalometric analysis: influence of patient scanning position,” European Journal of Orthodontics, vol. 31, no. 2, pp. 129–134, 2008. View at Publisher · View at Google Scholar · View at Scopus
  31. C. R. Moreira, M. A. O. Sales, P. M. L. Lopes, and M. G. P. Cavalcanti, “Assessment of linear and angular measurements on three-dimensional cone-beam computed tomographic images,” Oral Surgery, Oral Medicine, Oral Pathology, Oral Radiology and Endodontology, vol. 108, no. 3, pp. 430–436, 2009. View at Publisher · View at Google Scholar · View at Scopus
  32. K. Kamburoglu, E. Kolsuz, H. Kurt, C. Kiliç, T. Özen, and C. S. Paksoy, “Accuracy of CBCT measurements of a human skull,” Journal of Digital Imaging, vol. 24, no. 5, pp. 787–793, 2011. View at Publisher · View at Google Scholar · View at Scopus
  33. R. Nalçaci, F. Öztürk, and O. Sökücü, “A comparison of two-dimensional radiography and threedimensional computed tomography in angular cephalometric measurements,” Dentomaxillofacial Radiology, vol. 39, no. 2, pp. 100–106, 2010. View at Publisher · View at Google Scholar · View at Scopus
  34. O. J. C. van Vlijmen, T. Maal, S. J. Bergé, E. M. Bronkhorst, C. Katsaros, and A. M. Kuijpers-Jagtman, “A comparison between 2D and 3D cephalometry on CBCT scans of human skulls,” International Journal of Oral and Maxillofacial Surgery, vol. 39, no. 2, pp. 156–160, 2010. View at Publisher · View at Google Scholar · View at Scopus
  35. V. Kumar, J. B. Ludlow, A. Mol, and L. Cevidanes, “Comparison of conventional and cone beam CT synthesized cephalograms,” Dentomaxillofacial Radiology, vol. 36, no. 5, pp. 263–269, 2007. View at Publisher · View at Google Scholar · View at Scopus
  36. A. Daboul, C. Schwahn, G. Schaffner et al., “Correction: reproducibility of frankfort horizontal plane on 3D multi-planar reconstructed MR images,” PLoS ONE, vol. 7, no. 11, Article ID e48281, 2012. View at Google Scholar