Abstract
The main objective of the facial edema evaluation is providing the needed information to determine the effectiveness of the anti-inflammatory drugs in development. This paper presents a system that measures the four main variables present in facial edemas: trismus, blush (coloration), temperature, and inflammation. Measurements are obtained by using image processing and the combination of different devices such as a projector, a PC, a digital camera, a thermographic camera, and a cephalostat. Data analysis and processing are performed using MATLAB. Facial inflammation is measured by comparing three-dimensional reconstructions of inflammatory variations using the fringe projection technique. Trismus is measured by converting pixels to centimeters in a digitally obtained image of an open mouth. Blushing changes are measured by obtaining and comparing the RGB histograms from facial edema images at different times. Finally, temperature changes are measured using a thermographic camera. Some tests using controlled measurements of every variable are presented in this paper. The results allow evaluating the measurement system before its use in a real test, using the pain model approved by the US Food and Drug Administration (FDA), which consists in extracting the third molar to generate the facial edema.
1. Introduction
Inflammation is understood as the body's protective response to injuries [1]. If this response is not controlled, it could cause more harm than good, and this occurs in different inflammatory diseases. Chronic inflammatory diseases, collectively, represent the most significant component in terms of human suffering and economic cost in the first world, one in three people is affected by them.
The rapid advances in the inflammatory diseases knowledge and the related medicine have led to new medical treatments and have reduced the suffering in millions of patients [2].
In the anti-inflammatory drugs, development is necessary to identify specific biomarkers and measure the progression of the inflammatory disease to evaluate the medical results and the safety of the treatment; however, the developed measurement systems do not have an adequate reliability.
Currently there are five representative cardinal signs of inflammation [3], as follows.(i)Swelling: increased interstitial fluid and edema formation.(ii)Blushing: redness, due mainly to the phenomena of increasing pressure by vasodilation.(iii)Heat: increment of temperature in the swelled area due to the vasodilatation and increased local consumption of oxygen.(iv)Pain: the pain appears as a consequence of releasing substances capable of causing the activation of nociceptors, such as prostaglandins.(v)Lost or decreased function (trismus).
Figure 1 shows the inflammatory process and the location of its five cardinal signs.
The development of drugs to control and reduce inflammation caused by damage to the human body is continuously performed. These drugs, called nonsteroidal anti-inflammatory drugs (NSAIDs), are a group of drugs that block the prostaglandins synthesis provoking the following effects: anti-inflammatory, analgesic (reduce pain), and antipyretic (reducer fever), [4]. Products that have these properties differ in their side effects and the duration of the effect.
Within the research and development of anti-inflammatory drugs, there are generic models to analyze their effects. The third molar extraction pain model is recognized by the US Food and Drug Administration (FDA), and it is frequently used to evaluate analgesic and anti-inflammatory drugs [5].
The techniques currently used to measure the effectiveness of anti-inflammatory drugs are based on vernier calipers. Other techniques are based on using the length of a thread to measure the inflammation distance strategically located between some marked points in the face of the patient or to measure the distance between two points strategically located on the face.
Based on the need to obtain accurate measurements of the effectiveness of anti-inflammatory drugs, the techniques currently used to measure the signs present in the postoperative edema, due to a third molar extraction, require a technological improvement to get more reliable and trustworthy measurements of these variables.
This paper focuses on the current scientific interest over the area of pathological inflammatory diseases related to the development of a reliable system that allows studying the measurement of four main variables that influence an adequate edema evaluation: trismus, blushing, temperature, and inflammation, having as a possible application the measurement of the effectiveness of the anti-inflammatory drugs.
2. Current Methods Used in the Measurement of Variables That Represent a Facial Edema
After the extraction of the third molar, the attack on the oral and perioral soft tissues and detachment of the flap and bone trauma are responsible for the inflammation that, under normal conditions, is more intense 72 hours after the extraction. After this period, inflammation begins to subside and decreases during 48 or 72 hours [6]. This period is used to evaluate the variables present in the postoperative edema.
Determining the effectiveness of anti-inflammatory drugs is currently based on measuring only two of the variables presented in the postoperative edema: swelling and trismus, by using different techniques.
2.1. Trismus Measurement Methods
Historically, different methods to measure trismus have been used. van Gool et al. [7] used a compass instrument, later replaced with a millimeter ruler, to measure the value of the maximal mouth opening. Sánchez Sánchez [8] performed a direct measurement of the mouth opening with a millimeter ruler; other forms to measure the mouth opening are the Willis gauges employed by Yates et al [9].
Currently, trismus assessment is carried out by measuring the maximum interincisal opening using normal and digital Vernier calipers.
Figures 2(a) and 2(b) show the use of the mentioned tools to measure trismus.
(a) Caliper vernier
(b) Millimeter ruler
2.2. Swelling Measurement Methods
Inflammation is a three-dimensional volumetric change difficult to evaluate. Some of the methods that have been used to determine the swelling due to the third molar extraction are as follows.(i)Laskin method [10], which is still used by many researchers, involves performing three measurements during the evaluation period (7 days). These measurements are distributed at the following times: one immediately after surgery, another after 24 hours, and the last 7 days after retiring the surgical stitches. In order to measure the evolution of the inflammation at the determined points, these are marked with a dermographic pencil and a 00-thickness suture thread fixed with two clamps (surgical clips). Measurements are made between the interest points marked with the dermographic pencil. The reference points and distances to be measured are as follows. (a)The distance in centimeters from the bottom edge of the earlobe to the midpoint of the symphysis Hirota, called: horizontal distance to the symphysis (DHS). (b)The distance in centimeters from the bottom edge of the earlobe to the external angle of the mouth, called: horizontal distance to the corner (DHC). (c)The distance in centimeters from the palpebral outboard angle to the goniaco angle, called: vertical distance (DV).
Figure 3 shows the different distances used to measure inflammation by the Laskin method.
Other methods to measure inflammation are based on the following.(ii)Calibrators [11] which measure the swelling in one dimension (Figure 4). (iii)Photographic method and cephalostat [12] is perhaps the most accurate method; however, it is very complicated to implement and requires a series of auxiliary equipment that makes the system more expensive. This method was designed by van Gool et al. in 1975 [13] to demonstrate the lack of precision and consistency of the subjective measurements of the inflammation. Figure 5 shows the measurement of inflammation using the photographic method and a cephalostat. (iv)Facebows are complicated handling equipments, such as the ones used by Petersen [14]. Figure 6 shows the measurement of inflammation using a facebow.
3. System Proposed to Measure Variables in Facial Edemas: Trismus, Blush, Temperature, and Swelling
There is an area of opportunity to improve the application of new technologies to measure the different variables that determine the effectiveness of anti-inflammatory drugs in facial edemas. This improvement area is based on the measurement of trismus, blush, temperature, and swelling, present in the facial edema, using new technologies and techniques to obtain more accurate and reliably measurements than the ones provided by the commonly used techniques.
3.1. Method Proposed to Measure Trismus
The trismus is evaluated by measuring the maximum interincisal opening. The proposal presented in this paper is based on the measurement of trismus through digital image processing using the MATLAB software.
The developed technique requires locating the face of a person in a digital image, which is acquired at a fixed distance between the camera and the person by using a metric reference standardized to this distance. Once the image of the person mouth opening is ready, it is necessary to select the distance in the image to be measured using two points selected by the user.
The pixels of the selected distance are converted to centimeters by using the MATLAB software. The pixels to centimeters conversion resolution of the selected distance in the opening mouth image is defined by the distance between the camera and the captured face image. Figure 7 shows the flowchart used to measure the trismus by converting pixels to centimeters.
The proposed technique can achieve reliability similar to the techniques that use digital Vernier calipers. In the conversion process from pixels to centimeters, a centimeter can be represented by different pixel heights of an image, such as or pixels; the latter is the standard of the images used in this project.
The average of the opening mouth image is between 7 and 8 centimeters, for an image 1 pixel represents 0.013 cm, and this is small enough to obtain an accurate trismus measurement. The advantage of this proposal is that it is a noninvasive technique that only requires capturing an image of the mouth of the person and selecting the distance to analyze, resulting in a representation in centimeters or millimeters of the maximum mouth opening.
3.2. Method Proposed to Measure Blush
This research aims to measure the change in the color (blushing) of the area affected by edema by analyzing histograms of the facial color images.
A digitized image is simply an array of numbers, where every number represents the value of a pixel.
The histogram of an image (f) is a graphic representing the color intensity levels of (f) with respect to the number of pixels present in (f) corresponding to every color intensity; in other words, the histogram of an image is the plot of the values of its pixels. Thus, the histogram is a representation of the probability density of every color value and the derived statistical analysis of the histogram can be used to compare intensities between images.
All the colors are derived from a combination of the RGB (red, green, and blue) colors in different proportions. The color of every pixel in an RGB digital image corresponds to a value between 0 and 255 assigned to every RGB channel for every pixel; that is, every pixel has a value assigned to every color channel (red, green, and blue), and the combinations of these different colors emerge. If the value of a pixel is R255 + G255 + B0, red and green are mixed resulting in yellow. If the value of a pixel is the combination of , the result is the color black.
Histograms have very different aspects depending on the image content. A histogram is probably the best tool to adjust and to analyze digital images allowing knowing the range of the pixels for every RGB channel; in addition, it provides information about the changes in the intensity values that the same image suffers over time.
The methodology consists of the following steps.(i)A person is placed in a cephalostat that allows standardizing the position of the color images, allowing taking similar pictures.(ii)Different images are taken from the face of the person to whom different shades of blush are applied to simulate changes in coloration due to edemas.(iii)The obtained color images are processed using MATLAB to obtain the RGB histograms for every one of the images in order to analyze the changes in the color of the facial skin affected by edema.(iv)Comparisons are made about the RGB histograms of the images with different tones of blush; the result is a correlation between 0% and 100%. If the color is similar between two images, the correlation is approximately 100%; if the color between two images is too different, the correlation is about 50% or 60% for every channel.
3.3. Method Proposed to Measure Temperature
The application of infrared thermography to study the effectiveness of anti-inflammatory drugs is used for detecting temperature changes in the area between the jaw and the temporo-mandibular joint, and this measurement is performed during the evaluation period after the extraction of the third molar.
The images (thermograms) are captured by the FLIR i7 thermographic camera which provides information about the changes in the surface temperature of the patient's facial area once the third molar was extracted. The alteration in temperature corresponds to the radiation generated due to the changes in the internal cellular metabolism.
The process used to obtain the thermograms is as follows.(i)The patient is placed in a cephalostat to position and immobilize the patient's head in a fixed relationship to the digital camera. This equipment standardizes the thermal images (thermograms).(ii)Some thermographic images (thermograms) are taken and stored in the PC for further analysis.(iii)Every thermogram provides basic information about the temperature in every one of the areas of the face images that suffered injuries, having the option to select only a specific area of interest to obtain the average temperature of the specified zone as well as the maximum and minimum temperatures.
The application of a thermographic camera to evaluate changes in the temperature of an edema enables the user to obtain a comprehensive and reliable measurement about the edema evolution, so temperature measurement can be used in the evaluation of the effectiveness of anti-inflammatory drugs.
3.4. Method Proposed to Measure Swelling
The technique proposed in this paper consists in measuring inflammation through a three-dimensional reconstruction of the area affected by facial edema using the structured light technique (fringe projection). This reconstruction can provide measurements of the inflammation evolution through the variations in centimeters or millimeters of the inflammation.
The proposal consists in performing three-dimensional reconstructions of the interest area of the facial edema evolution after the extraction of the third molar, such that it generates the three-dimensional reconstruction difference between the initial state of the face before suffering an edema and the state after suffering it. Every difference between the reconstructions shows the evolution of the inflammation in centimeters or millimeters until the shape of the face returns to its initial state. The system is mainly constituted by the following elements:(i)a laptop or PC,(ii)a digital camera (CCD technology), (iii)a mini projector, (iv)a cephalostat to immobilize the head of the person and to standardize the photographs,(v)software image processing software (MATLAB).
The CCD camera and the mini projector are mounted on a base which allows the mobility and adjustment of both elements.
Figure 8 shows the three-dimensional reconstruction process. First, the devices must be placed as shown in Figure 8, then fringes must be projected on the object and through the phase shift technique, the wrapped phase map is obtained. The wrapped phase map is unwrapped by the path-dependent technique to finally obtain the height map by the triangulation technique [15].
In the inflammation measurement process, the person is placed in a cephalostat that allows taking photographs in the same position every time. Later, the structured light patterns (fringe grill) are sent from the PC to the projector, and pictures of the face are taken with the camera in the same plane of the projection.
Images are recorded and the necessary processing techniques to achieve the 3D reconstruction are applied. Using these techniques allows obtaining the wrapped phase map, the unwrapped phase map, and the real measurements in centimeters or millimeters using triangulation.
The result of the first step is a three-dimensional reconstruction of the affected area. It is necessary to compare these results to the three-dimensional reconstructions of the same area once the inflammation has evolved to obtain the difference in inflammation between two different 3D reconstructions, showing the inflammation evolution.
The proposed digitizing method uses an algorithm developed in MATLAB. A grayscale fringes grill is generated by using cosine values and a defined period, . This grill is projected onto the face surface to be the first reference of height. The phase shift technique is used to obtain the grill through a period of in 4 steps. Once the four pictures are taken, (1) is applied as where represents the surface wrapped phase map and represents the 4 pictures taken in every shift.
Equation (1) gives the wrapped phase map of the reference surface. The same process is applied to the second surface, in this case, the area of the face which suffered swelling due to the facial edema. Two phase maps are obtained.
Once both wrapped phase maps are obtained (reference and swelled facial surfaces), their subtraction is performed. The result of this subtraction is the swelling difference.
The next step consists in obtaining an unwrapped phase map, meaning that the phase map of the surface is represented by values between 0 and .
Applying the algorithm path-dependent method developed in MATLAB the unwrapped phase map is obtained. It is necessary to convert the values of the unwrapped phase map to centimeters or millimeters by applying (2) as
Once the unwrapped phase map is expressed in terms of (height), the matrix is plotted in a three-dimensional image corresponding to the actual measurements of the height difference of the compared facial surfaces.
4. Obtained Results
4.1. Trismus Measurement System Results
4.1.1. Validation of the Trismus Measurement Accuracy
To verify the accuracy of the trismus measuring system, some initial tests were made to measure the diameter of five different Mexican currencies and, based on the fact that their diameters are standardized and accurate, the measurement comparisons between the real value and the measured value are very reliable. Figure 9 shows the diameters of the five different Mexican currencies used to validate the algorithm developed in MATLAB.
Table 1 shows the results of the diameters measurements and the difference between the real and the measured value.
In Table 1 the maximum error obtained is 0.0064 cm. Based on ten different tests the average relative error is 0.1% which is equivalent to 0.003 cm, an acceptable value in the measurement of trismus.
Using the MATLAB guide interface a photograph showing the mouth of the person is selected. Once the image has been chosen, the distance to measure is selected by using the ginput command; this command allows selecting the image first point and saves its location in a [] vector, then the second point is selected and stored in a [] vector. A line is drawn with the command hline and distance between points is determinated in pixels by subtracting the values of over .
When the distance in pixels corresponding to one centimeter is obtained, a division between the distance in pixels of the mouth opening and the distance in pixels corresponding to one centimeter is made. The result of this division is the maximum mouth opening distance (trismus) in centimeters. Figure 10 shows an example of trismus measurement using the developed algorithm.
Several tests were conducted to verify the trismus measurement system. Since the average size of the opening mouth images, used to measure the trismus, is from 7 to 8 centimeters, if an pixels image is used, the final resolution corresponds to 1 pixel equal to 0.013 centimeters, obtaining an accurate trismus measurement.
4.2. Final Trismus Measurement Results
In the trismus measurement methodology a person opens his mouth at different distances (maximum opening, half opening, and almost closed).
Figure 11 shows the trismus measurement for an image showing an almost closed mouth. The distance in centimeters for the mouth opening obtained with the developed program is 2.9707 cm.
Figure 12 shows the trismus measurement for a half opening mouth image. The distance in centimeters for the mouth opening obtained with the developed program is 4.1716 cm.
Figure 13 shows the trismus measurement for a maximum aperture mouth image. The distance in centimeters for the mouth opening obtained with the developed program is 5.9584 cm.
4.3. Final Blushing Measurement Results
In the blushing measurement test three different shades of blush were used to simulate changes in skin color caused by facial edema. In this paper only the results corresponding to the third shade of blush (bigger color difference) are presented (Figure 14).
Figures 15(a), 15(b), and 15(c) show the RGB histograms comparison of the blushing measurements between the reference image (face before suffering edema) and the image with the shade of blush (which represents the last moment of edema evolution before it starts to decrease).
(a)
(b)
(c)
The results obtained in the blush comparison between the reference image and the image with the shade of blush are as follows:(i)correlation percentage of red channel: 93.8694%,(ii)correlation percentage of green channel: 90.379%,(iii)correlation percentage of blue channel: 88.8141%.
4.4. Final Temperature Measurement Results
In the final tests the temperature was increased and decreased in the area where the facial edema was simulated in order to generate the different temperatures present during facial edema.
Figure 16 shows the temperature measurement corresponding to a face at normal temperature. The interest area is marked within a rectangle and the obtained results are the maximum, the minimum, and the average temperatures.(i)Maximum temperature: 36.1°C.(ii)Minimum temperature: 32.4°C.(iii)Average temperature: 35.1°C.
Figure 17 shows the temperature measurement corresponding to the face of a person which suffered an injury.
The interest area is marked within the rectangle and the results are the maximum, the minimum, and the average temperatures.(i)Maximum temperature: 36.6°C.(ii)Minimum temperature: 34.3°C.(iii)Average temperature: 35.8°C.
Figure 18 shows the temperature measurement corresponding to the face of a person with increased temperature in the area of interest (a hot compress was used). The interest area is marked within the rectangle and the results are the maximum, the minimum, and the average temperatures.(i)Maximum temperature: 37.1°C.(ii)Minimum temperature: 34.4°C.(iii)Average temperature: 36.1°C.
Figure 19 shows the temperature measurement corresponding to the face of a person who returned to its normal temperature after the temperature increment. The interest area is marked within the rectangle and the results are the maximum, the minimum, and the average temperatures.(i)Maximum temperature: 36.3°C.(ii)Minimum temperature: 33.9°C.(iii)Average temperature: 35.5°C.
4.5. Swelling Measurement System Results
4.5.1. Accuracy of the Swelling Measurement Validation
To verify the accuracy of three-dimensional reconstruction system twenty reconstructions of a pyramid with a maximum height of 43 mm were performed. Figure 20 shows the grid of fringes projected on the pyramid to obtain the three-dimensional reconstruction.
Figure 21 shows the wrapped phase map of the pyramid.
Figure 22 shows de unwrapped phase map of the pyramid.
Figure 23 shows the three-dimensional reconstruction of the pyramid whit real value of its height.
Table 2 shows the twenty results of the heights measured for every one of the pyramid three-dimensional reconstructions.
Table 2 shows the average absolute and relative error, the standard deviation of the measurements, and the average of the maximum height measured for the twenty three-dimensional reconstructions of the pyramid. Based on Table 2, the accuracy of the three-dimensional reconstruction system is 0.01 cm or 0.1 mm
Figure 24 shows the equipment used at the experimental stage: the Fire-i digital camera and the UNIBRAIN Dell M110 projector.
(a)
(b)
An algorithm for the three-dimensional reconstruction using the fringe projection technique is developed in MATLAB. Figure 25 shows the flow diagram of the process used to reconstruct three-dimensional objects.
Several tests were performed using different objects to validate the three-dimensional reconstruction. Some of the used objects were a pyramid, a mask, and the shape of a child. Initially only simple three-dimensional reconstructions of the objects were performed without obtaining actual measurements of the objects height.
The advantage of the proposed technique is that it can provide more complete and accurate measurements, allowing obtaining the values of the whole area of interest affected by the edema.
4.6. Final Inflammation Measurement Results
Finally, to measure the swelling in the face of a person, the inflammation was simulated by filling the mouth with air (maximum and medium values), and the placing of a ball on the cheek, increasing the volume of the facial area allowing measuring the difference between two reconstructions (reference reconstruction and the increased volume).
Figures 26(a) and 26(b) show the fringe projection on the reference face (face without inflammation) and the face that presents maximum inflammation (through the placing of a ball on the cheek). Within the rectangle the interest area with inflammation is shown.
(a)
(b)
Figures 27(a) and 27(b) show the wrapped phase map and the unwrapped phase map corresponding to the selected area of interest.
(a)
(b)
Figure 28 shows the comparison between the three-dimensional reference facial reconstruction and the facial reconstruction with maximum inflammation through the placing of a ball on the cheek.
5. Conclusions
In the process to improve anti-inflammatory drugs, it is necessary to use techniques and equipment to obtain reliable measurements of the variables that determine its effectiveness: trismus, temperature, blushing, and swelling. Based on the need to improve the traditional measurement techniques an integral system is developed, and it can deliver the measurement of the four variables present in the inflammatory process once the model of the third molar extraction is applied.
To measure swelling the structured-light three-dimensional reconstruction technique is selected based on the study of its characteristics, advantages, and disadvantages. The main advantages are the scanning speed and the accuracy (±0.030 mm).
This technique cannot be used with transparent and reflective surfaces because light does not infer in them the same as in the opaque reflection. In this research the interest areas do not have the properties of transparency or reflectivity, so this technique is suitable for the desired application.
The implementation of this technique provides a 0.1 mm maximum error adequate to validate the precise evolution of inflammation and by consequence the effectiveness of anti-inflammatory drugs.
Based on the operating principle of the colorimeter which is the application of three color filters: red (R), green (G), and blue (B), the proposal of this research is to make a system to obtain the color difference of patient face images and to make a comparison of the histograms corresponding to the RGB channels, making it possible to identify changes in the face coloration under different swelling levels. It can be concluded that the system allows obtaining accurate results while the illumination of the environment is controlled, that is, under normal test situations because the measurements are usually performed under controlled protocols.
The implementation of a technique to convert pixels to centimeters, using digital image processing, allows a measurement equal or more accurately than the techniques using digital verniers but with the advantage that the proposed system does not require contact with the mouth the patient. By applying the proposed system the measurement of trismus allows obtaining a 0.03 mm error.
Finally, to measure temperature, the proposal is based on using a thermal camera which characteristics allow measuring temperature changes in the affected zone by the postoperative edema with an accuracy of 0.2°C and a sensitivity of 0.1°C.
The proposed system to measure the variables present in the facial edema requires placing the patient in a cephalostat and taking the pictures without touching the patient's face, avoiding modifying the affected area.
The processing time is not a drawback in this research because this application does not require real-time processing of the images.
Appendix
This section presents one of the algorithms developed for this project, the trismus measurement main algorithm as follows:(i)[] = ginput;(ii)[] = ginput;(iii) = [];(iv) = [];(v)hline = line(,, “color”, “”);(vi)pixel_distance = -.Conflict of Interests
None of the authors or coauthors of this paper have a direct financial relation with MathWorks, and the use of the MATLAB software is as a tool for the algorithms development.