Abstract

A method of extracting information in estimating heading angle of vision system is presented. Integration of grey-level cooccurrence matrix (GLCM) in an area of interest selection is carried out to choose a suitable region that is feasible for optical flow generation. The selected area is employed for optical flow generation by using Horn-Schunck method. From the generated optical flow, heading angle is estimated and enhanced via moving median filter (MMF). In order to ascertain the effectiveness of GLCM, we compared the result with a different estimation method of optical flow which is generated directly from untouched greyscale images. The performance of GLCM is compared to the true heading, and the error is evaluated through mean absolute deviation (MAE). The result ensured that GLCM can improve the estimation result of the heading angle of vision system significantly.

1. Introduction

Sensors are most the important components in measurement. With the data gathered from sensor, data analysis as well as control strategy implementation can be conducted. Due to the importance of the sensor, there are various works conducted in this field to cater issues that arise in this field, for instance, new design of force sensor [1, 2] and tactile display [3].

Particularly in the field of robotics, sensors provide information about a variable been measured in order to control the robot, which is most critical part. This information is processed in order to decide the way the robot should take an action with good performance. There are various kinds of sensors available for this purpose, for example, sonar sensor, position sensor, infrared sensor, and camera. Employment of a camera as a sensor, also known as robot vision system, is an interesting idea due to its capabilities to obtain useful information from surroundings through image acquisition. Depending on the application, extraction of information gathered from robot vision system can be executed in various ways. The extraction process commonly utilises image processing and analysis.

A research in the field of vision system is ranging from hardware development and testing, algorithm development specifically on image processing and analysis, and strategy proposal regarding the way to use vision system for a specific task and its implementation. Conducted previous works include manipulation of image acquisition capabilities of the optical mouse to be used as a sensor [46], utilization of panoramic image in visual navigation [7], log-polar imaging application in robotic vision [8], application of sparse visual data without odometry reading [9], obstacle avoidance strategy [10], visual mapping [11], and strategy for robust real-time vision [12].

In this paper, a new approach has been proposed in extracting information regarding the heading angle of vision system using optical flow, with the belief that this work will be beneficial especially in robot navigation. Since it is well understood that some flow vectors generated using this method also contained erroneous vectors which are not representing actual motion, it is necessary to improve the interpretation by integrating a new approach. Here, we proposed the use of grey-level cooccurrence matrix (GLCM) [13] together with optical flow computation as an improvement technique in heading angle estimation. By dividing the image into several areas of interest (AOI), the selection of the most suitable area will be handled by GLCM before an optical flow calculation is made. Additionally, from generated an optical flow vector, heading angle estimation is implemented, and this estimation is further smothered using moving median filter (MMF) in order to eliminate the outlier existed in estimation. The estimation performance is carried out using mean absolute deviation (MAE), and the results from the proposed technique are compared with the estimation result using unmodified optical flow vector field.

2. Grey-Level Cooccurrence Matrix

Grey-level cooccurrence matrix is one technique in texture feature extraction [14]. In general, texture in image shows a repeating pattern of local variation in intensity. It provides information in the spatial arrangement of colours or intensities in an image, characterized by the spatial distribution of intensity levels in a neighbourhood. GLCM is implemented based on statistical approach, where the texture is manipulated as a quantitative measure of the arrangement of intensities in a region. Various works have been done which take a benefit of GLCM, such as in [1518].

Specifically, GLCM is defined as tabulation of occurrence frequency with regarding to a different combination of pixel intensity values in an image. In this method, consideration has been given to the relation between two pixels at a time which is called a reference and neighbour pixel. This relationship is displayed in the form of cooccurrence matrix. Turning it into an equation, it can be described as

Here, is denoted as the number of occurrence of the pixel values posses at length in the image. The occurrence matrix has dimension , where is denoted as the number of grey levels in the image. The grey levels in the image are specified under quantization. The elements of are then normalized using the following equation:

Through normalization, elements in matrix have a value which is in range between zero and one, which allows to be manipulated as a function of probability.

Through GLCM, texture properties can be captured. Using cooccurrence matrix, feature vector can be extracted in several ways. In this paper, contrast has been selected in extracting feature vector. Contrast is a measure of the local variations present in an image, which are represented as

3. Optical Flow

Optical flow technique is commonly used in motion detection and estimation. Optical flow equation is derived based on the assumption that the brightness is preserved unchanged over an entire image sequence and is represented as

Here are denoted as spatial and temporal gradient which can be estimated from image sequence. Meanwhile are denoted as optical flow vector and they are unknowns. Since optical flow equation can be acquired from only a single pixel, hence, these unknowns cannot be solved directly. Regarding this matter, several methods have been proposed in helping to find a solution of these unknowns. In this paper, the method that has been selected is Horn-Schunck method [19].

In Horn-Schunck method, in order to solve these unknowns, optimization constraint is introduced with the assumption that(A1) motion stays constant within pixel neighbourhood;(A2) optical flow differs smoothly.

The optimization constraint is defined as

Here, is a weighting factor, and are partial derivatives of the optical components and . Moreover, computation of is implemented using the following equations:

In terms of partial derivatives of the optical components and , it can be computed from the 4-adjacent position to the position . Therefore,

In addition, through differentiating and minimizing (5) over and at position , the equation can be rewritten as

Here,

From (8), the unknowns are and . The relationship between , , and , can be observed. Therefore, iterative scheme can be constructed based on this relationship, as shown in the following equation:

In (10), is described as iteration step, and are described as the neighbourhood mean of the flow vector , respectively.

4. Heading Angle Estimation Strategy

In this study, a static environment is selected as an experimental scenario. A camera is shifted over horizontal flat surface from left to right, as shown in Figure 1.

Using this camera, a video is captured, and the image sequence is extracted. Extracted images have a resolution of pixels. At initial stage, all images are converted to greyscale format and then resized to pixels. Each converted image is divided into nine AOI. Since GLCM is employed, the intensity of the image needs to be scaled to several grey levels. In this study, the image has been scaled to eight grey levels. In each AOI, the GLCM is generated, and a feature vector is extracted using contrast criterion. By using computed feature vector, a suitable AOI is selected.

In the next stage, optical flow field vector is generated using Horn-Schunck method. Vector operation is employed where each flow vector is decoupled to horizontal and vertical components. Summation of each decoupled vector regarding horizontal and vertical axes is implemented, and angle estimation is computed. MMF is applied to acquire smoothness of the estimated angle.

In order to assess the estimation performance, the evaluation criterion used is mean absolute error (MAE). Selection of this criterion is due to the fact that MAE is more robust to outliers, compared to other popular methods such as root mean square error (RMSE) [20]. For that purpose, estimation gathered from the proposed method is compared with the actual value, which is 180°. MAE is defined as follows: where

Here, is the estimation value and is the true value.

5. Results and Discussion

In this experiment, ten images were captured by moving the camera in each second. One hundred images have been captured. The average processing time, starting from image preprocessing until angle estimation, is 0.02 sec. This is the average value based on the computation time to process one hundred images. This processing time is much less than the image sampling time of 0.1 sec. By using GLCM and feature vector extraction based on contrast, the AOI was selected for each image. The example of selected AOI is shown in Figures 2 and 3.

In the AOI selection, the values of feature vectors were calculated, and median value was selected. The reason why median value was decided to be used is because, after operation of GLCM and contrast, diagonal value of cooccurrence matrix is equal to zero, which means that similar grey-level values, occurred next to one another, were excluded. It is also understood that higher contrast value is caused by the effect of large contrast weights. This shows that large deviation on grey-level values occurred next to one another. In this test, the image contrast changed along with the camera movement. This is due to the changes in the image texture, which was affected by the changes of environment structure. Therefore, the AOI which shows the median value among the values generated by GLCM and contrast will be selected. However, the decision is made under the assumption that the contrast in the selected AOI does not have significant contrast difference over the time.

As a result of using the selected AOI, computed optical flow field is shown in Figure 4.

In addition, for comparison purpose, optical flow field created using an image without AOI is shown in Figure 5.

In Figure 5, there exist vectors whose directions are scattered. It is also observed that there are vectors whose its magnitude is too small. These types of vectors are called mislead vectors and irrelevant vectors, respectively. Mislead vectors may cause large errors in estimation because it is not related to an actual motion. The reason why these vectors exist is because of an assumption of brightness constancy, where, in a real situation, this is not easily maintained, especially in outdoor environment. On the other hand, irrelevant vectors show that there is almost small or no motion detected at vectors location. This is due to image texture at that location, which is remaining constant. For that reason, through the proposed method, improvement in optical flow generation is discovered.

Using these results, heading angle estimation is figured out. The estimation results are given in Figures 6 and 7, accordingly.

From Figures 6 and 7, it can be noticed that heading angle estimation shows an improvement when proposed GLCM is applied. Without GLCM, as shown in Figure 7, the estimation fluctuated over time, where extremely large spikes occurred that caused to poor estimation. Hence, through GLCM method, significant oscillation reduction is observed which makes the estimation result improved.

Related to estimation performance, Table 1 shows the results of evaluation conducted using criteria that have been mentioned in Section 4.

From Table 1, the evaluation results of the heading angle estimation which incorporates GLCM show better performance. MAE result without GLCM shows large deviation existence between estimation and actual value. It is because calculated absolute error is contributed from numbers of large oscillations as well as mediocre but rapid oscillations, as shown in Figure 7. On the other hand, the MAE value for proposed GLCM is smaller. This small MAE value is due to the estimation result in Figure 6 which shows large deviations between estimation and actual value that occurred at the beginning, and the deviations become smaller as the estimation becomes stable and reaches actual value. Therefore, most errors that contributed to MAE are from large variations between estimations and actual values at the beginning of the experiment.

In addition, estimation results without GLCM in Figure 7 show that the oscillation which occurred frequently caused difficulties to identify the heading angle of the vision system. This problem can be solved when our proposed GLCM method is applied.

6. Conclusion

In this paper, the effectiveness of the proposed method in estimating heading angle of vision system is presented. Each captured image has been divided into nine AOI, and GLCM contrast has been used to extract feature vector in order to identify the most suitable region that can be used for estimation purpose. Optical flow calculated using Horn-Schunck method has been used as a basis in extracting required information. Using generated optical flow, estimation on angle has been implemented and smothered using MMF. The result shows that an improvement in estimation is observed when compared to an angle estimation based on untouched optical flow. Through the proposed method, invaluable information regarding the system’s current status can be obtained.

Acknowledgments

The authors acknowledge the Ministry of Higher Education Malaysia, Universiti Kebangsaan Malaysia, and Universiti Teknikal Malaysia Melaka for their sponsorship in this study.