Abstract
A novel detection algorithm for vision systems has been proposed based on combined fuzzy image processing and bacterial algorithm. This combination aims to increase the detection efficiency and reduce the computational time. In addition, the proposed algorithm has been tested through real-time robot navigation system, where it has been applied to detect the robot and obstacles in unstructured environment and generate 2D maps. These maps contain the starting and destination points in addition to current positions of the robot and obstacles. Moreover, the genetic algorithm (GA) has been modified and applied to produce time-based trajectory for the optimal path. It is based on proposing and enhancing the searching ability of the robot to move towards the optimal path solution. Many scenarios have been adopted in indoor environment to verify the capability of the new algorithm in terms of detection efficiency and computational time.
1. Introduction
Path planning is an essential part of mobile robotic systems to find the optimal path [1–4]. The boundary values of velocity, acceleration, and deceleration of a mobile robot (MR) are defined as closed mathematical forms over the spatial path. These values can be calculated offline for any MR model [1]. In [2], an approach has been proposed to generate a time optimal velocity profile for any path in static or dynamic environments. The proposed algorithms are not time-consuming if the path is planned as smooth curves. A real-time robot path planning approach based on the utilization of inequality and optimization technique has been presented in [3]. The problem of finding a path has been transformed into finding a collision free path without calculating the configuration space obstacles. In addition, many heuristic algorithms have been developed recently to be used for path planning in real world. The fundamental similarities between these algorithms and the motivation behind each one have been introduced in [4]. However, these algorithms can be grouped in many categories such as static algorithms, anytime algorithms, and replanning algorithms [4].
However, computer vision (CV) has become an important tool in modern robot systems since it provides a useful tool for detection and decision-making during robotic navigation missions. In this context, many studies have applied CV for path planning tasks [5–9]. The object detection has been proposed for MRs to perform their missions in the environment [5]. The proposed algorithm in [5] is based on collecting data images from the surrounding environment during the training phase. Whilst the algorithm is measuring the errors in data image, it updates the classifier. As a result, it provides the means of classifying and more reliable object detection. The work in [6] addressed the problem of how making a robot learns natural terrain selectively and exploits its knowledge to estimate the terrain for optimal path planning. The author proposed a scheme that combines vision learning and interaction to give the robot an ability to understand the intricate environment. In addition, particle swarm optimization (PSO) algorithm was applied to search for the optimal path. In [7], a combination between CV and robot path planning has been presented aiming to program free robot applications. Image processing and path planning tools was presented using the Wise-Shop floor framework. The authors in [9] developed a CV system for autonomous MR navigation. A vision sensor was placed on system on-board to detect and localize another robot in the simulated environment. This proposed approach used algorithm to determine the path in order to find the best solution.
However, the classification of CV systems depends on the sophisticated degree of the system itself. Therefore, CV systems would be classified into several categories such as visual tracking and visual navigation [6–9]. Since visual tracking is highly dependent on the edge, many algorithms have been introduced in the literature to develop powerful edge detectors, that is, Canny and Sobel [10–13]. The modified Canny algorithm that uses different sigma values for different sections of the image has been proposed in [14]. Each image is divided into multi-sub-images to improve the accuracy level of the final output. The investigated pixel is calculated and processed by a Gaussian filter with different sigma values. The values of sigma were chosen carefully so that the output image has only the true edge. Moreover, it has been proved that the proposed method gives better results than the original algorithm. However, the histograms of the results were not distributed uniformly and the modified algorithm is still inappropriate for noisy images [15]. The authors of [15] proposed an edge detection process based on smoothing and morphological operations. This approach was executed based on Canny edge detection principle which is represented as a combination of erosion and dilation. In addition, they compared the results with other existing techniques and exhibited that their proposed technique can produce better results. However, the approach is based on probing an image with a 2 × 2 structuring element in which each of the pixels contains a value of zero or one. So, one of the limitations on this approach is that the frame rate cannot reach the required level for real-time applications. Although the proposed algorithms in [14, 15] can be successfully implemented to solve and optimize the edge detection problem, they have inaccurate results and some edges may be lost [14, 15].
Therefore, to avoid such limitations on these algorithms, many researchers have implemented intelligent algorithms such as the fuzzy edge detection. The method of fuzzy edge representation was proposed in [16]. This representation contains the ideal edges which are not affected by any degradation. This method is more general than the classical edge representation method because it defines the location of edges using the fuzzy set theory. Moreover, this model aims is to provide more information about the location of edges and enabling more flexibility in high level processing. However, further mathematical analysis and experiments with various membership functions are needed. In addition, fuzzy image processing is capable of dealing with the uncertainty in the given data by presenting a good mathematical framework to manipulate the drawbacks in edge detection [17, 18]. Fuzzy logic based edge detection has been considered for both smooth and noisy images in [19], where the proposed method utilizes a 3 × 3 mask guided by a set of fuzzy rules. The results were compared with other edge detection algorithms like Sobel, Canny, Laplace of Gaussian, Prewitt, and Roberts. In addition, the tools of fuzzy based edge detection have been described in [20]. In [21], an edge detection method has been proposed based on Sobel algorithm and the generalized type 2 fuzzy logic systems. In addition, to limit the complexity of handling the generalized type 2 fuzzy logic, α-planes theory has been applied [21]. Simulation results have been obtained with the Sobel operator and the generalized type 2 fuzzy logic system.
However, the GA is a very important technique in combinatorial optimization issues [22, 23]. The objective of such algorithm in mobile robot (MR) field is to search for the optimal path and trajectory. In [22], the authors developed the genetic operators for GA to be able to solve path planning problems. They claimed that their proposed algorithm has fast convergence towards the global optimum. Their algorithm depends on three ideas: new mutation operators, matrix coding, and visible space. Even though that algorithm is effective for static environment, it is not effective for dynamic one. Moreover, the modified algorithm has been presented in [23] in order to reduce the processing time. However, these researches showed that the GA can be improved in many ways in order to solve the path planning issues. In addition, some researchers are still investigating this problem to improve the algorithm and explore more solutions. However, all researches mentioned above were accomplished using MATLAB simulations and none of them was implemented in real-time applications. Also, most of them solved the optimal path in terms of single objective. In [24], the initial working path for the robot is obtained by using Dijkstra algorithm. Then ant colony algorithm is adopted to optimize this initial path and the global optimal path is obtained. The experiments are carried out but it was assumed to have a gas hazard area only if the gas concentration is greater than 2%. The global path planning algorithm, bidirectional simultaneous visibility graph construction, and path optimization by are presented in [25]. The algorithm is not constructing a visibility graph before the path optimization to improve the time efficiency of path planning. However, the algorithm was validated only using different simulation environments. In addition, it was assumed that the environments are known and static. Also, they assumed that obstacles positions should be known in advance.
In this paper, an intelligent edge detection algorithm is proposed for robot navigation systems. This work is extended to our previous work which takes the benefits of mixing robots system heterogeneity for navigation and path planning purposes [26]. The proposed algorithm is based on a combination between fuzzy edge detection and the bacterial algorithm. The bacterial algorithm is applied to optimize the membership functions and fuzzy rules to reduce the computational time. Moreover, the optimal path is generated by the modified GA with enhanced . This combination aims to convert the optimal path to trajectory path taking into account the most important objectives in path planning, that is, the minimum travelling time, safety, security, and energy consumption. Although the conversion process addressed in this paper has been ignored by most researchers, it is an essential part of any visual navigation system. In addition, the histogram equalization is presented to stretch the image histogram and allow the proposed approach to extract more details from the image. Unlike all researches mentioned above, real-time robot navigation system is adopted here.
This paper is organized as follows. After this introduction, the problem formulation is described in Section 2. The intelligent computer vision is described in Section 3. In Section 4, real-time results are shown and compared to a method used in [15]. Finally, conclusions and future work are addressed in Section 5.
2. Problem Formulation
The conceptual terms related to robot navigation systems are defined as follows.
Definition 1. The 2D map is the map that resulted from edge detection and includes information about the target object and/or any obstacle in the environment.
Definition 2. The path is the geometric trace that the vehicle follows to reach its destination avoiding obstacles.
Definition 3. Path planning is the problem of finding the geometric path from an initial point to a given terminating point (end point).
Definition 4. The trajectory is represented as a sequence of states visited by the vehicle based on time, velocity, and acceleration.
Definition 5. The trajectory planning for a robot concerns in the transition of one feasible state to another. It is referred to motion planning that is based on time, velocity, and acceleration. In addition, it should satisfy the vehicle’s kinematic limits and avoid the obstacles.
In this study, edge detection based object recognition is considered to identify specific objects such as robot and obstacles in unstructured environment. In addition, it provides important data to reconstruct the environmental map. To detect the edge in an image more accurately and efficiently, we need to design a robust approach that should work fine and deal with the noise in the image. These edges are so helpful for real-time applications such as robot navigation. Consequently, the histogram equalization is presented to stretch the image histogram to extract more details from the image. In addition, the intelligent image processing algorithm is proposed based on a combination between fuzzy edge detection and the bacterial algorithm that is applied to optimize the membership functions and fuzzy rules and to reduce the computational time of the proposed approach. The results are used to obtain the 2D map that is used in the motion planning, which is considered as an optimization problem. This means that the next step in the research is to find the multiobjective function to minimize the navigation time and the navigation distance. Moreover, a beneficial optimization should consider the safety during the navigation and successfully achieve the robot missions. Furthermore, the optimal path is generated by the modified GA with enhanced . This combination aims to convert the optimal path to trajectory path taking into account the previously mentioned objectives in path planning.
It should be noted that, during each planning cycle, the path planner module generates several trajectories from the vehicle’s current location towards the final destination point. This process is executed depending on the speed of the vehicle’s on-board sensors. In addition, each trajectory is evaluated with respect to some cost function to determine the optimal trajectory. It is assumed that the robot is placed on a plane environment and the contacts between the wheels and the ground have pure rolling and nonslipping conditions during the mission. The robot moves according to 2D map that might be static or dynamic. Also, the obstacles could be placed at any grid point in this map. The MR’s mission is to search offline and online for the optimal path for the robot from a start point to a destination point.
3. Intelligent Computer Vision
The proposed CV algorithm has two main stages as it is described in the block diagram in Figure 1. First, fuzzy based intelligent image processing algorithm is combined with the bacterial algorithm. This stage is responsible for detecting the edges in the captured images, where the bacterial algorithm is used in order to optimize the membership functions and fuzzy rule. The second stage is the modified GA and the enhanced , where this stage is applied in order to find the optimal path and optimal trajectory.
3.1. Intelligent Image Processing Algorithm (IIPA)
As stated earlier, the main core for the proposed IIPA algorithm is the fuzzy edge detection that starts detecting the edge points of the input image by processing each pixel with its 8 neighbors. In addition, the bacterial algorithm is applied to control the membership functions.
The definition of fuzzy image processing is simply explained as an approach which is capable of processing integral images with their segments values as fuzzy sets [27]. Even though this process has three main steps like the normal fuzzy, it is based on image analysis rather than on input values. For simplicity, before applying the fuzzy logic on any image, it is necessary to transform that image to perception domain. Fuzzy image processing describes the image characteristics as linguistic variables and maps them to membership functions. Hence, the fuzzification process transfers the images from gray level plane to membership plane. Then, the fuzzy inference system (FIS) maps the input to fuzzy output using fuzzy knowledge rules based on fuzzy operators. The last step is the defuzzification in which the crisp output is calculated based on the center of gravity method. Hence, the fuzzy operator generates the useful data and maps the fuzzy values to visualize them. The IIPA proposed in this paper is described in Figure 2. It starts by reducing the input image size and converting it to the gray scale. Then, an initialized integral matrix is created in order to save the values of each pixel in this matrix. Thereafter, the whole structure is tuned into enhancing filter function that segments the input images. The filter slides over each pixel, and it is guided by fuzzy rules to ensure that the current pixel is an edge or not. After that, the detected edges are collected and the integral image is converted into normal image.
The image is encoded with bits and converted to gray scale and then transformed to digital integral image. Each pixel consists of a value representing the intensity level. The histogram is the distribution of the discrete intensity levels with range , where is the length of image. This distribution is a discrete function () associated with intensity () and the number of pixels having this intensity (). In order to normalize the histogram, we need to transform the discrete distribution of intensities into a discrete distribution of probabilities. The normalization gives the properties of the probability density function. Therefore, it is required to divide each value of the histogram according to the number of pixels. It is clear that the digital image is a discrete set of values that could be represented by a matrix, so it is equivalent to divide each by the dimensions of the array.where and are the dimensions of the array.
In order to adjust the contrast of the image and modify the intensity distribution, the equalization of histograms is implemented. The main objective of this process is to give a linear trend to the cumulative distribution function (CDF) associated with the image. In other words, this technique aims to have an approximate uniform histogram and to assign a linear CDF to the resulting image. The CDF can be viewed as a cumulative sum of all the probabilities lying in its domain, and it is defined bywhere Card is the cardinality of a set or the number of pixels.
In the tested image shown in Figure 3, the floor and some parts of the furniture are clearly appearing white. Also, there are a huge number of pixels having intermediate gray level. The furniture and the robot are black, and they are occupying a big part of the image. So, it makes sense to have three different regions on the histogram: a black region with high peak, a gray region, and a white region, as it is shown in Figure 4. If restriction on the range of the histogram is applied, it will display and focus only on the regions corresponding to the right range of the histogram. So, we have considered only the range of values between 150 and 200 as shown in Figure 5. As shown in Figure 4, the histogram values concentrated in the right side have a lot of white intensities. Therefore, the equalization can stretch the histogram allowing us to really obtain more details from the image that we were not able to see before. Figure 6 shows the equalized histogram. The next step is implementing the fuzzy edge detection algorithm. The trapezoidal fuzzy membership functions have been used in this research work because they are capable of covering a wide range of the fuzzy space. The trapezoidal fuzzy membership function has the following form [28]:where and represent the rule index and the input variable, respectively, and are the breakpoints of the membership functions.
The fuzzy knowledge base has been designed to be a fuzzy filter, as shown in Figure 7. The investigated pixel (or ) contains binary value that is compared to the binary values of its eight neighbors. So, the fuzzy knowledge base has eight inputs and one output. Each input has two linguistic variables (B: black; W: white). The output is the edge with three linguistic variables (B: black; E: edge; W: white). The initial membership functions for inputs and output are given in Tables 1 and 2, respectively. Also, the initial rules are considered to be 35 rules.
After many experiments and researches, we had discovered that the computational time of IIPA is obviously longer than other techniques like Canny. As a result, we were looking for an optimization method that might be combined with fuzzy sets in order to reduce the computational time in addition to its capability of dealing with noisy images. The bacterial algorithm, which was proposed to discover the parameters of the fuzzy system and to design the fuzzy system, was the best choice for such combination [29]. This method discovers the combination of input variables of the rules, the parameters of the membership functions of the variables, and a set of relevant rules. Consequently, we have optimized the IIPA by combining the bacterial algorithm with the fuzzy sets.
However, there are two operations that can be applied to optimize the process using the bacterial algorithm [30]. The first one is the annihilation which means deleting the fuzzy rule if the membership functions are too narrow. The second one is removing the rule and the membership function by a fusion operator when the medians difference between two membership functions is too small. The optimized membership functions for the inputs and output are given in Tables 3 and 4, respectively. The optimized rules are assumed to be 15 rules as presented in Table 5. It has been found that the best fitness function value for 1000 generations is 0.5206. The fitness function which describes the absolute errors is shown in Figure 8.
The second tested image and the results obtained from implemented fuzzy edge detection are illustrated in Figures 9 and 10, respectively.
3.2. The Modified Genetic Algorithm
The soft computing GA can solve complex problems such as optimization of trajectory planning. It is based on proposing and enhancing the searching ability of the robot to move towards optimal solution. The formulation of such problem is concerned with two main steps. The first step is to find the multiobjectives optimal path from the start point to the end point. There are several purposes for introducing the multiobjectives optimal path such as minimizing the distance and time and ensuring the smoothness and safety of the path. Figure 11 describes these restricted tasks that should be done by GA and search algorithm. The second step is dealing with complex environment by converting the optimal path to trajectory. Moreover, the path planner is required in trajectory to compute a collision free path that is later converted into a time-based trajectory [31]. However, the first step converts the images from previous sections into 2D maps, then describes this environment model, and constructs a closed workspace with different numbers of static obstacles. In order to eliminate obstacles’ nodes from the map, a decreased operator has been used. The second step of GA is the initialization with population size 20, number of generations is 10, and mutation type is flip bits and crossover type with one point crossover. In the final step, the GA and the modified are used to generate a set of the suboptimal feasible paths in the given maps. The flow chart for the proposed GA approach is shown in Figure 12. The fitness values and average distances between the individuals in 10 generations have been shown in Figures 13 and 14, respectively.
4. Results and Comparison
In this section, we have illustrated the results of our proposed approach for different scenarios. Figure 15 shows the number of false edge pixels which are detected by other techniques: Sobel, Canny, and scan line approximation. These values are compared to our proposed fuzzy edge detection. When the proposed modified fuzzy edge detection was applied to two images with sizes, 512 × 512 and 256 × 256, false edge pixels were 78 and 128, respectively. As a result, it is obvious that the number of false edge detections is much less in our method.
Figure 16 shows a noisy version of the same image, and Figure 17 shows the histogram that describes the distribution of this image. However, the result of implementing the Canny edge detection on this noisy image was the unclear image shown in Figure 18. The equalization of this image is shown in Figure 19 whilst Figure 20 shows the result from our proposed algorithm. It is clear that Canny approach results in very bad edge detection when compared to our proposed approach.
Additionally, in order to compare our results with the existing methods, we have implemented the classical methods presented in [15]. Original image of the scene is shown in Figure 21. The detected edge of the image in Figure 21 using the detection algorithm of [15] is shown in Figure 22(a). On the other hand, the detected edge of the same image using our proposed approach is shown in Figure 22(b). It is clearly found that some of the edges were lost when the reference method (method of [15]) was implemented. In contract, these edges were represerved when our proposed approach was implemented.
(a)
(b)
Figure 23 shows a noisy version of the same image, whilst Figures 24(a) and 24(b) show the detected edges of the noisy image using the reference method and our approach, respectively. From these figures, it is clear that our approach is more immune to noise than the reference method. We have also measured the computational time for the considered two approaches (the reference approach and our approach) as illustrated in Figure 25. Both of them were tested 7 times for different images sizes. Regarding terms of computational time, it is shown that our method is faster than the reference one.
(a)
(b)
Moreover, depending on the shape and color features, the proposed algorithm will address the environment contents which contain some features such as the start and end points, target MR, and the environment obstacles as shown in Figures 26(a) and 26(b). The final result, which shows the optimal trajectory for the robot from its initial position to the target position, is shown in Figure 27. Another tested scenario was in our mechanical department corridor as it is illustrated in Figure 28. The final trajectory path for the mission inside the corridor is shown in Figure 29.
(a)
(b)
5. Conclusion and Future Work
In this paper, a strategy that takes the advantages of using artificial intelligent techniques has been proposed. The equalization of the input images has been implemented to get more details from the captured images; then, the fuzzy edge detection was optimized by the bacterial algorithm to reduce the computation time for the process. The algorithm has the ability to classify each pixel in the digital image as edge or not. Then, the results from this stage were used to construct an environmental and local 2D-map. This map is fed to the next stage that contains GA and search algorithm to find the optimal trajectory between the robot initial position and the destination position. In addition, the presented approach makes the robot capable of estimating its velocity based on the trajectory. It was noticeably shown that our approach is capable of detecting the edges more precisely along with reducing the noise in an image. Moreover, in terms of computing performance, the proposed method was better than the classical methods and the computational time was reduced. As part of future work, this method can be applied to optimize the speeded-up robust features technique (SURF) algorithm, that is, used for object detection and recognition. Moreover, the performance of the proposed algorithm can be studied in more complicated applications such as 3D-map reconstruction.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this article.