Abstract
In the process of inspection for the electricity transmission equipment using flying robots, an automatic way to find the best viewpoint to get the high quality of image would be beneficial to observe the state of the electricity transmission equipment. Finding the best viewpoint belongs to the viewpoint selection problem. A strategy is proposed in this paper to find the best viewpoint for the electricity transmission equipment inspection, according to which the candidate viewpoints can be generated based on the 3D model of the equipment. Then a scoring function is constructed which combines three factors including the amount of the information, the viewing quality, and the visibility of the information to evaluate the quality of the candidate viewpoint. Finally, the best viewpoint can be selected from all of candidate viewpoints according to the scoring function. The proposed algorithm is tested on the experiment to demonstrate its effectiveness.
1. Introduction
Electricity transmission equipment inspection is an essential job for the maintenance of electrical grids since the fault of the electricity transmission equipment easily causes the power outages, which can directly lead to the failure of nodes in the Internet communication network, transportation networks, social media networks, and many other relative networks [1], and it will give rise to the inconvenience of our lives. More importantly, the power outages always result in large economic losses [2, 3]. Early detection of potential anomalies and timely maintenance can help to avoid grid faults, thus reducing economic losses and benefiting customer, relative networks, and electricity company [4, 5]. In recent years, flying robot inspection has received increasing attentions [6–11] due to the advantages of low costs, easiness to control, and flexibility compared to the foot patrol and the helicopter patrol.
A wide variety of inspections are carried out by the flying robot, ranging from sagging spans, leaning poles, and tree encroachment to small-scale items such as broken insulators. An observer judges the state of electricity equipment by watching the videos captured by the camera which is mounted on the flying robot [9]. Therefore, the quality of the videos is one of essential importance. There are many possible 2D images of given 3D equipment and most people would agree that some images are more detailed or more “informative” than others. And these images could contain valuable information and always allow the observer to concentrate on defect recognition, hazard assessment, report preparation, and navigation. Thus, it will be very useful for the flying robot to be able to automatically compute these “best” viewpoints which can help to improve the quality of inspection and the efficiency of inspection.
This paper focuses on the quantification and measurement of the visual information presented in an image of the key electrical equipment, and then the location of the camera pose for the best viewpoint, which is always known as the viewpoint selection problem. In computer vision community, the viewpoint selection also can be useful for many other applications, such as the selection of thumbnails for objects databases [12], automatic camera positioning in CAD systems, automatic scene composition, object recognition, and surgery planning or training [13, 14].
The best viewpoint selection problem is addressed using a scoring function which evaluates a set of candidate viewpoints, and the viewpoint with the highest score will be the best viewpoint. Up to now, a number of scoring functions have been proposed for the viewpoint selection, but it should be emphasized that there has not been a general scoring function which suited all applications, and it may take into account different factors for different visual tasks or applications. For instance, in scene understanding or target recognition, a good viewpoint is essential for understanding the associated scene or object; thus the viewpoint which obtains the maximum information about the object is considered the best viewpoint; for the thumbnail generation for large 3D databases, the aesthetic criteria are always selected, and the three-quarter view is tended to be chosen as the best view (for most familiar objects, canonical view is a three-quarter view [14]); and others try to select the most stable views such as [15].
In this paper, the viewpoint selection must contribute to inspect the electricity transmission equipment, which means that the best viewpoint must be beneficial to judge the defect of the equipment (e.g., insulators broken and tower material broken). Reference [16] claims that it would be desirable if the equipment can be in the center of image in the inspection, which can be beneficial to the video surveillance techniques such as visual tracking and controlling; also it seems intuitive to consider a viewpoint to be good if it provides a large amount of information for the equipment (as Figure 1 shows); meanwhile, the equipment should be observed as detailed as possible (as Figure 2 shows). The viewpoint which meets these three conditions must be more helpful to observe the equipment. Taking these three factors into consideration, a scoring function based on viewing quantity, viewing quality, and visibility is constructed to evaluate the quality of the viewpoint. Thus, our main contributions include the following: a viewpoint selection strategy is proposed to find the best viewpoint which can be beneficial to observe the state of the equipment and a scoring function is constructed to qualify how desirable a viewpoint is. Given a priori 3D model of the objective equipment and the vision task, a best viewpoint can be found according to the proposed method. The effectiveness of this approach is validated here by experiment.
(a)
(b)
(a)
(b)
The remainder of this paper is organized as follows: related works on the problem of the best viewpoint selection are briefly reviewed in Section 2. The overview of viewpoint selection scheme is given in Section 3. Some considerations about generating a set of candidate local viewpoints and the scoring function of our approach are detailed in Section 4, and the evaluation of the achieved results is described in Section 5. Discussion is given in Section 6; finally some plans about future directions conclude the paper.
2. Related Work
Most of the viewpoint selection approaches consist of two steps: generating candidate viewpoints and constructing scoring function to evaluate the quality of a viewpoint.
For generating the candidate viewpoints, many works tessellate a sphere around the object and each grid is a candidate viewpoint [16, 17]. Reference [17] creates a sphere with a resolution of 80 faces, and the center of each face is a candidate viewpoint. In this paper, however, the object is the equipment on the electricity transmission tower, so candidate viewpoint sampled over a sphere is impractical. Thus, a part of cylinder surface is tessellated and each point is a candidate viewpoint in this paper.
Given the candidate viewpoints, the best viewpoint problem is solved by assigning scores to them based on the scoring function and finding the best viewpoint with the highest score. As already mentioned, some scoring functions have been proposed to evaluate the quality of a viewpoint, but there is no consensus about what a good viewpoint is.
Maybe the first attempt to compute good viewpoints is made by [18], in which a viewpoint direction is selected if it minimizes the number of degenerate faces under orthographic projection, but the amount of information obtained is not guaranteed.
Reference [19] introduces the term “visible projected area.” They argue that a good viewpoint is the one which maximizes the number of visible faces and the visible projected surface area. These two measures are weighted and summed to an objective function. Reference [20] extends the measure of [19] to operate on a per-face basis. A “probability” is defined for each face and defined as a fraction of its visible projected area relative to the total visible projected area. These probabilities are then combined using the information-theoretic entropy function. The scoring function, called viewpoint entropy, is defined to be the entropy of this distribution. Thus, the best viewpoint owned the maximum information. But it cannot provide more detailed information, and the drawback is that it depends on the polygonal tessellation. Reference [12] presents an algorithm based on the Kullback-Leibler distance for viewpoint selection. The KL measure only takes into account the proportion between the normalized projected area and the normalized actual area, trying to obtain a balanced vision of the object or scene that it can be insensitive to the discretization in contrast to [20], but it cannot guarantee the best viewpoint can obtain the maximum information. References [19–21] all do not take into account the amount of invisible surface area at all. Reference [11] describes a few descriptors to qualify the viewpoint like surface area entropy, visibility ratio, curvature entropy, Silhouette length, and others and concludes that none of them can be coined as universal.
Recently, a saliency measure is introduced in [22–25], of which [23] defines the best viewpoints as the ones which can discriminate the object from the other objects. The solution it proposes is based on the assumption that 3D models belonging to the same class of shapes shared the same salient features that discriminate them from other classes of shapes. Finding the best viewpoints of a 3D model can then be formulated as a feature selection problem. References [24, 25] argue that the factors which concur to determine a canonical view included salience and significance of features for the observer. Thus, they detect the regions of interest (ROI) of object first, then the best viewpoints can be the one which obtains the maximum information of ROI.
The methods mentioned above inspire the strategy proposed in this paper. There is a shared opinion that only by combining different approaches can we be able to meet the different users’ needs. In the fly robot inspection for the electricity transmission equipment, the main purpose of viewpoint selection is to capture the images which can help to judge the state of the equipment. Then, the viewpoints that maximize the amount of information and that provide the most detailed information and that ensure the objective equipment in the center of image are desirable. The methods mentioned above do not consider all the mentioned factors. Thus, in this paper, a scoring function based on viewing amount of information and viewing quality and visibility of information is constructed to qualify how desirable a viewpoint is for the inspection and a viewpoint selection strategy is proposed to find the best viewpoint.
3. The Overview of Viewpoint Selection Scheme
Before the overview of our method presented, some of the contexts must be introduced here.
First, this paper focuses on the inspection for the electricity transmission equipment which includes a wide variety of items such as insulators and traces of arcing, and the battery power supply limits the flight time of flying robot; thus each equipment can be assigned just for a short time; based on this, an inspection scheme must be designed to increase the inspection efficiency [26].
Then, the inspection scheme encapsulates the idea that dividing all of the electricity transmission equipment into many objective regions (OR) (e.g., an insulator string, power line, and fittings) and generating a local viewpoint region for each OR, finally, the best viewpoint will be found within every local viewpoint region. For example, the gray region as Figure 3 shows some of the local viewpoint regions generated for the objective insulator strings, the best viewpoint for the objective insulator string will be selected there. This paper focuses on addressing the problem that how to get the best viewpoint for an OR of equipment, which is similar to [27].
In this paper, the triangular patch is used as the basic element of structure data; the 3D model is represented by a triangular patch map. As mentioned above, the essential problem is to find the local best viewpoint from each of the local viewpoint region. Generally, special vision task is assigned in a local OR, which means some patches are main targets to inspect in this region. Then the triangular patches can be classified into two types: main patch and secondary patch; each triangular patch is labeled whether it is main patch or not.
The operation of our scheme consists of the following steps:(1)input a 3D model represented by triangular patches;(2)generate candidate viewpoints according to the proposed scheme and the 3D model data;(3)compute the value of scoring function for each candidate viewpoint;(4)select the best viewpoints from candidate viewpoints.
4. The Viewpoint Selection Scheme
4.1. Generate a Set of Candidate Viewpoints
As mentioned above, many algorithms tessellate a sphere around the object. The sphere is defined as a set of viewpoints. But when engaging on the inspection for the transmission equipment in the transmission tower, the inspection distance between the transmission tower and the viewpoint is limited by the requirement of power industry security rules [27], and also the flying robot is not allowed to fly over the transmission tower for safety reasons, so the candidate viewpoint sampled over a sphere is impractical. In this paper, the candidate viewpoint region is defined as a part of cylinder surface. And the candidate viewpoint is evenly distributed in this region as Figure 4 shows.
Reference [16] claims that the equipment projected in the center of the image can be beneficial to the video surveillance techniques such as visual tracking and controlling. Thus, the direction of each candidate viewpoint points to the center of the objective equipment.
4.2. The Scoring Function
The scoring function, , quantifies the property of the candidate viewpoint and it integrated the amount of information, the viewing quality of information, and the visibility of the information. Three factors will be introduced as follows.
4.2.1. The Amount of the Information
The amount of the information is used to choose the most informative viewpoint which provides the most information of the objective equipment. And it is represented by the viewpoint entropy which is introduced in [20].
The Shannon entropy, , denotes the entropy of a discrete random variable with possible values , and it is defined by where , for . The entropy provides the uncertainty of a random variable of . The logarithms are taken in base 2 and for continuity; the entropy is expressed in bits.
The probability distribution is defined as the relative area of the projected patches over the sphere of directions centered in the viewpoint , as shown in Figure 5, the patches are projected over the sphere to get the solid angle. The solid angle is the 2D angle in 3D space that an object subtends at a point. It is a measure of how large the object appears to an observer looking from that point. The viewpoint entropy is defined by where is the number of patches of the scene, is the projected area of patch over the sphere, is the solid angle of the sphere, and represents the projected area of background in open scenes. In a closed scene, or if the viewpoint does not see the background, the whole sphere is covered by the projected patches and consequently .
represents the visibility of patch with respect to viewpoint , depending on the property of the entropy that the entropy should be maximal if all the are equally likely, and for equiprobable the entropy should increase with the number of as (3) show. From the combination of (2) and (3), we can see that a viewpoint which gets a higher value of always includes more patches and each patch tends to get the same projected area. Thus, the most informative viewpoint can be chosen from the candidate viewpoints:
But a drawback lies in the fact that can be affected by the polygonal discretization way. For a same object and same viewpoint, the denser the object been discretized, the higher value of will be. Thus, a high discretized region will heavily attract the attention of the measure; as Figures 6(a) and 6(b) show, the two symmetric faces (surfaces A and B in Figure 6(a)(i) have the same area) of the cuboid are triangulated in two different ways like Figures 6(a)(ii) and 6(a)(iii); one is divided into two triangle surfaces and another is made into 18 triangle surfaces.
(a)
(b)
(c)
(d)
Figures 6(b)(iii) and 6(b)(iv) show face A and face B with different divided way; they obtain different value of ; it means the viewpoint entropy is sensitive to the discretization; the best and worst viewpoints obtained by the viewpoint entropy are shown in Figures 6(b)(i) and 6(b)(iv) and from which it can be seen that the viewpoint which includes the more faces is often chosen; from Figures 6(b)(i) and 6(b)(ii) we can see that the viewpoint entropy tends to select the one which includes that each visible face has almost the same solid angle.
Thus, Figure 6(b) shows that the viewpoint entropy will choose the most informative viewpoint, but it is sensitive to the object discretization, and it does not consider how detailed the information the viewpoint includes.
4.2.2. The Quality of the Information
Intuitively, the viewer should be as orthogonal as possible to every face of the 3D object to get the detailed information. Here the quality of the information stands for how detailed the information can be obtained from a viewpoint. It is based on the relative entropy.
The relative entropy or Kullback-Leibler (KL) distance between two probability distributions and over the set is defined as where , for continuity and the relative entropy is always nonnegative and is 0 if and only if . However, it is not a true distance between distributions since it is not symmetric and does not satisfy the triangle inequality. It is a measure of the distance between two distributions. In statistics, it arises as an expected logarithm of the likelihood ratio.
Then, the viewpoint relative entropy is defined as (5); it means is the relative area of the projected patches over the sphere of directions centered in the viewpoint and the is the relative area of patches:
It should be noted that is the number of patches which can be seen from . is the actual area of patch , , is the total actual area of the patches which can be seen from , and is the projected area of patch over the sphere, .
It can be deduced from the above definition that the viewpoint relative entropy can be interpreted as the distance between the normalized distribution of projected areas and the ideal projection, given by the normalized distribution of the actual areas. It means that when the normalized distribution of projected areas is equal to the normalized distribution of actual areas we can get the minimum value 0. Otherwise, increases when the proportion of projected areas leaves from the proportion of actual areas. Therefore, the best viewpoint always obtains the minimize value of ; for the convenience of computing, it will be normalized finally and is used to represent the relative entropy at last.
Figures 6(c)(i) and 6(c)(iii) show the best and worst viewpoints with the relative entropy, and Figures 6(c)(i) and 6(c)(ii) show the value of the two viewpoints looking straight into the surface. It can be seen that it tends to select the viewpoint directly facing the most patches. The viewpoint relative entropy measure only takes into account the proportion between the normalized projected area and the normalized actual area, which can stand for how detailed the information can be obtained from .
4.2.3. The Visibility of the Information
The two factors interpreted above have not considered the invisible patches, and the viewpoint entropy is sensitive to the object discretization that the visibility of the information factor is proposed. It is based on the percent of the real area of visible surface patches among all the areas of objective surface patches which is represented by , , and the visibility of the information is given by . It reaches a maximum when is equal to an optimum percentage , which is specified by the user ( when ) and it is determined by the shape of object. The function reaches a minimum when the percentage is either 1 or 0 as Figure 7 shows. A general cubic polynomial is used as the function, and it is divided in two parts as follows: Figures 6(d)(i), 6(d)(ii), and 6(d)(iii) give the viewpoints with the best visibility of information and the worst visibility information, Figures 6(d)(i) and 6(d)(ii) obtain the same value which means the two viewpoints can see the same surface, and it can be seen that the visibility of information tend to select the viewpoint which can observe more area of object.
4.2.4. The Scoring Function
The optimal function is a combination of the previous factors. And the optimal function is set as is the weight of the corresponding factor, and their value will be defined according to different requirement by users; it is noted that the sum of , , and must be equal to 1, while , , and should be normalized.
5. Experiment and Discussion
This paper focuses on finding a few of best viewpoints which can be beneficial to observe the state of the electricity transmission equipment. The insulator of the electricity transmission tower, as one of the most important equipment, is taken as the inspection target in this experiment. And the insulator failures always include insulator flashover, insulator broken, and ice accretion on insulators. In this experiment, the best viewpoint which can contribute to observe the insulators is obtained by the proposed strategy.
First, the 3D model of the insulator string represented by triangular patches is input as Figure 8 shows. And two insulators among them are extracted aiming to simulate the insulators dropped. The OpenGL toolbox in MATLAB is employed in this experiment.
Then, the candidate viewpoints are generated according to our strategy as Figure 9. And in the inspection for insulators, the best viewpoint will be found for each kind of failure such as insulators dropped and insulator flashover. This experiment will show the best viewpoints for these two conditions.
As mentioned above, the weight among the scoring function can be decided according to the special task. When the flying robot is inspecting the insulators to judge if they were dropped or broken, the viewpoint which can help to comprehend the shape and structure of the insulators will be desirable. It is intuitive that more information and more visible area of the object will be beneficial to understand the shape; therefore, the amount of the information and the visibility factors are important, and , are given larger value than . Owing to the drawback of the amount of information factor as introduced above that it is sensitive to the discretization, the is set a less value than . Finally, is set as , , and . Table 1 and Figure 10 give the results. Table 1 gives four viewpoints’ ; Figure 10 gives the corresponding image rendered. Figure 10(a) shows the viewpoint with the least value of , from which some insulators cannot be distinguished dropped or not, and Figure 10(d) shows the viewpoint with the highest value which can be used to judge whether the insulators dropped or not more clearly.
(a)
(b)
(c)
(d)
For inspection for the insulator dirty or cracking, it always happens on the surface of insulators; the viewpoint which can look straight into the objective face will be better for observation. It is intuitive that the viewpoint which can give the more detailed information about the surface will be more helpful for observation. Therefore, the quality of the information factor is important and is assigned the largest value. Meanwhile, the amount of information factor can supply some additional information to help to understand the object, so is set a value larger than . is finally set as , , and . The results are shown in Table 2 and Figure 11. Table 2 gives four viewpoints’ ; Figure 11 gives the corresponding image rendered. The viewpoint which obtained a higher score of can be more helpful to get the detail information. (a) shows the worst viewpoint which got the minimum value of , which obviously does not provide the clear image of the insulators surface. (b) obtains a higher value than (a), and with the higher scores the information can be seen more clearly. (c) gets a high score since that one side of insulators surface becomes more clear, and (d) with a high score clearly shows the other surface of insulators.
(a)
(b)
(c)
(d)
For the inspection of other transmission equipment, the best viewpoint can be achieved by setting appropriate value of similarly to the method mentioned above according to the specific target.
6. Conclusions and Future Work
Finding the best viewpoint to capture high quality of image is beneficial to the flying robot inspecting the electrical transmission equipment. The best viewpoint can contribute to observe the state of the equipment and judge if the failure happens or not. This paper focuses on finding these viewpoints.
This paper presents a strategy to find the best viewpoint for inspecting the electrical transmission equipment. Meanwhile, a scoring function is proposed to evaluate the quality of a viewpoint and select the best one from all of candidate viewpoints. The selection process is a combination of factors including the amount of information (represented by the viewpoint entropy), the viewing quality of information (represented by the relative entropy), and the visibility of the information. Given a priori 3D model of the objective equipment and the vision task, the best viewpoint can be found according to the proposed method by setting appropriate value of . The effectiveness of this approach is successfully validated by the experiment of insulators inspection.
Apart from the inspection, the scoring function also can be used in the selection best viewpoint of thumbnails for objects databases, viewpoint planning, automatic scene composition, surgery planning or training, and so on. Finally, further improvement will concern the development of methods for the generating candidate viewpoint region and defining the weights of each factor automatically.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work is supported by the National Natural Science Foundation of China (Grant no. 61105083), Program for New Century Excellent Talents in University (Grant no. NCET-11-0634), and the Fundamental Research Funds for the Central Universities (Program no. 12ZX16).