Abstract
As the current binary descriptors have disadvantages of high computational complexity, no affine invariance, and the high false matching rate with viewpoint changes, a new binary affine invariant descriptor, called BAND, is proposed. Different from other descriptors, BAND has an irregular pattern, which is based on local affine invariant region surrounding a feature point, and it has five orientations, which are obtained by LBP effectively. Ultimately, a 256 bits binary string is computed by simple random sampling pattern. Experimental results demonstrate that BAND has a good matching result in the conditions of rotating, image zooming, noising, lighting, and small-scale perspective transformation. It has better matching performance compared with current mainstream descriptors, while it costs less time.
1. Introduction
The local feature descriptor is the core of many computer vision technologies, such as object recognition, image retrieval, and 3D reconstruction. How to design a local feature descriptor that has excellent performance and low complexity is an important and difficult research. Many scholars have proposed a variety of descriptors in this area, such as SIFT (Scale Invariant Feature Transform) [1–3], which has good scale invariance and robustness to illumination and viewpoint changes. Because of constructing scale space and many other steps to improve accuracy, SIFT has high computational complexity. All of these make it prohibitively slow.
For these problems, many improved algorithms were proposed. SURF (Speeded Up Robust Features) [4] has similar matching rate with much faster performance by describing key points with the responses of few Haar-like filters compared with SIFT. PCA-SIFT [5] reduced dimensionality of the descriptor from 128 to 36 by using the principal component analysis, but the time of descriptor formation was increased. The GLOH descriptor [6] used the circular pattern in log-polar coordinates in order to enhance the robustness and uniqueness of the descriptors, but it is also more expensive to compute than SIFT. ASIFT descriptor [7] achieved matching feature points well when there were large changes in viewpoint.
To some extent, above-mentioned descriptors improved the performances, but they still have high computational complexity because of the local histogram statistics. Besides, each dimension of the descriptor is a decimal number, which makes them need a lot of memory. All of these make them difficult to be achieved on a low-power, low-memory application. In recent years, many binary descriptors appear, such as BRIEF (Binary Robust Independent Elementary Features) descriptor [8], ORB (Oriented FAST and Rotated BRIEF) descriptor [9], BRISK (Binary Robust Invariant Scalable Keypoints) descriptor [10], RIFF (Rotation-Invariant Fast Features) descriptor [11], and FREAK (Fast Retina Keypoint) descriptor [12]. These binary descriptors need lower memory than SIFT-like descriptors, and the similarity of descriptors can be evaluated by computing the Hamming distance, which is very efficient to be computed. But one of their shortcomings is that the shape of pattern cannot be changed. In other words, there are only circular pattern and rectangle pattern, which makes them have no affine invariance. In this case, the high false matching rate between different viewpoints images will appear. Another shortcoming is that one pattern only has one local orientation, and they are not fast enough because of the histogram statistics. In this paper we propose a new binary descriptor with low computational complexity. The orientations of descriptor are found by the five concentric circles, and the affine invariant regions are found by 16 rays which are emanating from the feature point. Experimental results show that the binary affine invariant descriptor has good matching result in the conditions of rotating, image zooming, noising, lighting, and small-scale perspective transformation. In this paper we call this binary descriptor BAND (Binary Affine Invariant Descriptor).
2. Method
We describe 3 key steps of BAND in this section, namely, affine invariant radius builder, multidirectional rotation invariant builder, and simple random sampling pattern builder.
2.1. Affine Invariant Radius Builder
There are many algorithms that extract the local affine invariant regions, such as MSER (Maximally Stable Extremal Regions) [13], EBR (Edge-Based Regions), IBR (Intensity Extrema-Based Regions) [14], and Salient Region [15]. Here we refer to the idea of IBR. With the aim of extracting local affine invariant regions, the 16 rays which are emanating from the feature point are extracted. The gray value of each sampling point located in the rays is obtained through quadratic interpolation. Here we define the affine invariant radius as where is the affine invariant radius in the direction of , is the Euclidean arc length along the ray , is the gray value of the radius at the position , is the absolute value of the , is the minimum value of the distance between local minimum of the and the feature point, is the maximum value of the distance between local maximum of the and the feature point, and is a constant, and we use a value of 0.76, which is determined by experiments.
By calculating the affine invariant radius, the original circular pattern is deformed into irregular shape pattern. It is shown in Figure 1.
2.2. Multidirectional Rotation Invariant Builder
In order to get the rotation invariant property, existing algorithms need to determine the direction of the local area and rotate the axis. In this process, the defects of these descriptors include(1)BRIEF descriptor ignores the rotation; therefore a lot of outliers appear in the case of large-angle rotation.(2)The algorithms, like SIFT and ORB, use the rectangle pattern. It is difficult to obtain the local orientation, and they have a high computational complexity.(3)The algorithms, like FREAK and BRISK, only consider one local orientation with a pattern, although using the circular pattern.
Based on the above three points and reference of the BRISK and FREAK, the pattern of BAND descriptor is as follows: construct five circles concentric with the feature point, and there are 16 sampling points in each circle homogeneously. Every affine invariant radius of every circle is obtained by the affine invariant radius builder.
Firstly, obtain the locations of sampling points on every circle by affine invariant radius: where is the index of the sampling points surrounding the feature point, is the number of sampling points, and is the location of the feature point.
Secondly, according to literature [14], LBP binary string of every circle is where is the index of the circle, is the location of one bit of the string, is the gray value at the position on the circle, and is the gray value of feature point. Finally, according to the rotation invariant LBP [16], we could obtain five local orientations. The rotation invariant LBP defined by Ojala is : where performs a circular bitwise right shift operation on the number with times. The pattern of BAND descriptor is shown in Figure 2.
2.3. Simple Random Sampling Pattern Builder
This step is very simple. We get a 256 bits vector by sampling point pairs from the five LBP coding strings randomly and orderly. The bit-vector descriptor is assembled by performing all the comparisons of point pairs (), such that each bit corresponds to where is the gray value of the point sampled from the five LBP strings.
3. Experiments and Analysis
Programming environment is Matlab 2013, Visual Studio 2010, and OpenCV 2.4.4. The datasets come from [17]. All algorithms are running on an Intel(R) Core (TM) i3 of 3.40 GHz. In order to unify the results, all of the results do not remove the mismatches and only show the best results by setting a threshold.
3.1. Single Performance Verification of Local Affine Invariant Descriptors
Each of the datasets contains a sequence of six images exhibiting an increasing amount of transformation. All comparisons here are performed against the first image in each dataset. Figure 3 shows one image for each dataset analyzed.
(a) Tress
(b) Ubc
(c) Rome
(d) Leuven
(e) Office
(f) Day_Night
The transformations cover blur (Trees), brightness changes (Leuven, Office, and Day_Night), JPEG compression (Ubc), and rotation (Rome). Match rate is defined as a ratio between the number of the correctly matching points and the total number of the matched points. In order to prove the easy integration of our algorithm, we use FAST [18, 19] for detecting the feature points in the sequence of the Trees and the SURF in the others. To make results be comparable, the threshold of Hamming distance for BAND, BRIEF, BRISK, and ORB is 50. We only show the best 50 matched pairs in SURF and SIFT. All of the results are shown in Figure 4.
3.2. Images Matching for Outdoor Scenes
We only test single performance of BAND descriptor in Section 3.1. In this section, we would use 6-pair images showed in Figure 5 to test its comprehensive performance. As it is shown in Figure 5, all of the 6 pairs contain view point changes in different degrees, such as in scale, brightness, and rotation. Besides, all of the images contain many structures which appear repeatedly. For example, the structures in image Church and the image Fountain both are symmetrical, and the buildings in image Brussels, Venice, Semper and Rathaus have similar doors, windows and statues individually. These conditions add extra difficulty to matching.
(a) Church
(b) Brussels
(c) Venice
(d) Semper
(e) Rathaus
(f) Fountain
Figure 6 shows the matching results of the BAND. The matched point numbers and match rates of different descriptors are shown in Figures 7 and 8. To make results be comparable as in Section 3.1, the threshold of Hamming distance about BAND, BRIEF, BRISK and ORB is 50 invariably. We only show the best 50 matched pairs in SURF and SIFT as before.
(a) Church
(b) Brussels
(c) Venice
(d) Semper
(e) Rathaus
(f) Fountain
We use SURF for detecting the feature points, and the threshold is equal to before. The 6-pair outdoor scenes are as shown in Figures 5 and 6.
3.3. Time Consumption Comparison
In Matlab 2013 platform, we compare the time consumption between SIFT and BAND descriptor per point pair. Test result is shown in Table 1.
If we set the time of BAND describing one feature point to be 1, then we would get the time consumption of other descriptors in Table 2 clearly.
3.4. Analysis of Experimental Results
In the single performance verification, BAND is demonstrated as it has a better adaptability than the other descriptors. In 6 experiments, the degree of transformations is increasing. It is noteworthy that the Leuven shows a transformation in brightness from normal to dark gradually. Similarly, the Office shows a transformation in brightness from dark to normal. Differently, the Day_Night shows a transformation in brightness from normal to dark, but some local areas become lighter than surroundings because of some lights, such as lamps, bulbs, and candles. In other words, it changes nonlinearly. Since the similarity is reduced, the direction of every line in Figure 4 shows downward. BRIEF shows the worst performance among all of the descriptors, because it does not note the variety of rotation and scale. In the light experiments, such as the Leuven, the Office, and the Day_Night, BAND demonstrates outstanding performance. BAND can maintain a high rate of correctly matched no matter how seriously changes of lightness. In other experimental conditions, performances of BAND are similar to the mainstream descriptors. In addition, no matter what feature points we use, like FAST and SURF, BAND descriptors can work well, which shows a high adaptability.
In the experiment of matching the outdoor scenes, we can get a lot of matched points by ORB descriptor, but its correctly matched rate is lower than BAND descriptor. One of its negative effects is that there will be more computation in the process of removing the mismatched points. Although the amount of the matched points we get by BAND is not as much as ORB, the amount is kept at a high level and the correctly matched rate of it is satisfactory. It is better than or as good as the correctly matched rate of other descriptors. The reason is that BAND considers the affine invariance and its pattern, whose shape is variable, can better adapt to the changes of viewpoint.
In the experiment of the time-consuming comparison, description time per point of BAND is as short as 0.1% of the time of SIFT approximately. The reason is that there is no process of establishing the Gauss Pyramids or too much computation of fitting operation. BAND computes the distance between two descriptions by using Hamming distance, but SIFT uses Euclidean distance. Obviously, calculating the Hamming distance has higher efficiency and lower computational complexity. So, together, BAND shows higher efficiency. And the comparison of the time-consuming with other descriptors also shows that BAND is faster. Therefore, BAND has advantage in the situation that demands high computing speed.
4. Conclusion
Experimental results show that BAND descriptor has significant advantages, such as low computational complexity, well adaptability, and good stability. It makes up for the disadvantages of other descriptors that have high computational complexity and have no affine invariance. BAND has a good matching result in the conditions of rotating, image zooming, noising, lighting, and small-scale perspective transformation. More specifically, it has a moderate number of matched points and a high correctly matched rate. Because of these, on the base of guaranteeing accuracy, BAND could improve the calculation efficiency and meet real-time requirement.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.