Abstract
Rotational symmetry is important for many applications in computer graphics, vision, and image processing. However, it remains difficult to design an effective algorithm for automatic symmetry recognition. In this paper, we present a rotational symmetry detection algorithm, which is easy to use and can determine both the center and the radius of the rotational symmetry supporting region without human interaction. Our algorithm is derived from frieze-expansions approach and improved through a radius-based expansion idea. Multiresolution pyramid is used to accelerate this detection process. We also discuss a solution to deal with rotational symmetry detection under slight affine transformation. Experimental results show that the method is effective for most nature images with rotational symmetry.
1. Introduction
Rotational symmetry is a pervasive phenomenon presenting itself in all forms and scales in natural and man-made environments. Although seeking symmetry from digital data has been attempted over the decades, a fully automated symmetry recognition system remains a challenge for real world applications. However, the recent interest in computational symmetry for computer vision and computer graphics applications has provided promising results.
Given an image, the general process of rotational symmetry detection can be stated as follows: evaluating rotational symmetry features for each pixel, selecting rotational symmetry centers from all candidates, and estimating local supporting regions. The major challenges are in evaluation, how to evaluate rotational symmetry likelihood of a pixel, and in estimation, how to develop an effective procedure to estimate the rotational symmetry supporting regions. Both the evaluating and estimating parts are essential for the success of symmetry detection: the accuracy of detection will depend primarily on the decision of center and supporting regions, while the efficiency of the evaluating procedure will directly determine the computational cost of the algorithm.
In this paper, we present an accurate algorithm that can detect a wide variety of images with rotational symmetry. The basic idea behind our algorithm is to convert the rotational symmetric image into expanded translational symmetric form through polar transformation, which will simplify the detection process. We first describe how the algorithm works in a single resolution, and then we extend it using a multiresolution pyramid to obtain improvements in efficiency. The proposed algorithm has a great ability to reflect the rotational symmetry feature in high-level semantics. Its effectiveness is verified by the experimental results.
The main contribution of this paper is an effective rotational symmetry detection method which has the following benefits. First, rotational symmetry can be more easily detected via radius-based expansion, rather than diameter-based expansion. It is more suitable for rotational symmetric detection of real images. Second, our algorithm serves better quality and faster computation speed than those previous techniques, especially for slight affine transformation cases. This allows us to apply our algorithm in areas where rotational symmetry detection has traditionally been considered too expensive.
The rest of this paper is organized as follows. We give a brief overview of previous work in Section 2. In Section 3, we describe the rotational symmetry detection algorithm in detail. We show experimental results and discuss our method in Section 4 and conclude the paper in the last section.
2. Related Works
2.1. Translational Symmetry Detection
Translational symmetry detection is crucial to rotational symmetry detection. This is because rotational symmetry can be transformed into translational symmetry when the center and radius of a rotational symmetry are given.
There has been substantial research on translational symmetry detection. Cai and Baciu introduced a bottom-up detection-inference approach which extends the versatility of current detection methods to a higher level segmentation [1]. They also demonstrated a framework of a new theoretical analysis of invariant repetitive patterns. But this approach has less coverage in previous literature on pattern detection and classification. Loy and Eklundh presented a method that could find symmetric constellations of features in images [2]. The method simultaneously considers symmetries over all locations, scales, and orientations and has proved to be reliable in detecting both bilaterally and rotationally symmetric figures in complex backgrounds. Michaelsen et al. employed Gestalt algebra as a tool for symmetry structure recognition [3]. But this method can only handle mirror-symmetry without any geometric distortion. Park et al. proposed a robust computational framework for automatic detection of deformed 2D wallpaper patterns [4]. They formulated the 2D lattice detection as a spatial, multitarget tracking problem, solved within a Markov Random Field framework using Mean-Shift Belief Propagation (MSBP) method.
2.2. Rotational Symmetry Detection
Recent studies have presented results on rotational symmetry detection. Prasad et al. built a symmetry saliency map named gradient vector flow and detected both reflectional and rotational symmetry [5, 6]. Lucchese introduced an interpretation of the ambiguities that arise in a locus in the form of additional pairs of orthogonal lines when applied to symmetric patterns and the development of other algorithmic steps which allow a simple and fast discrimination between cyclic and dihedral symmetries [7].
Object Symmetry Transform (OST) was used for bilateral symmetry in Di Gesù’s method which was rotation-invariant and provided interesting results. But the method is computationally expensive on conventional computers due to the computation of line intersection and distances [8]. Riklin-Raviv et al. proposed a method on perspective reflectional and rotational symmetry extraction and segmentation [9]. Cho and Lee expanded and merged confident local symmetric region matches by exploiting both photometric similarity and geometric consistency via new symmetry-growing framework [10]. Chertok and Keller presented a spectral approach for detecting and analyzing rotational and reflectional symmetries in -dimensions [11]. His contribution is the derivation of a symmetry detection and analysis scheme for sets of points. When there are more symmetries in one image, detection becomes much difficult. Kondra et al. proposed a global region-based method to deal with multiple rotational symmetries cases [12].
Affine distortion introduces some additional challenges when detecting rotational symmetry, and several researchers have considered addressing this problem from the perspective of geometric shapes. Some work focuses on the symmetry detection with perspective distortion [13, 14]. Yip [15] applied a Hough transform-based algorithm to detect and analyze rotational symmetry under affine projection. Cornelius and Loy presented detecting planar rotational symmetry under affine projection, which can deal with partial occlusion and is able to detect multiple rotationally symmetric surfaces in complex backgrounds [16]. A quantitative evaluation of the difference between (SIFT) feature-based and region-based symmetry detection methods can be found in [17]. Ni et al. gave a systematic extension to the fast radial symmetry transform to make it invariant to bounded cases of perspective projection [18]. Lee et al. proposed a frieze-expansion method that transforms rotational symmetry group detection into a simple translation symmetry detection problem [19]. Lee and Liu used the bilateral reflection symmetry approach to curve glide-reflection symmetry in 2-dimensional Euclidean space, so that classic reflection symmetry becomes one of its six special cases [20].
In the community of automatic symmetry detection, a variety of methods have been reported on cleanly segmented artificial shapes or images with dominant global symmetries. And only very few have considered the influence of affine distortion. In this paper we take the concept of feature matching proposed by [19] and extend it to address a more general planar rotational symmetry detecting problem. We also solve for the projection angles which describe the angular pose of the symmetric object, determine the rotational symmetry order, and segment the symmetric region. The proposed method has successfully demonstrated identifying rotationally symmetric figures in real images in the presence of partial occlusions and background clutter.
2.3. Evaluation of Symmetry Detection Algorithms
A quantitative evaluation scheme for rotation, reflection, and translation symmetry detection algorithms was proposed in a technology report from Carnegie Mellon University. The author verified and compared different symmetry detection algorithms based on a set of carefully chosen synthetic and real images containing both single and multiple symmetries, which had a diverse range of computational challenges.
In the year of 2011 and 2013, 1st and 2nd symmetry detection algorithm competitions were conducted as a workshop affiliated with the Computer Vision and Pattern Recognition Conference (2011: http://vision.cse.psu.edu/research/symmComp/workshop/index.shtml) (2013: http://vision.cse.psu.edu/research/symComp13/index.shtml). Organizers established a testbed for the evaluation of symmetry detection algorithms, devised evaluation metrics, and automated the evaluation process. They also tested the process on nine algorithms and established a performance baseline that can be used as reference for future work on symmetry detection.
3. Rotational Symmetry Detection
Our goal is to develop an algorithm that is able to extract the rotational symmetry region by detecting the rotational symmetry center and radius of local supporting regions. Our proposed process is a radius-based frieze-expansion (RFE) workflow, as described in Figure 1. We now describe the proposed algorithm and show how to construct it for a given image.
Given a natural image , Lee et al. [19, 20] proposed a diameter-based frieze-expansion (DFE) detection method for real images with rotational symmetry groups. The method could detect all the properties of a skewed rotational symmetry group, such as the rotation center, affine deformation, and the numbers of folds of the symmetry group. However, the DFE is well-specialized for rotational symmetric image detection only when the rotational symmetry is also reflection symmetry.
Inspired by the DFE rotational symmetry detection, we consider employing frieze-expansion process in a radius-based way, called radius-frieze-expansion (RFE), rather than in the diameter-based one. For the DFE, each diameter contains a pair of unconcerned radii of two rotational symmetric cells with opposite angles, while, for the RFE, we divide the diameter into two radii. Each radius only represents an independent rotational symmetric cell. The RFE is more suitable for real image’s rotational symmetric detection, because each rotational symmetric cell can be grouped with a subset of expanded consecutive radii.
This detection procedure is completely deterministic on the radius of local supporting regions and also can be applicable to slight affine transformations. So it is efficient and amenable to further acceleration with multiresolution pyramid.
3.1. Single Resolution Detection
The algorithm starts with an input natural image and an initial radius .
The resolution detection algorithm is based on the assumption that if a region is possessed of rotational symmetry with a radius and symmetry center , then the region is also possessed of rotational symmetry with a radius and the same symmetry center , and is not longer than .
The initial radius should not be longer than the actual radius of rotational symmetry regions, so we set the initial radius as 1/10 of the input image ’s short side. We force the detection process pixel by pixel in a raster scan ordering, that is, from top to bottom and from left to right. If the algorithm fails with the radius , the radius will update as half of the original radius value. Figure 1 shows a graphical illustration of the detection process.
3.1.1. Radius Frieze-Expansion
A key observation in our work is that the rotational symmetry can be detected and analyzed in polar space as a translation symmetry pattern. Given a candidate rotation center , a radius , and a polar angle-start , we rotate each radius in a rotational symmetric image and realign them consecutively from left to right in parallel (Figures 1(b) and 1(c)), to form the translation symmetries RFE on a frieze-expansion pattern. An arbitrary pixel in corresponds to a coordinate in the , and the relationship is as follows:
3.1.2. Rotation Center Detection
In [19], the detection of potential rotation center of rotational symmetries combines a set of rotational symmetry saliency maps. As in the diameter case, one-dimensional horizontal discrete Fourier transform (DFT) is performed on each row of the RFE . The energy density can then be defined as follows:where is the DFT coefficient for the row, representing the complex value of each frequency component of the spatial domain information. We also define Rotational Symmetry Strength (RSS) from DFT coefficients (Figure 1(d)) of the RFE image at each pixel :where radius is the total number of rows of the radius frieze pattern and represents the spectral density of the row for RFE centered at . We divide into two groups: , whose value is above the mean of , represents the primary density among the energy spectra; represents the secondary density. Higher values of RSS mean stronger likelihood of rotational symmetry at that location (Figure 1(e)). In addition, a Symmetry Shape Density (SSD) function is defined in terms of the phase information of the RFE image, and a potential rotation center map is built based on local feature matching.
3.1.3. Rotation Radius Estimation
To determine the rotation radius at symmetry center , RSS is computed for all possible radii at center . Starting from initial radius , the is recorded with increasing radius , until the nearest border is reached. The rotation radius can be estimated as the maximum value of corresponding set that enable RSS to obtain possible extreme values, as described in Figure 2.
3.2. Symmetry Detection under Affine Transformations
Due to the viewpoint change, the transformation interest for most natural images is affine, as illustrated in Figure 3. Clearly, a rotational region cannot directly be described with a circle. Instead, the shape of the region has to be adaptive or covariant with respect to affinities. That is, a rotational region under affine transformations should be delineated with an ellipse. The size of the regions is measured with both the short axes and the long axes of the ellipse, which corresponds to the radius of a circular region with the same area.
The RSS is also computed for all possible radii on the image under affine transformations. Starting from initial radius , the is recorded with increasing radius , until the nearest border is reached. The long axes can be estimated as the maximum of all extreme value points of radius , while the short axes are the radius with the maximum of RSS.
3.3. Multiresolution Detection
The single resolution detection algorithm captures rotational symmetry by using suitable radius length. The detection process works on all pixels of input image. However, image with higher resolution demands more computation. This problem can be solved by using a multiresolution image pyramid; computation cost will be saved because we can represent large scale structures more compactly by a few pixels in a certain lower resolution pyramid level.
The multiresolution detection algorithm proceeds as follows. A Gaussian pyramid is built from input image . The algorithm then transforms from low resolution to high resolution, such that each high resolution level is constructed from the detected low resolution level. Within each output pyramid level , the detection algorithm only needs to process in a way similar to the single resolution case where the symmetry center and radius are close to the low resolution results. The only modification is that for the multiresolution case, the detection process finds the extreme value point of the mapping pixels in higher resolution from the symmetry center in low resolution. When this process stops at the highest resolution, the candidate symmetry center and radius can be selected as the final result.
An example of a multiresolution neighborhood is shown in Figure 4.
3.4. Summary of Algorithm
We summarize the algorithm in Algorithm 1.
| ||||||||||||||||||
4. Experimental Result
We implemented the proposed approach by using MATLAB on a PC with 2.53 GHz Intel core i5 CPU and 4 GB RAM. We experimented with our algorithm on a variety of images with rotational symmetry, such as flowers, plants, tyres, dome, and still objects. Some representative results are shown in Figures 5–8. These examples come from symmetry competition sets (http://vision.cse.psu.edu/research/symComp13/index.shtml), which contain 96 real images with rotational symmetry under natural lighting conditions. The average runtime on a 2.53 GHz Intel core i5 CPU with 4 GB RAM is 1 minute. Some images with complex background take longer than 5 min due to the large number of repetitive detection processes for rotational center.
(a)
(b)
(c)
Figure 5 demonstrates some results of images with aligned rotational symmetry; note that the center and supporting region were marked with a white cross and blue circle, respectively. Our algorithm performs consistently well on these images. Flowers, dome, well cover, tyres, and tape can be detected with accurate centers and supporting regions.
Figure 6 shows results of images under affine transformations, where the supporting regions were marked with two concentric circles. In fact, we can employ an ellipse to describe this weak perspective case caused by affine transformation as shown in Figure 3. The results demonstrate that our method is applicable to images with slight skewing. It is a great improvement compared to previous method. Note that the bottle caps top and the second dome are marked with two concentric circles. Actually, there is something wrong with the bottle caps center detection; this is because obvious affine transformations always bring bad skewness.
Figure 7 exhibits the results of images with multiple rotational symmetry. When there are more symmetries in one image, the detection risk becomes a little difficult. The crucial point lies in how many extreme value points are extracted from the raster scanning process. A general way is to set an effective value as threshold. In this paper, We obtain a good result with default mean value.
Because our method is derived from Lee et al.’s algorithm [19], for single rotation center images, our method’s performance is equal or superior to current mainstream algorithms [5, 11, 15]. However, for images with multiple rotational symmetries, Kondra et al.’s algorithm outperformed ours in terms of precision [12].
The accuracy of our algorithm relies on the ability of center detection. One limitation of our algorithm is that it fails to find a correct center of rotation and supporting region when the symmetry object cannot be skewed into a plane. Our method also fails to detect symmetry with structural texture background (Figure 8(a)). Moreover, our method may not be able to handle image with incomplete rotational symmetry (Figure 8(b)). In certain types of images, the detection area is laid out in a manner that less important symmetric objects in background mislead the rotational detection process (Figure 8(c)).
5. Conclusion
In this study, we present an automatic algorithm for rotational symmetry detection. Our method successfully extracts the rotational symmetry region by detecting the rotational symmetry center and radius of local supporting regions. To detect rotation symmetry in a transformed space, we adopt a radius-based expansion idea on the frieze-expansion process. We also introduce multiresolution image pyramid to accelerate the rotation symmetry center detection process. In addition, we test our method on a variety of images from symmetry competition sets to verify the strength and weakness. Our approach shares some of the same limitations as frieze-expansion approaches: it does not work well on the rotational symmetry with strong viewing perspective. Potential directions for future research include detecting objects with partial rotation symmetry and making the algorithm robust in skew transformations.
Conflict of Interests
The authors declared that they have no conflict of interests regarding this work.
Acknowledgments
This work was supported by Tianjin Philosophy and Social Science Research Fund (TJSR15-008), the National Natural Science Foundation of China under Grant no. 61402332, and Tianjin University of Science and Technology Youth Innovation Fund (2015LG14).