Robust Keypoint Detection and Matching on Fisheye Images by Self-Supervised Learning

Figure 6

Overview of proposed self-learning architecture. The source fisheye image is firstly transformed into a viewpoint-changed version by undistortion, homography transformation, and warping, respectively. The keypoint network is applied on both source and transformed fisheye images to detect keypoints, interpreted by scores, relative positions, and descriptors. Based on the matching of keypoints, the homography transform between two fisheye images is further estimated and the losses are calculated (during training).