Morphable 3D-Mosaics: A Hybrid Framework for Photorealistic Walkthroughs of Large Natural Environments

Figure 4

When seeking a correspondence for a pixel $p$ of image ${ℐ}_k$, we reproject that pixel onto image ${ℐ}_{k+1}$ (using the approximate geometry of the local models and their relative pose) and then restrict our search only in a small region ${\text{SEARCH}}_p$ around the reprojection point. Furthermore, to take scale differences into account, we search not only inside region ${\text{SEARCH}}_p$ of image ${ℐ}_{k+1}$, but also inside the projection of ${\text{SEARCH}}_p$ onto the images ${ℐ}_{k+1}^s$, which are rescaled versions of image ${ℐ}_{k+1}$, that is, we search not only for a corresponding pixel, but also for a scale estimate. In fact, all the possible pixel-scale combinations will form the candidate labels of pixel $p$ and to choose the best one of them, we will use an MRF optimization procedure.