Matching Spherical Panoramas and Planar Photographs

My Master's thesis was based on finding effective ways to match panoramas to photographs.  This was motivated by the desire to be able to find a more accurate camera position for a device used remotely so that high quality augmentations could be placed on images captured on site.  With a large database of panoramic images available, a GPS location helps find the closest panorama.  Then image matching techniques would be used to align the photograph to the panorama.  Augmentations that have been previously calibrated for the panorama could then be transformed to the photograph and layered on top.

Abstract

Image matching and the epipolar geometry for a stereo pair has been a well-studied topic in the field of computer vision. There is a strong foundation for matching techniques between two planar images, and the case of two spherical panoramas has been more recently explored. This work establishes the geometry for a pair consisting of one planar image and one spherical panorama, while exploring matching techniques that will perform well for scenes with repetitive features. A pseudo-fundamental matrix is defined for use with one calibrated image and one uncalibrated. This allows a photograph to be used without calibration while a panorama can be more easily considered as a whole. A global context descriptor for Speeded Up Robust Features and Maximally Stable Extremal Regions improves matching results and automatically computed epipolar geometry for scenes with buildings having repetitive features.

Downloads

The dissertation is available to download as a PDF.

The conference publication based on part of this work, Global Context Descriptors for SURF and MSER Feature Descriptors, is available in the IEEE digital library.