This thesis presents a novel fast randomized matching algorithm for finding correspondences between small local regions of images. We also explore a wide variety of applications of this new fast randomized matching technique.
The core matching algorithm, which we call PatchMatch, can find similar regions or "patches" of an image one to two orders of magnitude faster than previous techniques. The algorithm is motivated by statistical properties of nearest neighbors in natural images. We observe that neighboring correspondences tend to be similar or "coherent" and use this observation in our algorithm in order to quickly converge to an approximate solution. Our algorithm in the most general form can find k -nearest neighbor matchings, using patches that translate, rotate, or scale, using arbitrary descriptors, and between two or more images. Speed-ups are obtained over alternative techniques in a number of these areas. We analyze convergence both empirically and theoretically for many of these image matching algorithms.
We have explored many applications of this matching algorithm. In computer graphics, we have explored removing unwanted objects from images, seamlessly moving objects in images, changing image aspect ratios, and video summarization. Because our technique for removing unwanted objects from photographs is both high quality and interactive, due to the fast matching algorithm, it has been included in Adobe Photoshop CS5 as a new feature "content aware fill." In computer vision we have explored denoising images, object detection, detecting image forgeries, and detecting symmetries. We also apply our algorithm to large collections of images. We conclude by discussing the limitations of our algorithm and areas for future research.
Cited By
- HaCohen Y, Shechtman E, Goldman D and Lischinski D (2013). Optimizing color consistency in photo collections, ACM Transactions on Graphics, 32:4, (1-10), Online publication date: 21-Jul-2013.
- Papas M, Houit T, Nowrouzezahrai D, Gross M and Jarosz W (2012). The magic lens, ACM Transactions on Graphics (TOG), 31:6, (1-10), Online publication date: 1-Nov-2012.
- Barnes C, Goldman D, Shechtman E and Finkelstein A (2011). The PatchMatch randomized matching algorithm for image manipulation, Communications of the ACM, 54:11, (103-110), Online publication date: 1-Nov-2011.
Recommendations
PatchMatch Filter‐Census: A slanted‐plane stereo matching method for slope modelling application
AbstractImage matching is a well‐studied problem in computer vision. Conventional image matching is solved using image feature matching algorithms, and later deep learning techniques are also applied to tackle the problem. Here, a slope‐modelling ...
PatchMatch: a randomized correspondence algorithm for structural image editing
This paper presents interactive image editing tools using a new randomized algorithm for quickly finding approximate nearest-neighbor matches between image patches. Previous research in graphics and vision has leveraged such nearest-neighbor searches to ...
PatchMatch: a randomized correspondence algorithm for structural image editing
SIGGRAPH '09: ACM SIGGRAPH 2009 papersThis paper presents interactive image editing tools using a new randomized algorithm for quickly finding approximate nearest-neighbor matches between image patches. Previous research in graphics and vision has leveraged such nearest-neighbor searches to ...