IMAGE: Two input images (from left) stitched together using the proposed method (right). (Image credit for original image: Blanka M. Lukes, Prague Private Guides; Image credit for modified image: Wei Wang and Michael Ng)
Philadelphia, PA--A paper published in the Society for Industrial and Applied Mathematics (SIAM) Journal on Imaging Sciences (http://epubs.siam.org/doi/abs/10.1137/110819871) by Wei Wang (associate professor at Tongji University in Shanghai, China) and Michael Ng (professor at Hong Kong Baptist University) describes an algorithm for image stitching (image fusion) that aims to improve panoramic photography by seamlessly blending two or more input images with overlapping regions into one picture.
Many different approaches for image blending are seen in the literature. "The traditional method is to search for a curve in the overlapping area in which the differences among the input images are minimal," explains author Michael Ng. "However, the curve may not be determined accurately because of light intensity, color inconsistency, parallax, occlusion, etc."
The software approach used in this paper instead minimizes seam artifacts by smoothing the transition between the images. The mosaic image here is a weighted combination of the input images. This means the pixel values from the two overlapping images are combined using a weighted average for qualities such as exposure, local contrast, and saturation.
How is this achieved? Many systems, both natural and man-made, seek out the lowest energy state, such as a ball rolling down a hill or a snow-laden tree branch bending to maintain the lowest possible energy in the system. The concept of minimizing the energy of a given system is also used in image processing. For a given image, an energy function is defined and minimized to get a better image (with less noise, better sharpness, and higher contrast, to name a few). This is the approach the authors use in the paper. Seamless combination of images is achieved by minimizing an energy function based on intensity or gradient differences of the two images.
"According to the model, we construct a weighting function over the overlapping area so that a panoramic image can be generated," says Ng. "The optimal weighting function can be obtained by minimizing the overall energy of the mathematical model." Thus, in the proposed model, both the weighting function and the final blending in the overlapping region are based on solving an energy minimizing problem. The authors show how to define an energy function and develop an algorithm to minimize it. This variational method--based on achieving the lowest energy or ground state--is seen to produce a more visually appealing photo in comparison to other existing methods.
Future work may extend the scope of this research beyond two-dimensional images. "It is interesting to consider extending the current variational approach to tackle the problem of three-dimensional image stitching in medical imaging applications and stitching video in computer applications," Ng says.
The Society for Industrial and Applied Mathematics (SIAM), headquartered in Philadelphia, PA, is an international society of over 14,000 individual members, including applied and computational mathematicians and computer scientists, as well as other scientists and engineers. Members from 85 countries are researchers, educators, students, and practitioners in industry, government, laboratories, and academia. The Society, which also includes nearly 500 academic and corporate institutional members, serves and advances the disciplines of applied mathematics and computational science by publishing a variety of books and prestigious peer-reviewed research journals, by conducting conferences, and by hosting activity groups in various areas of mathematics. SIAM provides many opportunities for students including regional sections and student chapters; see www.siam.org.
SOURCE: SIAM Nuggets; http://connect.siam.org/seamless-photography-using-mathematical-models-for-image-stitching/