Optical filters and math clarify underwater images

The challenge for nanophotonics is not simply one of fabricating nanostructures, according to Frank Schellenberg, a strategic marketer for photolithography software at Mentor Graphics (San Jose, CA).

Nov 1st, 2004
Th 162928

Studies have shown that certain marine animals actually use polarization to improve their underwater vision.1 This knowledge has been expanded upon and improved by researchers at the Technion-Israel Institute of Technology (Haifa, Israel), who have developed a computer-vision approach that combines mathematics with polarization-­filtering optics to remove degradation effects in underwater images.2

Unlike methods that use artificial illumination to improve image quality, the researchers use a passive computer-vision approach that exploits natural illumination. By associating polarization with the prime visibility disturbance (backscatter) that has to be deleted from the object, the image can be reconstructed rather than merely enhanced.

As amateur divers, Yoav Schechner and his student, Nir Karpel, applied their “physics-based vision” approach to the fundamental concept that more polarization at a pixel means more backscatter. Because traditional image-enhancement methods do not model the spatially varying distance factors that increase object degradation, they are limited in their ability to clarify underwater images. The image-formation model used by the researchers, however, automatically accounts for backscatter and its associated polarization dependencies on object distance and estimates a distance map of the scene as a byproduct.

In contrast to images in the open air that are illuminated by sunlight, which comes from many directions depending upon the time of day, underwater images are illuminated by a limited cone above the scene resulting from refraction of the Sun’s rays through the water surface, a phenomenon known as the “optical manhole” or “Snell’s window.” For this reason, underwater natural backscatter is partially polarized horizontally and its intensity depends on the distance of the object from the camera. By photographing an underwater scene in two polarization orientations, the amount of polarization-dependent backscatter as a function of distance from the original image pixel locations can be obtained.

A series of mathematical equations is used to approximate the contribution of backscatter and attenuation to image degradation, with the clear understanding that backscatter is the dominant contributor. The analysis shows that the recovered image radiance is related not just to the difference in polarization states of the two images, but is a complex inversion of the image-formation process in which polarization is related to backscatter and water transmittance as a function of image distance for each individual pixel in the underwater scene.

Estimating water parameters

To perform image recovery, an estimate must be made of the intrinsic parameters of the water and lighting that affect the amount of backscatter (such as suspended particles). This is done by measuring pixels corresponding to objects at infinity (for which signals are negligible due to attenuation). After the image degradation is compensated for, a white-balancing operation is performed to correct color bias in the recovered image. The resulting recovered image has a much improved contrast and color (see figure).

Click here to enlarge image

An underwater reef scene is viewed through an "aqua-polaricam." The raw, uncorrected image (top) is how the scene appears through the polarizer. The "unveiled" image (bottom) is the result of applying polarization ­filtration, color ­balancing, and a mathematical ­correction ­algorithm to the original image data, which removes back­scatter and attenuation, ­leaving a ­significantly clearer image with improved color and contrast while nearly doubling the visibility range.
Click here to enlarge image

A distance map of the underwater scene generated as a byproduct of the ­radiance-recovery process allows a ­quantitative determination of the improvement in visibility of the underwater scene when using the camera combined with the mathematical models. By viewing the image before and after the recovery process and identifying two points in the image at different distance locations, yet with identical ­contrast, the researchers were able to determine that their optical and mathematical method demonstrated an approximate doubling of the ­visibility range.


1. M. P. Rowe et al., Optics Lett. 20, 608 (1995).

2. Y. Y. Schechner and N. Karpel, Proc. IEEE Computer Vision & Pattern Recognition 1, 536 (2004).

More in Research