Algorithm decodes individual image frames from a single long exposure through an opaque medium

Sept. 30, 2018
The technique has applications in a wide range of fields from security to healthcare to astronomy.

Described in a Nature Scientific Reports article by researchers at Duke University (Durham, NC), coded apertures and memory effect techniques have been used to capture a number of individual still frames from a single long photographic exposure taken through a diffuse, non-uniform, mostly opaque medium. The technique has applications in a wide range of fields from security to healthcare to astronomy.

"When I explain to people what this algorithm can do, it sounds like magic," said Michael Gehm, associate professor of electrical and computer engineering at Duke. "But it's really just statistics and a ton of data."

When light gets scattered as it passes through a translucent material, the emerging pattern of "speckle" looks as random as static on a television screen with no signal. But it isn’t random. Because the light coming from one point of an object travels a path very similar to that of the light coming from an adjacent point, the speckle pattern from each looks very much the same, just shifted slightly.

With enough images, astronomers used to use this "memory effect" phenomenon to create clearer images of the heavens through a turbulent atmosphere, as long as the object being imaged is sufficiently compact. The technique fell out of favor with the development of adaptive optics, which do the same job by using adjustable mirrors to compensate for the scattering. A few years ago, however, the memory effect technique became popular with scientists again. Because modern cameras can record hundreds of millions of pixels at a time, only a single exposure is needed to make the statistics work. But while the memory effect can reconstruct a scattered image, it has its limitations. The object has to remain motionless and the scattering medium has to be constant.

Gehm's new approach to memory effect imaging breaks through these limitations by extracting a sequence of images from a single, long exposure. The trick is to use a coded aperture. Think of this as a set of filters that allow light to pass through some areas but not others in a specific pattern. As long as this pattern is known, scientists can computationally extract what the original image looked like.

Gehm's new technique uses a sequence of coded apertures to stamp which light is coming from which moment in time. But because each image is collected on a single, long photographic exposure, the resulting speckle ends up even more of a jumbled mess than usual. "People thought that the resulting speckle pattern would be too random to separate out the individual frames," said Gehm. "But it turns out that today's cameras have such amazing resolution that if you look closely, there's still enough of a pattern to computationally get a toehold and tease them apart."

In their experiment, a simple sequence of four backlit letters appeared one after the other behind a coded aperture and a scattering material. The shutter of a 5.5-megapixel CCD camera was left open for more than a minute during the sequence to gather the images.

While the best results were achieved with a 100-second exposure time, good results could still be obtained with much shorter exposure times. After only a few seconds of processing, the computer successfully returned the individual images of a D, U, K, and E from the sequence. The researchers then showed the approach also works when the scattering medium is changed, and even when both the images and scattering mediums are changing.

The best results were achieved when the letters appeared for 25 seconds each because the intensity of the backlight was not very high to begin with and was even further diminished by the coded aperture and scattering material. But with a more sensitive camera or a brighter source, there's no reason the approach couldn't be used to capture live-action images, Gehm said.

The technique has many potential applications. Not only does it work for light scattering through a material, it would also work for light scattering off of a surface--say the paint on a wall. This could allow security cameras to work around corners or even through frosted glass.

In the medical arena, many light-based devices look to gather data through skin and other tissues--such as a Fitbit capturing a person's pulse through their wrist. Light scattering as it travels through the skin and flowing blood cells, however, poses a challenge to more advanced measurements. This technique may provide a path forward. "We're also looking to see if this approach can be used to separate different aspects of light, particularly color," said Gehm. "One could imagine using coded apertures to gain more information about a single image rather than using it to obtain a sequence of images."

SOURCE: Duke University;

About the Author

Gail Overton | Senior Editor (2004-2020)

Gail has more than 30 years of engineering, marketing, product management, and editorial experience in the photonics and optical communications industry. Before joining the staff at Laser Focus World in 2004, she held many product management and product marketing roles in the fiber-optics industry, most notably at Hughes (El Segundo, CA), GTE Labs (Waltham, MA), Corning (Corning, NY), Photon Kinetics (Beaverton, OR), and Newport Corporation (Irvine, CA). During her marketing career, Gail published articles in WDM Solutions and Sensors magazine and traveled internationally to conduct product and sales training. Gail received her BS degree in physics, with an emphasis in optics, from San Diego State University in San Diego, CA in May 1986.

Sponsored Recommendations

Next generation tunable infrared lasers

Nov. 28, 2023
Discussion of more powerful and stable quantum cascade tunable infrared lasers, applications, and test results.

What AI demands mean for data centers

Nov. 28, 2023
The 2023 Photonics-Enabled Cloud Computing Summit assembled by Optica took an aggressive approach to calling out the limitations of today’s current technologies.

SLP feature for lighting control available on cameras offering

Nov. 28, 2023
A proprietary structured light projector (SLP) feature is now available on the company’s camera series, including the ace 2, boost R, ace U, and ace L.

Chroma Customer Spotlight - Dr. David Warshaw, About his Lab

Nov. 27, 2023
David Warshaw, Professor and Chair of Molecular Physiology and Biophysics at the University of Vermont (UVM), walks us through his lab. Learn about his lab’s work with the protein...

Voice your opinion!

To join the conversation, and become an exclusive member of Laser Focus World, create an account today!