IMAGING: Compressive sensing enables single-pixel digital camera

In the detector array of a conventional digital camera, each pixel performs an analog-to-digital conversion; for example, the detector on a 5-megapixel camera produces 5 million bits for each image. This large amount of data must often be reduced by a couple of orders of magnitude through jpg or other compression algorithms so as not to overburden typical storage and transfer capacities. This sample-then-compress process, however, wastes the battery power required to run electronic compression algorithms and the detector capacity used to collect the information from 5 million pixels, of which 80% to 90% can be discarded.

Richard Baraniuk and Kevin Kelly, engineers at Rice University (Houston, TX), have built a compressive-sampling (CS) camera in which compression is performed optically and simultaneously with detection. They described their research at the Optical Society of America’s Frontiers in Optics 2006 conference (Oct. 8-12; Rochester, NY).

The compression process in this device requires no battery-powered electronic calculations, and the compressed optical signal can be time-multiplexed, potentially enabling conventional digital-camera photography with a single-pixel detector.

Developed within the last two years, the enabling mathematics for CS are similar in concept to mapping a signal onto a frequency space of overlapping sine waves using a Fourier transform. But instead of sine waves, this mapping is done on the random speckle patterns of white noise. Because just a few measurements are taken (far fewer than the number of pixels), the pioneering mathematicians had to work backward though compression algorithms using linear-programming techniques in a manner similar to code-breaking, Kelly said.

The researchers found that random sampling essentially invoked a generalized Heisenberg’s uncertainty principle in relation to how compressed a signal (or wave function) can be in two domains simultaneously, Baraniuk said. While the implications of this finding are far-reaching, the upshot for digital photography is that the camera only needs to collect an amount of data proportional to the eventual output of the compression algorithm.^{1, 2}

Rather than collect 5 million pixels for an image, it might only be necessary to sample a factor of say three or four times the 50,000 pixels that the jpg compression might typically output. So only 200,000 single-pixel measurements would provide an immediate 25-fold savings in data collected compared with 5 megapixels.

A proof-of-concept camera developed at Rice replaces the CCD array in a digital camera with a digital-micromirror device (DMD) produced by Texas Instruments (Dallas, TX). The researchers place a sequence of random patterns on the micromirror array, bounce the image off of each pattern in the sequence, and collect the reflected light from each pattern sequentially with a photodiode sensor that acts as the single-pixel detector (see figure). After taking a sequence of essentially time-multiplexed measurements, they use an optimization algorithm based on the newly developed mathematics to tease the picture out of the collected sequence of single-pixel measurements.

The optical-compression process is inherently more efficient in the single-pixel camera in terms of power consumption because it occurs naturally as light passes through a lens. It is not necessarily faster, however, and is currently limited by the switching speed of the DMD, which is 15 ms. Commercial DVD systems in rear-projection TV sets, for instance, switch three orders of magnitude faster, which could enable a single-pixel camera to operate with exposure speeds similar to commercially available digital cameras.

The researchers have already prototyped faster systems in the lab and could also increase speed by using more than one photodetector in parallel-a 4 × 4 array, for instance. In addition, the DMD essentially acts as a shuttering mechanism for the proof-of-concept device, and different shuttering mechanisms might conceivably increase exposure speeds.

The intent is not to go after the digital-camera market, however. Near-term applications focus is on digital imaging for spectral regions such as IR and terahertz, in which large-area detector arrays would be prohibitively expensive. Even when using exotic detector materials, however, a camera based on a single-pixel detector could be orders of magnitude less expensive than one requiring 5 megapixels.

Elsewhere, CS algorithms are being explored for new camera designs, such as an almost totally flat, lensless camera. And new approaches to medical imaging in computed tomography and magnetic-resonance-imaging systems are under investigation, as well as DARPA-funded research on new approaches to analog-to-digital conversion.