BIOSPECTROSCOPY: High-resolution sensors capture application defined spectral bands in real time

GEORGE THEMELIS AND VASILIS NTZIACHRISTOS

Arecently developed method of multispectral imaging–which combines the advantages of imaging and spectroscopy–allows concurrent capturing of spatial and spectral information in a way that overcomes the limitations of existing technologies.¹ The approach was developed to serve the needs of medical diagnostic applications and is initially focused on measurement of diagnostic fluorescent probes and for tissue blood oxygenation. It can, however, be useful in other fields.

The technique enables creation of multispectral sensors that compare to state-of-the-art CCD sensors in terms of resolution and frame rate, and that also compare in terms of cost and size with 3CCD sensors (which use a prism to split incoming light and relay it to three CCDs that register red, green, or blue). It portends an advanced generation of cameras that add multispectral imaging capabilities to standard functions–for virtually no extra cost.

Overcoming limitations

Traditional multispectral imaging technologies can be divided into two general categories: scanning systems that acquire spectral and/or spatial data, and nonscanning systems that concurrently capture spatial and spectral information.

Scanning systems resolve spectra using interferometric methods, light dispersing elements such as prisms or grating, or, most commonly, tunable bandpass filters, such as liquid-crystal tunable filters. This category of system offers good spatial and spectral resolution and is well suited for studying stationary objects. However, scanning requires significant time, which compromises performance for studying dynamic phenomena or moving objects.

Nonscanning systems can be used to study both stationary and moving objects, and therefore are suited to a wider range of applications. Simultaneous capture of spatial and spectral information is usually achieved by means of prisms or mirrors, which split light so that the same field of view at different spectral bands is visualized on one or more imaging detectors. The limitation of existing nonscanning systems lies in the limited number of spectral bands captured (usually three to five), and in the complexity of the optics, which makes such devices bulky and expensive.

The key advantages of the new approach–developed at the Institute for Biological and Medical Imaging (IBMI), Helmholtz Center Munich, and Technische Universität München (TUM)–are that it captures a large number of spectral bands and it has a simple design that results in small and inexpensive multispectral sensors. It combines color CCD cameras and multiple-bandpass filters–for example quadruple-bandpass filters (QBPF), which transmit light in several discrete narrow spectral bands (see Fig. 1). Appropriate calibration of the system allows measurement of multiple images at defined spectral bands.

The color CCD is a matrix of sensors (pixels) covered by sets of n microfilters interleaved in a grid pattern. For example in CMYG (cyan, magenta, yellow, green) CCDs, for each group of four adjacent pixels, each pixel is covered by a different filter (see Fig. 1a). Assuming that a QBFP with central wavelengths λ₁, λ₂, λ₃ and λ₄ is coupled to a CMYG color CCD, the sensitivity of each pixel is altered as light is filtered twice, by the QBPF and by one microfilter (see Fig. 1b and 1c). Because CMYG filters are relatively broad, all pixels are sensitive to all four narrow spectral bands, but at different levels. The sensitivity of a pixel to each of the QBPF bands can be symbolized as: wXi, where I = 1, 2, 3, 4 refers to the QBPF spectral band and X = C, M, Y, G refers to the microfilter that covers the specific pixel.

Assuming that the light that reaches the CCD has intensities Iλi at the spectral bands λi then the signal measured by a CCD pixel covered by microfilter X will be given by:

S_X = W_X1 · I_λ1 + W_X2 + W_X3 · I_λ3 + W_X4 · I_λ4

For a CMYG camera, a system of four equations can be written to describe the intensities received by the n = 4 pixels, which can be written in matrix linear equation form, S = W · I, where S is the vector with the signals measured by a set of four (CMYG) pixels, I is the vector of light intensities at the four spectral bands, and W is the calibration matrix for the combination of one QBPF with one CCD sensor. If the calibration matrix W is known, then the last equation can be solved for I by simple matrix inversion: I = W^-1 · S.

Matrix W is a linear matrix that transforms a color measurement S into spectral intensities I. The calibration matrix W can be calculated experimentally by measuring m ≥ n different color samples with known reflection spectra, or more generally light sources with known intensities at the spectral bands λi, where n is the number of different microfilters.

Technology in action

This method allows common color CCD cameras to simultaneously measure four selected spectral bands, which can be very useful for applications that require imaging of multiple narrow spectral bands as an alternative to the broadband responses of standard microfilters in color cameras. Importantly, with the use of this method and appropriate beamsplitters, two or more color cameras can be used to image eight or more spectral bands (see Fig. 2).

FIGURE 2. A four-CCD prism, combined with four color CCDs–each with a different QBPF–enables simultaneous measurement of 16 spectral bands.

A simple experimental setup implements the multispectral imaging method described above (see Fig. 3). It incorporates a polychroic mirror that splits light by transmitting four spectral bands and reflecting the remaining light. The eight spectral bands generated by reflectance and the mirror’s transmission mode uniformly cover the visible spectral region, from 380 to 700 nm. Thus, the mirror serves both as a light splitter and as QBPF for a pair of cameras. The same principle can be applied with a more complex setup involving multiple polychroic mirrors and multiple cameras.

FIGURE 3. In the experimental multispectral imaging system using two cameras, a polychroic mirror serves both as a light splitter and as a QBPF for both cameras.

The performance of the experimental setup indicates that the method enables simultaneous measurement of eight spectral bands for a wide range of spectra. In a checkerboard consisting of a wide variety of colors, continuous lines represent the reflectance spectra captured using the spectrometer, while dots represent the multispectral data (see Fig. 4). The curved line of the reflection spectrum is successfully measured for all color samples, with an average error of 5%. These errors are likely attributed to the CCD’s limited sensitivity in the blue-UV region and to low contrast between transmitting and reflecting wavelengths of the polychroic mirror.

FIGURE 4. Reflectance measurements made using the experimental setup (dots) compare well with those made with a spectrophotometer (solid lines).

In practice, the setup enables differentiation between two similarly colored materials, for instance green leaves and green plastic, that have different reflectance spectra (see Fig. 5). The measurements differentiate the two reflectance spectra and display the spatial distribution of each material. A pseudocolor representation of the spectrally unmixed data shows the metameric materials displayed with different colors (see Fig. 5d). Capture and analysis can be done in real time.

FIGURE 5. A color image of green leaves on a green plastic background (a); map of the spectral component of the leaves (b); map of the background material (c); and composite color image of the leaves and background material (d).

Biomedical and beyond

The new technology was developed for, and thus is currently focused on, medical diagnostics. Fluorescent probes make cancer lesions visible when excited by light–but their effect is compromised by autofluorescence, which diminishes contrast between healthy and diseased tissue. Furthermore, because biological tissues tend to be highly inhomogeneous in the way they attenuate light, fluorescence signals often don’t represent concentration of the fluorescence substance. Imaging at multiple spectral bands, however, allows the correction of fluorescence measurements for both autofluorescence and for light attenuation by tissue. The technology allows simultaneous imaging of the required spectral bands and therefore can provide corrected images of the diagnostic fluorescent probe distribution in real time. Moreover, the capability of the method to measure up to 20 spectral bands allows the simultaneous use of two or more probes in order to increase the specificity and sensitivity of the method.

Multispectral imaging in reflectance mode allows measurement of the concentration of multiple tissue chromophores such as oxygenated and deoxygenated hemoglobin. Therefore, instead for just capturing color, we are able to measure parameters of great diagnostic value such as local blood concentration and oxygenation.

Other current biomedical applications include endoscopic and intraoperative imaging, which the Klinikum Rechts Der Isar, TUM, and Groningen University are pursuing cooperatively. Beyond that, several studies have indicated the great potential of multispectral imaging as an analytical tool in such areas as quality control, artwork analysis, surveillance and defense, security, forensics, and agriculture. The new approach brings greater flexibility and performance to all of these fields.

REFERENCES

G. Themelis , J.S. Yoo, V. Ntziachristos, Optics Letters 33(9): 1023-5 (2008)

GEORGE THEMELIS is a scientist with the Institute for Biological and Medical Imaging (IBMI), Helmholtz Center Munich. VASILIS NTZIACHRISTOS is Director of the IBMI, chair for Biological and Medical Imaging at the Technical University of Munich, and head of the university’s Medical Imaging Laboratory. Contact Dr. Themelis at [email protected].