Confocal imaging is a well-established technique for biomedical and industrial imaging in which light returning from a target is spatially filtered prior to detection. In typical confocal laser scanning microscopy (CLSM) systems, a spot is raster-scanned over the target. The light return from the target is descanned and focused through a pinhole aperture that is positioned at a conjugate image plane. The pinhole spatially filters the return light, which helps to reduce image artifacts, improve image contrast, and isolate features of interest at specific depths.
Despite the well-known advantages of confocal imaging, the technical complexity and sensitivity to optical misalignment of these systems result in high acquisition and maintenance costs for many users. To deal with these costs, many academic institutions pool their resources to provide group access to the technology; however, access can be infrequent and inconvenient because of instrument availability and facility location. To be more broadly and effectively used by the scientific and industrial communities, the next generation of confocal microscopes needs to balance costs and accessibility with the performance requirements for specific applications.
Aeon Imaging, LLC (Bloomington, IN) has taken a new and more cost-effective approach to confocal imaging that makes use of the rolling shutter method of detection, which is commonly used by CMOS sensor chips. During each frame, the rolling shutter progressively scans across the pixel rows of the active sensor region with a shutter width related to the total frame exposure time. Light incident on the sensor outside the active rolling shutter exposure area is not captured and therefore cannot reduce contrast. In this manner, the rolling shutter acts as a linear aperture that sweeps across the two-dimensional (2D) sensor array every frame.
To perform confocal imaging with the rolling shutter, a target is illuminated with a line that is scanned across the field of view.1 Rather than descanning the light return, as done in CLSM, it is imaged directly onto the 2D CMOS pixel array, where the line is synchronized to the position of the rolling shutter throughout the frame exposure. With spatial filtering provided by a narrow rolling shutter width, directly backscattered light is preferentially detected, while out-of-focus and multiply scattered light is rejected. A key benefit of the rolling shutter approach is that it allows real-time adjustments to the linear aperture position and width in small (pixel) increments through software, which enables dark-field imaging,2 and increased axial resolution and edge enhancement using subtractive imaging.3
Recent advances in digital micromirror device technology have permitted the use of low-cost digital light processors/projectors (DLPs) for confocal imaging instead of the more traditional laser illumination source and scanning mirror(s). For implementation with a rolling shutter confocal system, the DLP simulates line scanning by rapidly projecting a series of narrow adjacent lines that are temporally and spatially synchronized with the position of the sensor's rolling shutter.4-6 The lack of descanning, enabled by the use of the rolling shutter, allows the DLP red, green, and blue illumination sources and micromirror array to be used in their factory-designed state, thus reducing the optical complexity of the imaging system.
Confocal imaging using DLP illumination and CMOS rolling shutter detection is realized in Aeon Imaging's Digital Light Microscope (DLM) platform (see Fig. 1). The platform's essential imaging components include a LightCrafter digital light projector from Texas Instruments (Dallas, TX), a 5 Mpixel MT9P031 CMOS rolling shutter sensor from Aptina Imaging (San Jose, CA), a lens pair to collimate the illumination and match it to the sensor field of view, the necessary timing and control electronics, and the acquisition software.
These components can be readily combined with a wide variety of optics, providing cost-effective modular and flexible imaging functionality. To date, the DLM has been configured for brightfield, darkfield, ophthalmic, fluorescence, polarization-sensitive, and interferometric imaging (see Fig. 2).
A unique feature of the DLM is the ability to finely adjust the confocal aperture position through software in real time. By offsetting the aperture from the directly backscattered illumination, the DLM operates in its darkfield imaging mode, collecting predominantly multiply scattered and defocused light from the sample. Subtractive confocal imaging, in which images acquired with different pinhole sizes or positions are scaled and subtracted, is a well-known method of narrowing the confocal axial point-spread function (PSF).7 This approach can be readily applied to the DLM with no mechanical adjustment, resulting in a marked improvement in its depth sectioning ability.8
To illustrate, the DLM was configured for epi-illumination microscopy with a 10X 0.25NA infinite conjugate Olympus Plan N achromat objective. Using a 1951 USAF resolution test target, an imaging resolution of 1.0 μm was obtained, with a 317 × 317 μm field of view. The illumination line width at the target was 3.5 μm and an aperture width at the target of 4.4 μm was used. The PSF was measured by translating a mirror through the focal plane. Under standard brightfield illumination, the full-width at half maximum (FWHM) of the PSF was calculated to be 76 μm.
Subtractive imaging was performed in two configurations. In the first, an image is acquired with the rolling shutter set to a positive offset with respect to the illumination line. A second image is acquired with an equal negative offset. The average of the positive and negative offset images, corresponding to the multiply scattered light obtained from both sides of the illumination line, was subtracted from the brightfield image. In the second configuration, the DLP patters are modified so that the target is illuminated with a pair of lines. The brightfield image is acquired with the rolling shutter synchronized to one of the two lines. The rolling shutter is then offset so that it is positioned between the lines to collect light from predominantly outside the focal volume, which is then subtracted from the brightfield image.
With subtractive imaging, the FWHM of the PSF decreated by a factor of 2.3 to 33 μm and 32 μm for the single and dual-line configurations, respectively. The dual-line configuration was found to be more light-efficient since very little light from the focal volume was obtained in the darkfield image. However, as the mirror target was moved out of focus, the darkfield image intensity exceeded that of the brightfield image due to the contribution of light from both illumination lines, causing PSF sidelobes. When further decreasing the aperture width to 1.8 μm, a PSF FWHM of 21.0 μm was obtained using single-line illumination, which approached the diffraction-limited axial resolution of 17.5 μm predicted by CLSM theory using a NA 0.25 objective and 1 Airy disk diameter pinhole. The subtraction of unwanted out-of-focus light from a leaf target using dual-line illumination is shown in Figure 3.
Many confocal imaging applications require not just an optical section, but a complete depth profile of the sample. This is typically obtained by acquiring a series of images while translating the sample along the depth axis (commonly referred to as a z-stack). When imaging with high-numerical-aperture objectives, creating a z-stack is a relatively slow process since vibrations caused by the mechanical sample motion must be avoided.
To create fast and accurate depth profiles, the DLM was set up to perform optical triangulation using a slight variant on the traditional epi-illumination microscope optical design. Optical triangulation is a well-established depth-measuring technique, commonly used in three-dimensional (3D) structured light imaging, which creates depth profiles by analyzing multiple images of a target taken with spatially varying illumination patterns.9 As compared to current structured light imaging techniques, the DLM's use of single line pattern illumination with rolling shutter detection eliminates the problem of phase unwrapping, which results in minimal post-processing to generate the DLM depth maps.
On the DLM depth profiler system, the illumination and detection pathways are spatially separated using a half-mirror in the Fourier plane prior to the objective. The ray schematic in Figure 4 illustrates how the illumination and detection angles differ for samples with directional scattering. As the sample depth changes, the illumination line is proportionally shifted to one side, resulting in a loss of spatial overlap with the rolling shutter and a corresponding loss in detected light intensity.
To create a depth profile of a sample, the temporal or spatial offset between the rolling shutter and illumination line can be adjusted over a series of image frames. The sample depth at each pixel is proportional to the offset that yields the maximum pixel intensity. Since the offset can be adjusted electronically with high speed and precision, no sample motion is required to create a similar z-stack depth profile.
To measure the depth profiling performance, the DLM was configured with a 10X 0.25 NA infinite conjugate Olympus Plan N achromat objective. The relationship between the rolling shutter offset and sample depth was calibrated by scanning the row offset with a sample mirror positioned in-focus and at ±10 μm from the in-focus position. The central mean and standard deviation of the resulting depth maps indicated a 0.95 ±0.36 pixel offset/micron relationship. With a 10X objective, this method of depth profiling was found to perform best over a range of ±25 μm from the in-focus reference position.
Depth profiles were obtained for microfluidic sample molds provided by the University Health Network Microfabrication Centre (Toronto, ON, Canada). The fabrication of microfluidic devices requires a high level of 3D accuracy, which is commonly obtained in practice by iteratively adjusting the thickness of a spin-coated photoresist film and measuring the resulting channel height.
The facility's desired channel heights, typically ranging from 20 to 75 μm, are an excellent match to the profiling capabilities provided by the DLM configured with a 10X objective. The sample images shown in Figure 5 were acquired using a 10X Edmund Optics high resolution objective and a Point Grey (Vancouver, BC, Canada) Flea3 USB 3.0 CMOS rolling shutter camera operating at 27 frames/s. Depth profiles are obtained and displayed to users in under 5 s, depending on the offset scan range and scan increment.
The combination of DLP illumination with CMOS rolling shutter detection provides several advantages over earlier generations of confocal imaging technology, including low component cost, relative ease of alignment, and modular use in a variety of customizable optical systems. By configuring the optical system for triangulation, depth profiles can be rapidly measured to micrometer precision without computationally expensive post-processing.
ACKNOWLEDGEMENTS
Matthew S. Muller would like to acknowledge the support and assistance of Swept Image for depth profile imaging using the Flea3 camera, and the University Health Network Microfabrication Centre for providing microfluidic device samples.
REFERENCES
1. A. E. Elsner and B. L. Petrig, "Laser scanning digital camera with simplified optics and potential for multiply scattered light imaging," U.S. Patent #7,831,106 (2007).
2. M. S. Muller, A. E. Elsner, D. A. VanNasdale, and B. L. Petrig, "Multiply scattered light imaging for low cost and flexible detection of subretinal pathology," OSA Tech. Digest, FME5, FiO (2009).
3. M. S. Muller, A. E. Elsner, and B. L. Petrig, "Inexpensive and flexible slit-scanning confocal imaging using a rolling electronic aperture," OSA Tech. Digest, FiO (2008).
4. M. S. Muller, "Confocal imaging device using spatially modulated illumination with electronic rolling shutter detection," U.S. Patent #8,237,835 (2011).
5. M. S. Muller, "A pico projector source for confocal fluorescence and ophthalmic imaging," Proc. SPIE, 8254-7 (2012).
6. M. S. Muller, A. E. Elsner, and G. Ozawa, "Non-mydriatic confocal retinal imaging using a digital light projector," Proc. SPIE, 8567 (2013).
7. S. J. Hewlett and T. Wilson, Mach. Vis. Appl., 4, 233–242 (1991).
8. M. S. Muller, "Enhanced confocal axial resolution by subtractive digital light microscopy," OSA Tech. Digest, FiO (2013).
9. V. Srinivasan, H. C. Liu, and M. Halioua, Appl. Opt., 23, 3105–3108 (1984).
Objective lenses: At the heart of image quality
This novel rolling shutter confocal microscopy system relies upon precision alignment and exact synchronization to produce quality images. But, as with any optical system, the images can be no better than the optics. That's why it's important to select an objective lens that meets the requirements for numerical aperture, the degree of aberration correction, and the design wavelength range.
Numerical aperture (NA) is a function of focal length and entrance aperture. Higher NA enhances lateral resolution, increases light collection, and decreases both depth of field and working distance. High NA objectives-greater than 0.95-require oil immersion to gather a wider cone of light.
As NA increases, so does sensitivity to optical aberrations. To support improved resolution at high NA, the objective must correct for spherical and chromatic aberrations and for field curvature. The industry has adopted terminology to reflect the degree of correction. "Semi-plan" objectives have about 80% field flatness; "plan" objectives improve that to about 90%. "M Plan" objectives are corrected even further.
Microscopists must also consider the design wavelength range of the objectives. This is particularly important for epi-fluorescence microscopy, where the objective provides illumination and collects fluorescence, and must provide high transmission at both the excitation and emission wavelengths of the fluorophores.
The industry offers a wide range of objectives to meet a variety of system requirements (and budgets). Olympus Plan Achromatic Objectives are useful when cost is a constraint. They're excellent choices for visible light microscopy techniques, including basic confocal microscopy, where field flatness correction is necessary but not critical. Olympus Plan Fluorite Objectives provide excellent flat field images from the ultraviolet (UV) to near-infrared (NIR; 350–1000 nm). They're useful when an application demands higher NAs for low-light level detection across a broad wavelength range.
Mitutoyo's NIR, NUV, and UV infinity-corrected long working distance objectives combine superior aberration correction and field flatness over enhanced UV and IR spectral ranges. The NIR objectives are corrected from 480 to 1800 nm, making them good choices for visible confocal microscopy. These objectives fit applications that demand long working distances, extremely flat field imaging, and locations where NA is not critical.