A new, single-lens 3D capture technique breaks the magnification barrier for optical microscopy, enabling the combination of stereoscopy and high-magnification—with the capability for real-time interaction with specimens. The technology has also been adapted to endoscopy for application to minimally invasive surgery.
ByShawn Veltman and Paul Dempster
Recent advances in image capture, display, and processing technologies are helping drive a shift toward three-dimensional (3D) imaging in both digital and high-magnification microscopy. Key medical imaging approaches such as computed tomography (CT) and magnetic resonance imaging (MRI) have transitioned from 2D to 3D technologies already, and 3D is now widely accepted for these applications. Three-dimensional microscopy involving a single camera also offers benefits to other biomedical imaging areas that have traditionally been interpreted using only 2D.
Pairing 3D microscopy with virtual microscopy—which is the viewing of digital microscope images on a computer, over a network, or on a monitor—may enhance the speed and ease of certain types of diagnoses by adding depth information within samples. Like the shift from 2D to 3D, the move from looking through microscope eyepieces to virtual microscopy comes with a host of advantages. These include better ergonomics; less eye fatigue, neck strain, and headaches in operators; easier information-sharing and collaboration; and optimized teaching environments.
Overcoming limits in 3D microscopy
It is important to note that the ability to see in three dimensions through a microscope is not new. Stereomicroscopes, which display two separate optical paths (i.e., a separate objective and eyepiece magnification for each eye), have a centuries-long history. However, stereomicroscopes tend to reach their practical limit at 100x magnification; it has traditionally been difficult to produce the same image from separate optical paths at higher magnifications. Thus, stereomicroscopes cannot be used for diagnostic and research applications that require magnification in the 200–1000x range.
Depth information is not unimportant at these higher magnifications, though. Trained operators frequently use monocular depth cues to estimate depth information, and many use the popular "z-stacking" technique. Z-stacking takes multiple views of the same sample with different focus settings to obtain a rough idea of 3D space. Confocal microscopy, too, can create a 3D reconstruction of the cell, but while these kinds of approaches can aid depth understanding, they do not allow real-time interaction with the slide.
Ironically, the very factors limiting the magnification range of stereomicroscopes also point to the solution of capturing 3D at those higher magnifications. Axiomatically, each increase in magnification is a decrease in the area being viewed. As the area being viewed becomes smaller, the allowable distance between lenses to resolve a coherent 3D image decreases. At magnifications of more than 100x, the allowable distance becomes smaller than the diameter of the lens being used, and trying to have two lenses resolving on the same area becomes impractical.
It is possible, though, to capture two viewpoints through a single objective lens, while maintaining the ability to adjust the separation of those viewpoints to accommodate for different magnifications. This capability is being realized in a 3D imaging system developed by ISee3D (Vancouver, BC, Canada), which generates stereoscopic image pairs at very high magnification using patented image capture and optical switch technologies (see Fig. 1). The system incorporates a microscope adapter that can display a high-magnification stereoscopic image on any 3D monitor, enabling faster and potentially more accurate depth understanding than by the use of focus manipulation or z-stacking (see Fig. 2).
Two from one
On a conceptual level, the basis of this new technology is relatively easy to understand. Consider one technique to capture two images through a single lens: By blocking a portion of the lens, a new center point, closer to the edge of the non-blocked side, is created. If the left half of the lens is blocked and captures an image frame and then the right half of the lens is blocked and captures an image frame, two images from different viewpoints are created-in other words, a stereoscopic image pair.
ISee3D's microscope adapter is built upon a more sophisticated version of this technique, creating two viewpoints within a lens and optimizing the spacing between the left and right images for each magnification and numerical aperture (NA). Because the spacing required to see 3D comfortably at 200x magnification will be larger than at 400x, this intelligent shifting of the distance between images is a critical component of the system. Incidentally, this requirement to reduce the distance between images as magnification increases is the same challenge that prohibits stereomicroscopes from resolving images at these higher magnifications.
As the NA increases, so does the distance between the extreme left and right viewing angles. It is this property that allows for the capture of a stereo image pair through a single lens: The capture of two images through different viewpoints close to the edges of the viewing angle as opposed to a single image through the center of the lens. Initial testing of the adapter with different lenses showed that NAs at or above 0.75 provide the necessary angles.
By using a single lens and sensor, the camera and optics are synchronized as an integrated imaging chain, eliminating convergence alignment issues, which may be present in other stereoscopic systems using dual cameras and lenses. By setting the convergence point equal to the focus, the resulting output are carefully aligned and balanced 3D images that appear natural to the human eye, reducing eye strain and making it easier for workflow when switching between the 3D display and the microscope view.
Camera considerations
During development of the system, the engineering team conducted a battery of tests with several different cameras selected for their abilities to provide superior (crisp, sharp) images and produce very accurate color.
Toshiba Imaging's (Irvine, CA) 3CCD IK-HD1 camera led in the performance criteria that were critical to the success of the 3D microscopy imaging solution (see Fig. 3). The camera's prism block imaging technology features high-sensitivity three-chip CCD sensors and high-definition (HD) 1920 × 1080 video format, which-combined with advanced digital signal processing-effectively support 3D imaging. Already used in many microscopy and clinical video applications, the Toshiba camera features high signal-to-noise ratio and wide dynamic range at 30 fps for lag-free, real-time display. Its specialized optics make the camera a crucial component in achieving pixel-to-pixel alignment through the system.
In 3D, image fidelity, sharpness, color, and resolution are critically important. Since imperfections barely noticeable in 2D become difficult to resolve in 3D, every part of the imaging path from the lens to the sensor must be optimized to give the truest representation of the image. Feedback from several practitioners in histology and pathology indicate immediate uses for the technology and may prove beneficial for biological research, among other applications.
Using variations of the single-lens 3D technology, the ISee3D technical team has built working prototypes of a single-lens 3D rigid endoscope and a single-lens 3D microscope for use in minimally invasive surgery and medical research, respectively. The results from this technology promise new and advanced 3D imaging applications for a variety of disciplines.
ISee3D has developed a number of other 3D capture technologies for the medical device market as well, including a 2D-to-3D endoscope adapter and a single-lens 3D endoscope able to produce as much light or more than its 2D counterparts. Recent studies suggest that experienced surgeons prefer performing minimally invasive surgeries in 3D, and novice surgeons utilizing 3D experience significantly reduced error rates.1 While the company has demonstrated the ability to capture 3D through endoscopes with a <1 mm diameter, its most recent designs focus on 3D endoscopes with a 4 mm diameter.
REFERENCE
1. S-H Kong and B-M Oh, Surg. Endosc., 24, 1132–1143 (2009).
Shawn Veltman is product manager with ISee3D (Vancouver, BC, Canada; www.ISee3D.com), and Paul Dempster is director of sales with Toshiba Imaging Systems Division (Irvine, CA; www.toshibacameras.com). Contact Veltman at [email protected] and Dempster at [email protected].
More BioOptics World Current Issue Articles
More BioOptics World Archives Issue Articles