For any high-speed camera system, the sensor is very much at the core of its capabilities. Today, advanced CMOS-based camera sensors enable frame rates of one million frames per second.
ANDREW BRIDGES
Increasing demands for higher speeds and greater resolution are driving advances in high-speed digital imaging cameras. In one example introduced in 2008, the Fastcam SA5 from Photron (San Diego, CA) is designed to capture up to 7,000 frames/s at full resolution or 1 million frames/s with reduced resolution, surpassing previous high-speed performance records for similar camera designs. As with any high-speed camera, the sensor is very much the heart of the system.
It starts with the sensor
The SA5 is based on a complementary metal-oxide semiconductor (CMOS) sensor. Although CMOS imaging sensors have been around for more than 20 years, only recently has the technology advanced enough to enable the design of architectures that support the imaging speeds required for the most demanding applications.
The SA5 camera electronics have been optimized for the sensor to provide superior image quality. The video format used is based on a 1024 x 1024 advanced CMOS sensor with global electronic shutter. This megapixel sensor is divided structurally into a number of capturing sites or wells called pixels. Photons are collected at the pixel well and then transferred to a floating diffusion (shielded) photodiode, amplified, and then read out.
The sensor performs the task of capturing light energy (photons) striking its structure and converting them to electrical charges that are proportional to the amount of illumination. Our sensor has 1024 pixels in the horizontal direction and 1024 pixels in the vertical direction (see Fig. 1). There are two basic types of CMOS pixels: passive and active. The passive pixel allows XY addressing, while the active pixel also has XY addressing but the pixel is amplified at the pixel site. The SA5 sensor is an active pixel design and delivers full resolution up to 7000 frames/s, megapixel resolution (1024 x 1000) to 7500 frames/s, and reduced resolution to 1.0 million frames/s.
FIGURE 1. A graphic representation shows the Fastcam SA5 complementary metal-oxide semiconductor (CMOS) sensor active pixel design, with an array of 1024 horizontal pixels (yellow boxes) and 1024 vertical pixels. (Courtesy of Photron)
To achieve the 7000 frames/s bandwidth at 1024 x 1024 pixel resolution, the sensor must read out multiple columns in parallel. This bandwidth at full speed and full resolution is called the aggregate bandwidth (maximum transfer rate), defined for the SA5 as 7000 x 1024 x 1024 x (12 bits/8 bits) = 9.174 Gigabyte/second (GB/s). Note that the "12 bits/8 bits" in the equation above is to translate the 12-bit data from a pixel into bytes since memory is always discussed in terms of bytes.
Camera resolution is largely influenced by the architecture of the sensors and the light-sensing elements. While pixels vary in size from 5 µm square to as large as 20 µm for high-speed cameras, the number of pixels in the horizontal and vertical directions determines the resolving capability in the field of view. In addition, the size of an individual pixel determines the physical size of the sensor array. Because large pixels are more light-sensitive, the SA5 is designed around a pixel sensor with 20 µm square pixels to capture the maximum amount of light provided by the optics (20 mm square form). With a high fill factor, a larger portion of the pixel is gathering light, making the camera more efficient and reducing external lighting requirements (reducing both cost and the unwanted heat generated by the lighting).
When tested against ISO 12232, a standard that defines the speed and sensitivity of a camera sensor, the SA5's sensor is rated at 6400 ISO/ASA using the SNR40 noise-based definition for "excellent" image quality, and 25600 ISO/ASA using the SNR10 noise-based definition for "acceptable" image quality. With a high-resolution output averaging 33% better than the closest comparable camera, the SA5 also includes selectable resolutions such as 960 x 528 at 15,000 frames/s, 512 x 512 at 25,000 frames/s, and 256 x 256 at 87,500 frames/s.
All high-speed cameras can provide full resolution (1024 x 1024 pixels) up to a specific frame rate, but as frame rate increases, resolution decreases. Some sensors are limited in this area as they can only reduce the resolution in one axis, typically the vertical axis. This is fine as long as what you want to record fits within the wide and short aspect ratio. But when you need to maintain a more traditional square or rectangular aspect ratio, a lot of data is lost due to image cropping. The SA5 allows the user to select the desired aspect ratio in "blocks" of 64 pixels wide by 16 tall around the optical center of the sensor, or it allows the user to simply drag the mouse around the area of interest to be recorded.
Dynamic range
The 12-bit analog-to-digital converters (ADCs) in the SA5 are designed to provide excellent dynamic range across the full gamut of gray scales, with no detail lost in the shadows, while whites and blacks remain crisp and clear in the 12-bit monochrome version. The color fidelity of the 36-bit red/green/blue (RGB)-Bayer color version ensures blue skies and striking details that are clearly visible, regardless of the lighting conditions.
Given that many computer operating systems and file formats are limited to eight bits, the user can select which "adjoining" eight bits are to be saved. Of course, all 12 bits can also be saved in a selectable file format, or if the image is dark or over-exposed, the user can select a higher or lower eight bits for display and archiving. Alternatively, the record duration may be increased by as much as 33% by recording eight bits of data instead of 12. Obviously this will have some impact on the dynamic range; but if the length of recording time is paramount, this can be a highly effective and easy solution.
Some high-speed analytical applications require a very short interval between successive frames, so the SA5 has been designed with a minimal interval time of 230 ns. This feature is important for applications such as particle image velocimetry (PIV) analysis of fluid flows ranging from large flows to micro PIV purposes (see Fig. 2) and digital image correlation (DIC) measurement of solid object deformations when struck or stressed. In both these cases, the shorter the time between images captured, the greater the accuracy of measuring displacement versus time in the end.
FIGURE 2. A high-speed sequence shows fluid flow; a water-spray sequence is captured at 7000 frames/s. (Courtesy of Photron)
Shuttering
Although high-speed images can be captured without additional shuttering, portions of the image, when closely examined, can be blurred. The SA5 includes an optional submicrosecond shuttering function with a minimum shutter time of 369 ns to reduce or eliminate unwanted blurring. Lighting is an important issue with such a short exposure but applications such as ballistics—which tend to be self-illuminating—definitely benefit from this very short duration opening (see Fig. 3).
FIGURE 3. In a ballistics test, a pistol-firing sequence is recorded at 2000 frames/s. (Courtesy of Photron)
To accommodate the large amounts of data that need to be stored relative to the demand for higher resolution and higher-speed cameras, the SA5 camera has three memory configurations: 8 GB, providing 0.75 s duration at full resolution and 7000 frames/s; 16 GBytes for more than 1.5 s with the same parameters; and 32 GBytes, offering more than 3.0 s at megapixel resolution and 7000 frames/s. Obviously, these record times can be increased by reducing the speed, resolution, trigger method (burst), or pixel depth. While 0.78 seconds may not sound like a long time, it equates to 5460 separate images, each with 1024 x 1024 12-bit pixels. When these are replayed at the NTSC video standard rate of 30 frames/s, it takes more than three minutes to watch the entire sequence.
Fortunately, most high-speed events occur in a fraction of a second, with the average event duration being roughly 100 ms. To ensure the capture of all action, there are several trigger modes. The camera can record every frame after a trigger is received until the memory is full, or can operate in an endless record mode, continuously overwriting data until a trigger is received, at which point it stops recording and saves the last frames recorded. Another option allows the available memory to be divided 50/50 around the trigger for 50% recording up to the trigger and 50% after the trigger. The user can also define how many frames are saved before and after the trigger is received. Another mode gives the user a specified number of frames recorded every time a trigger is received until the memory is full. And finally after an initial trigger starts the recording at (x) frames/s, a second trigger increases the frame rate by a factor of two, four, or eight times, depending on the user's selection.
In addition to these options, the SA5 also offers the ability to partition its memory so that the user can define the available memory, independent of the configuration or trigger method selected, into from one to 64 blocks. As each block or partition is filled, the system can be set up to automatically increment to the next partition and wait for the trigger to fill the next one, until all are filled. The user can then configure the system to automatically download all data via gigabit Ethernet communication.
Advances in slow-motion imaging will likely continue with even faster cameras, smaller sensors, and improved image resolution, enabling the visualization of high-speed events, with many wonders yet to be discovered.
ANDREW BRIDGES is director, sales & marketing at Photron, 9520 Padgett Street, Suite 110, San Diego, CA 92126-4446; e-mail: [email protected];www.photron.com.