by MATT NIXON
Combined with a long-wave infrared camera, a custom processor and image-processing algorithms detect and highlight obstacles that aircraft need to avoid.
Military aircraft, which may be manned or unmanned and fixed or rotary wing, routinely fly at altitudes below 1000 ft in support of their missions. Flying at these low altitudes presents a safety hazard to the aircrew and aircraft due to the occurrences of obstacles within the aircraft's flight path. The pilot must rely on eyesight and, in some cases, on terrain-awareness warning systems (TAWS) and ground-proximity warning systems (GPWS) to avoid obstacles. Insufficient resolution and performance from these systems can make it very difficult or even impossible to react in plenty of time to avoid collisions. This problem is exacerbated at night and in bad weather.
Numerous systems address the problem of aircraft obstacle avoidance. However, none of these systems provides a passive, real-time capability. The digital terrain-elevation database (DTED) is a database system that can alert the pilot to known obstacles, but can lack sufficient vertical resolution, rendering certain obstacles virtually undetectable. Furthermore, if something has changed, then the DTED will fail to alert the aircrew of the threat, and will provide a false sense of security.
Millimeter-wave radar (MMW radar) and laser radar (LADAR) are common methods for detecting obstacles in the aircraft flight path. These systems are historically large and expensive, and they emit radiation that can increase the detectability of the aircraft by a threat (in the case of a military scenario) or even potentially interfere with nearby instruments. In addition, LADAR systems have the problem that, if the angle of incidence between the obstacle and the aircraft is low, there will be insufficient energy return for target detection. These issues often make active systems unattractive. Another method of detecting power lines is through the measurement of the electromagnetic radiation they give off, which is of limited utility for nonpowered lines.
Obstacle-detection algorithms
Boeing-SVS Inc. is currently working in phase II of a Small Business Innovative Research (SBIR) contract sponsored by Naval Air Systems Command at their China Lake, CA, site, for a passive obstacle-detection system (PODS; see Fig. 1). The company has designed a passive, small, low-cost (less than $100,000), gimbaled, infrared-imaging-based system with advanced obstacle-detection algorithms. Obstacles are detected in the infrared band and curvilinear features are analyzed by innovative cellular automata software. Processed images are presented to the aircrew on their display as color-enhanced features, along with wide field-of-view visible images (situational awareness) for pilot evaluation and evasive action as appropriate.
The design provides two basic options. One option is a complete, stand-alone PODS hardware and software solution including the payload system, which incorporates a gimbal, sensors, navigation, controls, and display. The other is a PODS computer vision-enhancement module (PODS CVEM), which consists of the image-processing boards and custom software for aircraft that already have a suitable infrared sensor and display installed.
The system is configured in a modular fashion to provide maximum compatibility with existing sensors, gimbals, and displays. For example, if the platform has an adequate forward-looking infrared (FLIR) on board, it is possible that this sensor could be used with the CVEM, provided that the existing sensor has a comparable instantaneous field of view with that of the PODS FLIR. This approach provides the most system configuration options and allows for a potentially low-cost solution.
An active scanning laser or radar option can be incorporated, if needed, to achieve higher probabilities of detection and lower false-alarm rates for scenarios in which covertness is not a concern. A passive solution offers the greatest military utility, for the purposes of covertness and the potential to minimize cost and weight by using the infrared sensors and displays already installed on military aircraft.
Subpixel curvilinear features
The core advancement for the PODS effort has been the development of image-processing algorithms capable of detecting subpixel curvilinear features and a custom processor to run the algorithms in real time (the frame rate of the digital camera). The camera itself operates in the long-wave infrared region (see Fig. 2). There were a number of factors that drove the algorithmic approach. These include limited a priori information, obstacles that may have varied brightness and contrast levels across the obstacle, obstacles that may have random orientation, other obstacles that may obscure parts of the object of concern (for example, a tree in front of a power line), and so on.
A new technique based on the application of a cellular automata-derived technique to the vector direction field of the gradient image was developed (see Fig. 3). This technique performed better than the previously examined conventional techniques, proving to be far more robust in the presence of adverse "real-world" factors. The technique can also be parallelized, which has proven to be important for real-time implementation. Numerous images have been collected and tested during phase II of the SBIR. These images have been analyzed on the test-bed computer system (see Fig. 4).
It is possible for a "don't care" scenario to arise in which the system would correctly identify a feature that is not a threat to the pilot. For example, a road on a mountainside at a significant distance could appear as a subpixel curvilinear feature and obviously would not pose a problem. There is, however, a discriminator that could eliminate these "don't care" detections. Obtaining range data for the detected features would allow for lowering of some of the detection thresholds, yet eliminating many of the false alarms. A passive ranging system that could utilize the same hardware as the PODS with some additional software would provide a robust and highly effective detection capability.
Incorporating passive ranging
As part of the PODS development, Boeing-SVS has conducted a study to examine the feasibility of incorporating a passive ranging solution with the obstacle-detection algorithms. This passive ranging capability will not only provide discrimination between power lines and other naturally occurring linear features, but will also provide ranging information for other features in the image. Controlled flight into terrain (CFIT) is a leading cause of both military and civil/commercial rotorcraft accidents. Ranging to other features could be invaluable in the detection of other obstacles within the flight path and therefore the prevention of CFIT accidents.
For relatively small obstacles that are discernable from the background from 1 km away, we fully expect to be able to provide range estimates at 600 m that will be accurate to within 50 m, based on our brief study of passive ranging. This error is well within the required warning baseline/timeline for the comparatively difficult scenario of the Osprey (V-22) aircraft for which the program was baselined. The validity of this ranging-error prediction was supported by range calculations performed on actual imagery gathered in the field. The techniques used to arrive at these range and error estimates were not particularly sophisticated. It is expected that significant increases in ranging performance may be obtained following some refinement of the algorithms.
Boeing-SVS's PODS system has the potential to save lives and expensive hardware by aiding flight crews in cable and power-line detection for missions involving low-level flight. The passive nature of the sensors used ensures that mission covertness will not be compromised. The modular design of the system, with the availability of the CVEM module for aircraft that already have the necessary sensors, provides a potential low-cost means of retrofitting PODS onto a wide variety of military aircraft. The innovative algorithms developed for the project, coupled with the inclusion of passive ranging to distinguish near from too-distant obstacles, offer the promise of timely warnings for obstacle avoidance. This system has further application to the emerging unmanned aerial-vehicle community.
MATT NIXON is an associate technical fellow at Boeing-SVS Inc., 4411 The 25 Way NE, Suite 350, Albuquerque, NM 87109; e-mail: [email protected].