Integrating 3-D vision with robotics enables flexible measuring systems

Combining 3-D vision with robots satisfies the need for flexible, programmable dimensional measuring systems for the assembly plant floor.

Aug 1st, 2003
Th 131019

In the 1980s, auto manufacturers began implementing totally automated in-line inspection of dimensions of car bodies using laser-based sensors. Systems for measurement of a body-in-white (an assembled, unpainted body before mounting of doors, hoods, and decklids) typically consisted of a large frame with 80 to 120 three-dimensional (3-D) laser sensors, with each sensor aimed at a specific critical zone on the body. Smaller subassemblies, such as underbodies, doors, hoods, and decklids, were measured with similar approaches, but with fewer sensors.

While these systems proved effective at providing quality information to the builder, they were limited by their ability to measure only one model. When a new model was introduced, the sensors were repositioned to the new critical measuring points.

More recent trends in auto manufacturing involve the implementation of flexible assembly lines capable of assembling many different models and providing rapid response to changes in customer demand. Flexible assembly requires a new approach to in-line dimensional measurement because the dedicated sensor approach does not provide the necessary flexibility to inspect multiple intermixed models.

Flexible sensors

Coordinate measuring machines, as found in almost all auto manufacturing plants, provide programmable flexible measurement capability, but typically are slow in operation and commonly are located in a temperature-controlled room, remote from the manufacturing operation.

FIGURE 1. A robotic system with a 3-D sensor mounted on the robot end effector measures components with different geometries .
Click here to enlarge image

The challenge of providing flexible, programmable measurement capability was met by integrating 3-D laser measurement sensors with industrial robots (see Fig. 1). The robot provides a robust, programmable sensor-positioning device to sequentially place the sensor at programmable measurement points. Robots are designed for reliable operation on the plant floor in the manufacturing environment.

While mounting a laser sensor on a robot seems a simple solution, making a reliable measuring system requires considerably more integration than simply bolting a sensor onto the robot end effector. Most measurements on a car body are complex, including edge and surface locations, vertex locations, hole positions, and locations of studs. Many of these items have positions that are specified in two or three dimensions. It is therefore desirable to use a sensor that has 3-D measuring capability.

The 3-D sensors used with robots are typically based on laser triangulation and include a 2-D CCD camera, laser line projector (a 670- to 680-nm laser diode), and light-emitting diodes (LEDs) for surface illumination. The sensor technology for robot applications is similar to that used in the fixed sensor applications. The laser line projector is used to determine standoff dimensions by triangulation, and the LEDs illuminate the surface to measure locations of holes and slots by gray-scale imaging (see Fig. 2).

Click here to enlarge image

FIGURE 2. The 3-D sensor uses two types of illumination—laser line and area projection—used sequentially to obtain 3-D data. The laser line projector determines standoff dimensions by triangulation, and the LED illuminates the surface in a full-field mode to measure locations of holes and slots by gray-scale imaging. For 3-D measurements, the laser line is used to determine surface data, then the laser is switched off and the LED is turned on to determine feature locations (holes, slots).

For 3-D measurements, the laser line is used to determine surface data, then the laser is switched off and the LEDs turned on to determine feature locations (such as holes and slots). If only one type of measurement is required, only that sequence is used to reduce overall cycle time.

The sensor has multiple image-processing algorithms for each type of measurement, with specific algorithms selected automatically for each measurement point. A graphical user interface is provided for a simple setup of sensor parameters.

3-D sensor plus robot

When a 3-D sensor is mounted on a robot end effector, it is subjected to significant dynamic forces as the robot moves, so all sensor components are enclosed in a rugged housing (see Fig. 3). The sensor cable is subjected to flexing as the robot moves, so a robotic-grade cable must be used to avoid breakage. A typical sensor weighs a few kilograms, easily accommodated by industrial robots. The 3-D sensor used for robotic applications has a standoff distance from the sensor to the center of the measuring zone of 100 mm and a three-sigma accuracy of 0.05 mm.

FIGURE 3. The 3-D laser sensor is mounted on the robot end effector (left). Heavy-duty packaging ensures reliable operation in the dynamic robot environment (right). The 120 × 78 × 70-mm sensor box has a standoff distance of 100 mm, a resolution of 0.005 mm, cycle time of 150 to 300 ms, weighs 1.2 kg, and contains a 670-nm laser diode, an LED, and a CCD camera.
Click here to enlarge image

Temperature compensation is important to successfully integrate a robotic measurement system. Industrial robots are subject to thermal expansion as the ambient temperature changes and to heat from the robot motors in the robot itself. Thermal expansion is on the order of magnitude of 0.5 mm. While unimportant when placing spot welds on a body, this thermal expansion creates unacceptable errors in measurements.

To eliminate effects of thermal expansion, automatic temperature compensation can be integrated with the measuring robot. Compensation includes correction software and fixed artifacts placed at known positions in the measurement volume. At times when the robot is not required to take measurements, such as during part transfer or at break time, the robot takes measurements of the artifacts. The software can then automatically compensate the system for thermal expansion.

To display the resulting measurements, storage, statistical process-control analysis, and graphic display capability can be used in a fashion similar to fixed sensor systems. The reporting package provides the ability to compare each measurement point to adjustable tolerances, and to display the results in color graphics for interpretation (see Fig. 4). A programmable logic-control capability provides the ability to stop the line or provide alarms for parts not conforming to tolerances.

FIGURE 4. Typical SPC reports from a measuring robot show color coding in simple graphic formats that provide easy-to-read information.
Click here to enlarge image

All of these elements must be integrated into the measuring system. Ideally, integration results in a common user interface for robot, sensor, diagnostics, and programming.

Plant integration

Most assembly plants have a specific robot used for the assembly operations such as welding and material handling. Generally it is desirable to use the same type of robot for measurement applications, because plant personnel are familiar with programming and maintenance of the robot and spare parts are readily available.

The result of integration of all these components is a programmable 3-D vision/robot measuring system. Such systems have found many areas of application in the body-assembly shop and at Tier 1 suppliers.

Off-line robotic systems are used to replace multiple hard-fixture gauges for subassemblies. The fixture gauges are expensive, take up significant floor space, and require skilled operators. A single robot, with programs for multiple part geometries replaces many fixture gauges. An added advantage to the flexible approach is that new models or changes to existing part geometries are accomplished with inexpensive program changes. Single robots can be installed in-line to perform 100% measurement of subassemblies with a mix of geometries.

For larger assemblies, such as a body-in-white or underbody, an in-line 100% measurement station usually consists of four robots to provide enough measuring volume and adequate cycle time to measure all critical points. An added requirement when multiple robots are used in a single cell is the ability to combine the data from all robots into a single database with a common origin. This coordination provides accurate reporting of differential values such as the width of the body by combining data from the left- and right-side robots. More than 150 of these types of measuring robots have been installed worldwide.

WALT PASTORIUS is technical & marketing advisor at LMI Technologies, 2835 Kew Dr., Windsor, ON N8T 3B7; e-mail .

More in Test & Measurement