Nearly every portion of the pharmaceutical manufacturing process, from drug synthesis to integrity testing as well as final packaging and label tracking, requires some form of machine vision inspection. Whether testing colored pills, foil packaging for absorption-based patches, syringes for missing or broken components, 2D data matrix codes, and even 3D Braille labeling, pharmaceutical manufacturers are depending on sophisticated optics, laser-based inspection, and software tools to meet rising volume demands for product inspection.
Smart serialization
Machine vision is fundamental to tracking serial numbers—whether bar codes, 2D quick response (QR) codes, or any variety of U.S. Food and Drug Administration (FDA), European Medicines Agency (EMA), or other internationally required codes—throughout the pharmaceutical manufacturing process. Drug counterfeiting is leading state and federal agencies to initiate legislation requiring item-level drug serialization and pedigrees (an audit trail that follows a drug from time of manufacture to final distribution at a pharmacy). In response, machine vision system manufacturers like Cognex (Natick, MA) are developing smart serialization technologies.
"It's costly and complex to implement traditional PC-based imaging systems into pharmaceutical manufacturing and packaging environments," says John Lewis, Cognex market development manager. "Rather than plugging discrete cameras and framegrabbers into a conventional PC, smart camera vision systems integrate a CPU, image sensor, control, and don't require a PC or even a monitor to run vision tools." Lewis says that the Cognex In-Sight Track & Trace smart camera system uses low-cost distributed processing with high-speed networking to provide infinite scalability and integrate easily with touchscreen displays for direct implementation on the factory floor where needed (see Fig. 1).
"Pharmaceutical vision systems manufacturers are continually challenged to adapt to increasing line speeds requiring faster processing," says Rick Roszkowski, senior director of vision system marketing at Cognex. "Small, complex injection-molded clear plastic parts used in drug delivery systems often require custom illumination techniques, and round or curved packaging challenges optical character recognition [OCR] software." In response, Cognex OmniView software uses images captured from four sides of a cylindrical object and combines them into a 3D model with very high accuracy. The software can then read bar codes and check cap skew, label quality, and safety seal presence. Cognex also offers the DS1100 3D laser profiling system that generates an image with brightness values denoting distance from the sensor so that users can measure heights of objects, depths of holes, and angles of different surfaces.
"While a good OCR system will read poor-quality text and make a decision as to whether the symbol being read is an 'o' or a 'q' for example, we offer something more—OCV or optical character verification," says Jonathan Ludlow, machine vision promoter at Microscan Systems' Northeast Technology Center (Nashua, NH). "OCR says little about label quality, while OCV not only accurately reads text, but judges quality and legibility to ensure that customers can accurately read product labels and insert pamphlets." Ludlow says that Microscan's fifth-generation OCV software tool images a good example of known text, "remembers" it, compares it to production labeling using a defined template, and applies rules about how much difference is allowed to verify that legibility requirements have been met.
Microscan incorporates its OCV technology into a full machine-vision hardware/software solution of cameras, lighting, and controllers that not only reads outer package labeling, inner pamphlet text, and blister-pack text, but also verifies that these discrete items belong to each other. In addition, Microscan says that manufacturers must ensure their system hardware and software continues to address next-generation developments such as the increasing use of laser marking on pharmaceutical packages (see Fig. 2).
Vision challenges
As if reading printed text or data matrices on curved and reflective surfaces was not challenging enough, the European Union has required that the name of medications be printed in Braille on pharmaceutical labels and on patient information leaflets for blind and partially sighted customers since late 2010, leading to the development of numerous ISO Braille standards that will eventually be implemented worldwide.
Among commercially available OCR-based Braille imaging technologies like SARA from Freedom Scientific (St. Petersburg, FL), Eye-Scan from Guerilla Technologies (Palm City, FL), and ScannaR software from Baum Retec (Wiesenbach, Germany), the DotScan inspection system developed by in-situ (Sauerlach, Germany) reads Braille dots and assesses their tactile quality with high accuracy. Non-contact testing uses shape-from-shading (SfS) technology based on a defined illumination model that reconstructs the 3D surface of a carton independently from its print.
DotScan SfS-based surface reconstruction is analogous to walking uphill and downhill through a landscape with the gradient vector g = (p,q) being the step heights. Starting at an arbitrary point (x0,y0) with an arbitrary height Z0(x0,y0), and integrating the gradients g along a path P from point to point through the entire picture yields the height map Z(x,y) of the surface as:
Z(x, y) = Z0(x0, y0) + P∫gds
At least three independent equations and three images with different light directions are required to calculate height of the Braille dots. The large field of view of the images requires correction of spherical lens aberrations, compensation of small deviations from telecentricity and homogeneity of the light sources, and low-pass filtering to suppress label bulges and optimize the gradient calculations amongst master markings, images, and logos that are usually color-printed on pharmaceutical labels (see Fig. 3).
Optical hardware
The smart cameras and OCR algorithms that process pharmaceutical images are only as good as the clarity and accuracy of the images presented. Consequently, machine vision systems depend on a variety of physical optics to obtain crisp images that are easily processed.
In the pharmaceutical industry, 360 degree vision is useful to inspect both the top and sides of an object (such as a vial, ampoule, or pill container) at the same time, with just one camera. Pericentric lenses from Opto Engineering (Mantova, Italy) are designed to inspect objects up to 60 mm in diameter and work with half-inch and third-inch detectors, granting the most appropriate optical magnification factor to achieve the field depth required for high-resolution 3D pericentric imaging (see Fig. 4). Opto Engineering also supplies a TCCAGE integrated optomechanical system that provides four orthonormal views of an object with a telecentric lens through an array of mirrors with onboard backlight and direct illumination.
"Pericentric or hypercentric lenses—also offered by Edmund Optics—can solve many 3D imaging challenges as long as the package is not too tall and standoff distances are reasonable; otherwise, the lenses get ridiculously large and expensive," says Gregory Hollows, director of machine vision solutions at Edmund Optics (Barrington, NJ). "Pharmaceutical manufacturers cannot tolerate defects or errors in labeling, packaging, or physical product, and consequently expect their machine vision systems to do everything-with ultrahigh resolution, wide field of view, and high contrast, even if it means incorporating bulky and costly optics."
Edmund Optics' numerous online tools and videos help users select the best optics to achieve a desired imaging resolution. Specific to the pharmaceutical industry, Hollows cautions that machine vision is all about tradeoffs; maximizing contrast at a desired resolution is not a trivial matter when considering optical components.
"Not all lenses are created equal; many optics advertise similar capabilities, but that doesn't guarantee performance," adds Hollows. For example, 2D codes on two sets of pharmaceutical bottles imaged with a similar lens under the same conditions (similar resolution and contrast) appear quite different; images from the first lens are robust while images from the second lens lack clarity—affecting both accuracy and repeatability of the system (see Fig. 5). Simply adding more pixels to a machine vision system like this will not increase the system's capabilities due to the lack of contrast; proper selection of optics fabricated with good design and highly controlled fabrication techniques are key to better vision results.
Counterfeit conundrum
Serialization goes a long way towards reducing the potential for counterfeit drugs to show up at your local pharmacy. But suppose a counterfeiter gains access to serial numbers and fabricates 'substitute' medications? Is there a way to ensure that the package received at the pharmacy is the same one that left the drug manufacturer? The answer is "yes" if a pharmacy takes advantage of laser surface authentication (LSA).
Developed at Imperial College London, improved for industrial use by Cambridge Consultants (Cambridge, England), and produced/marketed by Ingenia Technology (London, England), LSA instrumentation consists of a handheld reader that illuminates a package with red lasers and forms a two-dimensional image of an object's reflectivity as it is scanned (see Fig. 6). The proprietary reflection signature or "fingerprint" is compared against a database of signatures acquired by incorporating an LSA scanner within the machine vision system at the manufacturing site, authenticating the package. Because each reflection signature is unique and because it is impossible for a counterfeiter to replicate the surface structure of an item to obtain a matching fingerprint, the technology is considered 100% reliable.
Of course, the ultimate in anti-counterfeiting assurance is to physically verify that the drug in the package is the actual medication intended for the patient. One way to achieve this level of authentication is to incorporate spectral analysis capabilities within the machine vision solution. Today, spectral measurement sensitivity has improved to the point where drug contents can be analyzed at the distribution point or point-of-sale location without even opening the bottle; for example, Cobalt Light Systems (Harwell, England) uses spatially offset Raman spectroscopy (SORS) to analyze liquids at airports through clear and opaque containers.
Unlike other pass/fail inspection applications such as recycling or fruit sorting, the medical industry has particularly rigorous inspection demands. Microscan's Jonathan Ludlow reminds us that machine vision system manufacturers supplying pharmaceutical manufacturers must pay attention to the smallest details, even making sure that PC controllers or smart cameras are housed in enclosures to prevent the trapping of individual pills that could later spill onto a conveyor and contaminate a new batch of medication—a costly mistake with possible life-or-death consequences.
