2000: The Year Color Strikes Back

2000: The Year Color Strikes Back

If a black and white picture is worth a thousand words, then proponents of color machine vision systems would contend that an RGB picture is Moby Dick to monochrome's Cat in the Hat. Color images offer at least three times more data than simple gray scale pictures thanks to their red, green and blue or more color channels (or hue saturation and intensity for robotics). Manufacturers as a benefit for automated inspection often have touted this additional data. However, in the early days starting with the first industrial color inspection systems for the food industry in the 1980s, color-based vision gained a bad reputation as computationally intensive, expensive and quirky - too dependent on lighting and the 3D features of an object under test. 

Today, color vision proponents say that computer processors are more than a match for color applications. 'This 'slow' perception is wrong,' claims Steven Prehn, director of vision products at Integral Vision (Farmington Hills, MI). 'I could load you up with supporting data.'

Figure 1: Key Technologies uses near-infrared illumination in addition to RGB data to separate blueberries from Japanese beetles that greatly resemble the fruit in color, size and shape.

'In my opinion the idea of processing bottlenecks is a myth. Companies like Key Technologies have been using color very successfully in high speed food processing for years so it hasn't been stymied there. If you eat food you've undoubtedly benefited from their product& It's changing slowly. It's an attitude thing,' explained Robert McConnel, technical officer at Way-2C (Arlington, MA). 'Many people have been burned by difficulty of using traditional methods and/or marketers promises not kept.'

Sales data would seem to support Prehn and McConnell's points. According to Nello Zuech, author of the Automated Imaging Association's annual vision market report, 'The market for color-based machine vision may be materializing& North American companies generated revenues of $126 million based on the sale of 3501 units in 1999. This represented a 33.9 percent increase in revenues from such sales and a 26.7 percent increase in units sold over the previous year.'

Color conundrums
According to Automated Inspection Systems, Key Technology, Inc. (Walla Walla, WA) color vision changed significantly since it first entered the food industry and will continue to change. Specifically, today's latest systems not only have to perform multi-tiered processing to identify features and eliminate noise in RGB space, some systems are spreading beyond the visible spectrum into the near-infrared and ultraviolet (Figure 1). 

Key's efforts are indicative of an expanding understanding about the way color vision has to be adopted to fit the specific needs of industrial inspection. According to Way-2C's McConnell, although color CCD cameras and image processing boards capable of handling three color channels have been around for some time, the industry is only now coming into a fuller understanding of how color vision differs from monochrome. 

'There's plenty of computer horsepower around to do color-based classification. The main problem is that if one tries to extend traditional monochrome vision processing techniques to the complex color arena they fail miserably. You can cut wood with a
hammer but no one will like the product. If one uses appropriate tools & the problems disappear,' he said.

Luckily, the work of many years and vision experts are now resulting in appropriate vision tools including software and hardware that specifically targets color-based industrial inspection.

Algorithms get colorized
'Most essential software tools, those required for the classification of image objects based on their colors (color image segmentation), have not been around very long,' according to director of color vision development at Dimac (Højbjerg, Denmark), Peter Locht.

According to Locht, at an early stage in the development of color vision, the machine vision industry chose to conceive color image segmentation as a simple matter of performing traditional gray image segmentation of each of the three color channels and then somehow combining those results into a single segmentation. Unfortunately, Locht said that while thresholding is the correct segmentation method for a gray image or a single color channel, that concept is not trivially generalized into three-dimensional color space.

'A naïve combination of independent thresholding of the three channels means that a particular class of colored objects can only be represented as a box-type structure in color space. This is a serious restriction in any but the most trivial scenarios (e.g. involving a few artificially colored objects). A color class describing naturally colored objects tends to be accurately representable in color space only as a rather complicated shape, and simplifying that shape only leads to nasty class overlaps and subsequent misclassifications of image objects,' Locht added.

Dimac's approach has been to develop an algorithm that permits color classes to have any shape in color space - a class may even consist of several 'blobs' in color space. The algorithm has been further automated through self-training procedures that put the onus of color classification on the computer with minimal operator input. Dimac's color segmentation algorithm has not only been used in conventional industrial inspection applications such as quality control of baking and roasting foods and the production of cement and textiles, but its color approach has also led to automating some laboratory procedures such as the detection of pathological cells in microscopy. 

Figure 2: (Top) Image from a polished surface from a cement nodule showing three different mineral phases (Alite, Belite and Melt) plus two air voids. (Bottom) Same image after color segmentation using Dimac's algorithm showing the separate mineral phases.

Color cameras target machine vision
Color segmentation, although crucial, cannot be taken alone out of context. Akron, OH-based Applied Vision's president Amir Novini continues to define the different requirements for color vision by drawing attention to recent advances in color-based industrial cameras and illumination techniques.

'We have much better sensors these days. Sensors are less expensive, and more importantly the camera itself is made for automation purposes rather than close circuit or broadcast video. Abilities such as running them at frame rates higher than standard NTSC, to asynchronous reset and start the camera -- things of that nature have been an enabler. The basic color sensor -- the good one -- has been around for 10 years in solid state fashions, but initially just like most of the other sensors, they were intended for different usage, not necessarily for machine vision. [Today] there appears to be market for industrial color vision cameras, and companies such as Dalsa and Pulnix, etc., are focusing on automation. They're making cameras that you can asynchronously drive with frame rates that keep up with high speed throughput,' Novini said.

Novini, who plans to release his color version of the Genius system early next year, adds that the latest color hardware improvements also aid in calibration - a crucial element of a successful color system. 'Calibration of the sensor with the light source is a lot easier because you have the sensor under your control. The light sources are also a lot better. For instance, some of the stuff from Mercron comes to mind. Mercron produces high-frequency fluorescent drivers that can produce a fairly stable source of white illumination for good rendering of absolute color information. Ten years ago, you would have had to design that particular part yourself.'

Novini goes on to say that, 'we are becoming much smarter in terms of taking what is typically an HSI environment into our software and conducting mathematically complex analysis to determine what we're looking at [in color space.] In doing so, we're able to make better absolute and relative color measurements using smart algorithms that allow us to find minute flaws... and at the same time control the illumination in real time to ensure that what we are seeing is accurate and true. Those are the things that need to happen, and quite honestly, a lot of this is a learned through experimentation. It's more than fundamental science, but I'm not aware of too many universities that have a complete course in machine vision illumination or color vision. A lot of this stuff is new and learned through integrating it yourself.'

About the Author: Prior to accepting his position as managing editor of Wireless Design Online, Winn Hardin spent several years covering the vision industry for a variety of leading trade publications.