An Introduction to Color Machine Vision

Measurement and control of product color is critical in many industries, including food, pharmaceutical, and automotive assembly.  Color machine vision systems are replacing slow and unreliable human vision for inspecting colored objects, such as food or plastic parts.   Important issues in applying and integrating color machine vision (CMV) will be discussed, and, as an example, how CMV is used to inspect baked goods on a production line.

Human perception of color is influenced by genetic factors, experience, disease, age, adaptation, lighting, and the colors in the immediate environment.  For example, a color can appear different when put next to another color or as the size of a colored object changes.  These factors help explain why people disagree about paint colors and why a painted wall doesn't look like that little color sample from the paint store.  They also help explain why we are not very reliable color measuring instruments.

A photometer or other color measurement machine "sees" color based on integrated products of the illumination, the object’s surface reflectance, and the machine's sensors.  If we want to use a machine to approximate human color response, then the machine's response is calibrated in terms of a "standard observer" – an average of many people's perceptual responses to colors – such as the CIE color space.

In both machine and human color vision, it is common to have three sensors (receptors) with broadband responses in the red (long), green (medium) and blue (short) wavelengths of light.  Colors result from relative proportions of these three components.  The broadband response of the sensors means that colors with different spectra sometimes cannot be distinguished; these unintended matches are called “metamers.”  For example, light with a wavelength of 580 nanometers (nm) looks yellow, but a combination of green light at 540 nm and red light at 680 nm can also appear to be the same shade of yellow.  Multiple narrow-band spectral sensors are used to distinguish between metamers in some industrial processes, such as judging moisture content or pigment matching, but most applications of color machine vision are satisfied with three, broadband sensor types that approximate the response of the color (cone) sensors in our eyes.

A color machine vision application requires controlling both the intensity and spectrum of the inspected object’s illumination.  Incandescent light sources have a broad spectrum that makes color matching easier, but suffer from spectral shifts with temperature according to Wien’s law.  Most applications use fluorescent or “white” LED lights.  The spectrum from these lights is quite uneven, typically with intensity peaks in the blue end of the spectrum, but their spectrum is relatively constant over time and temperature.

Diffuse, white light illuminates colored pills in a blister pack while suppressing reflections off of the blister pack plastic.  The leading edge of a card of pills is detected by the part-in-place sensor and triggers image acquisition and analysis.  Incorrectly colored pills (purple, in this example) are detected and rejected downstream.  Information on errors are communicated to the plant’s quality control system.

A “reference patch” is usually included in the field of view of the CMV system.  This is a neutral, diffuse, white patch that reflects the illumination.  The CMV measures the spectrum of this reflected illumination and uses it to compensate for changes in the illumination spectrum.  Regular calibration and replacement of the illumination is also advisable.

The angle of the part or object with respect to both the light and the CMV system’s camera influences the measured color.  Diffuse objects, such as hamburger rolls, are less affected by angle changes, than surfaces with “depth,” such as painted surfaces.  For most parts, you should present the objects or parts to be measured to the CMV system in a known orientation to reduce the effects of angle changes.

Color Machine Vision Systems
The color machine vision system consists of a color camera or cameras, a processor, and software to make color measures and communicate these measures to the manufacturing and process control systems (see Figure 1).  Most color cameras use a monochrome sensor overlaid with a pattern of red, green and blue filters, called a “Bayer Pattern” after the inventor, to get the three broadband sensor types.  This reduces the camera cost but also reduces the camera’s spatial resolution.  You therefore should be careful when trying to measure part dimensions using a Bayer Pattern color camera.

The CMV system’s processor is typically a specialized computer, such as a DALSA’s Vision Appliance™, that interfaces directly to the camera and has control hardware, and digital inputs and outputs.  The control hardware and digital I/O are designed for easy integration of the processor into a production line.  For example, the digital outputs are used to “kick” defective product off a conveyer line.

Color machine vision software is specialized for making color measurements such as the location of objects based on color, the extent of a colored object, and color textures.  As noted, the software should also be able to compensate for changes in the lighting spectrum when using a “reference patch.”  The CMV software can report colors in many different “color spaces,” that is, standard representations for colors.  For many applications the red, green and blue values are sufficient but when we want to approximate human perception, then reporting is usually in a color space related to the CIE color space, such as L, a*, b* (Lightness, and a, b color components).

Inspecting Baked Goods
As an example, consider the inspection of baked goods such as hamburger buns, English muffins, or tortillas.  These baked products are sent by conveyer belt through an inspection station.  At the inspection station a laser profiler measures the product’s three-dimensional structure and a bright light and a color camera are used to measure product color (see Figure 2).  The product moves quickly, so the bright light is needed to give the camera enough reflected photons to form an image.  This video shows some of the process of making English muffins: http://www.sugden.ltd.uk/products/EMPLANT.asp

The red laser line is used for measuring 3D shape and the bright white line is used for measuring color.  The blue nozzles protruding from the box below the laser camera box are “air knives” that blow defective product down and out of the gap between this station’s conveyer belt and the next conveyer belt.  See www.montrose-tech.com

The color image of the baked goods is examined to ensure that it exhibits the proper colors.  Obviously incorrect colors, say green or orange, might indicate mold or contamination and must cause product rejection.  More subtle are colors that suggest to us that the baked goods are properly cooked.  Consider “toast marks,” on English muffins for example.  English muffins are baked in tins and the heel (bottom) appears brown with black marks.  The heel also has a dusting of farina or corn meal to prevent the muffin from sticking to the tin and to add taste and visual texture.

After the muffin is cooked, a “toast mark” is applied to the top of the muffin by briefly cooking it on a griddle (see Figure 3).  Without this “toast mark,” people think the product hasn’t been properly baked.  In fact, the product could be pasty white (assuming no contamination now!) and still be properly baked.  So these brown “toast marks” send the consumer a subliminal message that the product has been properly cooked.  Interestingly, the quality inspection ensures that the color matches the consumers’ expectation, and has little to do with the intrinsic quality of the product.

To be accepted, the “toast mark” has to be in a certain range of brown and a certain range of size.  Too light and you might think the muffin is not baked.  To dark and it looks burned.  Brown is a difficult color to measure – it is a “non-spectral” color as there is no wavelength in the physical spectrum that is brown, and our perception of brown is strongly influenced by surrounding colors.  Low intensity reds to orange wavelengths can be perceived as brown.

On English muffins, the brown is measurable by a color machine vision system and so the “toast mark” can be quantified by color and extent.  These measures could be used to adjust the temperature of the upstream toasting to give the desired mark color and extent.  Because we are matching human perception, color values are reported in as L, a*, b* components.

The vision software builds up the three-dimensional profile of the baked goods, checks the color and two-dimensional shape and might check for the presence of texture, decorations, etc.  The software also has to deal with partial images of the product – say a bun that is just “entering” the image – by computationally separating and remembering individual product items.  When defective product is found, the machine vision system signals a downstream mechanism to reject the product.

For example, as English muffins come out of the inspection system, defective product is blown off conveyer using one or more “air knives.”  This is shown in this video:  http://www.youtube.com/watch?v=6ScIeHMxyDI

The results of inspections are sent to the plant’s process monitoring system.  These results help the manufacturer adjust the production process, signal production problems, and can be the basis for a database of vision “recipes” or “solutions,” that makes changing products on a line relatively easy.

Using proper lighting, part presentation, processing, and communications, a color machine vision system can take over human visual inspection of colored products.  With easy-to-use color machine vision software and hardware designed for factory integration, subjective human color inspection can easily be automated.