« Back To Vision & Imaging Industry Insights
AIA Logo

Member Since 1984


AIA - Advancing Vision + Imaging has transformed into the Association for Advancing Automation, the leading global automation trade association of the vision + imaging, robotics, motion control, and industrial AI industries.

Content Filed Under:



From Raw Materials to Packaged Goods, Machine Vision Keeps Food on the Table

POSTED 03/05/2013  | By: Winn Hardin, Contributing Editor

Photo courtesy of The University of Auckland, New ZealandMachine vision solutions for the food processing industry typically fall into one of three categories: raw materials, food processing, and packaged goods. Thinking of the industry in three categories helps the end user understand how machine vision can help their operations, but the truth is that within each category are many subtypes of inspection and quality-check applications for everything from separating rocks from nuts to verifying portion sizes and coverage to verifying foil seals and that the right cork is on the right wine bottle.

In the end, the random, organic nature of food in general and the way it is processed mean that the food processing industry needs custom solutions to benefit from automated quality assurance systems. The keys to choosing the right integrator and getting the correct solution without paying too much requires the customer to understand the nature of his or her application and use that knowledge to find the right technical partner.

Raw and loving it
Processing food is like any other manufacturing process: raw materials come in one side, they’re processed into a product, and packaged goods go out to customers.

“The first step is usually to inspect the incoming product,” explains Fred Turek, Chief Operations Officer for FSI Technologies (Lombard, Illinois). “In some cases the food is processed at a different location, but in our experience, many incoming product inspections deal with grading of fruits, vegetables, meats, and tobacco, for example. Color analysis is a big part of inspecting raw goods. In fruit inspection, we used to use a neural network for color analysis, but in our architecture the neural net is just one of many tools in the software engine.

“In the case of tomatoes, we use a whole set of attributes, such as perimeter, size, and color, as inputs to the neural net; this includes an image transformed based on degree of match of each pixel in 3D color space.  This can then be process by a range of tools.  For example, this may include using the size and degree of color difference of off-color areas as inputs to the neural net,” Turek adds. “We then ran the same sample against professional human inspectors and found out that our system was doing a great job at automating this step. In the past, we’ve even used hyperspectral imaging to find variances that are barely visible to the human eye. By looking at tobacco in different spectral bands, we were able to find specific bands that would differentiate bad product from good. In some cases, such as meat, we combine that information with geometric measurements and color to guide automated cutting and portioning machines.”

Once a specific color or spectral band is determined to be indicative of good or bad product, you can reduce the cost of the product by using monochrome cameras with notch or bandpass filters, according to David Dechow, President of Aptúra Machine Vision Solutions (Williamston, Michigan). “Color is often used in food inspection and processing for quality assurance,” he says. “Fish cutting is a classic application for structured light applications, where you use both 3D and structured light. But in the case of cherries, apples, or chicken, or even bacon for slicing, it’s either a color solution or grayscale with a bandpass or notch filter. This is a good way to keep costs down. For example, [when] using infrared lighting and cameras sensitive to this area of the spectrum, bruises in fruit that are below the surface show up clearly in the resulting images. Certain lighting will really enhance spillage or defects in the product.”

Not only is the color of the light important, but also the ability to fully light the camera’s field of view. “It can be hard to tell the difference between a bruised fruit and a fruit in shadow,” Turek notes. “This isn’t an exclusive problem to the food industry – it’s the nature of machine vision in general. But in other cases, for example, we had to inspect incoming fowl. They were alive at this point in the process and running up a chute, but occasionally, one would turn around and mess up the automated count. By focusing on the color and shape of the head, we were able to account for the stubborn bird.”

Processing and packaging
While many food processing steps take place in tanks and enclosed machines where vision inspection doesn’t offer much benefit, there are many processing steps that do require a vision solution. For example, machine vision systems that use x-ray imaging can locate foreign objects in processed food before they reach the customer, as well as measure portion sizes in items like TV dinners and other prepared foods.

In the case of cutting and portioning meats, however, food processing applications can become very complex indeed. “It could be a simple blob analysis but usually requires a lot more image processing to isolate important features,” explains Dechow. “Water-jet cutting of chicken parts is one example where you need to develop complex preprocessing routines including dimensional information to portion the meat to best effect and guide the cutting tool, much like a vision-guided robotic application.”

Inspecting the barcode or color quality of a finished food package may be relatively straightforward, but when packaging has a functional aspect of keeping the food in a pure state for consumption, it can become one of the most important and challenging steps of automated food production and quality assurance.

“There’s a whole category within packaging that involves the package seal and leaks that is extremely challenging,” notes Dechow. “Has the wrapper been sufficiently closed to avoid contamination, or the foil on a yogurt cup completely sealed around the perimeter of the container? There is no single answer to these challenges. You simply have to figure out a solution for the specific application.”

Foods with clear plastic wraps and bottle inspection pose specific challenges. For example, plastic-wrapped pizzas with their curved surfaces are a challenge to light adequately to image the highly reflective surface. And while bottle cap, fill level, and thread inspection are fairly straightforward applications, the speed of bottling lines is a challenge itself. “When you’re inspecting six to 18 bottles a second, and have to check for five or 10 features, you have to find ways to shave milliseconds out of the cycle time while communicating with a pretty fancy reject mechanism that can remove the problem bottle at that speed without causing a train wreck,” says FSI’s Turek. “Those challenges make bottle inspection pretty fun for an integrator.”

The non-standard solution
While every customer would love to go the machine vision store and buy a low-cost, high-volume solution that works perfectly on their application, the truth is that all machine vision solutions require some degree of customization, particularly in the food processing industry.

“Every system is unique,” says Turek. That’s the “bad” news, but the good news is that it’s about application engineering, not an R&D project. It’s like a carpenter cutting molding for doors and windows. You don’t tell the carpenter that you don’t want “custom cut” moldings while still demanding that they fit properly. Small things such as the different sizes and ranges of defects, how accurate do we need to be, how is the part presented to the camera are all important and unique to the application.”

In food processing, says Turek, there is less control of part presentation, which directly impacts lighting requirements. And the fact that the processes often move quickly or involve large batches of raw materials or finished products can make the entire process more difficult.

On the plus side, what represents a defect may not be as stringent. “One one-thousandths of an inch variation in the electronics industry is a big deal, but in tobacco inspection, not so much,” Turek notes.