Machine Vision in the Food Market

I recently vacationed with my son Matthew visiting the hometown of my parents, a little town of 700 people called Cloz in the Trentino Province of Italy. Turns out that the economy of the entire valley is based on apples. The main warehouse for the apples is in this little town. My cousin is the operations manager so I visited the warehouse. Was I surprised at the sophistication of their control system to monitor the oxygen levels in these huge rooms, each the size of an airplane hangar!  And, I was delighted to find a machine vision system used to sort the apples by color.

The vision system was manufactured by Grufa Machinebouw out of Holland. A single camera was mounted over the conveyor, which conveyed 10 rows of apples across the scanner area. My cousin noted that the system does not really replace the inspectors. Turns out typically two inspectors stand on each side of the conveyor. As the apples pass, each has a stick that they use to touch apples that are misshapen or otherwise not up to the standards of the quality apple they strive for. Seems that the vision system also recognizes when an apple has been touched by a stick and sorts those apples accordingly.

Food sorters for the fresh-pack market in Europe are apparently in widespread use. The adoption has been driven by two factors - the aging workforce resulting in fewer workers available to perform sorting tasks and the higher cost of employing those workers. Consequently, not only is machine vision-based sorting technology in greater use in Europe, but also there are more companies in Europe pursuing that business than in the US.  Many of these companies are also pursuing the US market, often having acquired domestic counterparts.

Some of the European-based companies offering application-specific machine vision systems for the fresh-pack and/or food processor market segments include: Aweta B.V. (acquired Autoline), Barco Machine Vision (largely the combination of the former Elbicon and Pulsarr companies), Belgian Electronic Sorting Technology (BEST), MAF Industries, Odenberg Engineering, Qsort and Sortex. US-based companies in this market include Delta Technology, Durand-Wayland, Ensco, FMC Food Technology, Focused Technologies, Key Technology, Produce Sorters International, Sunkist, TTI/Exeter Engineering and Woodside Electronics. In addition, Satake, out of Japan, having acquired US-based ESM several years ago, is also in this market in the US as is Compac, a company out of New Zealand. As large as this list is, this list may not be complete but includes those companies known to be pursuing stateside business by VSI.

Table 1 lists many of the players in the North American machine vision-based food inspection market. It attempts to distinguish between those players targeting the fresh-pack market (farmers, cooperatives, etc.) from those targeting the food processor market or companies that add some value such as freezing, canning, packaging, roasting, etc. Virtually all the companies in this market offer color-based versions today, though not all of their catalog products are color-based. While virtually all of these companies offer products designed around conventional line scan or area scan approaches to image capture, some of these companies also offer laser scanner-based versions, which compete with conventional machine vision techniques. Some of these companies also offer versions that are based on infrared images, which is often a tactic used to detect pits in products or to grade products.
Virtually all of these companies offer products based on bulk transfer using conveyors. Some also offer channel sorters, which are typically used when granular dry products are to be sorted, such as rice. While many of these companies specialize in dry product handling or wet product handling, some offer the capability to handle both types of products. The companies specializing in fruits or vegetables, like onions and potatoes, generally channel the product into multiple rows, using cup or pocket arrangements, under the camera or arrangement of cameras depending on how many rows are involved.

Table 1 - Players in North American Machine Vision-Based Food Inspection Market

Typically multiple cameras are arranged to capture the images of the entire surface of the product. Where products are more or less round, mechanisms are incorporated in the pockets to rotate the product under the camera arrangements. Size decisions are generally made based on the volume data derived. Shape sorting may be based on maximum/minimum diameters, ratios, etc. Color decisions are generally based on the entire surface scanned. Various schemes are used to assess color properties: simple percentage ratios, intensity value histograms, minimum or maximum areas defined, etc.

Machine vision technology has benefited from advances in microprocessor and DSP technology, higher resolution cameras both line scan and area scan, color cameras both line scan and area scan, developments in lighting leading to lamps with unique spectral outputs, and much research driven by our land grant agricultural colleges and universities and the US Department of Agriculture as well as research institutes in Europe and elsewhere. The result is that today's machine vision systems are more capable of addressing the demanding requirements of the food industry.

Many of the systems are semi-custom. That is, they often incorporate lighting whose spectrum is specific to the product being inspected so as to enhance sorting of foreign objects or grading based on appearance that could connote blight, frost bite, rust or other skin blemishes or disease.

Not all these products, even those competing within a class of applications, are equal. It is important to understand the comprehensive performance you expect from a food sorter/inspection system. For example, grading may be based on color (reflecting ripeness, for example), color distribution, shape, size, surface conditions (bruises, digs/gouges, insect infestation, etc.). Several of the companies also offer weighing stations that could also provide size information in addition to the physical size derived from the vision system. Some also provide X-Ray-based machine vision systems, which would also be able to detect internal flaws or conditions that might not otherwise be detectable by simply scanning the outer surface of the product.

While virtually all those in this business offer color cameras, not all applications actually require color-based analysis. In some applications, creative lighting arrangements can yield sufficient contrasts readily discerned and distinguished by monochrome cameras. Some systems take advantage of near-infrared properties as well as properties in the visible spectrum. Color-based systems may make it possible for the same system to be adapted to different products thereby increasing the return-on-investment.

Sorting machines have been used in the food industry since the 1930s. The early versions used simpler detector arrangements. In the 1990s many of the companies offering these simpler machines began to investigate and then develop machine vision approaches. However, it wasn't until color cameras came down in price and they replaced monochrome cameras in the more demanding applications that rigorous performance was achieved.
Performance of these application-specific machine vision systems for the fresh-pack and food processing markets continue to improve with the adoption of digital line scan technology and digital area cameras with features like exposure control and asynchronous scanning.
Versions of these systems for the fresh-pack market have been developed for: apples, citrus (lemons, oranges, grapefruit, limes, tangerines, tangelos, mandarins, clementines), tomatoes, peaches, plums, nectarines, persimmons, tamarillos, kiwi, avocados, pears, pomegranates, papayas, mangos, melons, onions, potatoes, sweet potatoes, cucumbers, etc.

Versions of these systems for the food processor market have been developed for: nuts, cereals, snack foods, rice, oats, sunflower seeds, rye, lentil, wheat, raisins, dates, prunes, corn, green beans, lima beans, peas, French fries, rutabagas, carrots, spinach, shrimp, meat product cubes, cranberries, cherries, olives, etc.

Some of these systems have also been adapted to sorting tobacco, cotton, plastic chips and  flakes for recycling and pulp for paper industry.

The use of these systems can greatly enhance the consistency of grading. People are notoriously ineffective in performing inspection tasks that require examining apparently similar products. After a relatively short period of time their visual acuity becomes dulled even if simple sorting is involved - separating good product from bad or foreign product from the stream of good product. Asking people to be consistent in grading a product is more than can be expected. While not perfect either, these application-specific machine vision systems perform significantly better in sorting bad from good, foreign matter from acceptable matter and classifying grades.

These factors alone should justify the purchase of this equipment. As the population ages, compounded by the fact that the younger generation abhors manual labor, makes it increasingly more difficult to find workers to perform these boring inspection tasks. This only provides further justification to consider these systems.

While machine vision-based optical sorters and graders represent the most widespread application of machine vision in the food industry, there are other applications. A number of companies today offer water-jet cutting portioning systems that use 3-D machine vision systems to characterize the volume of the product being portioned. These include portioning systems for seafood, chicken, turkey, pork and beef. Two companies offering such products are FMC Technology/Stein (www.fmc.com) and Marel (www.marelusa.com).

Other interesting applications in the food industry include those related to egg inspection, poultry inspection and poultry part sorting and baked goods inspection. Diamond Automation (www.diamond-automation.com) is one company that has developed machine vision-based systems to inspect eggs for cracks. Qsort (cited above - www.qsort.com) in addition to vegetable sorting also offers a vision system to check for spots on poultry parts - breast, leg or wing.

Another segment of the food industry where machine vision is rapidly being successfully deployed is the baked goods segment. Dipix Technology (www.dipix.com) has delivered systems using 3-D and color-based techniques to monitor products such as bread, buns, crackers and cookies. As the industry adopts automatic packing machines, machine vision becomes more important to avoid jams due to misshapen product. The lines are getting faster so that people have difficulty keeping up. Ultimately, product consistency improves. Three-dimensional techniques are important where product is stacked and fit into size-specific bags. By monitoring the height of the individual product, the stack height can be determined to guaranty the stack will fit.

By continuously monitoring product, data can be derived that spots trends that could lead to product outside tolerances for both dimension, shape and shade. By providing immediate feedback, yield can be improved. Even small improvements in yield for an average size company in the baked goods business will make a significant difference to their bottom line. While color techniques are sometimes used, often, sufficient performance can be obtained by using monochrome cameras and product-specific filters and lighting.

Success in applying machine vision in the food industry requires that both the end-user and the machine vision vendor become intimately familiar with all the nuances of the application. Foremost it is important that the vendor appreciate that the installation involves blue-collar workers ultimately to take ownership of the system for the installation to be successful. When the system is perceived to empower the line operator, that is, provides him the tools to do his job better and makes his job easier, than experience suggests the installation will be successful. Hence, appropriate training is critical.

Throughout the food industry one finds more automation moving into the plants. Clearly this means opportunities for machine vision as automation eliminates human vision. At the same time, machine vision is becoming more powerful, more capable and less expensive. Hence, even more cost-effective applications will be possible.

Special thanks for assistance in this article are given to Marc van Gerven, Barco Machine Vision and Paul Pearl, Dipix Vision Systems