Machine Vision on Packaging Lines
| By: Nello Zuech, Contributing Editor
From the earliest days of machine vision the technology has been used to address applications on packaging lines. Even the most primitive machine vision technology was able to perform relatively simple tasks like verifying that the label was on the package, that it was straight and that the cap was on the container. The challenge was that a project or application was described in rather simplistic terms (suggesting that simple machine vision techniques would be adequate), and as one engaged in the application it was clear there were far more variables than most simple machine vision approaches of the early days could handle: various colors (hue, saturation, intensity) of the base package itself, various colors (hue, saturation, intensity) associated with the label, throughput, some mis-registration was acceptable, concerns included aesthetics, was the label the correct one, repeatability of the presentation of the package, etc.
The result was often that more rigorous machine vision technology was required. In one early case I recall that while color was important the actual shade was permitted to vary quite a bit. In trying to apply a system that was quite capable of detecting subtle gray shade changes, as the packager changed package batches and presented a perfectly acceptable package with a slight shade variation, the vision system would proceed to reject everything that came down the line. It had to be fine tuned for every batch change and many times to accept the subtle shade variation during the running of a single batch. Needless to say it was not a success.
Another major challenge in the early days was the cost of a system. Even the most primitive system (making decisions based on background pixel/foreground pixel variations, for example) cost upwards of $25,000. I say upwards because that was generally the price for the basic hardware itself. Since virtually every application required pioneering effort, the engineering cost frequently doubled the price for a packaging line application.
It was always interesting to engage in one of these early packaging line applications and observe that the plant ultimately would need to equip another 20 lines. And, by the way, this was only one of 10 plants. Needless to say the sales person always had visions of large commissions! The challenge was that while the local plant manager could generally make a decision on a $50,000 investment, when you started to talk about equipping 20 lines the total capital budget for such a project was now $1,000,000 - far more than most plant managers could approve. Hence, a corporate decision was required and given other options for the same money; more often than not the decision did not favor the proliferation of machine vision within the plant.
Today, of course, that is no longer the case. Most of the machine vision applications along a packaging line can be addressed for under $20,000 and this includes engineering cost. Given the availability of proven packaging line application-specific software, many companies are able to perform the installation with internal personnel. Today’s machine vision systems can cost - effectively guaranty the performance of every operation on a packaging line: labeler, filler, capper, cartoner, etc.
To get some insights into the current landscape in packaging line machine vision applications, companies known to offer systems that address such applications were asked to complete a questionnaire. What follows are the responses of the following who graciously accepted our invitation to participate in this article.
- Blake DeFrance, Product Marketing Specialist, In-Sight Vision Sensors, Cognex Corporation
- Ben Dawson, Director of Strategic Development. DALSA IPD
- Jim Anderson, Product Manager, Machine Vision, SICK, Inc.
- Jeff Snyder, Vision Products Promoter, Siemens Energy and Automation
1. Can you summarize the packaging line applications that you have addressed with machine vision technology?
[Blake DeFrance – Cognex] Cognex has addressed a wide variety of inspection applications on packaging lines over the last twenty-five years. The most common applications include:
- Inspecting labels (presence, tears, print quality, OCV/OCR)
- Inspecting cans, boxes, and cartons for dents or unclosed flaps
- Verifying safety seal presence and correct application
- Checking for cap presence or skewed application
- Matching product with the label via OCV/OCR or ID software tools
- Verifying that a package is assembled with all components in the correct position
- Guiding robots for pick and place or palletizing operations
- Verifying liquid fill level in bottles or containers
[Ben Dawson – DALSA IPD] Consumer and pharmaceutical packaging.
[Jim Anderson – SICK, Inc.] SICK has addressed a number of applications on packaging lines using machine vision and vision sensors. Applications include verification of printed date codes using both the SICK CVS4 Vision Sensor and IVC-2D Smart Camera on neutraceutical packages… and verification of label placement with the SICK CVS1-Easy. SICK also has the added ability to solve a number of applications with our wide range of vision and other standard photoelectric solutions, giving us more flexibility when trying to solve a customer’s application.
[Jeff Snyder – Siemens] I have grouped example packaging applications based on industry:
1. Date/Lot Code Inspection (OCV)
A date/lot code is almost always printed on-line. We have inspected the
following types of on-line print:
Continuous ink jet
HP ink jet
2. Component ID (OCV to ensure correct component is being used)
3. Print Quality Inspection (PQI)
Clinical trial label
4. Component ID (non OCV)
1D bar code
1D bar code
1D bar code
1D bar code
5. Symbology Print Quality Verification
Bar codes via ANSI standard
Data Matrix via AIM, ISO 16022 and ISO 15415
6. Bright Stock
Visible Data Matrix
UV Data Matrix
7. Blister Pack
Date/Lot code OCV
Component code OCV
Blister to print registration
8. Finished Product Inspection
Label registration control
Presence/absence of powder for inhalation medication
Roundness of cap for crimp top vials
1. Finished cigarette pack inspection
2. Date code inspection
3. Correct component via OCV verification
4. Package sortation of returned cigarette packs
1. Kit completeness
2. Serialization via Data Matrix
3. Component ID via Data Matrix
1. Date/Lot Code Inspection (OCV)
Many laser marked codes directly on product
2. Component ID (OCV to ensure correct component is being used)
Personal care products (deodorant, etc.)
3. Component ID (non OCV)
1D bar code
Data Matrix code
4. Symbology Print Quality Verification
Bar codes via ANSI standard
Data Matrix via AIM, ISO 16022 and ISO 15415
Personal care products via Data Matrix code
Glue pattern on carton
Fill level via IR light (to penetrate black glass bottle)
Plastic bottle cap defects and liner completeness
Component presence for cosmetics
2. What has been the most difficult packaging line application you addressed and why? What were the specific application issues (throughput, package variables [appearance and position] handled, line integration issues, etc.) that had to be overcome? How did you overcome the challenges?
[Ben] Checking the label quality and reading the date, lot and barcode on pharmaceutical bottles. The inspection rate was fast, the bottles could be in any angular position around their vertical axis, and the label inspection had to be detailed. The bottles were mechanically spun as they passed a line-scan camera to scan the label. Then DALSA’s Sherlock™ software aligned and unwrapped the label for inspection and reading.
[Jim] The SICK IVC-3D has allowed us to solve a number of applications that may have otherwise been nearly impossible. One application that proved to be very difficult was the confirmation and measurement of dough on a conveyor line. The challenge occurred because the placement of the dough was random and the customer required not only the length in the x-direction but also the actual length of the dough roll even if the roll was in a curved or “S” shape. The speed of the line was quite high and the amount of data the customer wanted proved could only be solved with the SICK IVC-3D Camera. This is due to the ability of the camera to track the roll based on height - even when in the form of difficult shapes.
[Jeff] A blister machine application that required complete OCV and PQI on the lidding material as well as print to blister inspection and component ID for the associated carton. The lidding material was either a tyvek type or foil material. A platen printer was used to print all the information on the lidding material. The date/lot code and dosage was inspected with OCV tools. All other text and graphics were inspected with PQI tools. The web was approximately 210 mm wide. Four high-resolution area cameras were used across the web. The cameras were triggered every 70 mm of web travel.
After the individual blisters were die cut, the lidding material registration was checked with respect to the edges of the blister. This was to ensure the thermoform machine had not lost its registration and that the required text was present on each blister. Lastly, a Data Matrix code on the carton was inspected to ensure the correct carton was being used.
[Blake] There are four challenging packaging applications that Cognex has addressed recently. These are high-speed applications, more demanding inspections, limited space requirements, and corrosive food and beverage environments.
The need for manufacturers to increase throughput while improving quality was a driving force behind our development of higher speed vision sensors such as the new InSight 5600 platform. This sensor processes data at more than double the rate of our previous vision sensors – thus allowing manufacturers to perform inspections at a faster rate or to use additional software tools to inspect more features in each image.
More demanding inspections are defined here as those inspections which require higher vision sensor resolution to process. For example, if the sensor must be placed far from the target to be inspected, then the resolution of the sensor must increase to correctly image the features targeted. Cognex has developed resolutions as high as 1600 x 1200 pixels for this reason.
A third difficult application Cognex addressed was limited space availability prevalent in today’s factories. The footprint of machines has shrunken to make more efficient use of expensive production real estate, forcing automation component manufacturers to design smaller devices. To meet this need, Cognex developed a “remote head” vision sensor that allows a tiny remote camera head to be placed in the inspection area, separated from the vision processor.
A final challenging packaging environment is found in many food and beverage production facilities. These facilities typically require 'wash down' of production lines with liquids that can be corrosive. Cognex designed ALL of its InSight family to be IP-67 to withstand normal wear and tear on the plant floor. Additionally, the company offers several of its most popular vision sensors in a 'stainless steel' version which is impervious to harsh chemicals and is encased in a smooth housing to resist trapped dirt or impurities.
3. Can you briefly describe your solutions for packaging line applications? Specific machine vision hardware (frame grabbers, smart cameras, embedded vision processors, etc.), lighting arrangement, camera details? And Why?
[Jim] SICK’s line of CVS Vision Sensors have enclosure ratings of IP 67 which allows them to be used in applications in almost every area of the packaging facility. The IVC-3D Camera has a high rating of IP 65…and a soon to be released IP 69K enclosure, making it ideal for even the harshest of environments.
[Jeff] Siemens uses a broad range of machine vision products for packaging line solutions. This includes standalone vision sensors, smart cameras and PC based systems. The hardware choice is typically driven by the application requirements as well as industry specific requirements such as 21 CFR Part 11.
For example, the MV220 area color sensor can be used to verify that a correct component is being used based on color. The MV230 profile sensor can be used to verify presence based on a z-axis profile. The HawkEye 1500 series smart cameras can be used for Data Matrix and 1D bar code reading and print quality verification. The HawkEye 1600 series smart cameras can be used for standalone vision inspections utilizing the full Siemens Visionscape machine vision tool kit. The Visionscape family of PC based products includes frame grabbers, accelerated frame grabbers and embedded vision processors. The Visionscape I-Pak is a pharmaceutical industry targeted vertical product that has a user interface tailored for the pharmaceutical and packaging markets.
Via Siemens NERLITE, we have access to a broad range of LED based lighting solutions including patented devices such as the CDI (Continuous Diffuse Illuminator). As with most machine vision applications, lighting is extremely application dependent. A common thread for many packaging applications is the use of white light so that product graphics can be inspected. As an example, using a red LED source on a label with red graphics causes the red label graphics to appear white. Also, much brighter LEDs are now available that make illuminating large areas with LEDs much more practical.
[Blake] See question 2 above.
[Ben] DALSA makes cameras, frame grabbers, vision appliances, embedded vision processors, and vision software. Our iLabel™ software runs on the vision appliance™ hardware (a small embedded system with vision processing and remote cameras) and is used in most packaging line inspections. Challenging applications can tap higher performance hardware, software, and cameras.
4. What is the role of a customer in a machine vision packaging line application project?
[Jeff] There are several very important roles that the customer needs to play to ensure a successful application. Below are the most critical:
1. Create a specification describing the application AND the parameters that will allow the project to be considered complete. Describe the parameters that will dictate success.
2. Provide a complete set of samples, showing examples of good, bad and marginal.
3. Be an active participant in the application evaluation and solution review process.
4. Provide timely access to the packaging line for the installation.
5. Ensure operators are available for training.
6. Assign and train at least one person to be the vision expert.
[Blake] The customer plays an important role for Cognex in evaluating a machine vision application on a packaging line. It is important that, as a supplier, we understand the 'big picture.' Having a firm grasp on our customers’ overall goals for any given production line allows us to present solutions that might not otherwise be specified by our customers.
Additionally, many times production problems are forwarded by our sales team to our software and hardware designers to consider in next generation products.
In addition to giving Cognex a big picture of production goals in a facility, it is helpful if a customer can define firm boundaries for a vision application. The more clearly specific problems are communicated, the more precisely and quickly an acceptable solution can be proposed. For example, it is useful to prioritize the types of problems that a vision system can address into “A” or “B” categories. This way, a vendor can focus on solving the most critical problems and recommend solutions that would work in the future for “B” issues.
[Ben] Simple package inspection (label quality, label presence, position, bar code, date-and-lot code reading, “flags” and torn labels, etc.) can be quickly set up by the customer using our iLabel software. More challenging package line inspections, such as ones with high speed or multiple, coordinated views, are generally developed in cooperation with a vision integrator or a distributor and the customer.
[Jim] The customer’s role is to define the parameters and requirements of the inspection to be completed. In some cases the customer will even act as the integrator for the application. These cases tend to be on simple applications and applications where a vision sensor is the solution.
5. What are some mistakes buyers of machine vision systems for packaging lines make?
[Blake] Oftentimes buyers of vision systems don’t fully consider the importance of a long track record of solving vision applications. Not only is the time to implementation more rapid when a vision company has an extensive history of solving similar problems, but the learning curve is accelerated. An experienced sales force and proven, rapid after-sale support go hand-in-hand with a successful installation. These are the hallmarks of any successful organization, including machine vision suppliers. Sometimes customers do not consider their migration path as vision is implemented into packaging applications. For example, what are the goals of the company in three to five years, and does the vision supplier have the products and support needed to meet those goals? What about lower level vision applications; does the supplier offer a sufficient product breadth to meet various levels of vision application complexity? Usually sufficient product breadth is found in companies specializing in vision alone – not dabbling in vision along with other automation products such as controls or photoelectric sensors.
[Ben] I think two common mistakes are compromising the vision system’s view of the product by adding vision to an existing line, and judging character readability by human standards – a slanted, smeared dot-matrix printed date code on a specular material might be read with some effort by a human, but not with high reliability by a machine vision system.
[Jim] One of the most common mistakes made by buyers of machine vision solutions is concentrating on the cost of acquisition of particular hardware components. It is important to look at the entire cost of the application solution not just the hardware. In many cases the hardware cost is only a fraction of the entire cost of the application solution and the cost savings of the solution should be considered when making the buying decisions with regard to machine vision.
1. In the pharmaceutical industry, dealing with a vendor who has previous experience is critical. Validation issues, FDA compliance and knowledge of the industry are critical for ensuring successful projects. A general-purpose product in a pharmaceutical application often has hidden costs and delays when validation is performed.
2. For applications where a broad range of products run on a single production line, it is imperative that the vision vendor be provided with samples of all the products. Camera fields of view and illumination must be designed so that all products can be inspected.
3. While a specification can be time consuming to prepare and get approved, this is the one document that defines the project, expectations and success factors.
4. Lighting is the single most important aspect of a successful machine vision project. Perform a thorough lighting evaluation for all applications.
6. What changes in the underlying machine vision technology (vision engines, lighting, cameras) do you anticipate in the next 2 – 3 years?
[Ben] Lighting is always important for machine vision, not only to illuminate the part but also to act as a high-speed 'preprocessor' to select desired components – such as a pill in a blister pack. DALSA continues to push camera and vision engine technology, as well as vision software. We are incorporating more FPGA and other hardware accelerated processing in our vision engines. As always, I think the development of better (smarter, easier to use, faster) algorithms will be an important underlying change in machine vision technology.
[Jim] I think that the biggest changes will be the creation of more and more application-specific vision sensors. There is a growing need in the packaging market for solutions that are just beyond the capabilities of standard sensors and below the capabilities of traditional vision solutions. This gap in technology will be filled with fast, robust vision sensors that solve specific applications very well and are fairly simple to implement at the user level.
Another change that packaging customers are making is the use of more and more networked devices. SICK is implementing Ethernet/IP into its Smart Cameras making it easier for customers to monitor and install vision technology onto their machines.
[Jeff] Both PC based systems and smart cameras will continue to get faster. Digital cameras will become more common. As with all technology-based products, prices decline and performance increases.
[Blake] Increasingly, we expect our customers to demand smaller, more powerful machine vision systems. Packaging manufacturers are being asked to make more products, of a higher quality at a lower cost than ever before. Cognex is expecting that similar expectations will be placed on vendors of automation devices so we are anticipating these requirements by thinking about how to make machine vision more cost effective, powerful, smaller, and easier to use!
7. How will those changes impact packaging line applications?
[Jim] These changes will allow the packaging lines to do more things in less space - potentially saving valuable floor space. The new vision sensors will also potentially allow customers to have more traceability to verify their process over time.
1. Lower cost solutions will mean companies will be able to cost justify vision inspection more easily.
2. Vision inspection will be installed on very high-speed lines where it may not be practical or cost effective today. An example would be canning lines and breweries.
3. Rather than inspect products at the end of a packaging operation, more inspections will be moved upstream, closer to the point where errors can be corrected and scrap/rework minimized.
[Blake] Packaging lines will be expected to improve throughput, quality, and cost as previously mentioned. One effect on the line is the ability to more rapidly changeover products. Increasing packaging market segmentation means that more subsets of customers require specific packaging, so manufacturers will have to alter lines more frequently. Cognex will attempt to make this line change process smoother with our next generation products.
[Ben] Better processors and algorithms will make it easier and faster for the customer to take charge of their packaging line applications. The DALSA IPD product line offers customers easy-to-use hardware and software solutions so that they can quickly add vision without the expense and time of using a machine vision expert.
8. Are there market/process changes that are taking place in packaging line technology that are driving the adoption of machine vision?