The Latest on OCR/OCV Machine Vision Applications

One of the earliest applications of machine vision was optical character recognition, used more than 30 years ago. It required reading the alphanumeric serial numbers on rifles and pistols being returned to inventory for the U.S. Army. The condition of the characters varied throughout the lot. The motivation for automating the reading was to avoid transcription errors when read by an operator.

Given the condition of the characters (consider that the advanced lighting and high resolution solid state cameras of today were not available), the system was designed to operate in a fail-safe mode. If it could not read a character in the string it would highlight that specific character on a display and then the operator would be flagged to make the reading. The initial system was delivered with a bell that would ring to alert the operator that he needed to make a judgment call. The very first modification to the system was to change the bell to a light because the operator assigned to the system was deaf. Just goes to show you can never know too much about your customer!

Optical character recognition (OCR) and optical character verification (OCV) are two distinct requirements though with similar application issues. Verification reflects a condition where one knows what the character string is going to be and the requirement calls for making sure that the specific character exists at a specific location.  Recognition, on the other hand, implies that one does not know which character is present at a specific location, but that it is one of 26 characters or one of nine numerals.

Besides engineering details, there are two basic issues related to OCV application:

• Is the application satisfied by verifying that the character is correct?
• Or, is it also the requirement that the system must be able to assure the quality of the character – print quality inspection (PQI)?

In the way companies have approached this application, these are not mutually exclusive.

In the pharmaceutical industry, date and lot code verification not only calls for the system to verify that the characters are correct, but also that they are complete and not potentially subject to misinterpretation as a result of a printing error (contrast declining, character obliteration, etc.). Consequently, vision systems that are used for this application should generally have the ability to examine sub-features of the character, such as specific lines and loops.

In OCR/OCV applications, because there are often translation errors in the imprinting head, the vision system must first do a location analysis before it can actually do a read or verification. Generally, the entire code is first located and then each individual character is located. Once each character is located it can be checked for such features as line width, hole fill-in, breaks in the character and completeness of the character. Sensitivity of the system is therefore generally established ahead of time.

To gain insights in the most recent technology being applied to perform OCR/OCV applications, input was provided by the following suppliers of machine vision systems:

Bryan Boatner, Product Marketing Manager for In-Sight, Cognex Corporation

Ulises Salas, Vision Support Specialist, Automation Control System Division, Panasonic

Jim Anderson, Machine Vision Product Manager, SICK

1. Can you describe your OCR/OCV product line and discuss how you differentiate your products that address these applications, if you offer more than one?

[Bryan Boatner, Cognex] Cognex offers a variety of OCV/OCR solutions tailored to specific applications and industries. First, Cognex offers high-speed vision sensors capable of less than 1 millisecond (ms) per character reading times with a grayscale-based algorithm that tolerates poorly formed text and noisy backgrounds. Next, Cognex offers PC Vision software that can tolerate drastic nonlinear scale changes in OCV applications; OCV/OCR solutions for packaging operations employing non-rigid containers, such as pouches and bags where severe perspective changes and random spatial distortions result from the object changing shape during the process; and multi-camera OCV/OCR solutions for randomly oriented cylindrical surfaces such a bottles, vials and cosmetic containers that allow the entire surface to be inspected simultaneously. Finally, Cognex offers wafer ID readers that read codes and characters that are hard-marked, soft-marked, or super soft-marked on semiconductor wafers.

[Ulises Salas – Panasonic] Our A230 OCR/OCV Image checker character recognition function distinguishes the product type, and the character verification function checks the printed quality. Combining these two powerful features, the A230 is easy to implement. A230 offers background suppression functions that allow characters to be read under noisy backgrounds. It can even
recognize Dot Matrix, jointed characters, chip or blurred characters

[Jim Anderson – SICK] SICK offers multiple solutions for OCR/OCV, ranging from simple vision sensors such as our CVS product line, to our IVC Smart Cameras, to streaming cameras, such as our Ranger Cameras, which use applied tools to achieve results for nearly all types of applications.

The CVS4 is a sensor entry into the OCR/OCV applications. It is installed with 13 standard fonts so setup is easier, reducing the time needed to train the sensor. That is not to say that if the font needed for the customer is not pre-installed, you are stuck; it is possible to teach and store a user library of up to 56 characters. The IP 67 housing and integrated lighting makes the CVS4 a good solution and very cost effective on OCR/OCV applications in the packaging industry where date code reading is a growing requirement. The CVS4 also has a built-in clock and calendar so it can continue to work even as the date and time that the sensor may be looking for changes.

If the complexity of the job increases or there are other inspections to do along with the OCR/OCV, the IVC-2D Camera with reader capabilities is a great solution. The IVC-2D allows for greater field-of-view and different standoff. The IVC2-D allows for varying position of the text and even for rotation and still to be decoded properly. The text specifics are user defined, so it is more flexible than a sensor-based approach.

2. What specific OCR applications (and in which industries) do your products address?

[Ulises] Date/lot numbers for candies and bottles: Packaging
VIN numbers and license plates: Automotive
Lot numbers/codes on wafers: Semiconductor
Codes on stamped metal parts: Stamping industry
Lot numbers on black plastic molds: Molding industry

We apply the A230 mainly to the packaging, automotive and semiconductor industries.

[Jim] OCR applications seem to be on the rise with our food and beverage customers. They are even starting to put pressure on OEMs to include this as an option on printing and labeling machines. For OEMs in this field, adding cost to their machine is very difficult, so providing this added capability and keeping costs down lends itself well to a sensor-based solution like the CVS4 from SICK. 

Because it is a true OCR application, the camera doesn’t know what the next string of characters will be, so it needs to be flexible to adjust for different data on each item and give a reliable read-out of the text it finds. SICK offers multiple solutions for more difficult OCR applications.  The IVC2D allows the user to define where to expect the text and do other inspections at the same time – like measure the position of a label on a package.  It is also capable of reading both 1D and 2D bar codes, making it a really versatile solution.

SICK also offers a compact camera sensor called the ICR803 that is also capable of reading barcodes of nearly all types and do OCR.  This is a great sensor – as the lighting is integrated in the housing.

[Bryan] Most ‘‘text reading’‘ in food, beverage, pharmaceutical and consumer goods packaging applications are really OCV, however, some food and beverage aplications sometimes do ‘‘reading’‘ because they want to know not only if the code is wrong, but what it is. The automotive, semiconductor and electronics industries use a lot of OCR. For example, Cognex products address part number traceability in the automotive industry and part number reading and verification in the semiconductor and electronics industry.  

3. What specific OCV applications in which industries do your products address?

[Jim] Customers are looking at OCV applications to help them increase the overall presentation of their products. Matching the date code or serial number to a taught-in or saved image is something that is done quite often in the bar code industry, but customers and consumers are growing more critical of appearance. If the date is printed but not clearly legible, this can lead to an overall feeling that one product is better than another. By using OCV the presentation of the printed material can be more closely monitored.  SICK has a number of solutions, including the CVS4 and ICR803 on the sensor side of the spectrum, and IVC2D in the smart camera field.

[Bryan] Cognex products address date/lot verification in the pharmaceutical, food, and beverage packaging industries. Pharmaceutical manufacturers need to ensure that product information is correctly printed and that labels are placed on the right products. Mislabeled allergens in food processing cause expensive product recalls, and FDA traceability directives require manufacturers to track ingredients such as peanuts, soybeans, milk, eggs, shellfish, tree nuts and wheat. Cognex vision systems are used to verify that the correct allergen label is present. They also read can codes and match product labels to prevent mislabeled products. Cognex vision systems are also used for address and sequence verification in postal/printing and document handling industries. In many cases these are hybrid reading/verification applications because even though we may know what the expected string is, customers usually want to know what character was read if it didn’t match the expected string.

4. What parametric recommendations do you make to your customers? Font style?

[Bryan] Cognex can read any type of printed font, but we find that machine-readable font like OCR-A or SEMI usually provides the optimal score separation between similar-looking characters. Height and width don't necessarily matter, as long as the stroke-width requirements are met. As far as spacing is concerned, characters should not be touching, however our higher end products can be configured to tolerate touching characters. Cognex software supports fixed-width or variable-spaced fonts. A two-pixel minimum stroke width is typical if the print is clean; four to five pixels if it is not cleanly printed. There are no string-length limitations as long as the characters have ample stroke width. While it is always recommended to have as clean a background as possible, our tools are capable of reading characters even in the presence of very confusing backgrounds. It is usually recommended to have at least one character's worth of quiet zone between the characters and the border, however, the software can be configured to tolerate conditions where this is not the case. Ink jet printing usually presents the most challenges due to the large skew and non-linear scale change that can result depending on the shape of the product, and any acceleration/deceleration the product may undergo while the ink-jet printer is printing.

[Ulises] The only thing that we request from the customer is the original font of the characters so we can create the dictionary on the unit and start inspecting the pieces.

[Jim] Obviously the more influence we can have in the selection of things like font, character size, and where or how it will be printed on the object, the more confident we can be in the effectiveness of the entire solution. Using fonts like OCRa, OCRb, and other fixed width fonts helps as these types of fonts have specified widths for all characters and similar characters like Bs and 8s are printed in way to make differentiation more reliable.  High contrast between the characters and the background is always a plus. Solid characters are preferred over dot matrix characters, but the repeatability of the print is much more of an influence than the type of print.

Our cameras have functions built into them to adapt to all of these types of printing irregularities, but having to adjust for fewer of them can dramatically influence the speed of the entire system.

5. Can you provide some insight into the principles embedded in your OCR/OCV products?

[Ulises] Our software uses a character learning function based on Neural Network technology, which allows you to register a variety of characters for recognition. Once the unit learns the characters, they can be recognized even if the size changes.

[Jim] During the teaching process, an ROI (region-of-interest) is automatically created around each separate character and tolerance is set for the grayscale threshold at which the camera sets its binarization point. That is really a general overview of the concept.

[Bryan] We design our algorithms to address the variety of run-time variation that a user may experience when setting up an application. We have separate parameter controls for scale variation, confusing backgrounds, as well as an optimized reading mode for high-speed applications. We provide fielding capabilities for OCR in order to optimize reading times. 

6. In the case of your OCV products, besides verification does your system perform anything else? Analyze legibility? Aesthetics?

[Jim] When running the IVC2D and the CVS4, it is possible to get a percent match or ‘‘readability’‘ value for each character in the decoded string. This is a feature that is common in the smart camera arena, but is a very ‘‘high end’‘ feature that is included in the CVS vision sensor.

[Ulises] No.

[Bryan]  Our In-Sight and VisionPro product lines are general-purpose machine vision products, so they have the capability to not only perform OCR, but also perform rough print quality inspection, as well as read 1D and 2D codes to verify that they match the human-readable text on the product.

7. What are the recognition rates in OCR applications? What are the verification rates in OCV applications?

[Bryan] Recognition and verification rates are highly application-dependent. With our recently released In-Sight 5600, the OCV and OCR tools can run at rates faster than 1ms per character, and in some cases, as fast as .3ms per character. That’s about 6ms for a 20-character string.   

[Ulises] For OCR, it depends the size of the character and how noisy the background is. Assuming the character is 1/4 the FOV under a good contrast, if would be around 10ms per character

[Jim] The recognition rates for vision sensors like the CVS4 depend on the setting for the lighting.  The more contrast there is, the faster it can be triggered. The average time for finding and decoding a string is 40ms (20-30 characters).

The IVC-2D camera rate of recognition depends on the field-of-view or the amount of area in which the camera is expecting to find characters. The more defined the field, the faster the camera. The read step of an IVC-2D program typically adds around 65ms (20-30 characters). The IVC is a bit faster when doing OCV; it typically adds around 40ms to a program (20-30 characters).

8. In the case of OCR, what are your accuracies under the highest contrast conditions? No reads? Mis-reads?

[Ulises] Usually we are running between 99.5% to 100%.

[Jim] Under good conditions, the CVS4 and IVC-2D give very high repeatability. It gives a no-read output so the user gets a confirmation signal either way. On good and bad reads the data that is decoded is sent out over the serial line along with a factor of reliability for each character. This allows the customer to see which characters are causing issues and make any changes to the sensor or the print itself.

The IVC 2D camera is also very repeatable, especially under constant conditions (i.e. good lighting, repeatable text position and good print). The OCR and OCV tools both give the option to output the ‘‘readability’‘ or percent match of each character that the camera tried to decode for good and bad reads.

[Bryan]  Again, accuracy and no-read measurements are very application dependent. I would recommend that users perform their own testing on their actual part to determine the accuracy, no read, and mis-read rates. With that being said, our OCV/OCR tools are deployed in many mission critical applications such as semiconductor and pharmaceutical applications and have achieved up to 99.99% accuracy.

Our software outputs a % match, as well as a ‘‘difference accept’‘ score, which indicates how close the read character scored relative to the next-highest scoring character. Additionally, we provide a confusion matrix when training the font, so that users can anticipate which characters will be potentially confusing so that they can set their ‘‘accept’‘ and ‘‘difference accept’‘ thresholds at optimal levels.

9. What advances associated with the technology infrastructure of your OCR/OCV products has lead to more rigorous performance in your newest products (optics, lighting, vision hardware, vision software, cameras, etc.)? And, what are the specific advantages of these advances in terms of price/performance?

[Jim] The technology in machine vision is improving on all fronts, making applications like OCR/OCV more standard than ever before. Higher quality lighting and optics available at more competitive prices continue to make the possibility of saving money by using machine vision more apparent to new customers. Increases in technology, like using grayscale images instead of binarized images, makes lower contrast application possible where they would not have been in the past. 

[Bryan] Our ability to integrate PatFlex and PatMax (our patented pattern finding algorithms) into our OCVMax algorithms on our PC vision products has led to the most improved performance for difficult inkjet OCV applications where there is a significant amount of non-linear distortion skew and scale change to the code.

For our vision sensors, advances in hardware performance have enabled us to read and verify codes at extremely fast speeds.

[Ulises] Since OCR/OCV is a hard task for any vision system, we had to develop a more robust software, which includes preprocessing filters as part of many options to create a better contrast.

10. What changes in the underlying machine vision technology (vision engines, lighting, cameras) do you anticipate in the next 2 – 3 years that will yield even better OCR/OCV performance?

[Bryan] As DSPs and microprocessors continue to improve in speed, so will the speed and performance of our OCV/OCR tools.

[Ulises] I believe that the software will get better on eliminating noise in the images in order to get a good reading.

[Jim] SICK continues to offer cutting edge technology for machine vision with the use of 3D. 3D vision will allow users to solve applications that they were not able to do in the past. Characters or codes can be added to the product without making it obvious to the consumer. This may allow them to have a more repeatable process during production. This leads to more applications in the automotive and manufacturing industries where labels are not added, but the text is stamped or etched into the parts. These types of applications have proven very difficult for standard 2D systems, while 3D data extraction will continue to make these solvable applications.

11. Are there still barriers to the adoption of machine vision technology for OCR/OCV applications and what are they?

[Ulises] Usually companies do not know how much a vision system could cost and price is not attractive to medium or small organizations.

[Jim] There are still some barriers to machine vision for OCR/OCV applications but they are not too different than objections given for other machine vision applications. There are still a number of customers who have had bad experiences with vision solutions in the past – whether it was high cost or lower than expected performance…

It is important to point out the major advances machine vision has made since the 1980s when early adopters tried the first systems. The ease of programming of vision sensors and smart cameras has really improved, making the solutions not seem like magic, but like a robust and defined process. These are the same issues that face OCR/OCV applications. Customers can get a very solid OCR/OCV solution for a very reasonable price without having to make performance-related concessions.

[Bryan] I think the ease of use of machine vision has always been a barrier to adoption. Software features implemented to help address ease of use include pre-trained fonts and tuning capabilities. With pre-trained fonts, customers that use standard machine-readable fonts like OCR-A, OCR-B, and SEMI do not have to manually train characters. For customers that do require custom characters, we provide an easy to use training wizard. Once the font is trained, we provide a tuning button that automatically compares the trained font to the run-time characters and applies a one-time scaling change to provide optimal results.

12. What advice would you give a prospective customer for a machine vision-based solution addressing OCR/OCV applications? 

[Jim] I think the best advice to give a potential OCR/OCV customer is to give all the information you can about the application. How defined the application is will determine the speed at which a solution can be created. If there are parameters such as placement of the text, color of the text, and font type that are open for discussion, let that be known. The part of vision applications that most often causes issues is the actual definition of what is required. Knowing the parameters in which we can work allows the engineers at SICK to help create a cost-effective, reliable solution using a variety of products from photoelectric sensors to bar code scanners, and obviously machine vision.

[Bryan] As a former application engineer, I would recommend a thorough evaluation of the products you are looking to perform OCV/OCR on. Be sure to test the system under varying lighting, scale and speeds, so that you can be assured that the vision software has the required performance for your application.  Also, look for a company that sells a broad range of OCV/OCR solutions rather than a single product. That way you’ll be assured of a solution that addresses your specific application requirements.

[Ulises] We can find in the market many excellent OCR/OCV inspection software and vision machines, but they will be useless if you do not have good illumination. However, since lighting is mostly constant, there are other factors that interfere with the images, not only lighting fluctuations but noise in the background, too. In this case, the vision system software has to be good enough to suppress that noise and have a consistent reading of the characters.