Machine Vision-Based Proofreading. . . .by Nello Zuech, Contributing Editor

Accuracy, accuracy, accuracy is the name of the game in all printed materials produced for the pharmaceutical industry. This includes labels, shelf cartons, inserts and outserts. The requirements are especially challenging for inserts and outserts. Many of these can be 20 – 24’‘ x 20 – 24’‘ printed in 4 point type. The result is lots of characters and symbols.

Major concerns are omissions, illegible print due to smearing, ink splash, etc. – pretty much any condition that could go wrong in a press run that would lead to inability to read the text accurately or otherwise be subject to misinterpretation. Additional concerns relate to the accuracy of the printed material as compared to original data today found in one or another computer format. The placement of a decimal point is critical. There is a big difference between 1.0 and .10 or 10.0.

In order to detect both subtle and gross concerns a number of companies have developed machine vision-based proofreading systems. Typically these will use a high-resolution scanner to input an image of the printed material. Processing to achieve precise registration within the computer memory is critical as typically image subtraction is performed – the current image being subtracted from the image of the printed material stored in memory. The difference regions between the ‘‘golden standard’‘ and the current image are flagged so an operator can quickly determine by viewing the display and toggling to the scene location where the difference is detected whether or not the difference is real and a genuine concern or a false alarm.

When one is detecting subtle differences in scene changes, any changes stemming from misregistration between the current image and the ‘‘golden standard’‘ will lead to false alarms. In the past the number of such false alarms was so high that it discouraged the adoption of these machine vision-based proofreading systems. Today false alarms are down to more acceptable levels, often less than the number of real regions of concern that are detected. Significantly, no escapes of real regions of concern are acceptable. Consequently, these systems are set to be more sensitive to potential regions of concern, which, in turn, leads to some false alarms.

Using these systems has reduced the amount of time it takes to proofread printed materials from what were often hours to minutes. Significantly, these systems permit proofreading against the actual original computer generated artwork and printed information rather than against an original print, which itself could have been flawed.

To gain insight into proofreading systems that are on the market, companies known to offer such systems were canvassed for input to this article. The following made contributions:

• Gary Parish, President, Complete Inspection Systems
• Miriam Glassman, Marketing Communications Manager, Global Vision, Inc.
• Dr. Mike Negin, President, Mnemonics
• Dr. Stephan Krebs, President, Nyquist Systems

Dr. Negin in his response makes some comments about what is meant by electronic proofreading: ‘‘Maybe the best way to start this discussion is to ask, What is not a proofreading system?

• It is not a spell checker.
• It is not a grammar checker.
• It is not a style checker.
• It is not software that treats only a single document.
• It is not a ‘‘print quality’‘ inspector.
• It is not an OCR system.
• It is not an OCV system.

Some of the above features are built into word processors; others are features of vision systems.

So what is a proofreading system?
It is a collection of comparison engines that performs the comparison of two or more documents. The documents can be from an electronic source or a physical source. For example:

• Electronic
• Word processor files like Word
• PDF, PS, EPS
• TIF, JPG, PNG, images
• RIP (raster image processing) file from prepress operations
• Physical
• Printed pages (from laser printers, inkjet, lithographic)
• Press sheets, proof sheets, inserts, outserts, labels
• Printing plates (metal or flexographic plates)
• Blue lines (or blue prints)
• Photographs

The goal of a proofreading system is to compare two or more documents to:

• Find content deviation between files of a mixed format (e.g., a Word file to a PDF file)
• Content comparison is performed by extracting the internal coded information from the document (e.g., the ASCII or Unicode character representations) and then comparing the strings. Since this is not a vision problem, this proofreading requirement will not be discussed in detail.
• Find content and format deviations between files of similar format and layout. This is the classical proofreading task required when comparing a source document to a sample document. The documents can be electronic or physical provided they can be converted to have similar layout.

This task is performed by performing image comparison between a standard and a sample.’‘

1.  Can you provide a brief general description of your proofreading system? Basic theory, brief implementation description (cameras, optics, lighting, material handling, file compatibility, online/offline, etc.).
[Gary Parish – Complete Inspection Systems] We offer a number of automated proofreading systems for both Image and Text Comparison. Our main product called AutoProof Pro can be used to scan, capture, or compare electronic files. We convert the images to a Bitmap and provide inspection up to 1200 DPI. It is an automated system noting the defects. Another system called AccuProof Pro allows users to manually align and compare images including resizing. A third called Inspec is a workstation that allows users to use either a CCD camera or Digital Camera to capture and compare. AutoProof Matrix provides the ability to grid large documents for comparing sections. Another called AutoProof Pharma provides a complete audit trail to meet CFR 21 Part 11 standards.

[Miriam Glassman, Global Vision] Proofreading is a huge challenge for most pharmaceutical and printing companies around the world. It is often done manually. This method is extremely time consuming and lends itself to a great margin of human error. With over 15 years of experience, Global Vision is proud of its history as the pioneers in automated proofreading and inspection solutions.

There are three main ways our inspection solutions are handled. Which type is used depends on the particular needs of the organization (example: working with hardcopy only, degree of technologically savvy users etc…) our solutions include:

• Camera Based Solutions
• Scanner Based Solutions
• Software Solutions
• Secure Workflow Consultancy Services 

Which system you use depends on the client’s environment. Our systems work in environments where the workflow is:

• Digital to hardcopy
• Hardcopy only
• Digital only

The concept is the same regardless of the tool (Camera, Scanner, or Software Only). By electronically comparing revisions, our solutions automate the proofreading process by locating the difference from one document to the other (whether it be a master and sample or revisions).

Our recent product launch, DOCU-PROOF™ C³ is a breakthrough technology that compares unlike (dissimilar) document layouts in any language. In the past, text inspection was always limited to comparing like (similar) documents, where embedded text and graphics were impossible to inspect and proofread.

[Dr. Mike Negin – Mnemonics] MNEMONICS proofreading system, AVIA (Automated Visual Information Analysis) removes the human visual detection process from the proofreading task, but leaves the decision task to the reviewer. This is a classical separation of feature extraction and decision rule steps found in classical pattern recognition. AVIA finds the flaws, differences, and errors so that the human is left with the task of making the decisions. AVIA’s operator interface is designed so that navigation through the resulting images and tabulated results is integrated and enables several different review paradigms handling exceptions as well as systemic decision-making.

AVIA is fundamentally a software algorithmic image comparison engine. As such, the images can be from any source and can be used for any image comparison application. For proofreading, the imaging devices are usually large format scanners, flatbed scanners, or CCTV cameras. Because of the AVIA capability for general image comparison, the same engine can be used for a wide variety of uses in a variety of scenarios. For example, documents and inspection profiles can be set up off-line (e.g., in a pre-press application, or sampled QA use) and then be moved to on on-line application (e.g., in label inspection). The AVIA engine is the same engine regardless of the application. The configuration of the comparison and the source of the images may change, but the inner working of the comparison are identical.

The basic theory is to compare two ‘‘images’‘ of the two (or more) documents. Documents in this context can be from any source provided the documents can be converted to an image that consists of a two dimensional array of vectors. The vectors themselves may be points (a gray scale image) or they may be color pixels (e.g., C, M, Y, K) or other more complicated vector data structures. The vectors may be of a mixed data type structure. For practical purposes, thinking of the images as gray scale images is the simplest way to understand the process.

The implementation is based on non-linear image comparison techniques that minimize energy differences between the images. Because of physical printing issues such as paper stretch and warp and ink bleed through, the non-linear comparison techniques are required to prevent excessive false alarms.

AVIA is designed to treat an image as a large multidimensional vector. AVIA is therefore independent from the imaging device, as long as the image can be made available in memory. As a practical matter, the image sources should be of high quality (e.g., large format scanners up to 54 inches wide can generate images on the order of 300 megapixels, electronic cameras from 640x480 (300 Kbytes) to megapixel cameras, electronically generated images from documents (e.g., pdf to image conversion). File compatibility is not a significant issue, as long as the file source can be loaded into computer memory.

[Dr. Stephan Krebs – Nyquist Systems] Our company provides inspection from prepress, via press to postpress applications. A critical issue is the validation of the reference image for the inspection on press or on the rewinder. Our proof reading system bridges the gap to the original artwork by comparing the image taken from the camera of the web inspection system back to the artwork pdf image. The images are taken from high-resolution high-speed line scan cameras. Typically, we use cameras with a resolution of 6144 pixels.

Two or three cameras can be used across the web. The high resolution does not allow for color proof, as such cameras are not available today. However, as our systems are mainly used for critical pharmaceutical or security applications where this is not an issue, as grey scale images work very well. Furthermore, we can scan a sample from the press and compare it back to the artwork. In this case we use flat bedscanners(<17’‘) or wide format scanners (up to 48’‘) to obtain the image.

2.  What have been some specific applications of your proofreading systems?
[Miriam] 80% of our business is dedicated to providing automated proofreading solutions to major pharmaceutical companies around the world. Global Visions solutions can be found all around the world and their applications vary in terms of complexity.  Some Departments that apply our solutions in the pharmaceutical industry include:

• Regulatory
• Marketing
• Graphics
• QA/QC 
•  Production
• Pre-Press

[Mike]