Configurable Machine Vision Systems

It has been most amazing to see how the commercial underlying infrastructure hardware and software technology for machine vision has evolved over the years. Piggybacking largely on Intel and Microsoft, the machine vision products now being offered can truly be configured to address a multitude of applications.

Before the Automated Imaging Association even existed, a number of companies existed with machine vision products touted as being ‘‘general-purpose’‘ offering the ability to be configured to address any application. Most of these products came from one school of thought reflecting how specific machine vision implementations could mimic the configurable performance of the eye when it comes to inspection applications. Companies usually touted ‘‘send us your parts and we will show you how our system will handle your application’‘. I remember the president of one early machine vision company remarking at a conference that his company was probably going to be the first machine vision company to be able to build a complete car from all the sample parts they were receiving.

Some of these early approaches included the geometric feature analysis approach out of the Stanford Research Institute – basically an approach that segmented the scene into black and white based on a threshold and developed over 40 geometric descriptors to serve as the basis of pattern recognition. Then there was the morphology approach out of the Ecole des Mines and University of Michigan based on Boolean math and shape analysis. Then there was the histogram approach used by several companies. Several companies were also using syntactic analysis approaches – assessing specific part features and their relationship to each other. Others were using simpler approaches such as background/foreground pixel counts, gray scale projections, template matching.

Those companies that lived and died by commitment to a single approach mostly died along the way. The companies that survived were the ones that ultimately recognized they needed to offer a box of tools and not just a single tool. Essentially, if you’re building a house, you need carpentry tools as well as plumbing and electrical tools. Those that tried to make every application fit only the tools they offered disappeared.

Today these surviving companies offer a family of tools usually in a variety of form factors. Along the way the technology has also become less hardware-centric and more software-centric. In fact, perhaps the most configurable machine vision approach today is one based entirely on software resident in a personal computer. With any number of digital camera connectivity approaches one form factor for a machine vision system is a digital camera and personal computer. In some cases the personal computer might house a frame grabber with versions capable of handling analog or digital cameras.

Another version where more compute intensive applications are involved is a personal computer with an intelligent frame grabber or a frame grabber that includes embedded compute capability in the form of DSPs, FPGAs or application-specific integrated circuits. Another version is a standalone box with embedded PC and frame grabber functionality generally referred to as an embedded vision processor or embedded vision computer or vision appliance. In this case the camera is tethered to the box. A standalone box that incorporates embedded PC and intelligent frame grabber functionality is generally referred to as a vision processor or vision development system.

When the camera and embedded PC and frame grabber functionality are housed in a single box one has a fully self-contained machine vision system or smart camera. A version of the smart camera that usually has limited compute functionality as well as lower resolution imagers and less configurability for a variety of applications is referred to as a vision sensor.

In all cases, what makes these products machine vision systems as opposed to image processing and image analysis systems, for example, is the software. The software makes the underlying technology and principles totally transparent to the user. It speaks to applications instead of algorithms. Significantly, many of these products also provide access to the underlying image processing and image analysis software so more knowledgeable users can further optimize code for specific applications or even develop entirely new code for new applications.

To provide insights into what has been happening in the area of configurable machine vision, input was solicited from all known suppliers of one or more of the type of products reviewed. In the case of one company we received responses from two different product groups reflecting their respective positions on the questions coming from the perspective of the respective products. Those that specifically responded to our questions were:

• Steve Maves – Senior Application Engineer, Vision – Banner Engineering
• Steve Cruickshank - Product Marketing Manager, PC Vision – Cognex Corporation
• Lisa Eichler - Director Product Marketing, Vision Sensors – Cognex Corporation
• Steve Geraghty – Director of ipd – DALSA Corporation
• Ulises Salas - Vision Support Specialist, Automation Control System Division - Panasonic Electric Works of America
• Andy Harkison - Product Manager - SICK
• Matthew Linder – President - Valde Systems
• Endre Toth - Director of Business Development - Vision Components GmbH

What follows is their responses to the questions raised.

1. Can you provide a one-paragraph description of your line of configurable machine vision products?

[Steve Maves – Banner Engineering] Banner offers full-featured and single-purpose self-contained vision sensors that do not require a PC to operate. They all use the same PC based programming software for operator convenience and have a simple four-step setup.

[Steve Cruickshank – Cognex Corporation] Cognex® VisionPro® systems combine machine vision technology with a flexible and powerful PC-based vision application development. VisionPro makes it quick to evaluate, implement, and deploy PC solutions for challenging machine vision applications. The extensive Cognex vision tool library provides extremely robust, reliable, and repeatable performance while the VisionPro acquisition engine supports a wide variety of Cognex frame grabbers and industrial camera options. VisionPro is a PC-based system suitable for system integrators, machine-builders, OEMs, and advanced manufacturing engineers.

Cognex also offers ProofRead, an OCV and ID solution for optical character verification, even when the print is extremely distorted.  ProofRead enables manufacturers in the pharmaceutical, food, beverage, consumer goods, and other industries assurance their products are accurately labeled. Applications for ProofRead include: verifying a date/lot or product code, validating that the correct label is placed on a product, reading 1D and 2D codes for product traceability, ensuring the correct product is placed in a carton or box, verifying the accuracy of the dosage on a pharmaceutical bottle.

[Lisa Eichler – Cognex Corporation] Cognex's In-Sight & DVT vision sensors address factory floor machine vision applications.  Versions of the DVT vision sensor family include standard, high resolution and color.  In-Sight vision sensors offer higher performance capabilities and rugged die-cast aluminum and stainless steel cases.  Both product lines provide a complete library of vision tools and Ethernet communications.

[Steve Geraghty – ipd, DALSA] ipd coined the name ‘‘Vision Appliances’‘ for its line of easy-to-configure machine vision solutions. These packaged products, including controller, sensor(s) and software, offer ease-of-use with advanced inspection capabilities. In a nutshell, Vision Appliances abstract the complexity associated with machine vision, making the benefits of the technology available to all users.

[Ulises Salas - Panasonic Electric Works] We provide a complete line of reliable and Fast 2D Vision Systems designed to inspect and recognize complex images under very low contrast. Panasonic offers Gray Scale and Color Image Processing devices implemented with unique image processing filters that make the inspection process even easier.

[Andy Harkison – SICK] SICK offers a wide variety of products for vision applications.  The SICK ICR Bar Code Scanner uses vision to read and decode 1D and 2D bar codes for applications, including automotive, electronic/PCB, and high-speed document handling.  SICK also has a line of innovative Smart Cameras, the IVC-3D and IVC-2D, which are used for high-speed object classification, dimensioning, and robot vision.

[Matthew Linder – Valde Systems] Valde Systems products are designed to be stand alone vision processors unto themselves. Although they are part of a larger overall system, they typically perform the entire vision processing task as opposed to a frame grabber that only inputs the data and the PC does the processing. This makes them much easier to embed in larger machines or autonomous robots.

[Endre Toth – Vision Components] Our product program embraces more than 30 different Smart Camera models in three product groups: Single board smart Cameras for OEMs, Sensor Smart cameras in round IP65 housing and standard smart cameras. VC smart cameras are complete machine vision systems with built-in fast processor, image sensor, frame grabber, image memory, interfaces. They are self-contained, small size units in a robust industrial housing.

2. What specifically differentiates the respective models that you offer?

[Steve C.] VisionPro provides a wide range of frame grabbers and acquisition platforms, offering an extensive range of price/performance options for virtually all types of image capture: analog, digital, color, monochrome, area scan, line-scan, high-resolution, multi-channel, multiplexed.

[Lisa] Within the DVT and In-Sight product lines, models vary by resolution, monochrome vs. color, and processor performance.  All of the vision sensors provide the a complete library of inspection, measurement and gauging tools, while character recognition and verification (OCR/OCV) and advanced objection location algorithms are additional optional functionality.

[Steve G.] Our products are differentiated by performance, number and type of attached sensors (cameras). The entry level product, called the VA20, supports up to 2 mono cameras with good inspection performance (typically 600-900 ppm). Moving-up the product line allows more sensors and greater number of inspections per minute, along with color and line scan processing capabilities.

[Ulises] Besides the difference between Gray scale and Color vision systems, we divide the line according to the number of cameras, speed and filters needed to make the image processing 100% reliable. Even though all our products have filter function, we have developed vision systems that have special algorithm and filtering function to address low contrast images.

[Andy] The models are differentiated by the technology that they use - either laser- or camera-based technology.  The 2D bar code readers use CCD or CMOS to image an area and then decode the image with minimal setup time.  SICK Smart Cameras have more image processing abilities and a software development environment for checking geometries, detecting and reading figures, letters, bar codes, and inspection in two and three dimensions. 

[Matthew] Our products all differ from the competition in that they offer passive stereo two camera image processing. They differ from one another in form factor. Some are the processor, some are the processor and cameras together (a stereo smart camera) and some are PCI plug in cards.

[Endre] The models differ in resolution and frame rates of the applied image sensor, the processor speed and the interfaces. Resolution ranges from 640x480 to 1600x1200 pixels in the area scan smart cameras. Many of the models are also available not only in black and white but also in color - using a color sensor with Bayer pattern. The VC2065EC color smart camera model is a full-color smart camera with on-board 24-bit true-color VGA live display. The VC40xx family uses a 400 MHz 1200 MIPS DSP processor from TI. The New family VC44xx models can process images executing 8 instructions per cycle reaching an 8000 MIPS.

The most important technical characteristics of the different models:

• Processor: TMS329C64xx TI
• Memory: 16 Mbytes SDRAM
• Processor speed: 400 MHz/1 GHz  
• CCD resolution: 640x480, 752x582, 1024x768, 1280x1024, 1600x1200
• Frame rates: 16, 40, 63, 110, 250 fps
• Interfaces: serial, Ethernet, 24V I/O (PLC), SVGA
• Lens mount: CMOUNT

Vision Components also offers a high speed line scan smart camera.

[Steve M.] Banner’s vision sensors differ in the type and range of functions they perform and in image resolution. Single-purpose vision sensors feature one key tool, or algorithm, such as the Edge tool for detecting the edges of uniformly shaped objects or the GEO tool for recognizing patterns. Full-featured vision sensors include all tools available, for applications that require additional flexibility and robustness. All models are available with standard or 1.3 megapixel resolution.

3. What are some reasons for configurable machine vision products becoming as capable as they are today?

[Steve G.] For starters, the market demands it. Quality is on the front door of every manufacturer today, making machine vision an integral part of the operation. Machine vision companies are responding to this need by building products that are both capable and suitable for factory floor deployment. These products have to do much more than inspect, they have to integrate and communicate with the entire factory enterprise. 

[Ulises] Industry requires reliable and faster systems for inspection because quality control is very important or even crucial for some companies. Not only that, the vision system has to be easy to configure and to maintain.

[Andy] Hardware is smaller and more capable than it was a short time ago.  Coupled with more powerful software algorithms, these different products can solve problems for a large number of applications.  This is also the primary reason that there is more overlap in capabilities.

[Matthew] Primarily the increasing performance of the hardware. PCs have been moving so fast that it was easy to use them as platforms, but since they are burdened with non-real time operating systems, and architectures not conducive to image processing, they have their limitations. DSPs and FPGAS however, continue to improve in their performance.

[Endre] Faster and more integrated DSP processor chips became available from manufacturers such as Texas Instruments. The New VC44xx smart camera utilizes the 1GHz 8000 MIPS DSP processor, which is able to perform image processing functions faster than a PC in a smaller size, without a fan or moving parts. The smart camera is operated by a real–time, multitasking, and network enabled operating system with optimized image processing software tools.

[Steve C.] Today's PC based machine vision systems benefit from advances in both software technology, such as Microsoft's .NET products, and digital camera technology such as FireWire and Camera Link®. When you combine those technologies with advances in operator interface generation and integration with machine vision programming toolkits, the result is a PC-based machine vision system that provides the ‘‘ease of use’‘ typically associated with vision sensors with the vision tool performance and flexibility usually found in machine vision products that require programming.

[Lisa] As the algorithms for the vision tools become increasingly sophisticated, vision has become simpler to apply and provides more reliable, repeatable performance than ever before.  For example, Cognex's patented PatMax technology has improved the vision sensor's ability to repeatably locate objects under difficult and varying conditions.  This makes the solution less sensitive to lighting variations, image distortions and changing environments.

4. What are the specific application issues that one must be attentive to when applying a configurable vision product? For example, camera, input/output, resolution issues, etc.? Does it vary with product type?

[Matthew] Simply that the application must be better understood ahead of time. In a PC-based environment, there are many software packages and other options available since there are many suppliers. Typically in a configurable product if there are lighting or performance issues later, there are not as many options.

[Steve M.] Regardless of product type, lighting is the most critical factor in the success of a vision application, because it determines which traits of the target object the sensor detects. The quality of the lens is also important for image clarity. In addition, resolution must be sufficient for the application: there’s no point having higher resolution than needed, but some lower-end units are not adequate for applications that involve a large field-of-view or require accurately measuring features.

[Steve C.] When applying a configurable vision product, you must pay attention to not only the traditional concerns (lighting, optics, cameras, etc.) but also the machine control interface. Accepting triggers, making decision, sending results and machine control signals out through I/O lines all must be accomplished without programming. The application development environment must generate not just prototype for evaluations, but a ‘‘ready to go’‘ operator interface that is capable of inspecting products on high-speed manufacturing lines and sending reject signals fast enough to reject the correct part.

[Lisa] Many factors must be considered when applying a vision sensor to an application, including lighting, line speed, resolution, color, and communication to third party devices.  A knowledgeable vision provider should be involved in the definition and selection of the appropriate vision sensors.

[Ulises] Resolution and inspection time are very important and must be understood as related to the application.

[Andy] When choosing a product for an application, it is important to consider how much vision capability is needed or will be needed.  Understanding attributes of an application, such as decoding or object finding, is important:  is the object moving, how large is the code, is it in a particular orientation?  With a camera, this helps determine the best product capabilities such as lighting, focal distance, image resolution and even image reading technology-- matrix or line-- for object movement.  These vary with product type and there are different SICK products for a variety of applications.

[Steve G.] With any vision application, it is essential to understand the inspection requirements and the surrounding environment. Specifying inappropriate hardware, or software, is a sure recipe for disaster, such as selecting a low-resolution sensor for a measuring application requiring single digit micron accuracy. Similarly, lighting should be a primary consideration for any vision application.

In the factory world, it is also important to consider where the product will be installed. If the environment is harsh, the equipment will need to be protected, either in a stainless enclosure or cabinet. Furthermore, consideration must be paid to how the operator intends to interact with the product, both for setup and monitoring. Likewise, it is necessary to know what third party products will need to be interfaced and how.

5. What is it that you require from a prospective buyer of a configurable vision product for an industrial application to assure that it will satisfy their application?

[Endre] Take full responsibility of your implementation plan. Commit the appropriate expertise, resources, time and financial means. Early discussion and decision should include whether the customer will need outside assistance from a system integrator or software consultant.

[Steve M.] The buyers must clearly understand the application—what features to detect on the target, how fast the target is moving past the sensor, and the full range of variation in the optical appearance of the parts. Also, the customer must be willing to learn how to configure the sensor and be willing to support having configurable sensors in their facility.

[Steve C.] I don't believe a configurable system adds any requirements for the prospective buyer. As with other types of vision systems, they still need to understand their application and select a tool that will accomplish the desired result, taking into account all the usual factors: inspection task, line rates, lighting, communication, acquisition styles, and many others.

[Lisa] Typically, we need to see the parts to be inspected, the environmental conditions, and the speed required.  An evaluation of the application in a vision lab should be provided by the manufacturer, distributor or integrator for the project to determine the specific lighting and lensing required as well as to help select the specific model best suited to the application. 

[Steve G.] What is the problem the vision product is required to solve? Many times, manufacturers will have a difficult time defining a specification for the application. There is a problem with acceptable and unacceptable results, but defining acceptable can be a challenge. If the problem and expectations are well understood, the solution will be also.

Before recommending a solution, we like to understand more about the problem: What is it? How much is it costing the manufacturer? What have they done to try and solve it? What are their expectations for a vision solution?

[Ulises] As mentioned before, first, resolution and speed have to be satisfied. Then, after making the proper lighting and optics tests, the next step is the physical environment. We require the space necessary to install the camera and lighting. 

[Andy] Typically a questionnaire and/or site survey is provided to the customer to best determine a solution.  Some applications only require reviewing the questionnaire with a SICK Applications Engineer.  Other times, an in depth site survey and a test setup is performed to evaluate different products and samples to determine the best solution.

[Matthew] That they have a good understanding of how the specific product will address their application.

6. What are the skills required to integrate a configurable vision product? Do they vary with product type?

[Steve C.] Yes, the skills required to integrate a configurable vision system will vary somewhat with product type. For example, a PC-based system has different integration concerns than a vision sensor.

[Lisa] Both vision and automation skills are required to solve any application.  Having the expertise to configure the vision sensor is important, but also having a qualified engineer or integrator who is knowledgeable in the complete automation system and provides start up services can make the difference in whether the vision system works reliably. 

[Ulises] Programming our vision products is very simple and does not require special knowledge or high level language to program them. However, lighting and optics are the most important part of a vision system. Furthermore, lighting and optics skills are required most of the time.

[Andy] The skills vary depending on the application.  Vision bar code scanners essentially read codes out of the box with minimal setup.  Other products such as Smart Cameras have some pre-configured capabilities but also rely on a software development environment to tailor the tools to the particular application. Both the IVC-2D and IVC-3D Cameras are programmed via a graphical interface (IVC Studio). This decreases the learning curve, especially for a customer interested in transitioning to 3D applications. Once configured, the camera can work in stand-alone mode, without the need for a PC, enabling the user to easily monitor the camera.

[Matthew] The skills are similar to what is needed to integrate a regular vision system/sensor they aren't that different.

[Steve G.] Knowledge of industrial automation, at least for factory floor integration, is essential. This is because the vision product has to be configured to interface at some level with other equipment, be it a PLC, rejecter solenoid or factory network.

Knowledge of machine vision is a big plus, but not essential with Vision Appliances, at least for many applications. More difficult tasks, involving line scan techniques, for example, would definitely benefit from an experienced installer.

[Steve M.] Vision products overall are becoming easier to use, although there’s still a wide range. Some vision sensors, such a Banner’s, are designed to be easy enough for operators on the factory floor to use. Complex vision systems require highly trained staff, or a consultant. Products with the same capabilities can vary in their ease of use, so it’s important for users to request demos of several comparable vision products.

7. How do you support your configurable vision product line? E.g., help with set-up? Post service support? Training? Warranty? Documentation? Are these free or is there a fee?

[Ulises] Panasonic supports all our customers and provides training at our facilities, which is free of charge. However, if the customer requires more detailed training or service in the field, then there is a service charge. 

[Andy] SICK is committed to meeting the needs of its customers through innovative customer programs. Within each of these programs are key areas essential to meeting customer requirements and providing a competitive advantage in the marketplace. SICK conducts in depth training sessions for Distributors and field sales representatives throughout the year. We also offer trial and/or demo units to our customers for use in the field.  And, we offer a warranty, similar to other standard sensors. We provide a variety of consulting services to help customers meet their needs. We also offer field services including before and after field installation through an established network of field personnel. SICK can also provide product training, start-up assistance and discounts on service and repair. The type and amount of training or support determines if there is a charge and what that charge is.

[Matthew] We provide setup and post service support, but presently no formal training classes. Warranty and documentation are included. Support is free of charge.

[Steve G.] All of the above. The software application is fully equipped with context sensitive help and online manuals. The products are backed by a worldwide selling network offering local support and engineering services. In-house or onsite training classes are available for a fee.

[Steve M.] Most of our pre-sales support is provided free by our distributors around the country. Post-sales training is offered by our salespeople. A limited amount of startup assistance is often given to a customer, usually one day or less. Additional support is quoted on an hourly or daily basis. Like all of our products, the vision products carry a 1-year warranty. Our documentation is freely available for download on the Internet. 

[Endre] Free training courses are regularly offered for customers in different topics and levels. Training courses are held in traditional classroom format, and more recently we also offer training in a webcast format. Vision Components provides two year warrantees for smart cameras on labor and parts, and repairs the cameras for a flat rate after expiration of the warranty time.

[Steve C.] Cognex supports VisionPro and ProofRead the same way we support all our products ... our experienced team of engineers provides support via phone or email, and can be contracted for personalized on-site assistance.  The comprehensive Cognex online Support Center includes a Download Area for software releases and related material, a searchable Knowledge Database that provides answers to frequently asked questions, and Case Submissions for customers to send issues directly to Cognex. Online and classroom courses are available to meet specific needs. Cognex offers a standard 1-year warranty on all hardware, and extended warranties are available.

[Lisa] Cognex offers many training options to help customers new to vision learn the basics.  We offer free machine vision seminars across the country and our Automation Solution Providers host workshops monthly.  Also, we have free classroom and online training.   Documentation and trial software are available at no charge on our website.  And finally, pre- and post-sale technical support is offered in conjunction with our Automation Solution Providers

8. Are there some emerging technology changes associated with the underlying technology used in configurable vision products that will impact the performance of those products? Will some of these emerging technology changes have more of an impact on some product classes than others?

[Matthew] I think some of the new communication standards and protocols will begin to find their way into machine vision. Presently there are the old analog approaches, but no standard networked digital interface exists to other parts of a vision systems such as a photodetector or ejector.

[Steve G.] Configurable machine vision products benefit from the same technology advancements that drive the component industry. GigE technology, for example, will make new cameras available and provide faster video streaming between server (product) and client (network monitor). Similarly, newer, faster, smaller processors will improve processing performance and shrink hardware form factors. 

[Steve M.] As processor speeds improve, performance improves. As performance improves, features wi