Smart Vision Sensors, Consultancy

Today manufacturers of machine vision systems are conveying the impression that machine vision has been ‘‘dummied’‘ down. One sees companies promoting the concept of ‘‘smart vision sensor.’‘ The impression given as that like proximity and other related sensors frequently found in various machinery controls in virtually every manufacturing industry, one can ‘‘plug and play’‘ a smart vision sensor.

More often than not smart vision sensors come as integrated units including camera, lighting, optics and ‘‘intelligence.’‘ The challenge, however, is that in machine vision, ‘‘one size does not fit all.’‘ Every application has variables that frequently dictate the complement of machine vision hardware as well as software that is required. Nevertheless, these smart vision sensors do have applications for which they are well suited. More often than not they come with a simple set of algorithms making them suitable for presence/absence tasks where significant contrast changes dictates the condition to be sensed. Whether or not the lighting arrangement that comes with the sensor can provide the contrast required still requires experimentation. It may be that for the specific application another lighting arrangement would lead to more rigorous performance. Lighting is just as critical in the application of these smart vision sensors as in any machine vision application.

The low cost of these smart vision sensors makes it possible to consider applying them after the different value adding functions one might find on an assembly or packaging line. In such cases, the individual sites will be less demanding in their inspection requirements than would be the case if just an end-of-line inspection were being performed. There definitely is a role for these smart vision sensors, but one must be attentive to the requirements of the application and understand that one does get in performance what one pays for. This axiom is true in machine vision as well.

Recognizing that the distinction between smart vision sensors, smart cameras and embedded vision computers is becoming murky, several questions were posed of the vendors offering these products asking them for clarification. What follows are their responses, each coming from the perspective of their respective product lines. The following were kind enough to contribute to this article:

Jeffrey Schmitz – Banner Engineering
George Blackwell – Cognex
Robert Settle – DVT
Joshua Jelonek – Keyence
Sreenivas Rao – MicroView
Mark Sippel – Omron
Endre Toth – Vision Components

What are ‘‘smart vision sensors?’‘ How do they differ from a ‘‘smart camera’‘ or ‘‘embedded vision computer?’‘

Mark Sippel: ‘‘Omron typically uses the term ‘Vision Sensor’. The purpose of this name is to imply to users that this technology is being approached as a usable ‘‘industrial’‘ sensor as opposed to a vision system, which implies that integration of parts or products like a PC and software will be required. In Omron's case, a ‘‘vision sensor’‘ is a stable industrial sensing platform that can solve advanced noncontact quality, inspection and measurement applications using machine vision or imaging as the sensing method. The term ‘smart camera’ implies a video camera with a self-contained processing capability. An ‘embedded vision computer’ implies a PC based system embedded in a process for a very specific purpose.’‘

George Blackwell:  ‘‘We would typically call these vision sensors because intelligence is implied if it can do machine vision. The vision sensor is becoming more and more like a small version of a general-purpose machine vision system. It used to be that you had to compromise on tools and speed and interface, but the products that we’ve released in the In-Sight product line are for the most part just like general-purpose vision systems, but in smaller form factors and at lower price points.’‘

Jeff Schmitz: ‘‘Smart vision sensors are easier to use and cost less than traditional machine vision systems. Banner vision sensors--because they typically cost less than $3000--address many new applications that previously could not be cost-justified with machine vision systems. Smart vision sensors typically incorporate a digital imager, a central processor and I/O. They do not require a separate computer for operation. Smart vision sensors are configured using GUI that guides novice users (factory electricians and maintenance personnel) through the process of image setup (focusing and exposure time/lighting), building an inspection (selecting vision tools to find features of interest and driving I/O) and analyzing inspection results. Typically the GUI can be run on Windows PC through an Ethernet network card or serial COM port. Some smart sensors, like Banner's PresencePLUS line, have built-in ‘Teach’ functionality that learns the parameters of ‘‘good’‘ parts. The Ethernet and serial communications hardware built into smart sensors can be used to send inspection results to other computers on a network for systems process control.’‘

Bob Settle: ‘‘When DVT introduced the first smart camera in 1991, the term SmartImage Sensor was also introduced to differentiate this new product from existing vision systems. While it was, in fact, a vision system, the word ‘‘sensor’‘ was used because it was positioned at the low end of the market and was promoted as an upgrade to simple sensors. Since that time, DVT’s SmartImage Sensor has evolved into a sophisticated, stand-alone, high-end vision system that offers features and flexibility rivaling any general purpose product in the marketplace.’‘

Joshua Jelonek: ‘‘Smart vision sensors are an all-in-one package. There's no need to purchase any software for a PC. This makes the vision sensors simple to learn and very easy to integrate.’‘

Sreenivas Rao: ‘‘Smart Vision Sensors are basically miniature Vision systems that can be used in small to medium sized applications. They are supposed to be space saving, low cost and easy to setup. Real time performance to key to any Smart Vision Sensors.’‘

Endre Toth: ‘‘There does not seem to be a clear-cut separation of the three in the industry.’‘

What machine vision image pre-processing tools might you find in a typical smart vision sensor?

Schmitz: ‘‘Banner includes exposure time and gain adjustments, focus (contrast) monitoring and feedback, trigger timing (delay, pulse width).’‘

Sippel: ‘‘There are several preprocessing features Omron provides in its vision sensors. These range from electronic image position compensation, shutter speed adjustment, automatic or manual electronic background suppression, multiple electronic image filters like edge smoothing, edge enhancement, edge extraction, dilation, median and erosion. These preprocessing features allow the user reposition and adjust an image for movement, adjust for a noisy image background and with filters to enhance particular features to be inspected in the image.’‘

Jelonek: ‘‘The Keyence CV series vision system contains pre-processing tools that can perform a wide variety of functions including extracting edges in a specific direction, compensating for uneven lighting on cylindrical parts, and expanding targets that are normally too small to detect in a specific field of view.’‘

Rao: ‘‘Blob Detection, Edge Finders, Correlation & Filters.’‘

Toth: ‘‘Neighborhood averaging, lens distortion compensation, edge enhancement, white balance, median filter, Prewitt filter, Laplace filter and others.’‘

Blackwell: ‘‘On a typical vision sensor you might find only a few image pre-processing tools such as Open, Dilate, and Close. However, the In-Sight machine vision sensor family includes a full toolbox with those pre-processing tools plus others including Bot Hat, Edge, Magnitude, Erode, High pass filter, Low pass filter, Top Hat, Binarize, Clip, Equalize, and Stretch. It should be noted though that when you have a powerful and robust object location tool such as PatFind, the amount of preprocessing required is minimized in many applications.’‘

What machine vision orientation tools might you find in a typical smart vision sensor?

Schmitz: ‘‘Our CV series vision system contains pre-processing tools that can perform a wide variety of functions including extracting edges in a specific direction, compensating for uneven lighting on cylindrical parts, and expanding targets that are normally too small to detect in a specific field of view.’‘

Blackwell: ‘‘In a typical vision sensor you might be able to use general purpose Edge or Blob tools to determine the orientation of an object that is translating with only limited rotation. However, for high accuracy applications or applications where object movement is more complex, users need the robustness of a geometric-based model finder such as PatFind, which is specifically designed for object location.’‘ 

Toth: ‘‘The most typical orientation tools are locate point, locate line, locate circle, locate a pattern, locate blobs, locate contour, locate angular position, locate edges on circle, locate transition at line, integrated basic robot protocols to send and receive geometrical or status information, calibration functions’‘

Sippel: ‘‘For Omron's vision sensors, several orientation tools are available. These include center of gravity, edge detection and model search for X and Y locations, gravity and axis, circular angle and model rotation search for X, Y and Theta orientation.’‘

Jelonek: ‘‘Edge and pattern location tools.’‘

What machine vision segmentation tools might you find in a typical smart vision sensor?

Blackwell: ‘‘Typical vision sensors have Blob analysis. Others will include Color and Projection tools.’‘

Toth: ‘‘Rectangle, circle, elliptical ring, circular ring, contour, pixel counting, thresholding techniques, binary morphology based on RLC coding.’‘

Schmitz: ‘‘Edge extraction.’‘

Sippel: ‘‘Vision sensors offer multiple setup scenes with multiple segmentation windows in each scene. These window types include box (rectangle), circle, circumference, and arc and polygon shapes. The segmentation windows can be used in two ways, as a active measurement area or as a masking area allowing a user to exclude parts of the image not required for measurement or inspection.’‘

What machine vision image analysis/decision-making tools might you find in a typical vision sensor?

Toth: ‘‘Pixel counting, texture (surface) analyses, best fit line, best fit circle, gray scale correlation, focus.’‘

Settle: ‘‘Since DVT’s products are totally self-contained, all image processing is done onboard the camera. Ethernet connectivity is provided on all DVT cameras, and full reader functionality is provided on all models (1-D, 1-2, OCR, etc.). A full complement of soft sensor tools include: Translation (line and fiducial), Rotational (circular arc, elliptical arc, edge of parallelogram), Intensity (line, area), Edge count, Feature count, Measurement, Math Tools, Blob Tool, Readers, Object Find, Pixel Count, Color Monitoring, Segmentation.’‘

Sippel: ‘‘In Omron's vision sensors, a user can use pixel counts or area size, number, location and size of blobs or areas, X, Y and Theta location, percentage match or correlation, line width or length, quantities, reference values and overall judgments as well as basic and advanced mathematical expressions from measurement data to make decisions and determine results.’‘

Jelonek: ‘‘A capable vision system will have a wide range of tools that can be used to inspect a target. Various things like X/Y coordinates, presence/absence, and part dimensions can be output from the sensor. Typically the better vision systems will allow you to use the data from the tools to do calculations and measurements.’‘

Blackwell: ‘‘Image analysis and decision-making tools are two different things. The last three questions refer to image analysis, which includes pre-processing, orientation, and segmentation tools. For decision-making, typical vision sensors require either a scripting language or an external device such as a PLC. In-Sight, in contrast, offers decision making as a built in part of the vision spreadsheet. Because the spreadsheet is inherently designed for manipulating data, In-Sight vision sensors are capable of both discrete decision-making and trend analysis decision-making. For example, users can latch data and make decisions based on the standard deviation of the last 50 units.’‘

Schmitz: ‘‘Measuring (from any points identified with other vision tools). I/O logic tools that set-up acceptable windows of tolerance and tie results from other vision tools to outputs.’‘

What are the highest throughputs you might expect a smart vision sensor to be able to handle? What contributes to throughput limitations?

Rao: ‘‘Depends on the complexity of the application. For e.g.: A simple CHIP CAPACITOR inspection algorithm to measure length, breadth and terminal dimensions should not take more than 15 ms (including a picture take time of 5 ms). It is impossible to embed a P3 or P4 Pentium chip in an Embedded system, because of the size and heat dissipation. So the processor of choice must be small, low power dissipation, preferably with image processing instructions & with built-in flash memory, cache and high memory bandwidth. With this requirements the limitations also start dropping from the processing point of view.’‘

Blackwell: ‘‘Throughput is highly dependent on the application.’‘

Schmitz: ‘‘The fastest application Banner's PresencePLUS Pro is running is 1300 inspections per minute on a brewery bottle conveyor monitoring a rinsing station. Image pre-processing of a full, decent resolution image (640x480 or better) limits the number of inspections. The more vision tools that are required to inspect features of interest the more response time from trigger to output. Most inspections running on the PresencePLUS Pro, however, have total response times (trigger to output) of less than 100ms (600 inspections/minute).’‘

Toth: ‘‘In high-speed label inspection, more than 400,000 labels per hour can be checked using normalized gray scale correlation based search tools.’‘

Settle: ‘‘Theoretically, inspection times of simple 1-D barcodes could exceed 10K/minute.’‘

Jelonek: ‘‘The top speed of the CV series is close 15,000 part per minute, practically speaking though, using a few common tools the throughput would be more like 2000-3000 parts per minute.’‘

Sippel: ‘‘Vision sensor throughput rates are determined by the complexity and quantity of measurements and the type of camera used. The more image manipulation and measurements attempted in a single measurement cycle, the slower a throughput rate will be, similarly to the performance of a PLC. Throughput rate can very between 240 to 6000 parts or inspections per minute.’‘

Market Intelligence News & Insights: 

How Will Apple’s $500 Billion Investment for US Manufacturing, Education, and AI Play Out?

On February 24^th, Apple announced a “commitment… to spend and invest more than $500 billion in the U.S. over the next four years”, with plans for new manufacturing plants, enhanced support for manufacturing partners via their Advanced Manufacturing Fund, and a myriad of other initiatives. As part of this plan, Apple projects 20,000 new hires, with roles focused on R&D, silicon engineering, software development, and AI/ML engineering.

Read More

Have you introduced anything new as a smart vision sensor in the last year? If so, please describe that development.

Settle: ‘‘The latest iteration of DVT products consists of the Legend family of SmartImage Sensors and FrameWork 2.5 software. DVT’s recent product introductions include high-speed models; color (640 X480); high –resolution (1280 X 1024); high-resolution color (1280 X 1024); a low-cost CMOS version for indexed lines and simple inspections; a new, robust reader product; and a spectrographic system for precise color matching. Setup is accomplished through a PC utilizing FrameWork software. Once setup is complete, the PC is disconnected and the camera functions completely stand-alone. DVT does not charge for software (initial releases or updates), and upgrades are available off the DVT website.’‘

Blackwell: ‘‘Cognex introduced the In-Sight™ 4000 Series vision sensors within the last year. The Series offers speed improvements of up to 5x over previous In-Sight models, and includes three models: an ultra-high performance vision sensor offering fast frame rates and accelerated vision tool performance; a high resolution (1024 x 768) version for applications that require increased resolution for inspecting small objects, or capturing images of larger parts; and a compact, remote head camera (1.25’‘ L x 1.25’‘ D) version of the In-Sight 4000 targeted at applications where a small and/or lightweight camera head is required, or for harsh industrial environments where protection from dust and wash down is necessary. The development is significant because it broadens the range of high-speed applications that can be solved with a low-cost product. Its dual-processing architecture provides maximum vision performance, even in network-intensive applications such as remote process monitoring.’‘

Jelonek: ‘‘Keyence is currently working to meet the consumers ever increasing demand of high end capability in a small, easy to use, inexpensive package. Our current push is to make our systems faster, more accurate, and more network friendly.’‘

Toth: ‘‘VC2038E, VC2048E, VCM50 models.’‘

Schmitz: ‘‘Banner introduced the PresencePLUS Pro™-- a full-function vision sensor. Like larger, more complex and expensive systems, PresencePLUS Pro captures images and analyzes them using one or more vision tools to solve inspection applications.’‘

Sippel: ‘‘Omron is introducing a new vision sensor called an F210 that provides a one or two camera platform and features high-speed measurement performance with Omron's advanced measurement capability, including menu based flow-chart style configuration, high-accuracy Edge Code technology based measurement tools and Quest OCR tools.’‘

Do you offer smart vision sensors with general-purpose tools and what are those tools?

Blackwell: ‘‘Yes. In-Sight offers a full library of proven Cognex vision tools for locating, inspecting, measuring, and identifying parts and a spreadsheet interface for fast, easy application set up. The complete suite includes all the tools from all of the categories listed above, plus unmentioned tools such as ID tools and OCV tools. In addition, In-Sight offers tools that aid in communication and ease integration such as I/O, SMTP, FTP, DHCP, DNS, and Telnet.’‘

Sippel: ‘‘All of Omron's F-series vision sensors offer general-purpose measurement tools like binary based pixel counting area, gravity and axis, blob detection and scale-scale based edge detection, model searching or pattern matching and density data.’‘

Schmitz: ‘‘Linear/rectangular vision tool families with adaptive thresholding and filtering--Locate, Edge,  Object Area (rectangular or circle with multiple masks) vision tool families--Pattern Find, Pattern Count (normalized grayscale correlation using sensor's built in DSP), Average Grayscale and BLOB analysis Analysis--Measure, Test (sets up pass-fail tolerances and conditions and drives outputs) Communications--selects, packages and sends out data to Ethernet ports or RS232 devices.’‘

Jelonek: ‘‘Many of our tools are geared towards the automotive industry. For instance, looking for dents and defects in metal parts can be done with our STAIN tool. However, our tool base is broad enough to cover applications across any industry. A couple examples would be looking for short shot or flash on injection molded parts with our BLOB tool, or inspecting part dimensions with our edge tools.’‘

Toth: ‘‘Our image analysis tools include locate line, locate circle, blob analysis, pattern matching and detect contour. In addition we offer a group of test commands: surface test, gray value test, brightness test, brightness offset, focus, color test, count edges, count edges on a circle, read characters, teach character set, read DataMatrix code and curvature test. We also offer gauging tools - Distance, angle, define point, define line, define circle, straightness of contour, roundness of contour, calculation of center of gravity and area of contour, contour distance, contour extreme point and rotational contour position.’‘

Do you also offer smart vision sensors with application-specific tools and what are those applications and associated tools?

Sippel: ‘‘Omron offers application specific tools like Classification for multi-product sorting and recognition, advanced defect tools like Edge Code Defect for advanced edge and surface defect detection for products from electronic components to sealing parts, Edge Code Positioning for high-accuracy position detection and location for robotics and positioning equipment, Quest Optical Character Recognition for difficult character reading applications and Fine Matching for precise detection of label or graphics defect detection on packaging.’‘

Toth: ‘‘Print inspection, surface inspection, flat cable inspection, pin inspection, stamping inspection, date code verification, screw thread tool, etc.’‘

Blackwell: ‘‘Yes. Cognex has offered for some time the In-Sight 1700 Series of wafer ID products as well as the In-Sight 1010, which is an application specific vision sensor for 1D and 2D bar code identification. However, Cognex just announced our most recent application specific vision sensor product introduction in April. This innovative vision technology will soon be incorporated into low-cost vision sensors for new markets and applications where machine vision sensors have never before been successfully employed. Because of the increased performance and decreased price of computer hardware, we can now design vision-based products for high-volume applications that require high speed, high accuracy and high reliability, but very low cost. Our first product designed specifically for these new, non-industrial applications is called the Cognex CPS-100. CPS stands for Cognex People Sensor. The CPS-100 is a vision sensor designed for door security; it will detect and count people as they pass through an access-controlled doorway. The CPS-100 utilizes Cognex’s existing vision software, as well as patented 2D and 3D vision technology that Cognex has recently developed specifically for ‘people sensing’ applications.’‘

What peripherals are offered as an integrated smart vision sensor module? Lighting? Optics? Monitor? PC?

Jelonek: ‘‘We offer lensing, lighting, cables, and monitors for all of our vision systems. Some of our systems even have a built-in monitor for easy setup and troubleshooting.’‘

Toth: ‘‘We provide vision sensors with integrated lighting, lens and UI.’‘

Settle: ‘‘A full range of lights (integrated and separate) and lenses are available from DVT. DVT provides training on lighting and lenses as part of its comprehensive classroom training program (also free).’‘

Blackwell: ‘‘As a full-line supplier of machine vision sensors and systems, Cognex offers a number of optional In-Sight-compatible accessories including LED-based lighting modules, an Ethernet workstation, network gateway modules, and industrial camera enclosures.’‘

Schmitz: ‘‘Full line of LED and fluorescent lighting and C-mount lenses.’‘

Sippel: ‘‘Omron offers Intelligent Light Source control that allows users to control the lighting built into our cameras or control external lighting sources using the vision sensors menu system to adjust light intensity and angle as well as to save and reproduce the lightings settings. With the use of Omron's F150-M05L color LCD monitor or our NS series touch panel displays, video imagery can be displayed in ‘‘real-time’‘ using a rugged and integrated industrial display. Omron also offers cameras with limited built in lighting and lens for some applications.’‘

How does one interface to a smart vision sensor?

Blackwell: ‘‘In-Sight vision sensors can be easily integrated with PC-based factory automation devices and with PLCs on the factory floor via discrete I/O, serial communications, or using TCP/IP-based communications such as Ethernet, Ethernet/IP, DeviceNet, ModBus, and ProfiBus.’‘

Settle: ‘‘DVT supports a variety of networks are supported, including TCP/IP, Modbus, and EtherNet/IP. We also support Profibus and DeviceNet through a SmartLink module.’‘

Schmitz: ‘‘With a Windows PC over the Ethernet (network card), or a COM serial port. Or with a dedicated hand-held controller (see our PresencePLUS2 PPCTL).’‘ 

Jelonek: ‘‘Some systems interface with a PC, some can be accessed directly from the system controller; the best systems can be accessed through both.’‘

Sippel: ‘‘All of Omron's vision sensors allow users to interface and configure these easy-to-use products using either a push-button interface or a on-screen, drop-down type menu system without the need for any external interfacing device or software like a PC. All that's required is a removable keypad on our menu-based products. No PC software or network experience is required.’‘

Rao: ‘‘Ethernet, USB or IEEE1394.’‘ Endre points out, ‘‘Interfaces:  multiple digital I/O lines; Programmable push button on the smart camera hosing; Programmable Multi-color LED series on the camera housing; Direct video output from the smart camera  (RS170, VGA, SVGA with color overlay, sometimes with touch screen); RS232; Ethernet/IP; Device Net; Fieldbus systems’‘

What are the main industries using smart vision sensors and for what applications?

Rao: ‘‘Security, Semiconductors, Traffic management, Automotive and inside Aircrafts.’‘

Blackwell: ‘‘The main industries that are using vision sensors are general manufacturing and automotive for assembly verification, guidance, gauging, inspection, and identification. The pharmaceutical and medical industries use vision sensors for date and lot code reading, as well as inspection and verification of components in diagnostic kits. The packaging industry uses vision sensors for everything from inspecting tamper evident safety seals to checking bottle caps and labels. A big user of our application specific vision sensors for wafer ID is the semiconductor industry.’‘

Toth: ‘‘Mechanical engineering, electronic manufacturing, medical.’‘

Schmitz: ‘‘Automotive--error-proofing processes; Packaging--label presence and position, cap skew, print verification.’‘

Settle: ‘‘Primary industries using SmartImage sensors include automotive, plastics, electronics, packaging, food/pharmaceutical/ medical. Typical applications include presence/absence, precision measurement, coordinate transformation, color sorting, label check, pattern matching, etc.’‘

Sippel: ‘‘Automotive uses vision sensors to inspect and insure repeatable high-quality in parts before and during final assembly. Industries requiring packaging of parts or products, for example, food and beverage, pharmaceutical and consumer goods use vision sensors to check labels and label information, contents and overall appearance to maximize or insure safety, self image and customer satisfaction. Semiconductor and electronics industries use machine vision for quality inspection, alignment and final assembly from wafer manufacture to circuit board finishing and packaging.’‘

What advice would you give to a prospective user of a smart vision sensor to guaranty a successful installation?

Blackwell: ‘‘It’s important to work closely with a vendor to make sure that the product is correctly matched to the application. Take into account all requirements for inspection as well as all variations in part appearance that normally occur due to part rotation, changes in optical scale, and inconsistent lighting conditions, among others. If you need to establish a communication link between vision sensors and PCs at the enterprise level, the vision sensor must support a broad range of standard network protocols. Make sure the vision sensor makes it easy to set up applications and create custom graphical user interfaces at the same time within the same environment. Make sure the vision sensor has sufficient image pre-processing tools. Look for a vision sensor with its own family of compatible accessories. Find out what types of product support services are offered and make sure that the representative assisting you is a full-time machine vision specialist. It’s almost a certainty that as you gain experience with vision, you’ll add additional tasks, so make sure that the vision sensor has adequate headroom for growth.’‘

Toth: ‘‘You are going to make a big favor for yourself by taking control of the implementation process.’‘ 

Rao: ‘‘Do not compare a Smart sensor with a High End vision system. Expectations on performance should be realistic. Try before you buy.’‘

Schmitz: ‘‘Purchase vision sensors a from local, technical, industrial distributor who can help you select the right lighting, optics and vision tools for your application. Designate an internal ‘vision sensor champion’ who has the aptitude and will take the time to be trained and learn how to properly install, configure, maintain and trouble-shoot vision sensor inspections.’‘

Sippel: ‘‘Lighting and image quality are the most important keys to succeeding with an application. No amount of vision product horsepower can reasonably compensate for an improperly lit or noisy image environment. Also, simply buying a vision sensor does not ensure success. Working with a customer like a partner to help develop the application, determine the lighting required for example, and providing the proper training required to understand their installation are all key to a successful installation.’‘

Jelonek: ‘‘Don't spend $4-6K on a vision system and then $800 on installation and lighting. The long run cost benefits of installing the correct lighting and creating proper fixtures are tremendous.’‘

Settle: ‘‘Lensing and lighting is the most critical factor in a successful application and we provide technical assis