3-D Vision Improves Performance of Challenging Inspection Applications

Manufacturing has made increasing use of traditional 2D machine vision inspection, optical character recognition (OCR) and ID reading systems, which increase quality, improve throughput and reduce costs. But there are numerous applications that are better suited for 3D vision such as measuring height, volume and tilt of parts or reading raised characters against a similarly colored background, and performing inspections in variable lighting conditions. These challenges are being addressed by a new generation of 3D vision systems that utilize laser triangulation in combination with a 2D imager to generate a 3D topographical profile of an object that can be used for measurement, inspection, or character recognition. 3D vision systems are able to perform all types of 3D measurements that read characters and codes, independent of contrast or lighting conditions. Another advantage of 3D vision systems is that they can provide real world measurements to micron level accuracy without being specially calibrated for the application. For many manufacturing applications where 2D inspection is limited, 3D vision systems provide substantial performance upgrades that can improve quality, throughput, and efficiency.

While traditional 2D machine vision systems can provide excellent performance on the vast majority of inspection and barcode reading applications, there are a considerable number of applications where vision systems run into the inherent limits of 2D technology. Typical examples include applications that are inherently 3D, such as, measuring the height of an object, parallel to the surface of the sensor, measuring the slope of a surface that is tilted, measuring the volume of an object, or evaluating the profile of an object. Another concern in OCR and applications lies where characters or codes must be distinguished against a background that is either of a similar color or where it is difficult to distinguish the foreground from the background. There are also many machine vision and ID reading applications that consist of challenging lighting problems which can be addressed through the use of 3D technology.

Advent of 3D machine vision systems
A new generation of 3D vision systems is addressing these challenges by using laser triangulation to extract 3D information from parts. A laser displacement sensor projects a beam onto the object to be measured. The beam is displaced by the 3D shape of the object. The image sensor captures a 3D topological representation of the 3D image where the footprint of the object is represented by the extent of the image, and the z height data is stored. A complete 3D image is generated from a series of acquired profiles where each row of the image corresponds to one profile, resulting in a 16 bit greyscale image. The resultant 3D image of the object can be used to both locate the object, and accurately register vision tools for the purpose of measuring 3D and 2D features, such as, length, width and height. It can be used to easily determine presence or absence of an object regardless of color or lighting and yield a high-contrast image from very subtle changes in height.

Consider the sprocket gear shown in Figure 3 that looks good in the x and y dimensions. But a 3D inspection reveals that a tooth is broken with the top 10 millimeter of height being missing. One could also imagine a case where only the top surface of the tooth is missing, as opposed to the entire tooth, and clearly 3D inspection is required to identify a defect of this type. Another typical 3D inspection application involves measuring the flatness of a critical component such as a printed circuit board (PCB) or a substrate in a carrier. Skewing or warpage of components cannot be detected with 2D inspection. However, 3D machine vision can easily determine the complete topographical profile of the part surface. Detecting the presence or absence of components whose color is similar to the area where they are supposed to be installed is another very common 3D inspection application. There are also many robotic guidance applications where 3D vision is required, such as, picking parts out of a bin or guiding a robot arm around multiple surfaces of a part.

In the electronics industry, inspection of substrates in carriers is required to determine if a substrate is missing, is too high or low, or is tilted. An advantage of 3D machine vision in this application is that it can determine the height of the substrates with respect to the carrier. For example, if the carrier itself is higher than normal, it will not cause all of the substrates to be mistakenly identified as too high. Likewise if the entire carrier is tilted by a few degrees, the overall tilt of the carrier can easily be factored out to measure what is really important – the tilt of each substrate with respect to the carrier.

Automotive manufacturers need to record the Department of Transportation (DOT) codes on the sidewalls of tires that are installed on vehicles. The codes are molded and raised on the surface of tires, but are the same color as the tires. 2D vision systems can be used to perform OCR on the codes but require precision lighting which must frequently be re-adjusted when new tires are introduced. 3D vision systems, however, produce a high contrast image from very small changes in the geometry so they can provide a much more reliable and robust solution.

3D vision systems
Figure 6 shows how the working area of the sensor depends on its field of view (FOV). The area that can be measured by the sensor consists of a trapezoidal tunnel and as long as the part or feature to be measured is contained within this tunnel the sensor will be able to perform 3D measurements on it. Sensors with a small FOV offer the highest resolution so they are well suited for inspecting smaller objects at the highest possible levels of precision. On the other hand, sensors with a larger FOV are capable of inspecting larger objects with a greater clearance distance, although with less precision than sensors with a small FOV.

Calibration is important for all vision systems but especially for 3D systems that need to make accurate measurements for dimensional gauging and robot guidance applications. Because the measurement range is fixed, the new generation of 3D vision systems can be rigorously calibrated at the factory to real world measurement units, such as, millimeters as opposed to conventional 2D vision systems that require calibration on the factory floor for each application in order to convert from pixels to real world units. 3D vision systems also handle extrinsic calibration, which refers calibrating the position of the camera in the world, which is particularly applicable to situations where the camera is mounted on a robot and moved to the part being inspected. Software is also available to support hand-eye calibrations which are used to convert vision results into the robot’s motion coordinate system.

3D inspection adds an extra layer of complexity to machine vision programming. However, 3D programming environments have been upgraded to simplify the process of building maintainable 3D vision applications. The typical development environment enables users to drag and drop 3D and 2D vision tools, use interactive editing controls to set vision tool properties and add plug-ins for external hardware. A graphical flowchart-based programming environment results in a diagrammatic representation that’s easy to understand, and can be supported by factory engineers. Typical 3D vision tools include a plane to plane angle calculator, a height calculator that can be used to determine minimum, maximum or average height, a plane estimator used to create a reference plane, a volume calculator and cross-section tool that captures a slice of an object for dimensional inspection. Typical 2D vision tools used in 3D applications include pattern matching tools that can be used to identify the presence and location of a specific geometric pattern, regardless of its orientation, as well as specialized tools for performing OCR. Programmers can create a runtime operator interface to monitor and control the vision application simply by dragging and dropping controls and images. Results or decisions can be communicated through physical I/O, factory floor communications such as Ethernet/IP or ProfiNet, and monitoring equipment can also be connected. The final step is to deploy the application on a dedicated vision controller or industrial PC. Critical to the successful deployment of a 3D vision application is having the right hardware, vision tools, development environment, and communication to the factory floor that has been assembled and tested as a complete solution.

Conclusion
In contrast to traditional machine vision systems that provide information in two dimensions, 3D simply provides users with more information about their manufacturing processes and products, allowing manufacturers to apply machine vision to a larger range of applications on the factory floor.

By Robert Tremblay
Product Marketing Manager