3D, OCR, and Robustness Drive Machine Vision Software Evolution

For most technology companies, customer requests — and often funding — are the driver behind product development. Today’s custom hardware is tomorrow’s off-the-shelf product. The same can be said for image processing software.

For example, the growth of robotic assembly has pushed the development of 3D-imaging systems and associated software. Vision-guided robotics (VGR), along with 3D machine vision, is a “huge trend” in the machine vision industry, says Johannes Hiltner, HALCON product manager for machine vision software specialists MVTec (München, Germany). Part of the growth in VGR applications is the increasing availability and cost-effectiveness of 3D sensors, paired with decreasing computational costs. But despite the availability of more processing power, it’s still critical for machine vision software to be optimized to handle a broad range of data sources, says Hiltner. Equally important are industry standards that help grow markets for machine vision equipment.

Simplified VGR
“Customers don’t want to cope with camera-specific sensors when it comes to developing 3D vision applications,” Hiltner says. “So that’s a trend when it comes to standardized interfaces, not only between cameras and the machine vision application but also in standards which connect the technologies like the robot controllers or the SPS controllers with the machine vision application itself. We love standards at MVTec because they help us to expand the hardware support we offer for our hardware and simplify the integration process for our customers.”

MVTec representatives are working with MATROX and other companies through the AIA and European Machine Vision Association (EMVA) to expand the GigE Vision, GenICam, and GenTL standards to include 3D hardware and their associated interfaces, including new classes of time of flight (ToF) sensors.

To illustrate his point, Hiltner points to HALCON’s latest version (12), which now has the ability to perform many machine vision algorithms on true 3D point cloud data sets, rather than project the 3D object into 2D space and then apply 2D algorithms. While doing so may provide a more accurate result, downsides exist. For example, Hiltner says, the underlying mathematics become more complex, and run times typically are slower compared with software that manipulate just 2D data. By offsetting those drawbacks, however, the 3D point cloud software is more robust in terms of reducing false positives and miscalculations, important considerations for VGR applications such as bin picking, in which a collision can be costly and lead to extensive robot downtime.

“We have more than 2,000 features in HALCON’s image processing library, all based on 2D,” Hiltner says. “We’ve selected the most common features to be used in 3D applications, such as centroids, searches, and others and modified them to work in true 3D space.”
At the same time, MVTec has given HALCON the ability to work with local deformable surfaces within 3D point clouds. Designed for applications like food handling, the algorithm starts with a golden part, and then allows for localized deformations to occur to help robots to adapt to organic shapes, Hiltner says.

Faster, Better OCR
Improved 3D algorithms are also part of newest release of the Matrox Imaging Library (MIL) version 10. “We found that with a lot of 3D applications you have a 3D data set but you are analyzing a cross section,” says Pierantonio Boriero, product line manager for Matrox Imaging (Dorval, Quebec, Canada). In practice, this means a 2D image. “So we’ve developed the means to easily extract a cross section from a 3D data set, point cloud or depth map, and feed it to our metrology tool to do measurements against tolerances.” This allows users to examine a cross section of a thickness and an angle of their choosing across the object, and not just flatten an object on a reference plane.

Matrox has also optimized its geometric pattern-recognition tool for certain standard shapes, including ellipses and circles. Adding these shapes strengthens and speeds the tool’s ability to find these features compared to general geometric pattern search routines.
This month, Matrox will roll out MIL 10’s latest processing pack, including a new optical character recognition (OCR) tool for dot-matrix printers.

When it comes to dot-matrix characters, the conventional method uses a filter to fuse the dots into strokes. The problem, says Matrox’s Boriero, is when the letters are spaced closer than dots within a letter because of distortion the printed material. “When you fuse the dots to form characters you actually fuse the characters,” says Boriero.  “So any angle variation from the perpendicular will result in distortion, and just the variation in the material underneath and the line speed will affect dot-matrix printers.”

The new Matrox CR tool is designed to work out of the box with the user responsible for only a limited number of settings. Says Boriero: “The only thing they have to do is tell us what the dot size is of the text in the image and the dimension of that text” — in other words, the length and height but not the x-y coordinates.

Users can create and edit fonts, but predefined fonts are supplied, including fonts for at least one industry-leading ink-jet printer manufacturer. What’s more, the tool offers the possibility to set user constraints, such as expiration date and lot-number parameters. As a result, the tool knows that the image will start with EXP and LOT characters that have definite character positions with pre-defined variations. This further refines the positive recognition rate, Boriero says, by “constraining the read. You know that what you are getting is being read with higher confidence.”

While MVTec and Matrox are juicing their 3D and OCR algorithms, Cognex (Natick, Massachusetts) is adding a turbo boost to its mainstream PatMax geometric shape search algorithm.  Pattern matching represents a first step in most machine vision applications.

Because previous tools imposed certain speed limitations, many applications resorted to lower-resolution cameras to keep pace with production lines. Cognex says that its PatMax RedLine technology performs faster on high-resolution vision systems, eliminates the tradeoff between speed and performance, and enables customers to increase resolution and gain accuracy without sacrificing speed.

“Our customers have told us that they have an increasing need for color inspection and visualization of very small defects, even in a large field of view,” says Joerg Kuechen, vice president and business unit manager, Vision Products. The technology aims to address what Cognex says is the slow speed of conventional pattern-matching algorithms when tasked with high-resolution applications.

The technology is available in the Cognex In-Sight Explorer software version 5.1, and is built to run on three vision systems: In-Sight 5705, In-Sight 8405, and In-Sight 5705C.  The 5-megapixel models come set up with In-Sight Explorer software and a spreadsheet view that offers efficient system configuration and handling of vision-application data. A drag-and-drop vision tool palette and tool property sheets help to make the effort more intuitive. The new vision systems also use JavaScript to simplify complex and data-intensive vision tasks.

Taken together, these software developments help solve the true mission of machine vision – helping customers to do their jobs better, faster, and cheaper. Today, the mutually beneficial symbiosis between microprocessor and image processing software is making yesterday’s hardest applications de rigueur while opening up new applications that will further expand the machine vision industry.