Speed, Simplicity Steer Vision Software Development

As machine vision customers continue to expect swift vision application development, ease of use, and adoption of new technologies, image processing software manufacturers are updating their products to meet that demand. That cycle is resulting in numerous opportunities for software makers to show off their products’ versatility and muscle.

“The market is not changing as much as it is expanding,” says Steve Geraghty, Vice President, Industrial Products Division, of Teledyne DALSA (Waterloo, Ontario). “The incorporation of newer technologies, such as infrared and 3D imaging, is increasing the range of applications where machine vision can be applied.”

Dr. Olaf Munkelt, Cofounder and Managing Director of MVTec Software GmbH (Munich, Germany), says that machine vision development is “driven by the rapid changes due to the Industrial Internet of Things (IIoT), or Industry 4.0, demanding integrated production processes enabled by the digital interconnection of systems, machines, objects, and people.” 

Deep learning, in which computers learn from experience, is poised to meet these demands on the factory floor and beyond. “New software technologies such as deep learning extend application possibilities by learning normal product variation such that robust classification is possible,” Geraghty says. “Deep learning allows machine vision to be used to segment and classify objects, or even beings, from cluttered or changing backgrounds, rendering very difficult applications possible.”

Within its HALCON and MERLIC software products, MVTec has introduced deep learning technologies for optical character recognition (OCR). “With deep learning algorithms becoming more and more manageable, we were able to cast them into a feature that is very important in industrial applications and also within the IIoT — reading characters of any shape and under harsh industrial conditions robustly, fast, and without long training processes by the user,” Munkelt says.

At the end of 2017, MVTec will release a new version of HALCON, which offers a large selection of functions for the use of deep learning out of the box. For example, for the first time it enables customers themselves to conduct training of convolutional neural networks based on deep learning algorithms.

Matrox Imaging (Montreal, Quebec) has added a dedicated OCR tool called SureDotOCR™ to its Matrox Imaging Library (MIL) 10 software development kit. “SureDotOCR reads challenging dot-matrix text, because the historical approach of using morphology to connect the dots before reading — using a traditional solid-stroke OCR — is simply not robust enough in practice,” says Pierantonio Boriero, Director of Product Management for Matrox Imaging. “What’s more, the use of dot-matrix text for product identification is still extremely common in the packaging industry given the continued effectiveness of inkjet printers.” 

The company also boosted its barcode and 2D code reading tool in terms of initial recognition ability for first-time users or when tackling new applications.

Other additions to the MIL reflect customers’ drive for even more precise vision inspection tasks. Matrox Imaging started to add dedicated shape-finding tools to its vision library after finding that many pattern-recognition applications required locating only basic, as opposed to complex, geometric shapes. A new ellipse-finding tool complements an existing circle-finding tool. 

“These dedicated shape-finding tools outperform general geometric pattern recognition in terms of robustness and speed while providing greater degrees of freedom,” Boriero says.

For 3D vision, Matrox added a method for extracting and then measuring a cross section from a point cloud, as many profiling applications are solved simply by analyzing cross sections using 2D metrology. “We also made calibration more practical with the support of a self-describing calibration target and support for partially visible and extended calibration targets,” Boriero says.

Another factor influencing image processing software development is the proliferation of embedded vision systems. MVTec recently released a HALCON version (13.0.1) that runs on ARM®-based platforms as a standard, since numerous embedded vision software applications are based on ARM architectures. The new software version “helps customers to solve their embedded vision tasks within these architectures without any need for customization,” Munkelt says.

Easy Does It
Because their customers continually request user-friendly products, vision software manufacturers are easing developers’ programming burden. Using data-flow models on a graphical user interface, VisualApplets software from Silicon Software (Mannheim, Germany) simplifies the programming of application-specific image preprocessing on field-programmable gate array (FPGA) vision processors. The technology allows software and application engineers to program FPGAs without knowledge of VHSIC Hardware Description Language (VHDL), one of the standard languages used to describe digital systems. 

“Writing FPGA-based image processing algorithms with VHDL can take months, since modifying algorithms is also a long process,” says Mike Faulkner, Silicon Software’s Director of Business Development for the Americas. “With VisualApplets, you can complete these same tasks in days.”

Because most of the image preprocessing takes place directly on the FPGA, vision systems achieve real-time results that PC-based image processing cannot. VisualApplets software not only allows high-speed and low-latency image processing but also reduces the volume of data to be transmitted and processed and consistently delivers repeatable performance. It also lowers system development costs by greatly accelerating the development cycle.

A vision system relying on multiple high-bandwidth cameras will benefit even further from VisualApplets, according to Faulkner. For example, one of Silicon Software’s clients was doing 3D laser profiling that required four cameras, each interfacing with its own PC that fed data into a master computer. The Camera Link application transmitted 850 MB of data per second, per camera, per PC, for a total of 3.4 GB/s. 

Silicon Software’s OEM customer jettisoned the four preprocessing computers and interfaced four FPGA boards to the four cameras feeding data into the one master PC. “Instead of our OEM doing all his 3D laser profiling in these PCs, they created their 3D code with VisualApplets and put it into the FPGAs of each of these four boards,” Faulkner says. “They still had 3.4 GB of data coming into our four V-Series boards, but by doing all of the preprocessing laser triangulation algorithms on the FPGAs, it came down to a total of 200 MB/s into the bus for the PC processor to complete its final processing tasks.”

In the previous system design, the customer wanted to add more features but couldn’t because the PC processors were near maximum capacity. “Now, with this massive data reduction, our OEM keeps their existing algorithms inside the FPGA, which has liberated processing capacity to add more functions,” Faulkner says.

Matrox Imaging also has released a program geared toward those who do not want to, or cannot, code. The newest version of the company’s Matrox Design Assistant flowchart-based interactive development environment aims to “further streamline the process of developing and maintaining a vision application,” Boriero says.

Matrox Design Assistant achieves this objective by introducing an image-centric approach to configuring measurements, moving events and actions processing to sub-flowcharts, adding ready-to-go communication with automation controllers, and bringing in persistent recipes for creating a common flowchart that can handle different part variations.

MVTec’s MERLIC takes the same line, as it is designed for quickly building machine vision applications without programming. Features such as easyTouch, which helps guide the user interactively toward the solution, are introduced for a simple and fast application configuration.

Great Expectations
The changes in image processing software have been more gradual than grandiose, and despite machine vision customers’ growing list of refinements and requirements, the rules of engagement remain the same. “Developers who use a vision library still want, first, access to tools that are robust, fast, and practical,” says Matrox’s Boriero. 

Not only do developers expect a library’s application programming interface to make it easy to join employee tools together through support of various coordinates’ reference frames, for example, they also expect a library to help them with the integration and implementation of the rest of the application, such as through graphical image annotation in the operator interface.

“The vision tools themselves need to be ready to deploy, needing little to no adjustments to settings in order to work in specific cases,” Boriero says. That said, “developers do want the comfort of well-thought-out controls, just in case.”

Boriero further indicates that developers don’t want to code just to try or investigate something. “They want code generation to get them going more quickly when they are ready to proceed with their application,” he says. 

Ultimately, software manufacturers attuned to their clients’ needs will develop solutions that tackle every vision task from the simple to the complex. “While most applications we encounter can be easily solved with our standard tool set, we do encounter plenty of applications that require something unique,” says Teledyne DALSA’s Geraghty.

That could include developing custom algorithms or tool sequences specific to a customer’s application, or changing how to interact with customer programs, since some customers have preferred or predefined methods of integrating with third-party applications. 

“We try not to overcomplicate the sales process with a la carte ordering, but we also recognize that users want to purchase what they use,” Geraghty says.