Spreadsheets, Gateways, Continuous Training: Deep Learning Gets Easier

Deep learning enables machine vision solutions for applications where defects or inspection criteria cannot be easily quantified or mathematically defined. It offers machine vision designers a powerful new tool for advanced inspection, assembly, and quality assurance applications because the software essentially helps the designer develop the best possible algorithm. But until recently, creating deep learning software required traditional programming skills, preferably combined with a strong grounding in statistics and traditional machine vision system design. 

Today, thanks to deep learning–enabled smart cameras, gateways, and application-specific coding, deep learning for the masses may be just a few clicks away. 

Smart Cameras Get Deep
The deep learning process is broken into two parts: training and inference. Training involves the optimization of a neural network by feeding it images of “good” products and “bad” products. Human operators tag the images as good or bad, and the deep learning software performs a statistical analysis to create a weighted table. The training process is very computationally intensive, prompting many designers to use high-end workstations for the training step or to leverage cloud resources. 

The resulting statistical matrix from the deep learning software is the heart of the second step in deep learning: inference. Inference involves using the deep learning algorithm to solve real-world problems. This part of deep learning software can run on a much wider variety of computer platforms, ranging from high-end PCs to smart cameras. Now, rather than using TensorFlow, Caffe, or another complex deep learning data visualization program to train a deep learning algorithm, users can turn to other solutions. For example, Cognex’s newest smart camera, the In-Sight D900, makes deep learning programming as simple as creating a spreadsheet. 

Cognex’s In-Sight series of vision systems pioneered the use of spreadsheet programming to simplify the design and deployment of machine vision solutions using a smart camera platform. By combining its ViDi deep learning software with the In-Sight platform, Cognex hopes to bring deep learning solutions not just to defect detection and optical character recognition (OCR) applications but also to more complex assembly verification.  

“The D900 is the first deep learning machine vision system that does not require a PC to deploy,” explains Brian Benoit, Senior Manager Product Marketing at Cognex. “Users are able to train the deep learning software through the In-Sight software interface on a PC. Once the training step is complete, the job is loaded on the camera and you’re ready to deploy.”

According to Benoit, the D900 is specified for applications running up to 150-200 inspections per minute. Cognex achieved this performance by upgrading In-Sight’s internal processing power while leveraging the speed inherent to an embedded solution. 

Gateway Brings Deep Learning to Every Camera
Not every machine vision customer that wants to leverage deep learning will be willing to replace existing machine vision hardware with new platforms, however. To bring deep learning to existing installations while simplifying the image acquisition and tagging that is so important to deep learning success, machine vision network specialist Pleora has launched its AI Gateway. Based on the NVIDIA GPU framework, AI Gateway sits like a frame grabber between the camera and the processor or other computational engine. Gateway provides Pleora’s interface expertise, accommodating GigE Vision, USB3 Vision, Camera Link, MIPI, and more, as well as TensorFlow deep learning software, the OpenCV traditional machine vision image processing library, and some Python to bind the solution together. 

With a single gateway, Pleora can provide deep learning machine vision capability for multiple “dumb” cameras, according to Jonathan Hou, Pleora’s Chief Technology Officer. “Customers may have the best deep learning algorithm, but how do they deploy it to the factory floor? Our solution allows them to keep their installed machine vision hardware base without forcing customers to choose any particular manufacturer’s deep learning solution path.” 

For customers who want a head start on leveraging Pleora’s deep learning capabilities without writing code, the company has partnered with Neurocle, a software solution provider for industrial machine vision inspection applications, and perClass, specialists in hyperspectral imaging. According to Hou, using the perClass plug-in, for example, a novice can extract intelligence from dense hyperspectral data cubes without any prior spectroscopy expertise. Industrial applications include differentiating visually identical pills in pharmaceutical manufacturing. Pleora has partnered with other AI experts in military imaging applications too, including vehicle rollover detection systems to address changing road conditions, IED detection based on newly tilled earth, and camo versus foliage differentiation. 

Never Stop Learning
While most deep learning systems have a hard stop between the training and inference steps, so that an algorithm’s effectiveness isn’t deluded by images that haven’t been expertly tagged or graded by humans, some solutions might be improved if they never stop learning. 

For example, barcode and traceability experts at Datalogic first applied deep learning technology to their image-based identification systems. “We’re heavily focused on anything around traceability, so we’ve applied deep learning to identification, code reading and some pattern recognition in general,” explains Bradley Weber, Product Marketing Manager at Datalogic. “When we look at a label on a package, obviously we want to read the barcode. But the most robust solution will also read the numbers under the barcode, to and from shipping information — any and all marks on the box. Every bit of data you collect on a package increases the chances that you’ll be able to trace the package, even if the barcode or labels are damaged.”

Additionally, Weber says that Datalogic’s focus has been on embedding the power of deep learning in its products so that customers do not need to access the underlying code. “When we look at who installs traceability solutions, they are not going to be a vision engineer,” Weber says. “So the solution needs to be simple and error-proof. For example, we will soon be releasing our Smart-VS, a vision sensor that uses artificial intelligence.  Without programming, a  user can push a button to teach what is a good item, push the button again to teach what is a bad item, and have the sensor will learn how to reject or sort the item.” 

According to Weber, traceability applications like code reading and OCR could benefit from continuously learning about how a customer’s packaging changes over time. The system could learn new fonts for OCR — possibly even handwriting. 

“Right now, processing is so cheap that we’re focused at putting computational power at the edge, which is enabling deep learning but also limiting our ability to train on the fly,” Weber says. “As 5G technologies roll out, we can imagine the momentum shifting back to the network and away from microprocessors. I suspect the back and forth between more powerful computer systems and faster networks will continue to guide how we deploy and use deep learning in machine vision applications. Anything that allows us to automate transportation and traceability — especially during these crazy times of COVID-19 — is a very good thing.”