Deep Learning, Industrial Applications & Looking Into The Black Box

Designed for easy integration with existing machines, PLCs and automation, Radian provides deep learning-based high-speed inspection functionality. Credit: Elementary

Over recent years, deep learning solutions have gained significant traction in industrial applications, particularly those involving vision. A subset of machine learning, deep learning is based on artificial ‘neural networks’ that learn in much the same way as human neurons do.

More than half of Europe’s top manufacturers are already implementing at least one AI use case in manufacturing operations, according to 2019 research from the Capgemini Research Institute, with that number dropping to 30% and 28% in Japan and the U.S. respectively. The report also identifies quality control, predictive maintenance and demand planning as applications well suited to deep learning-based approaches.

Deep learning has found a sweet spot in vision-based applications, particularly inspection, where they outperform both human operators and traditional machine vision systems.

While traditional machine vision systems can typically only handle hundreds of thousands of parameters at a time, deep learning-powered systems can handle millions of parameters. This enables faster, more powerful, and more flexible vision-based applications. Deep learning helps solve some of the most pressing challenges in image processing such as classification, segmentation, and detection.

The volume of data involved in deep learning also gives rise to the ‘black box’ phenomenon, in which there are so many layers of complexity and so many data points being processed that it is not always possible to explain precisely how the system derives its outputs from its inputs. Likewise, it is not always possible to interpret the exact chain of cause of effect that plays out within a deep learning system.

So, what types of application are deep learning-based vision systems used for? And does the black box ‘phenomenon’ really matter when these systems perform so well in real-world applications? 

Deep learning-powered vision
Vision-based inspection and QA applications have proven to be an excellent fit for deep learning-based approaches. For example, the Prism software platform and Radian camera developed by pioneering California-based AI and automation firm Elementary uses deep learning strategies to enable flexible inspection of items on moving production lines. The advanced technology allows fast deployment on applications that are beyond the capabilities of traditional vision systems.

Unlike conventional vision systems, which take considerable time to retrain on new tasks, Prism provides user-friendly ‘drag and draw’ features that support quick and easy deployments on a wide range of QA applications. Credit: Elementary

One of Elementary’s customers has deployed Prism™ and Radian™ to detect foreign objects in bulk granola. Another uses the platform to ensure uniform distribution of toppings on food products such as pizzas and candy sprinkles on ice cream bars.

“Deep learning-powered approaches are challenging the assumptions of traditional manufacturing in fundamental ways. I think what we are doing at Elementary is very disruptive and it’s time for industry to embrace deep learning, because the value derived from adopting this modern form of artificial intelligence is really profound,” says Krishna Gopalakrishnan, Senior Director of Platform & Vision at Elementary, noting that systems like Prism take the complexity out of artificial intelligence and machine learning by providing interfaces that can easily be used by non-expert end users.

Elsewhere, Korean firm Neurocle has developed Neuro-T and Neuro-R --deep learning vision trainer software solutions specially designed for easy adoption by non-experts. (The company also provides Neuro-X training software for experts.)  The software’s intuitive interfaces allow non-expert end users to design processes that automate processes such as appearance defect inspection, image analysis, and logistics packaging inspection. Neurocle's software is deployed by several sectors including automotive part manufacturers, cosmetics logistics firms, and in the medical field where it is being used to improve endoscopic and microscopic cell image analysis.

“Manufacturing and automation processes typically involve repetitive tasks, making them an excellent fit for deep learning technologies,” says Neurocle CEO Hongsuk Lee. “The neural networks that support deep learning systems need data to learn from and when that data is almost repeating, it makes it easier for the deep learning system to do its work.”

In Dec 2021, Gartner identified Neurocle as a ‘Cool Vendor’ in the ‘AI for Computer Vision’ category.

Deep learning powers several classes of vision application from inspection and QA to object detection and classification. Credit Neurocle.

Peeking inside the black box
The level of complexity and millions of parameters involved in deep learning systems makes it difficult to explain the precise path a deep learning system took to achieve its outputs.

However, it’s a mistake to think of these systems as being ‘mysterious’ in any meaningful sense, says Elementary’s Gopalakrishnan.

“Deep learning is based on sound science. And there are really good mathematical models to provide explainability as well as back propagation algorithms that have been known and studied for many years,” explains Gopalakrishnan.

“Is research in deep learning done? No. There are a lot of interesting questions that people are asking, including at our company, and the field is constantly evolving. At the same time, a worker on the shopfloor or manufacturing line doesn’t need to know which state-of-the-art deep learning models are being used – they want to know that it works, without having to worry about training the system themselves.”

‘Explainability’ is an ongoing research question, with research being driven by mission-critical applications such as self-driving cars, says Dat Do, Director of Machine Learning at Elementary.

“Self-driving car developers want to know why the algorithm made a certain decision, and there are a lot of methods developing in this space right now. Elementary is working on this too. Our anomaly detection models can explain where they believe the defect to be and also the level of anomalousness of the defect, which enables us to understand the black box to a much better degree,” explains Do. 

For Neurocle’s Lee, the black box means that there is a small element of trust involved when deploying deep learning strategies.

“We create the neural network, define some mathematical parameters and then let the deep learning algorithms do their work. Even deep learning engineers and experts have to trust the neural network,” says Lee.

Nevertheless, the black box phenomenon is “not a big problem” for industrial vision applications.

“From the practical perspective, the black box is not significant. It’s totally OK not to understand everything about DL and to rely on neural networks. Our goal is to create and train high performance deep learning models that bring value to our customers. We are achieving that goal and constantly striving to improve the power and functionality of our deep learning software,” says Hongsuk, noting that Neurocle recently launched updated versions of its core software that introduced a new ‘fast retraining’ function.

Neurocle uses convolutional neural networks (CNNs) as the basis of its approach to deep learning in vision applications.

“From the technology perspective, CNNs provide many improvements compared to conventional deep learning algorithms. Rather than analyzing the image itself, we use convolution to track specific features on which the neural network then focuses. Not only is the performance better than other deep learning approaches, but it also provides more clarity,” explains Lee. 

Credit: Neurocle

Future research into explainability and how to interpret chains of cause and effect within deep learning systems will further illuminate the black box. And in the meantime, adoption of deep learning technology in industrial and business applications is set to rise, driven by the business case but also by the ease of use and flexibility associated with today’s deep learning systems. The global deep learning system market, including vision, is predicted to reach a value of USD93.34 billion at a CAGR of 39.1% by 2028, according to analysis by Emergen Research.

Meanwhile, the global computer vision market revenue is predicted to grow from USD2.9 billion in 2018 to USD33.5 billion by 2025, driven in large part by advancements in deep learning-powered systems, according to research from Omdia.