Choosing Off-the-Shelf AI for Manufacturing and Automation Applications

In everyday life beyond the walls and labs of those within the industrial automation landscape, the definition of artificial intelligence (AI) varies significantly. In truth, “AI” as a technology does not necessarily exist.

Rather, it refers to a science that encompasses several different technologies and engineering disciplines, including machine vision, computer vision, machine learning, and deep learning. When a system based on a combination of these technologies is competently designed — from application analysis to final validation — it can add tremendous value on the plant floor.

AI’s Emergence in Manufacturing

John McCarthy, a computer science professor at Stanford University who is considered the “father of AI,” said that AI can be defined as the “science and engineering of making intelligent machines, especially intelligent computer programs. It is related to the similar task of using computers to understand human intelligence, but AI does not have to confine itself to methods that are biologically observable.”

In this context, AI can provide manufacturers across different industries with a valuable tool for automated inspection in machine vision systems. Within AI are the subsets of machine learning and deep learning. Machine learning uses techniques that enable machines to “learn” to improve at different tasks. One such technique is deep learning, which uses artificial neural networks such as convolutional neural networks (CNNs) that mimic the learning process of the human brain.

As a subset of machine learning, deep learning (and thus machine learning) has become popular in the industrial automation space for its ability to “learn” from the continual analysis of models over time. The deep learning process begins with data. To help machine vision spot product defects, for instance, manufacturers would create an initial dataset by uploading images depicting classes of defects or features that must be detected along with “good” images. By collaboratively labeling the initial dataset, training the model and validating results with test images from the original dataset, testing performance in production, and retraining to cover new cases, features, or defects, deep learning ensues.

When all factors have been considered and the proper steps have been followed to implement a deep learning tool into a new or existing automated inspection system, the software provides value in many applications, including tasks such as defect detection, feature classification, and assembly verification. Specifically, these AI techniques can aid in subjective inspection decisions where a human inspector would otherwise be required. AI can also help inspect scenes where identifying specific features are difficult due to high complexity or variability.

Augmenting Machine Vision Systems

Not every application benefits from AI, and it’s not stand-alone technology. Rather, AI technologies represent a powerful tool for the automated inspection toolbox that can be deployed in several different industries, and when it comes to choosing a solution, manufacturers have options. They can program a solution in-house by using frameworks such as PyTorch or TensorFlow, purchase an off-the-shelf solution, or opt for an application-specific AI-enabled product or system.

Several off-the-shelf AI solutions exist in the market that allow end users to build their own model that is not tied to a particular application. Elementary's QA platform, for example, provides what the company calls a “full-stack vision system” that features both cameras and machine learning software with advanced analytics designed to identify problems, continuously improve, and unlock new insights into various manufacturing processes. The system combines traditional machine vision tools such as barcode reading and optical character recognition along with machine learning capabilities to add additional inspection capabilities to the system.

“AI is not magic, and it doesn’t do everything, but it can add powerful new capabilities to automated inspection systems,” said Mike Bruchanski, VP of product. “Anomaly detection — such as finding a piece of clear plastic in cereal — is a well-defined example of a machine learning–based vision tool that can work in tandem with a machine vision system for quality control.”

Common inspection applications for Elementary’s vision system include consumer packaged goods — including label, cap, and kitting —medical devices, automotive parts and assembly, and food and beverage products, which often involves a unique version of assembly inspection, according to Bruchanski.

“In prepackaged breakfast sandwich inspection, for example, it’s difficult to build a pattern that allows the software to understand if the cheese is not in the correct spot or not there at all, but our machine learning tools allow the vision system to look at the stacked sandwich to make a quick determination,” he said. “Our platform provides similar approaches in medical device assembly inspection while also performing a range of automotive inspections from regulatory label identification to inspecting welds for pits, voids, or cracks.”

Robotic Adoption on the Rise

Several more application-specific AI-enabled products have emerged in recent years, with the goal of streamlining and simplifying certain tasks. In some cases, this may involve an entire system that can be up and running in hours. Rapid Robotics’ Rapid Machine Operator (RMO) represents a prime example of such a system. Designed to handle common machine operator tasks, each RMO comprises a 6-axis robot arm, 3D depth sensor, grippers, and a control box for edge computing and AI processing. RMOs, according to the company, come with pretrained AI algorithms. 

“Each RMO is designed to meet unique customer production requirements,” said Juan Aparicio, VP of products, Rapid Robotics. “These modular work cells let manufacturers scale automation quickly, at low cost, and with little risk.”

Advancements in AI, according to Aparicio, are to thank for making robotic automation easier and more effective to deploy than ever before.

“One of the most important value propositions for AI in our field is the diversification of who can automate. The common narrative is that automation has already made its way through American manufacturing. Through our work, we have found that this is most certainly not the case,” he said.

He added, “A recent MIT report on the Future of Work found — to the researchers surprise — “very few” robots existed at small-to-medium-sized manufacturers. 

Many opportunities exist for AI-based robotic deployments, including quality inspection, autonomous mobile robots, assembly, and generative design, suggested Aparicio.

Staying within the realm of robots, Photoneo uses AI approaches in its automation solutions for the recognition, picking, and sorting of mixed types of items. The company leverages CNNs and trains them on a large dataset of objects to recognize items of various shapes, sizes, colors, or materials. If the software encounters an object it hasn’t seen before, it can recognize and classify that object based on similar objects it has previously encountered or been trained on. Furthermore, the software can be trained on specific datasets if a customer needs to pick unusual or custom items that may cause the model to decrease in performance.

“Customers often need a robotic object picking system that can recognize, pick, and sort items of various shapes, sizes, colors, or materials,” said Andrea Pufflerova, PR Specialist, Photoneo. “Integrating AI into such a solution enables customers to localize and handle mixed object types, including organic items such as fruit or fish.”

She added, “This may even include items that are typically difficult to recognize, such as bags, which are flexible, deformable, and full of wrinkles and irregularities.”

Comprehensive Custom Solutions

Companies looking to deploy AI-enabled software into their operations can take it even further and have a company like Prolucid build and integrate custom machine learning models, including support with data collection and labeling, model training, and deployment.

“As a systems integrator, our focus is on applying advanced computer vision and AI-based models to help with complex manufacturing inspection applications as well as a variety of non-manufacturing customers, including nuclear and medical,” explains Prolucid CEO Darcy Bachert. “Our typical approach is to solve the problem in as simple a manner as possible using computer vision or other existing tools. If we run into an application where these are not well suited, then we would look at AI as an option and start by looking for off-the-shelf models that suit the specific use case, such as anomaly detection or feature classification.”

Bachert notes the significant positive impact that open-source platforms such as TensorFlow, which come with pretrained models designed for relevant use cases, as well as the overall Python ecosystem, have had on the adoption of AI in manufacturing and other applications.

“It can be very time-consuming to develop a model from scratch, which tends to be impracticable for manufacturing customers,” he explains. “However, if a pretrained version can be utilized, then it greatly simplifies that initial investment.”

Into the Future of AI in Manufacturing

The future of AI in manufacturing, automation will likely involve using advanced analytics to identify defect trends early on and eventually prevent them from occurring. Machine learning can identify when a company produces more defects during certain parts of the day or when date-code labels are starting to fade because a printer is low on ink, for instance. The technology will recognize when processes are trending toward bad and send a command to the system or an operator to make an adjustment, according to Bruchanski.

“In the future, machine learning can help optimize processes by detecting defects, identifying trends in the root causes of errors, provide this data to the manufacturer, and eventually help move them further toward a defect-free environment,” he said.

Evolving hybrid AI models that combine model-based and AI-driven approaches also offer potential for industrial applications, suggests Pufflerova.

“It may not be enough to train a system today that works reasonably well on a limited set of examples – one also needs to understand its internal representations,” she said. “Hybrid AI models offer faster and simpler learning as well as better explainability compared to the traditional black box machine learning or deep learning methods.”

For Aparicio, it’s difficult to have a conversation about robotic automation without also talking about the workforce of the future.

“In terms of AI and automation making human roles obsolete, robotic innovations will cause changes, but will ultimately lead to more opportunities for people,” he said. “For example, robotic deployment will always require the use of engineers given the unique demands of orchestrating the variety of integrated processes, mixing of hardware and software, and designing a reliable system.”

These roles will likely become more infused with IT as software becomes the primary vehicle of robotic training and support. Given the speed at which these techniques are developed, companies may decide to partner with vertically integrated solutions providers, allowing them to focus more on growing their business while the vendors manage the robot fleet. In this scenario, explains Aparicio, the robotics workforce shifts from a distributed team to a central approach that enables a robots-as-a-service company to tap into economies of scale and centralize training.

When it comes to overcoming the hurdles that stand in the way of more rapid AI adoption, Bachert concludes that AI is just another tool that can be used in industrial automation. “As the open-source community continues to grow and more pretrained models become available, the barrier to entry for these techniques into real-world applications decreases.” He cautions, however, that this adoption requires the end customer to invest in training within their team, as AI has very distinct challenges that don’t always exist in simpler computer vision or inspection applications.