Quality Inspection with AI Vision

The future of quality inspection with AI vision, including machine learning and deep learning, will transform the way manufacturers control quality assurance.

The majority of quality inspections today are done manually or by using traditional machine vision systems. Neither are without challenges.

Manual inspection relies on people. People — even experts with years of experience — can make decisions influenced by mood and environment, are error-prone, and have limited attention spans.

Rule-based algorithms are the basis of traditional machine vision. These industrial systems analyze and interpret image data and take actions based on the results to ensure manufacturing quality assurance. Rule-based machine vision systems tend to be rigid. As a result, they must be programmed to cope with every variation — and during manufacturing, there is a lot of variation.

Figure 1: AI vision is inserted alongside conventional machine vision, which has formed the brain of computer vision systems for many years (2), however the objective remains the same, to extract information from images to support quality control. Once image processing has been executed, a means of acting upon the result is required (3) – this is typically achieved by communicating the result to a reject mechanism, a PLC or an industrial robot (6). (Image courtesy of Agmanic Vision Inc.)

In contrast, vision systems that deploy artificial intelligence (AI) can process and visualize information more like a human would. Deep learning is really the essential AI breakthrough that has enabled the creation of visual perception systems that can deal with very messy, unstructured, and complicated real-world environments.

Rule-Based Machine Vision Challenges

“Rule-based programming techniques require a vision engineer to manually determine the algorithms used to process the image,” explains Peter McLaughlin, co-founder, Agmanic Vision Inc. This approach presents a number of challenges.

In order for rule-based techniques to identify defects, all defect types must be known and predefined as part of the image processing algorithm. Therefore, rule-based techniques are flawed when they are used to carry out inspection tasks that require interpretation and judgement; this is particularly true when inspecting agricultural and natural products such as food and lumber.

A second disadvantage of rule-based algorithms is that they are easily perturbed by changes in the environment, such as fluctuations in lighting or grease on machined parts. Another challenge of rule-based machine vision techniques is that they are highly application-specific, making them a challenge to transfer across various applications and industries.

“The principle of deep learning is to learn by example, rather than manually deriving the rules,” McLaughlin continues. “A set of example images is prepared with defects already identified.

These example images are used to train a model, which extracts patterns. Once trained, the model can be used to find defects in new images. This process of learning by example allows human-like judgement to be combined with the consistency and reliability of automation, overcoming the challenges of rule-based computer vision.”

AI Vision Provides the Best of Both Worlds

The benefit of AI vision in quality inspection is that it provides the best of both worlds. As with rules-based machine vision systems, it is very consistent and is more capable at handling subjective tasks.

A manufacturing line generally consists of products progressing along a production line where it is possible for a wide variety of problems to arise. But because computer AI vision systems are painstakingly trained to detect and identify defects in the particular products, they can quickly detect anomalies and keep the production line moving.  

There are many different forms of defects, such as surface defects or missing components on a manufactured item. There are also instances when parts are incorrectly assembled. Among the many benefits of AI vision for quality inspection are consistency, speed, accuracy, ease of use, and repeatability.

“Our goal is to match human accuracy on their best day and do that day in and day out,” explains Quinn Killough, computer vision specialist at Landing AI. “Consistency comes from having AI do the work, which is more consistent than humans will ever be. A well-trained computer AI vision system will be faster than humans, too. AI also can be iterated over time to improve performance with human supervision, since manufacturers need to trust quality inspection results.”

If the same defect is presented to an AI vision system 10 times, it will be recognized as a defect 10 times. This may or may not be true of a defect scanned by any given person. Also, AI vision can more easily repeat the same level of excellence from job to job, plant to plant. Another benefit of AI vision solutions is ease of use. These systems tend to make it easy to label defects on images and train the model. This means a company can have more people training and using models — not just data scientists or AI experts.

Quality Inspection Across Industries

The use of computer AI vision for quality control is becoming more and more prevalent in a variety of industries. According to Killough, Landing AI use cases span industries ranging from automotive, pharmaceutical, medical device, and life science to consumer electronics, semiconductors, and food and beverage.

For example, Landin AI works with semiconductor and Micro-Electro-Mechanical System (MEMS) manufacturers around the world. Because of the platform’s automatic defect classification solutions, one leading manufacturer posted an 80% drop in labor costs. Much of the reduction came from streamlining workflows.

Figure 2: Agmanic Vision has applied AI to a wide variety of quality control use-cases. Examples include detecting stains on automotive bearings, locating cracks in grinding disks, and grading chickens on a food processing line.  (Image courtesy of Agmanic Vision Inc.)

Formerly, a chip maker would rely on workers to review images of defects. The process was slow, expensive, prone to accuracy issues and, for one company, required 12 inspectors working in shifts around the clock to handle the volume of images from a single fab, according to Killough.

Other quality inspection industry examples include:

• Monitoring every stage of food and beverage production. If pizza mix passes through a production line, LandingLens monitors for quality such as appearance, color, etc. This results in improved end-product quality.
• Detecting failures in automotive parts. Early identification of cracks in things like camshafts, brake pads, or brake discs is so critical because it helps avoid adding more value to a product that is ultimately going to be scrapped or require expensive rework.
• Ensuring quality for medical devices. For instance, surgical tools, heart valves, ventilators, and stents are sealed in sterile packaging before shipping. LandingLens enhances detection of contamination and improper or incomplete seals.

Human and Object Movement in Manufacturing

Not all AI vision for quality inspection involves analyzing images. Some AI vision technology focuses capturing data from human workers and manual production to look for quality issues and improve processes/explains Prateek Sachdeva, co-founder and chief product officer of Invisible AI.

“When a cycle does not match the benchmark timing/movement the AI has been told to look for, an alert is sent,” says Sachdeva. “Quality control managers review alerts and watch videos of abnormal cycles to identify potential quality issues, monitor adherence to standardized work, and use video footage for root cause analysis.”

Quality professionals across industries have been interested in ensuring standard work processes for manual production are followed, as standardized procedures minimize risks of defects.

“Invisible AI customers use our outlier cycle tools and workforce performance analytics to monitor adherence to standardized work. QA professionals benefit from streamlined root-cause analysis and video footage of any cycle along the line,” says Sachdeva. “They can type in a product number and pull up all associated videos and audit stations to ensure operators adhere to standardized processes that minimize quality risks. Videos can also be saved and used as examples for training purposes.”

Figure 3: Invisible AI acts as an AI coach that helps augment and support the work being done by quality assurance professionals so they can get more done in less time. (Image courtesy of Invisible AI.)

AI Vision Quality Assurance Application Differences by Industry

Industry-to-industry, the basic concepts of AI vision in quality inspection applications are the same.

Image analysis is being done by companies across various industries to classify non-conforming features in images. The application only differs based on how the computer vision solution is integrated and what is done with the information.

According to Sachdeva, industries dealing with discrete units — such as in automotive and electronics manufacturing — benefit from automated object quality inspection, taking images when the object is in a particular position. For discrete products, AI systems find defects that happen repeatedly, whereas with continuous units, vision systems look for a blemish or some kind of error in the product’s texture.

Consider the case of a steelmaker looking for blemishes on steel plates. Once the AI identified the blemish, the steelmaker may cut out the blemish and keep the unblemished part of the steel plate. In contrast, food and beverage manufacturers may simply throw away defective products, while quality inspections in medical device manufacturing, which should be performed quickly, require that high-recall escapes be low. So, the perfection tolerance varies from industry to industry.

Depending on the industry, McLaughlin has observed significant differences in the extent to which AI is decisional. “In highly regulated industries, such as aerospace and life sciences, AI may be used to support a decision ultimately taken by a qualified inspector,” McLaughlin explains. “In less critical industries, where a degree of false negatives and false positives can be accepted, AI can take the final decision regarding part quality.”

Challenges of AI Vision Implementation for Quality Control

Many companies across industries face challenges when it comes to obtaining high-quality images of defects. Just like with a machine vision system, the first step for AI vision systems involves acquiring a good image of the part that clearly shows the features of interest.

It is impossible to label an image and train a model if the defect can’t be seen in it, Killough notes. “On top of this, good manufacturers don’t create many defects, so models need to be trainable with small data sets. And finally, we see a lot of computer AI vision platforms that help manufacturers train models but do not help to get the models deployed into production settings.”

According to McLaughlin, the principal challenge is obtaining a sufficiently large and representative dataset, especially for defective parts. “Manufacturing managers looking to implement AI for quality control usually have good process controls in place, meaning that the percentage of defects is already low. Even if some defect samples are available, it is often not possible to gather physical samples representing all shapes, sizes, and colors, etc., of the defects.”

Pros and Cons of Synthetic Data

One of the ways companies overcome this shortage of data for defect inspection is through the use of synthetic data. Synthetic data is artificially generated information, created by an algorithm or simulation that mimics real-world data. By creating synthetic defect data, manufacturers can enrich their datasets, providing a comprehensive range of examples for training machine learning models. This approach allows for the development of more robust and accurate inspection systems, capable of identifying and classifying defects with high precision.

But this data often still needs some work.

“The general consensus is that, if you're able to build an AI model that creates realistic synthetic data, you don’t need to train a model anymore because you have a model that understands the defects,” says Killough. “With that said, we have seen that synthetic data can be helpful if a rigorous review process is used to make sure that what is coming out is actually realistic.”

The AI-created images most likely need to be touched up by a human to ensure they look right. That can be done, but it takes a lot of time and money. It may be worth it to apply only in some applications, but not for others. Fortunately, this is a rapidly evolving field and there will be many innovations in the near future that will make the process more automated and reliable.

McLaughlin asserts that synthetic data has the advantage of reducing the effort and time usually required to gather real image data, but also notes that synthetic datasets must be carefully managed, manually reviewed, and adjusted, often requiring real image data to complete certain areas of the envelope.

Agmanic Vision recently used synthetic data to create a dataset of welded part images from their CAD models. The objective of the application was to classify parts into categories as they exit a painting booth. “Rather than manually capture images on-site,” McLaughlin explains, “we were able to train a classifier using synthetic images generated from CAD.”

Ethical Considerations When Using AI Vision and Synthetic Data for Quality Inspection

Concerns over privacy and algorithm bias are two of the principal ethical concerns associated with widespread adoption of AI vision and synthetic data for quality inspection. It’s a given that it is possible for AI to introduce bias into decision-making if it is not correctly trained and tested.

“The addition of synthetic data, if not correctly applied, can exacerbate dataset bias,” says McLauglin. “If the deployed AI is decisional, i.e. making the final decision in a quality inspection context, the impacts of false negatives and false positives can be significant.”

However, McLaughlin notes that a growing number of AI training tools such as NVIDIA's TAO toolkit offer integration mechanisms with synthetic data generation tools such as Unity. “The emergence of generative AI is influencing synthetic data generation, providing a means of creating realistic textures and surface finishes.”

Killough agrees that AI vision for quality inspection is getting easier, faster, and more accurate, and generative AI will really boost its use. “For instance, Landing AI now helps companies build domain-specific large vision models (LVMs). Businesses can bring AI to their huge libraries of images to create applications that meet their specific business needs. While LLMs [large language models] enabled text prompting, LVMs will enable a vision revolution.”

Figure 4: Through Landing AI's large vision model (LVM) solution, companies can create domain-specific LVMs from their unlabeled image data, which can number millions or billions of images. (Image courtesy of Landing AI)

Through Landing AI’s LVM solution, companies can create domain-specific LVMs from their unlabeled image data, which can number millions or billions of images. In this way, a vast range of computer vision problems can be solved. As the images are trained, they can be used to improve inspection solutions, such as identifying cancer cells in histopathology images or manufacturing defects, according to Killough.

“Visual prompting also enhances use of genAI capabilities in computer vision for quality inspection,” Killough adds. “Visual prompting takes the framework of text prompting in technologies such as ChatGPT and brings it to computer vision. It reduces the time to get a computer vision result from days or months to minutes or even seconds.”

The Impact of AI Vision on Manufacturing Quality Assurance

Quality assurance processes have been transformed by AI vision integration in the manufacturing sector. Before deep learning was invented, machine and computer vision systems were limited to applications that did not require much judgment, such as metrology and relatively obvious defects. AI vision systems can now achieve human-like levels of inspection using deep learning, facilitating faster, better, less expensive, and easier quality assurance processes.

That’s not say that human-expertise is left behind. AI offers the ability to augment a human expert’s skills, taking them away the monotony of the inspection line, and turning their attention to edge cases or the fine-tuning of the data sets used by the vision system. Their knowledge continues to help drive quality forward.