« Back To Industry Insights
Association for Advancing Automation Logo

Member Since 1974


Content Filed Under:



How AI is Transforming Industrial Human-machine Interfaces

POSTED 12/08/2023  | By: Roy Sarkar, A3 Contributing Editor

No matter the machine, it's the user experience that can make or break manufacturing production lines. Recent advancements in AI have made it easier for workers to interact with their tools and systems – optimizing productivity and improving outputs at the same time – but the potential of this technology must be considered in the context of where it can do the most good.

Industrial human-machine interfaces (HMI) support workers by simplifying the control and monitoring of manufacturing systems. This traditionally means making machine inputs, status information, real-time alerts, and reports available through physical controls and touchscreens. With AI, industrial HMIs are shifting toward more flexible processes and less restrictive ways for workers to interact with their systems. Techniques like natural language processing (NLP) and generative AI are enhancing user experience, improving productivity, and enhancing accessibility.

How AI expands possibilities for human-machine interface design

Advancements in AI are bringing manufacturing interfaces in line with the intuitive and personalized experiences that workers have come to expect from their smartphones and tablets.

Holger Kenn, director of Business Strategy for AI and Emerging Technologies at Microsoft, considers individual workers to be the driving force behind emerging HMI technologies. “We’re seeing an infusion of consumer technology into the industrial field, where techniques like large language models (LLM) can transform massive printed manuals into interactive real-time training and provide HMI developers with more options to design for accessibility.”

AI’s biggest impacts will be in areas underserved by traditional industrial HMI technologies, such as:

  • Intuitive interfaces with familiar, consumer-like features, such as voice commands and gesture-based inputs to reduce learning time and task workloads for increased overall efficiency.
  • Personalization and customization that caters to the needs and preferences of individual workers to streamline workflows, reduce errors, and boost engagement, resulting in less frustration and higher productivity.
  • Intelligent guidance and training to mitigate worker shortages and skills gaps, enabled by devices that simplify complex tasks and provide step-by-step instructions, real-time troubleshooting, and interactive learning content.
  • Accessible interfaces to support workers with physical and developmental disabilities, accommodating individual accessibility requirements through techniques such as image recognition for the visually impaired and lip-reading recognition for the hearing impaired.

Despite all the buzz around AI triggering a massive, wholesale transformation of manufacturing operations, Tom Hummel, VP of Technology at Rapid Robotics, believes that actual use cases will focus on the human part of the equation, not the machines.

“As AI is introduced on the manufacturing floor, it won’t necessarily change what robotics have been doing a million times over for years,” Hummel says. “Determining precision welding feed rates and plotting robot paths are classically defined problems with very little room for techniques like machine learning to improve upon. Rather, LLM and similar methods will make the entire system easier for the operator to use and more adaptable to the needs of the manufacturer.” 

Key technologies behind the future of industrial HMIs

In industrial applications, HMI technology decisions must consider the physical environment in which operators interact with the system, on top of the actual tasks. Such decisions are pivotal in optimizing operator effectiveness and safety.

“The ability for AI-based systems to recognize human voices in noisy industrial environments has improved significantly in recent years, simply because of the amount of training data and examples we now have,” Kenn says. “This will lead to multimodal systems that can combine inputs like voice and gestures to recognize context, enabling operators to deviate from prescribed procedures to get what they want done faster.”

Four technologies are driving the future of industrial HMIs:

  • Natural language processing (NLP) is an AI-based technology that can understand, interpret, and generate human language in ways that are meaningful to operators. It enables machines and equipment to process and respond to spoken or written language, making control and feedback more intuitive and efficient.
  • Gesture-based inputs enable body movements or hand gestures to be used for controlling machines and interacting with digital interfaces. With the assistance of AI, gesture recognition systems become more accurate and adaptive to individual differences, allowing any worker to manipulate virtual interfaces, control robotic arms, and navigate through complex process visualizations.
  • Generative AI uses training models such as blueprints, user manuals, and a machine’s performance metrics to learn the underlying patterns and relationships within data so as to generate new content or solutions. For example, generative AI algorithms can learn from a database of existing machinery designs and specifications to generate a new design or help workers identify the most efficient and effective approaches to a production challenge.
  • Image recognition systems can become more intelligent and accurate through AI, helping to automate quality control processes. Automotive manufacturers already use image recognition to inspect painted car bodies for signs of scratches and paint irregularities. Similarly, electronics manufacturers use such systems to detect soldering defects and component misalignments on printed circuit boards.

Whether AI-enabled or not, vendors and manufacturers must carefully choose the HMI technologies that best fit a desired outcome, rather than just jumping to capitalize on the AI bandwagon.

“AI tends to prove the adage that says, ‘when you have a hammer, everything looks like a nail’”, Hummel explains. “It’s really good for open-ended problems, like improving the accuracy of parts inspection on a production line, but less suitable for closed, repeatable tasks, like pick and place for a pad printer. If the machine or automation already in place is easy to use and doing the job well, manufacturers should look to other areas where AI can help.”

The big question for HMI providers and users is how to determine the use cases where AI can best improve worker effectiveness, efficiency, and accessibility. Those who get it right will help the entire industry to successfully address worker shortages, skills gaps, employee satisfaction, and other pressing challenges.