For some, artificial intelligence (AI) brings to mind HAL 9000 from “2001: A Space Odyssey.” For others, it’s everyday tools like Siri or Google Home. Today, though, AI is doing much more — powering advances in industry, science, and medicine while driving innovation in numerous automation solutions. The recent surge in the use of AI for real-world applications is especially prevalent in the fields of robotics and physical AI, where intelligent systems are enabling machines to sense, move, and respond to their environments with greater autonomy.
Traditional AI vs. Physical AI
Artificial intelligence has been used for several years to analyze big datasets quickly. Its applications in theoretical research are leading to advances in technology, science, biomedical research and academic research that benefits from the rapid identification, cataloguing and analysis of reams of information to a more manageable subset. Theoretical applications in the digital world of AI, which is intended to provide clarity and answers about the universe, the world, or medical and scientific research are the best understood uses of AI for most people.
Recently, there has been a surge in the use of AI for real-world applications, especially in the field of robotics. Whereas digital AI focuses on theoretical applications, physical AI employs machine learning and AI to “teach” a robot how to do tasks that were historically too complex for a machine to complete. Rather than relying on complex foundation models and coding, this AI application relies on machine learning and reinforcement learning to improve the robot’s ability to complete complex tasks.
For AI to function properly, it needs real-world information. Traditional AI relies on reactive, rule-based coding that stipulates the predetermined rules and expected response at each step of a task. Any deviation from the predetermined parameters requires human intervention to update or change the rules so the robot continues to function.
Physical AI in Action
Physical AI applications bring artificial intelligence out of the digital realm and into the physical world, where machines integrated with AI in the physical body can sense, move, and act in real and dynamic environments. These systems combine AI with robotics, sensors, and advanced motion control to perform tasks that once required human intervention. That’s because, in addition to being able to analyze data, physical AI can translate the intelligence gleaned from the data into physical action.
In healthcare, robots like Moxi assist nurses by transporting supplies, while exoskeletons help patients regain mobility. In logistics and manufacturing, AI-driven robots streamline warehouse operations, handle packages, and work safely alongside people. Autonomous vehicles and drones extend physical AI into transportation and delivery, enabling efficient, contactless movement of goods and people.
Even social and companion robots demonstrate physical AI by recognizing emotions, responding to speech, and interacting naturally with humans. Across industries, these applications highlight AI’s growing role not just in thinking, but in doing, bridging digital intelligence with real-world action.
Challenges with Integrating Physical AI
Bringing physical AI into the real world isn’t easy though. Companies face challenges with safety, reliability, cost, and making sure these intelligent machines can work smoothly alongside people and existing systems.
“There have been challenges with applying AI to solve problems in the physical world. There has not been enough information available to program for every exception,” said Kristi Martindale, chief commercial officer with Palladyne AI. Prior to focusing on a software AI model, Palladyne AI was known as Sarcos Technology and Robotics Corp, which had 30 years of experience with dexterous robots, including animatronic robots, and exoskeletons. “We applied deep knowledge and expertise with complex robots and used large language models (LLMs) to apply to a broad range of robots and develop a low code/no code approach to AI and AI-powered motion control.”
Building on this foundation, one of the first tasks required for physical AI is to define the use cases, manufacturing processes, and personas for AI to replicate. “Technology is reaching a certain point in maturity where there are no longer enough application kits to deliver value quickly and easily across industries. That is impeding progression at the moment,” added Rohit Khanna, chairman of the board, 3D Infotech.
Educating the Industry in Physical AI
But identifying the use cases is only the beginning. To fully leverage physical AI, the industry itself must be brought up to speed. Most people in the manufacturing and research sectors have a basic understanding of AI, but that often extends to digital AI applications. The first challenge, then, for automators deploying AI is to educate the industry about the possibilities of AI.
“Physical AI is a relatively new approach, and there are thousands of robot integrators worldwide who could benefit from adding AI applications to their robots once they understand it,” says Martindale. She explained that in an industry “hardwired to point A-to point B-to point C hard-coding,” the flexibility of AI to train individual use cases can lead to higher profits and lower costs. This shift “requires a complete shift in thinking,” as it allows robots to be reprogrammed and redeployed without taking them offline, rather than relying on traditional recoding methods.
Both colleges and universities have many robots in their labs, and adding AI applications is an ideal way of training the next generation. “When you get tech in the hands of the ‘young’uns’, they learn it quickly, provide great feedback, and enter the workplace already comfortable with the new technology,” said Khanna.
Building and Training LLMs for Physical AI
Before an AI application can be trained, there must be a source to pull training information. One option for doing so is by creating large language models (LLMs). These models are trained using synthetic data, such as CAD and engineering drawings, as well as data collected from vision systems with multiple camera angles. Use cases and generative AI are then applied to extract information from existing documentation, which helps build the LLMs that train the AI system.
“Everyone will fail if the AI application is not simple to use, but it also must be robust,” said Khanna. “Once you have created the LLMs for general use cases, machine learning can ‘learn and teach’ the application, creating more specific LLMs unique to the operation. This allows continuous learning and improvement, which results in the robot being able to perform more and more complex operations.”
One of the advantages of using synthetic data to create the LLMs, is by using generative AI tools, factor tags and labels are automatically created, resulting in better LLMs. Then, if there are revisions or version changes, software updates or additions, the “parametric effect to the LLM” adapts without the need for additional programming. “We are a few years away from that sophisticated implementation. I expect the big players will actively create and maintain synthetic data sources for LLM creation,” continued Khanna.
While LLMs are a common way to deploy physical AI, they’re not the only way. Another way is to focus on task-specific intelligence. This utilizes real-time sensing, decision-making, and actuation. Some examples include training the robot through simulation, utilizing computer vision models, or relying on other adaptive intelligence techniques that don’t involve language understanding.
Autonomous Decision-Making with Agentic AI
One of the challenges with physical AI is the updates, testing, and learning that the back-end of a robot system must process when there is a new application, a software update, or a change. Agentic AI is a system designed to operate semi- to fully-autonomously within specific parameters, allowing the robot to make decisions and take actions that will meet specific goals with minimal or no human intervention. In other words, it simulates the decision-making that a human would follow in a similar situation.
For example, an autonomous mobile vehicle (AMV) with agentic AI capability and ML will divert to a different route if the normal path is blocked by an obstacle without requiring human intervention to program a different route. It allows the system to adapt to changing situations by leveraging tools and proactively identify and solve problems through ML of the environment, interaction with other AI agents and a variety of tools including analyzing data from large language models (LLM).
“Agentic AI is a means of teaching ‘tribal knowledge’ such as industry best practices and practical knowledge to the LLM so that the data can then be fed to the LLM and the robot can learn from it to improve performance over time,” continued Khanna.
Agentic AI uses “retrieval augmented generation” to proactively analyze, formulate, and implement new actions based on learned information and retrieved data. Rather than traditional AI’s reliance on predetermined rules and decision-tree, ‘if-then’ responses, agentic AI is proactive, capable of complex decision making in a dynamic environment.
“Artificial intelligence is only as good as the data you feed it,” said Sarah Andrzejewski, software project manager for Yakasawa America Inc, Motoman Robotics Division. “There has been a huge shift in interest in AI, and everyone wants to talk about it.” One of the trends is pairing AI with 3D vision that become the “eyes of the robot.” They have applied agentic AI using an autonomous 3D path to “teach” a robot piece by piece tasks such as bin picking and placing.
As part of the AI application in the Motoman NEXT robot line, which includes a fully autonomous control unit, parcel induction was used in tandem with a human employee to teach the robot the best places to pick up an object. In addition, several central processing units were broken in different ways, or different parts to teach the robot sensors how to inspect the CPU. Instead of pendant-based teaching, the application is skills and task based.
For example, if a robot is sorting and packaging parts, the skill would be to pick the correct part and place it in the correct bin, while the tasks would be identify, orient, pick and place. The skills then form the different components that a robot can do, and as more tasks are learned, the robot can complete more skills. “This particular robot is not suited to a high speed operation. Rather, it is intended for slow, close, careful operations or inspection that is traditionally completed by a human,” concluded Andrzejewski.
The Human Component
Automation discussions can cause concern amongst existing employees, concerned that the “robots will be taking their jobs.” When you add a robot that can ‘think, problem solve and learn’, is it only a matter of time before the robots take over completely? Short answer, no.
First of all, humans are responsible for identifying and providing the inputs, controls, protocols, quality parameters and safety measures that form the basis of the LLMs. They are instrumental in both initial and on-going training and evaluation of the robot’s performance, providing oversight for all the AI-powered tasks. “In a high mix/low volume environment, employees will be adapting the system any time a new workflow is required. One of the benefits of AI in particular, and Palladyne AI’s low code/no code application, is work that could only be completed by a control or applications engineer can now be handled by a technician working the line. This provides higher value opportunities for employees who used to be stuck with tedious tasks,” said Martindale.
Humans are adept at troubleshooting and problem-solving on the fly, and experienced employees have workarounds for common issues. If robots are performing some of the functions, then quality control oversight becomes more important. Robots can be programmed to send an alert if the sensors identify an issue with part conformity, for example, but the employees are more adept at determining how far upstream the issue may lie, or what kind of a bottleneck could result while the issue is resolved. “Humans will always be the final oversight, whether in terms of consistency, quality, or safety. They will simply be fulfilling higher-value tasks rather than the boring ones. In addition, every manufacturing facility must conform to the strict health and safety protocols, and in addition to sensors and shields to protect workers, there is always an emergency stop button that employees can access. Humans remain in control of the system at all times,” added Khanna.
In addition, with agentic AI, an employee could work through the range of motion required for a task to allow the system to “learn it”. For example, an employee would move a robot arm through a pick, move, place motion to move parts from a bin to a box, and then generative AI would add the parameters, dimensions, and tolerances to allow the robot to pick the right part for the right box from the right bin. “It all begins and ends with human oversight. While the robot may operate autonomously, humans will always have final oversight. Any task can be stopped by human intervention. Robots aren’t taking over…yet.” continued Martindal.
There continues to be a great deal of curiosity about how AI-driven applications can benefit the robotics automation industry. Advances in physical AI, agentic AI, and generative AI will continue to push the possibilities, and continue to provide proof of context use cases for lowering costs and increasing profit.
