Researchers Say AI-Enabled Robots Need New Safety Frameworks

More thorough frameworks are needed to assure robots don’t harm humans, according to a new paper jointly published by Carnegie Mellon University (CMU) and the University of Oxford researchers in Science Robotics.

AI-enabled chatbots often have safety features, called guardrails, embedded within their software to prevent them from producing potentially harmful, biased, or otherwise problematic outputs. Generally speaking, however, robotics has not been afforded the same level of safety.

Without an embodiment, chatbots can't inflict direct, physical harm, but issues arise when AI systems are allowed to control robots. If hacked, safety risks can be substantial. The researchers have noted that one extreme is that by framing instructions as movie dialogue, they could persuade a chatbot to deliver an explosive device, even with manufacturing guardrails in place.

The safety built into traditional chatbots also doesn’t extend to AI-enabled robots because chatbots cannot interact with the physical world, unlike robots. Therefore, AI systems need to be better aligned with human values when being implemented in robots as the guardrails are designed for the digital world, not the physical world, where the actions are based on inertia and momentum rather than textual output.

Context is important. Chatbots can refuse a universally dangerous request, such as building a bomb, but robots must judge whether an action will be dangerous in each situation. This requires contextual awareness of the action and how it applies to that specific situation, and that specific situation only. This means that guardrails cannot broadly accept or deny requests for AI -enabled robots; there needs to be some subtlety and nuance to them for the many situations where robots and humans interact.

ROI Calculator

Discover the potential cost savings of robotic automation over a 20-year system life

This calculator compares your current manual labor costs against the total cost of owning and operating a robotic system over its 20-year lifespan.

EXPLORE TODAY

The need for contextual safety is increasingly critical as robots integrate into human environments. While they have traditionally been used in warehousing and logistics alongside some humans, robots are increasingly showing up in hospital and home settings. As their presence increases, the need for improved AI-robot safety is critical.

But what can be done to improve safety?

Researchers outline new safety guardrails that should be implemented to ensure that AI-enabled robots are safe to operate around humans on a daily basis.

1. The first is clearer and more explicit rules for the AI system itself, including in the system prompts that govern the behavior of the AI.
2. The second is safety checkpoints across multiple stages of an AI-enabled robot so that no single point of failure can compromise the entire robotic system.
3. Third is training algorithms need to use data that contains safety information, as this will help the robot to better understand when specific scenarios, situations, and actions are safe or not.

Overall, the researchers argue that safety can’t rely on a single guardrail, and these guardrails need to extend across the entire robotic system to ensure safety at all levels. This includes the rules that govern a robot’s decision to check and monitor its behavior to ensure that the robot understands context when faced with a potential safety risk. In the past, robots relied on simpler and more static assumptions because robots operated in simpler environments.

With robots getting smarter and working in closer proximity to humans on a regular basis, these traditional safety measures are no longer suitable, and more robust measures are needed going forward.