Guide to AI Middleware

AI middleware delivers connectivity and acceleration that enables AI to solve industrial problems that stymie traditional automation.

Smart systems help industrial organizations achieve lower costs, safer workplaces, and greater business agility. Industrial processes are awash in data that could help companies make better decisions and act faster.

Getting that data to where it needs to go is the role of middleware. AI middleware provides a pipeline for moving data efficiently from its sources to AI systems that analyze and infer, and then automates systems that act. 

In this guide, part of a series from A3 that introduces AI software , AI middleware and AI hardware , you will learn what AI middleware is, the pros and cons of various sources of AI middleware, and how AI middleware can address common challenges of AI integration. 

You also will learn about recent use cases that have delivered a timely return on investment.

AI middleware connects data to AI models and automation 

Software developers and engineers use AI middleware to integrate and interface sensors, edge devices, AI models, and automation scripts. 

An AI model uses past data to interpret and make inferences about current events or predict future occurrences. Training data includes production processes, machine tolerances, known failure mechanisms, and even photos of workers with and without hard hats.

 “Once the model sees enough examples, it can be given new data and, based on what it has already learned, the AI model provides an informed guess,” explains Seth Clark, co-founder and head of product at Modzy, a leading production platform for machine vision. “While the logic is complex internally, in an industrial setting, as long as the AI model provides reliably accurate predictions for that particular business, it’s a good model.” 

As Clark explains, there are three types of AI middleware: AI accelerators, model-serving middleware, and connectivity middleware. AI models often involve machine learning, which requires heavy-duty processing. AI accelerators, such as NVIDIA TensorRT and Intel OpenVINO, optimize AI inference on a specific hardware device. 

Model-serving middleware, by contrast, enables an AI model to receive input, process the model, and return results. Finally, AI connectivity middleware uses APIs, software development kits (SDKs), and internet protocols to deliver data to the AI model and distribute its results to a system that can act, such as a ventilation system, robotic control system, or an operations center. 

What largely distinguishes AI middleware from AI software is who interacts with it, adds Clark. Business users and industrial workers use AI software applications to do their jobs with higher productivity and efficacy. 

By contrast, AI middleware is used behind the scenes by automation engineers, supply chain teams and IT and operational technology (OT) operators to provide connectivity. “AI software is like the cell phone and AI middleware is like the 5G mobile network,” Clark explains. “Both are essential for an effective AI solution.” 

Sources of AI middleware affect cost and flexibility

Closed- and open-source libraries are two sources of AI middleware, each with pros and cons. “Open-source AI software, such as TensorFlow, Open Computer Vision Library (OpenCV) and PyTorch, requires more expertise for effective use than closed-source AI libraries but is more powerful and free to use,” says Peter McLaughlin, cofounder of Agmanic Vision, an expert provider of computer vision solutions. “This can mean higher initial development costs, but lower licensing costs and greater flexibility over the long term.” 

By contrast, closed-source AI middleware libraries from companies, such as Halcon and Matrox Imaging, charge licensing fees but provide higher-level functionality, and can be more convenient for developers to use than building a solution from the ground up. 

As McLaughlin adds, vendors that produce a product that contains AI middleware must factor other considerations into their decision to use open- vs. closed-source components: product support and future development. “The lack of visibility and control over closed-source middleware could impact their ability to support their customers and develop new products,” he says.

Deploying AI is complex but doable

Deploying an AI model in real-world settings, such as on a factory floor, on a robot, or in the field, requires significant integration of IT systems and industrial Internet of Things (IIoT) devices. These systems generate inputs for the model and utilize its outputs. For instance, a computer vision application needs cameras, lenses, lighting, brackets, and electrical panels, along with a system to process and use the information generated by the model.

“When creating an inspection system, for example, you also need a way to feed and present the part to the camera,” McLaughlin says. “Then, to use the AI’s output, you need a reject mechanism that takes results from the AI model and uses them to separate bad parts from good parts.” 

Expertise is vital to solve integration challenges, but it need not reside in-house, as McLaughlin explains. “For any industrial AI solution, I recommend that project managers reserve time in the timeline to bring in an expert from the solution vendor or an external consultancy,” he says. “While much industrial AI innovation occurs in Fortune 500 companies that have software architects on staff, solution providers like Agmanic Vision help businesses of all sizes to create highly integrated, end-to-end AI vision and robotics solutions.”

For those that want to speed AI software development further, a full-featured open-source development platform for edge AI and robotic automation applications may be the solution of choice.

Solutions, like the NEPI platform from Numurus, provide a software toolset designed to accelerate edge AI and robotic automation solution development through ready-to-use remote and local user interfaces, hardware drivers, ROS robotic interfaces, IoT connectivity, data management, and AI and other applications. 

“Today, every solution vendor looking to integrate modern AI and automation into their solutions is building these middleware tools from scratch and spending a lot of time creating and maintaining them,” Jason Seawall, Numurus CEO says. 

These types of solutions help experienced coders speed up edge AI development and also enables anyone to start creating AI-enabled solutions without years of coding experience.

Smart use cases for AI middleware ROI

The key to finding a good AI project is to identify a costly business problem that cannot be solved by traditional automation but can be solved by smart automation that learns from examples and makes a judgement call. Following are accounts of recent projects delivered by AI middleware vendors Agmanic Vision, Modzy and Numurus, respectively, that addressed common challenges on the factory floor and in a mobile setting.

Effective and efficient defect detection and sorting

On the factory floor, one customer wanted to automate the detection of cracks in inexpensive grinding disks, but traditional automation did not work because the disk surface was reflective, and the disks varied from one to the next with wide tolerances. An AI solution created by Agmanic Vision enabled the customer to identify even the tiniest cracks more effectively than they could using a human worker,” McLaughlin says. 

Another Agmanic Vision project involved a paper mill that produced diaper material. A blunt or broken blade failed to perforate a roll, causing problems that might show up days or even weeks later when processed by downstream equipment. The AI solution integrated mechanical, software and electrical systems to produce a system that constantly scanned a roll to immediately identify incomplete perforation. “The solution stopped production when it detected a defect, vastly reducing the amount of scrap produced,” McLaughlin said. “Consequently, it paid for itself in just a few months.” 

Maintaining industrial safety

Another factory-floor project, this one addressed by Modzy, resolved a potentially serious safety issue. In one area of a production plant, the US Occupational Safety and Health Administration (OSHA) flagged a build-up of noxious fumes that put workers at risk. “Using a predictive AI model, Modzy created and implemented a predictive air-monitoring solution that processed air-sensor data and took action to maintain or restore air quality,” Clark explains. “This included alerting workers hours in advance of a likely build-up and gave them the ability to open doors and turn on industrial ventilation fans.” 

Fast-tracking mobile AI development 

AI middleware also solves industrial challenges far from the factory floor. To address one such challenge on the ocean, Numurus helped to develop an AI solution for a vendor of commercial-fishing sonar systems for use with large nets. 

“To stay competitive, this vendor, like other sensor vendors, needed to modernize a 15-year-old product,” Seawall explains. 

Working with Numurus and a team of two engineers, the sonar vendor quickly brought to market an AI-enabled system and smartphone app that made it easier for their customers to find and catch more fish.

The connectivity and model acceleration provided by AI middleware makes simple and complex industrial processes smarter. When business problems stymie traditional automation, industrial companies may want to look at AI.