Machine Vision and AI Help Retail and E-Commerce Companies Keep Up With Demands

From retail to restaurants, COVID-19 had significant impacts — good and bad — on industries of all types. The global pandemic forced companies to rethink strategies and act on their feet, but even prior to the outbreak, many businesses were already shifting their strategy toward advanced automation technologies such as robotics, artificial intelligence (AI), and machine vision. Retail and e-commerce businesses represent two such examples, and this article looks at some of the latest technology trends in these spaces.

Robots Help During COVID-19

As the virus caused global shutdowns, people became increasingly reliant on e-commerce from major retailers including Amazon, Walmart, Target, Home Depot, and Best Buy. While online shopping was popular prior to the pandemic, Americans spent 32.4% more in 2020 than 2019, for a total of $791.7 billion. The growth spike in e-commerce as a percent of total retail sales was a significant 3% – increasing to 14% of all U.S. sales in 2020, compared to 11% in 2019, according to CNBC.

RightHand Robotics’ RightPick system autonomously handles items in a consumer-packaged-goods company’s warehouse.

In warehouse settings where the predictable flow of products is the difference between happy customers and lost opportunity, advanced automation technologies have helped these businesses keep pace. RightHand Robotics, for example, produces the RightPick^TM system, an autonomous item-handling robot system featuring RightPick AI software, which lets the robot handle thousands of SKUs from totes, bins, boxes, and cases, and a hardware platform comprised of an intelligent gripper, a high-performance industrial PC leveraging a dual-GPU architecture, a robot arm, such as the UR5e or UR10e robot from Universal Robots, and an Intel® RealSense™ D415 3D depth camera. The company’s RightPick MCP application programming interface (API) also provides simple integration to commercial warehouse management software to support a number of common warehouse automation workflows.

Capable of picking and placing individual items as part of a range of fulfillment center processes, RightPick is currently in operation in Japan at consumer-packaged-goods company Paltac Corporation. The company deployed robots to automate the fulfillment process for more than 20,000 SKUs, and the systems have worked throughout the pandemic. When a retailer carries thousands, or even tens or hundreds of thousands of different products, the item transfer process becomes very complicated, explained Vince Martinelli, Head of Product & Marketing, RightHand Robotics.

“If a vision-guided robot system has to transition from picking a toothbrush in a blister pack, to a pill bottle, then over to a box, you’ve got items of different sizes, shapes, and weights that can be presented in totes at arbitrary orientations relative to one another,” said Martinelli. “Our system is designed with the idea that every time a robot needs to pick and transfer an item in a warehouse from one container to another, it’s going to be a slightly different task. Fulfillment centers require systems with the hand-eye coordination to handle this variability in real-time, with no advance knowledge of what’s coming next.”

Automating with Edge AI  

In many ways, COVID-19 may have helped increase machine vision adoption in e-commerce and retail settings. With foot traffic dropping in brick-and-mortar stores everywhere and instead more people shopping online for products, the dynamics of supply and demand altered considerably, so companies turned to automation, said Nash Wong, Product Marketing Manager, ADLINK Technology.

“In retail, we’ve seen increased adoption in self-checkout with touchless options and robot-based grocery delivery, for example,” he said. “On the e-commerce and fulfillment side, there has been an increased adoption of autonomous mobile robots as well as machine vision technologies for identifying and inspecting products during shipping and receiving.”

AI deployed at the edge — whether on retail floors, on the road, in the warehouse, or elsewhere — is poised to become even more important to retail and e-commerce companies over the next few years, said Wong. To help meet this demand, in-store software solution provider Proppos recently developed an intelligent and autonomous checkout system called Proppos FastPay. The system integrates a hardware inference platform from ADLINK with AI software from Pervasive Technologies.

ADLINK’s edge AI platform delivers autonomous checkout capabilities for the Proppos FastPay system.

The software runs on ADLINK’s DLAP-201-JT2 industrial-grade edge AI platform, which features an NVIDIA Jetson TX2. During operation, customers slide their tray under a camera and in 1.5 seconds, the checkout system automatically identifies all items on the tray and presents the total for the customer, who can then pay with a credit card or smartphone application.

“AI-related applications will continue to advance and provide better overall customer experiences across multiple retail and e-commerce settings,” said Wong.

In another recent fulfillment application, Evans Distribution Systems – a third-party logistics and supply chain solution company and sole online fulfillment provider in the United States for Girl Scout cookie orders – deployed ADLINK’s AI-powered machine vision solution at its warehouse in Michigan. By doing so, the company doubled it shipping volume while decreasing error rates, while providing a means for order fulfillment during the COVID-19 pandemic.

Advancing 3D Capabilities

Other factors have led to increased automation deployments as well, including higher labor wages, labor shortages, and build-to-order and lower cycle times for two-day shipping. Machine vision, according to Alex Koepsel, Technical Sales Manager, Zivid, has become necessary for the robots used in such applications to perceive their surroundings and manipulate unstructured piles of objects and packages in 3D space. Preprogrammed and “blind” robots simply don’t work for the 21st-century warehouse, he explained.

“More and more companies use machine vision and robots for accurately picking, moving, placing objects, and then properly kitting and palletizing packaged goods,” he said. “Within machine vision specifically, one major trend is the adoption of 3D cameras that are accurate, affordable, robust, and can meet the requirements of handling various SKUs, quickly, with 99% reliability or better.”

In pick-and-place applications specifically, AI solutions will help improve image quality, object recognition, pick planning, and motion planning. To compliment the improvements in intelligence, the market may see more robot-mounted sensors to enable mobile manipulators, better image quality, and reduced cost of ownership with fewer sensors per system, said Koepsel.

Zivid’s first-generation 3D camera helped with e-commerce warehouse automation for DHL’s logistics center in Beringe, Netherlands, and the camera technology has since been improved. Now, products such as the Zivid One+ is used in picking applications and palletization applications while the Zivid Two has already picked millions of pipettes, syringes, and swabs in fulfilling orders for medical kits, for example.

“In pick and place for fulfillment, the biggest challenge lies in the ability to adapt and quickly handle a variety of products and new objects that were never seen in the lab, and this begs the need for 3D sensors that are accurate, capture all the details [including color], and can quickly capture high-quality point clouds,” said Koepsel. “Our camera solves these problems by generating high-definition point clouds that are extremely accurate [0.2% trueness error], integrating color data with the 3D data, and capturing point clouds in less than 100 milliseconds.”

“Just Walk Out” Technology

One well-known example of machine vision and AI technologies being deployed in this space is the Amazon Go stores, where multiple camera types and complex AI software track selections and offer a “cashierless” experience. The company’s “Just Walk Out” Technology allows users with the Amazon Go app to enter a store, shop, and walk out of the store without lines or checkout. The technology automatically detects when products are taken or returned to the shelves while keeping track of them in a virtual cart. When the customer is done, they leave the store, and their Amazon account gets charged shortly after.

Amazon boasts a chain of Amazon Go stores, with several locations open in Seattle, New York, Chicago, San Francisco, and London. And the company now offers its technology to other retailers, whose customers can have the same “Just Walk Out” shopping experience. Other stores use their own variety of automated technologies, including Alibaba’s Hema supermarket, Albert Heijn’s NanoStore, Standard Cognition’s Standard Store, and several others.

Into the Future

Technologies including robots, 3D vision, AI software, and edge computing devices will continue to advance and push the limits of what is possible in e-commerce and retail automation. A3 will hold a number of upcoming events related to deploying advanced technologies in warehousing settings. 

Furthermore, A3 has several relevant webcasts, including Technologies Used in Automation: How to Get Started and Advances in Robot Picking, Grasping & Manipulation, Collaboration and Robotics: The Latest Solutions, Trends, Innovations and Technologies in End-of-Arm Tools, and Smart Automation: How AI is Changing Industry Today.