« Back To Vision & Imaging Industry Insights
AIA Logo

Member Since 1984


AIA - Advancing Vision + Imaging has transformed into the Association for Advancing Automation, the leading global automation trade association of the vision + imaging, robotics, motion control, and industrial AI industries.

Content Filed Under:



Bin Picking and the Challenge of the Elusive 15 Percent

POSTED 08/10/2016  | By: Winn Hardin, Contributing Editor

Machine vision has been guiding robots to pick up parts on conveyors for assembly and inspection for decades. Practically every automobile on the road has been built with help from vision-guided robotics (VGR). But when it comes to pulling those parts from a bin to place on the conveyor, the machine vision designer has to sharpen his virtual pencil and gear down.

Despite a growing number of hardware and software tools for 3D robotic guidance, bin picking remains one of the most challenging volume applications for VGR workcell designers. “From the OEM packages that I’ve seen, they’re still more set on 2D types of pattern matching for robot guidance: ‘Here’s a product and pick it up on a level surface,’” says Steve Wardell, Director of Imaging at ATS Automation (Kitchener, Ontario). Life isn’t always on the level, however, and parts in a bin are seldom stacked in ideal order for machine selection.

Wardell acknowledges that strides have been made in getting 3D information to bin-picking robots. However, he says, “the problems we find with bin-picking applications is not the 85 percent situation where you can find the most available part and go tell the robot where to get it but picking from the corner of a bin, scrap left over, or parts sitting at an angle that can’t be justified with software. You still have to pick 100 percent of the parts out that bin. Getting that last 15 percent of parts is what’s most difficult.”

Bin picking is catching on among manufacturers, product distributors, and packagers, and these end users are “getting smarter” when it comes to imaging parts in deeper, larger bins and with higher resolution, says Nick Tebeau, Manager Vision Solutions at LEONI Engineering Products & Services (Lake Orion, Michigan).

According to Tebeau, end users are realizing that to be successful with bin picking, they may need to change other parts of the process, precisely because “they won’t be able to pick 100 percent of parts out of a bin.” One possible solution is to redesign the bin itself. “That usually means a concave bin so you force parts to settle in the middle,” he says. But that can be a difficult and expensive approach. “Not every end user is willing to redesign the bin,” he says.

Bin-Picking Toolset Grows
While robot suppliers acknowledge the challenges of bin-picking applications, they add that dedicated VGR systems are helping to make it easier thanks to a greater number of hardware and software tools.

“End users and even software manufacturers often underestimate what it takes to get parts out of a bin,” says David Dechow, Staff Engineer for Intelligent Robotics/Machine Vision with FANUC America Corp. (Rochester Hills, Michigan). In some cases, the answer lies with choosing the best possible algorithm for that application. FANUC has developed a suite of 3D algorithms that can be applied anywhere from single-point location to matching 3D planes to 2D finds.

FANUC offers two 3D products. One is a 3D point product that delivers 3D point on the object, which is called the 3D laser sensor. “It’s a unique camera system using a pair of angled lasers not to do scanning but to do a single-point analysis of a planar surface to provide a 3D point in space,” he says. The second product is a 3D area sensor, which he calls the “classic point cloud device.” It uses multi-image linear structured light to offer up a full point cloud of the image.

Model Everything
While every VGR application requires accurate part location – whether it be 2D data for a part on a flat plane or 3D data for free-moving parts – bin picking needs to consider much more than just the location of the robot and the part. Gripper-related issues also come into play. “Making sure a gripper can fit into the corner of a bin and doesn’t collide with walls to pick up different parts has always been a problem,” Tebeau says.

It is critical to include end-of-arm tooling (EOAT) models into the bin-picking simulation. “If you don’t know how you’re going to pick up the part, you miss a credible component to the solution,” Tebeau says. That can include exact physical dimensions for the EOAT as well as the bin.

The same applies to the geometry of the bin. All structures within the workcell need to be taken into consideration. As an example, Tebeau offers up a scenario in which a cart holds a globe that can be picked up with a suction cup. “You can say you want to pick it up from top, side, or front, but depending on the part geometry, you might have a 30-degree ability to pick it up.” However, if you don’t know the EOAT dimensions, then the ability to estimate the pick success rate is at best a guess.

Today, all major robotic suppliers offer their own in-house bin-picking applications in answer to the steadily growing demand from customers for more accurate, repeatable assembly solutions. As bin-picking algorithms, 3D modeling software tools, and camera resolution all continue to improve – driven by steady increases in affordable computational power – bin-picking workcell designers will find a simpler path to success for this more challenging of VGR applications.