Camera, Light Advances Push Machine Vision Outdoors

Like the Nosferatu, sunlight has long been the bane of outdoor machine vision applications.

While the sun is physically one of the best sources of illumination for machine vision – providing a stable, flat light source across the visible and IR spectrum – clouds and obscurants are a wooden stake in the heart of the aspiring machine vision system designer. Wildly fluctuating ambient light conditions caused by the sun’s path across the sky or changing weather conditions can challenge the most advanced on-chip camera controls. The usual response for brave souls bent on solving surveillance, agriculture, or other outdoor application challenges is to bury the system in an enclosure to hide the camera from the sunlight and replace the sun with more predictable manmade light sources to fill the darkness.

Today, advances in camera sensor designs and solid-state illumination are helping to overcome the obstacles facing outdoor machine vision applications in entertainment, agriculture, security, and military.

“We’re seeing a lot more of our customers willing to put machine vision systems outside,” says Dr. Rex Lee, president of Pyramid Imaging, Inc. (Tampa, Florida). “Some are traditional surveillance types of clients in border security and unmanned vehicle applications. Others are in the intelligent transportation industry, and we’ve had some great applications recently in the entertainment industry.”

Given Pyramid Imaging’s location in Tampa — just a short drive down the road from Orlando, one of the busiest tourist destinations in the U.S. — it makes sense that the company would be on the forefront of outdoor entertainment applications. According to Dr. Lee, his team was surprised to discover that an imaging system designed to capture people’s faces as they scream their way along a roller coaster would be among the most challenging.

“The entertainment industry is very critical when it comes to color rendering,” Dr. Lee says. “They learned a long time ago that if you appeared to have a sunburn, or if the colors were wrong on your favorite sports jersey, [customers] probably wouldn’t buy the image.”

The solution was choosing the right camera with solid color response in all light conditions with a consistent white LED light source set to strobe during the camera’s open exposure period. This allowed the camera to capture very consistent images regardless of the time of day or ambient light conditions. “The new high-powered LEDs have really helped strobing applications without adding to maintenance or replacement costs,” Lee says.

When it came to choosing the right cameras, data sheets weren’t enough. Dr. Lee’s team had to test cameras against varying fabrics and textures as test targets. “Corduroy was a real problem, especially varying shades of red, as well moiré effects due to repeating patterns in the material,” he notes.

Finally, timing was everything when it came to system design. “That’s probably one of the most important tricks to strobe with a bright light and expose the camera during the strobe duration, which requires very accurate triggering of the camera,” Dr. Lee explains. “That’s one major reason why machine vision cameras are employed in these applications instead of consumer-grade cameras.”

He sees the same for intelligent transportation systems (ITS) and license plate readers, “where the light has to be strobed very quickly so as to not distract the driver but capture a readable image. It’s still early, but the new cameras with advanced dynamic range and very fast auto gain and exposure control may allow us to get a good image without strobing a light. But so far, we’re not seeing a lot of call for these newer cameras.”

New CMOS Sensors Enable Outdoor Applications
Few announcements have had the far-ranging impact of Sony’s intention to discontinue production on one its most active 200-mm CCD production lines.

According to Marci Garcia, FCB & MV product marketing manager for Sony’s Visual Imaging Solutions, Component Solutions Business Division, several factors have contributed to the cessation of CCD sensors in favor of CMOS sensors. Sony has acquired a number of companies that specialize in wide-dynamic-range CMOS sensors, and with the introduction of the Pregius image sensor, Sony can now offer CMOS dynamic range while surpassing CCD-level image quality — and with a global shutter, instead of CMOS’s traditional rolling shutter architecture.

“The global shutter CMOS cameras work great in intelligent transportation systems, as well as traffic and waterway monitoring where analog cameras were traditionally used,” says Garcia. “But it’s not just about improving the sensors. It’s also about putting more image processing on the chip for defective pixel correction, shading correction, area gain – all these features help you reduce noise and boost performance.”

Garcia also points to Sony’s recently launched Starvis color sensor. These back-illuminated sensors offer incredible light sensitivity and color rendering, even in extreme low-light conditions – the perfect solution for next-generation roller coaster scream captures as well as critical surveillance and ITS systems.

“With Starvis, you can work with shutter speed and be able to capture license plates in the dark with very little ambient light or possibly without the use of strobe lights at all,” Garcia says. “A great example is Sony’s A7S II sensor, which has an ISO sensitivity rating of more than 409,000. This ultra-low-light performance is impossible using CCD sensors.”

Sony’s position in the marketplace, along with its plan to end CCD production, has helped other CMOS sensor makers. “Our industrial sensors, like the Sapphire and Ruby families, are widely used by our machine vision camera manufacturing partners because they offer both global and rolling shutter without requiring a compromise on performance, which is critical to industrial applications,” explains Gareth Powell, marketing manager for e2v (St Egreve, France). “Although for rolling-shutter CMOS sensors the scene needs to be illuminated during the entire frame exposure period, with the global shutter, you can illuminate it for less than the frame period and save considerable power, so It’s not only about motion.”

Francis Pang, e2v Application Engineer, says “In general, that’s what the machine vision industry does best: They take a sensor and apply it to many different applications. At e2v, our application-specific standard product team serves those customers, but we also look for applications where we can further optimize the sensor at the pixel level rather than just make a one-size-fits-all sensor.”

As an example, Pang references e2v’s Onyx 1.3MP image sensor with range-gating built in. “This low-light sensor is designed for surveillance, and by adding time gating, we allow the sensor to see through fog and obscurants,” Pang says. “This product is derived from our Ruby sensor, which has 50% quantum efficiency above 850 nm, which is great for industrial and outdoor surveillance applications that use both visible and infrared light.”

Despite momentum behind CMOS imagers, CCD still has a few tricks up its sleeve for low-light applications with area-specific processing capabilities like the latest CMOS sensors, according to XIMEA (Marianka, Slovakia) CEO Max Larin. Larin points to the KAE-02150 CCD sensor from ON Semiconductor, which has the ability to sample the charge captured by a single pixel without destroying it, and then directing the charge through a low- or high-gain amplifier depending on the light level in that pixel.

Imaging experts like Pyramid’s Dr. Lee are quick to indicate, however, that having a camera sensitive in the IR and visible is only part of the challenge. “If you’re going to look at objects and you want to capture as much light as you can in the IR, you need to make sure your lenses are corrected to those specific wavelengths,” he says. “Otherwise, there will be some blurring.”