Industry Insights
« Back To Vision & Imaging Industry Insights
Member Since 1984
AIA - Advancing Vision + Imaging has transformed into the Association for Advancing Automation, the leading global automation trade association of the vision + imaging, robotics, motion control, and industrial AI industries.
Content Filed Under:
Industry:
N/A
Application:
N/A
Latest Sensors Offer More than High Rates - Resolution
POSTED 06/23/2015 | By: Winn Hardin, Contributing Editor
AIA
We’ve also talked a lot about GigE Vision, 10 Gigabit Ethernet, USB3 Vision, and Camera Link HS and the need for high-speed network protocols to handle the torrent of data out of the newest industrial cameras.
But when it comes to the evolution of the industrial camera, higher resolution, die size, and frame rates are only part of the story. Sensor manufacturers also are developing new ways to design color sensors — as in hundreds of colors, not just red, green, and blue — that practically eliminate fundamental noise limits to low-light applications and massively expand dynamic range, which could make outdoor machine vision systems a lot easier to design.
Seeing In the Dark for Semiconductors - Spectroscopy
Xenics (Leuven, Belgium) has designed and manufactured a 640 x 512 pixel, 20-μm-pitch InGaAs focal plan array (FPA) for fluorescence imaging and photon emission failure analysis applications with sensitivity in the short-wave infrared (SWIR, 0.9–1.55 μm) range.
According to Patrick Lamerichs, Xenics’ hardware engineering and group leader, the sensor’s extreme low-light sensitivity is due to several factors. “One of the biggest problems in designing sensors and cameras for very-low-light-level imaging is the auto-illumination of the detectors by the transistors in the readout circuit,” Lamerichs says. “Besides proper shielding of the detectors, our ROIC is optimized for minimal electrical activity during the integration time of the very weak signals coming from the circuit under test. For this reason, we designed the sensor using a source follower per detector (SFD) architecture, resulting in a background-limited performance of the detector.”
The sensor, which has been under development for four years and is designed and manufactured by Xenics, has been installed in the company’s Cougar-640 camera with a liquid nitrogen Dewar cooling system with ultra-low-noise readout electronics. With correlated double sampling, the camera’s dark noise is below 25 e- and can collect 2.5 frames per second at full resolution, although that rate can go higher if the sensor is windowed down. Users can select between read while integrated (RWI) or integrate then read (ITR) for maximum low-light detection.
The readout electronics also come with a 24-bit analog-digital (AD) converter to keep up with the camera’s extreme sensitivity for applications such as semiconductor failure analysis, photon emission microscopy, and nanotube fluorescence, adds Myriam Gillisjans, Xenics marketing manager for the Americas.
While specialized sensors like Xenics’ aren’t likely to set new volume sales records for industrial cameras, highly optimized sensors can be very valuable, if not critical, to certain classes of applications. For example, both the Cougar-640 and XIMEA’s (Marianka, Slovakia) xiSpec hyperspectral cameras, which are developed around a unique sensor from IMEC, target sophisticated applications in the semiconductor industry, among other applications.
“The magic of the IMEC invention is that they have developed a cleanroom-compatible process to build Fabry-Perot cavity filters directly on top of each pixel independently from the other pixels on the wafer of a commercial CMOS sensor,” says Max Larin, CEO of XIMEA.
By applying the filters on top of the sensor, Larin says IMEC has removed the need for a complex optical setup of dispersive elements, such as prisms with slit apertures. This allowed XIMEA to build what it says is the world’s smallest hyperspectral camera in a housing that measures roughly 1 square inch per side. This opens vast new opportunities for deploying hyperspectral imaging (HSI) technology, according to Larin. The xiSpec HSI camera line currently offers three models.
A line-scan version has more than 100 bands and spectral resolution of 4 nm. By scanning across the object, the camera outputs information that forms a full 3D hyperspectral data cube in which the different wavelengths make up the third dimension. Another model uses a snapshot mosaic sensor, where the filters are arranged in 4 x 4 patterns much like a Bayer filter on a visible color camera, covering a total of 16 spectral bands. This model is easy to integrate, as no special optics is required, Larin notes. Meanwhile, a snapshot tiled version of the sensor collects 32 spectral bands at 256 x 256 spatial resolution in each band with the requirement for dedicated optics. XIMEA offers HSI cameras based on each of the hyperspectral imaging (HSI) sensor models.
“This 1-cubic-inch camera also can be put on a small UAV for precision agriculture applications, surgery monitoring applications, and a plethora of others,” says Larin, adding that XIMEA is already in talks with a number of precision agricultural service providers.
Future versions of XIMEA’s HSI cameras also will include processing capabilities similar to the company’s CURRERA line of smart cameras. “Post processing is critical to hyperspectral imaging,” Larin says. “We already offer APIs for hyperspectral image processing software from Perception Park [Graz, Austria] and NICTA [Canberra, Australia].”
CMOS Continues to Advance
Low-light inspection of wafers for electrical faults isn’t the only application that will benefit from low-noise sensors. According to Keith Wetzel, director of new product development at IMPERX Inc. (Boca Raton, Florida), major CMOS sensor makers are overcoming traditional CMOS sensor drawbacks such as fixed pattern noise and “black sun” artifacts caused by oversaturated CMOS pixels overdriving the internal analog electronics, resulting in black pixels when they should be white.
“We’re seeing better image quality coming from CMOS sensors along with the higher frame rates and resolution,” says Wetzel. “In the past, you could use a look-up table to overcome fixed pattern noise in a given temperature range or spend a lot of time completely characterizing a device over temperature. But today, several image sensor manufacturers have virtually eliminated the influence of fixed pattern noise across a broad operational temperature, eliminating the need for correction in many applications.”
Wetzel adds that charge coupled device (CCD) cameras aren’t going away anytime soon despite the advances of CMOS. “CCDs are easier to make in large formats above 4/3rds of an inch and higher, so we expect them to dominate the large-format sensor size for some time to come.”
Another exciting advance for CMOS technology is the ability for sensors to increase dynamic range to more than 14-bit without having to use look-up table compression techniques. By collecting short and long exposures for each pixel during the same frame period, CMOS sensors can combine these exposures and extend the sensor’s dynamic range without any motion artifacts.
Companies like ON Semiconductor (Phoenix, Arizona) are using other experimental techniques to essentially allow frame-by-frame multiplexing between two independent acquisition frames, making one sensor into, effectively, two sensors on the same die. “You can imagine a lot of applications, such as surveillance, where you want to capture imagery at low resolution, and then capture high-resolution images when an event happens without having to worry about registering one frame to the next since the two frames were acquired at nearly the same time,” Wetzel says. “High-speed, high-resolution CMOS sensors can trade frame rate for this type of functionality. IMPERX has implemented this capability and wide dynamic range functionality into our new Cheetah C4080 and C2880 cameras. We see a lot of excitement around these new technologies, and we’ll soon see if these designs catch on in the marketplace.”
Using the power of semiconductor manufacturing, especially as it trends toward more complicated 3D structures, there is no telling what magic sensor manufacturers will develop next for machine vision applications. The question is: Will the market be ready to support the new features when they come online, or like so many brilliant ideas, will it take years before the technical advances find a paying and receptive audience?
We’ve also talked a lot about GigE Vision, 10 Gigabit Ethernet, USB3 Vision, and Camera Link HS and the need for high-speed network protocols to handle the torrent of data out of the newest industrial cameras.
But when it comes to the evolution of the industrial camera, higher resolution, die size, and frame rates are only part of the story. Sensor manufacturers also are developing new ways to design color sensors — as in hundreds of colors, not just red, green, and blue — that practically eliminate fundamental noise limits to low-light applications and massively expand dynamic range, which could make outdoor machine vision systems a lot easier to design.
Seeing In the Dark for Semiconductors - Spectroscopy
Xenics (Leuven, Belgium) has designed and manufactured a 640 x 512 pixel, 20-μm-pitch InGaAs focal plan array (FPA) for fluorescence imaging and photon emission failure analysis applications with sensitivity in the short-wave infrared (SWIR, 0.9–1.55 μm) range.
According to Patrick Lamerichs, Xenics’ hardware engineering and group leader, the sensor’s extreme low-light sensitivity is due to several factors. “One of the biggest problems in designing sensors and cameras for very-low-light-level imaging is the auto-illumination of the detectors by the transistors in the readout circuit,” Lamerichs says. “Besides proper shielding of the detectors, our ROIC is optimized for minimal electrical activity during the integration time of the very weak signals coming from the circuit under test. For this reason, we designed the sensor using a source follower per detector (SFD) architecture, resulting in a background-limited performance of the detector.”
The sensor, which has been under development for four years and is designed and manufactured by Xenics, has been installed in the company’s Cougar-640 camera with a liquid nitrogen Dewar cooling system with ultra-low-noise readout electronics. With correlated double sampling, the camera’s dark noise is below 25 e- and can collect 2.5 frames per second at full resolution, although that rate can go higher if the sensor is windowed down. Users can select between read while integrated (RWI) or integrate then read (ITR) for maximum low-light detection.
The readout electronics also come with a 24-bit analog-digital (AD) converter to keep up with the camera’s extreme sensitivity for applications such as semiconductor failure analysis, photon emission microscopy, and nanotube fluorescence, adds Myriam Gillisjans, Xenics marketing manager for the Americas.
While specialized sensors like Xenics’ aren’t likely to set new volume sales records for industrial cameras, highly optimized sensors can be very valuable, if not critical, to certain classes of applications. For example, both the Cougar-640 and XIMEA’s (Marianka, Slovakia) xiSpec hyperspectral cameras, which are developed around a unique sensor from IMEC, target sophisticated applications in the semiconductor industry, among other applications.
“The magic of the IMEC invention is that they have developed a cleanroom-compatible process to build Fabry-Perot cavity filters directly on top of each pixel independently from the other pixels on the wafer of a commercial CMOS sensor,” says Max Larin, CEO of XIMEA.
By applying the filters on top of the sensor, Larin says IMEC has removed the need for a complex optical setup of dispersive elements, such as prisms with slit apertures. This allowed XIMEA to build what it says is the world’s smallest hyperspectral camera in a housing that measures roughly 1 square inch per side. This opens vast new opportunities for deploying hyperspectral imaging (HSI) technology, according to Larin. The xiSpec HSI camera line currently offers three models.
A line-scan version has more than 100 bands and spectral resolution of 4 nm. By scanning across the object, the camera outputs information that forms a full 3D hyperspectral data cube in which the different wavelengths make up the third dimension. Another model uses a snapshot mosaic sensor, where the filters are arranged in 4 x 4 patterns much like a Bayer filter on a visible color camera, covering a total of 16 spectral bands. This model is easy to integrate, as no special optics is required, Larin notes. Meanwhile, a snapshot tiled version of the sensor collects 32 spectral bands at 256 x 256 spatial resolution in each band with the requirement for dedicated optics. XIMEA offers HSI cameras based on each of the hyperspectral imaging (HSI) sensor models.
“This 1-cubic-inch camera also can be put on a small UAV for precision agriculture applications, surgery monitoring applications, and a plethora of others,” says Larin, adding that XIMEA is already in talks with a number of precision agricultural service providers.
Future versions of XIMEA’s HSI cameras also will include processing capabilities similar to the company’s CURRERA line of smart cameras. “Post processing is critical to hyperspectral imaging,” Larin says. “We already offer APIs for hyperspectral image processing software from Perception Park [Graz, Austria] and NICTA [Canberra, Australia].”
CMOS Continues to Advance
Low-light inspection of wafers for electrical faults isn’t the only application that will benefit from low-noise sensors. According to Keith Wetzel, director of new product development at IMPERX Inc. (Boca Raton, Florida), major CMOS sensor makers are overcoming traditional CMOS sensor drawbacks such as fixed pattern noise and “black sun” artifacts caused by oversaturated CMOS pixels overdriving the internal analog electronics, resulting in black pixels when they should be white.
“We’re seeing better image quality coming from CMOS sensors along with the higher frame rates and resolution,” says Wetzel. “In the past, you could use a look-up table to overcome fixed pattern noise in a given temperature range or spend a lot of time completely characterizing a device over temperature. But today, several image sensor manufacturers have virtually eliminated the influence of fixed pattern noise across a broad operational temperature, eliminating the need for correction in many applications.”
Wetzel adds that charge coupled device (CCD) cameras aren’t going away anytime soon despite the advances of CMOS. “CCDs are easier to make in large formats above 4/3rds of an inch and higher, so we expect them to dominate the large-format sensor size for some time to come.”
Another exciting advance for CMOS technology is the ability for sensors to increase dynamic range to more than 14-bit without having to use look-up table compression techniques. By collecting short and long exposures for each pixel during the same frame period, CMOS sensors can combine these exposures and extend the sensor’s dynamic range without any motion artifacts.
Companies like ON Semiconductor (Phoenix, Arizona) are using other experimental techniques to essentially allow frame-by-frame multiplexing between two independent acquisition frames, making one sensor into, effectively, two sensors on the same die. “You can imagine a lot of applications, such as surveillance, where you want to capture imagery at low resolution, and then capture high-resolution images when an event happens without having to worry about registering one frame to the next since the two frames were acquired at nearly the same time,” Wetzel says. “High-speed, high-resolution CMOS sensors can trade frame rate for this type of functionality. IMPERX has implemented this capability and wide dynamic range functionality into our new Cheetah C4080 and C2880 cameras. We see a lot of excitement around these new technologies, and we’ll soon see if these designs catch on in the marketplace.”
Using the power of semiconductor manufacturing, especially as it trends toward more complicated 3D structures, there is no telling what magic sensor manufacturers will develop next for machine vision applications. The question is: Will the market be ready to support the new features when they come online, or like so many brilliant ideas, will it take years before the technical advances find a paying and receptive audience?
Vision in Life Sciences
This content is part of the Vision in Life Sciences curated collection. To learn more about Vision in Life Sciences, click here.