Imaging Takes a Front Seat in Futuristic Automobiles

(Barrington, NJ) and Raytheon (Lexington, MA), grabbed headlines in the popular press with the first night vision heads up display enhancement for passenger vehicles. Low-cost microbolometers developed for the US Army extended the driver's night vision three times farther than standard high-beam headlights while making IR imaging affordable enough to be offered as optional equipment on luxury automobiles.

Figure 1: Advanced airbag systems use infrared imaging systems to detect the passenger's position by flooding the passenger compartment with eye-safe invisible light, while CMOS cameras keep a watch out for children and small adults and adapt airbag deployment accordingly.

Since then, cameras inside vehicles have been limited mostly to TV shows such as COPS. However, new Federal regulations under consideration by the National Highway Traffic Safety Administration (NHTSA) will require airbag designs that can protect drivers and passengers at higher collision speeds. These new systems - expected to hit the consumer market by 2006 -- will need to deploy faster to absorb the higher collision speeds. Unfortunately, faster deploying airbags means increasing the risk of airbag-dependent injury to children and smaller adults. 'The balancing acts is that if you make the airbag more aggressive, you have to then take additional measures so that it's also safe for the fifth percentile and smaller folks,' explained Bill Futz, systems engineer at Delphi Delco Electronic Systems (Kokomo, IN).

Imaging continues to look forward
Despite a retreat from the public eye, night vision, or passive IR imaging, looks like it will continue to garner interest from the automobile sector, adding to a general momentum in the uses of imaging and image processing in automobile systems. In December of 2001, Indigo Systems Corp. (Santa Barbara, CA) joined forces with automotive safety systems designer, Autoliv (Farmington Hills, MI) and General Motors (Detroit, MI) to announce development of a thermally sensitive focal plane array (FPA) night vision system for automobiles. Officials at Indigo added that GM is interested in pushing standards for the night vision systems in addition to new designs.

While the actual demand for optional equipment such as the night vision systems is debatable, demand for Federally mandated safety equipment is not. Analysts predict that by 2006 - the suggested deadline for new NHTSA requiring advanced or 'smart' airbag systems - occupant sensing systems will be a US$5.5 billion market.

Scott Upham, analyst for global automotive safety systems and president of Ann Arbor-based Providata, said that advanced airbag systems are likely to use a fusion of sensor systems, many of which could include some form of imaging system. 'There are many types of sensors from force sensing resister (FSR) matrix sensors that create a thin map under the seat and sense the weight distribution of the occupant, to ranging sensors and image processing algorithms that determine whether the occupant is a bag of groceries, a rear-facing child seat or an occupant,' Upham said. 'No one technology is 100 percent…so most car makers are utilizing redundant technologies that compliment each other, like weight sensors and [infrared].'

Active rather than passive imaging
While the automobile vision enhancement systems such as the Cadillac and Autoliv systems are passive technologies that sense heat sources in front of the car, advanced airbag systems tend towards active IR systems with reduced imaging capabilities, such as those from Delphi Delco Electronic Systems and Siemens. Bosch has also developed a system using ultrasound transducers to generate 3D occupant location data.

According to Delphi's Fultz, 'What we've developed is an active IR imaging system that scans the passenger side of the interior of the vehicle using IR light beams in order to develop a spatial representation of what's in the passenger seat based on ranging point-to-point information from individual beams.' Delphi uses a single IR source in the eye-safe range around 1.3 microns. The light is projected through a segmented optic that creates 30 individual beams, which are reflected off the passenger (or object in the passenger seat).

Delphi is evaluating both position-sensitive photodiodes and CMOS cameras. 'It will boil down to what are the ultimate functional requirements of the system and the cost constraints whether we do photodetectors or cameras.'

Both Delphi and Siemens are developing 'stand alone' systems with embedded processors, however, Siemens' design reflects greater flexibility and functionality than just advanced airbag deployment. Siemens is also using an eye-safe diode laser with a CMOS sensor. According to Derrick Zechmair, vice president of sales and marketing for Siemens VDO electronics division (Schwalbach, Germany), the multiple double short-time integration (MDSI) advanced airbag system is evaluating 8x8 and 16x16-pixel CMOS sensors with on chip A/D conversion. 'We went with CMOS because of the dynamic range, A/D conversion, low cost and standard manufacturing process,' explained Zechmair.

Processing the load
The Siemens system collects range data from approximately 1000 points every 80 ms. 'We're shooting for 20 to 30 ms for dynamic out-of-position measurements,' Zechmair said. To accomplish this amount of processing in such a short time, Siemens uses a Motorola PowerPC processor.

The PowerPC offers several advantages, Zechmair said, including quick time to market. 'We want to be the first ones to bring this product to market, so we are focusing on standard components because they give us the flexibility to introduce the product quickly. After that, maybe we can think about ASICs or a DSP to reduce chip costs.' Siemens hopes to have the system in production models by 2003-2004.

Delphi is using a 16-bit fixed-point processor, although 8-bit would work fine. 'The industry has made it so that 16-bit processors are cheaper than 8-bit. It's like memory, it's cheaper to buy 128 MB than 32 MB,' said Delphi's Fultz. 'Ultimately, the system technologies will be more based on the final [NHTSA] requirements…the actual regulations are as yet undefined…[the final decision] will depend on the manufacturers familiarity and toolsets. DSPs are becoming lower in cost and the tools are more available for automotive systems than they have been in the past,' Fultz said. 'In many cases the algorithm will guide the choice. If you're heavy on pattern recognition in image processing, you're going to push to DSPs that have dedicated mask processes embedded in them. This has to be an embedded solution, no framegrabbers and PCs here.'

Tier-one system developers such as Delphi are interested in any technological developments that improve pattern recognition, image processing and the algorithms behind them, added Fultz. 'In essence, it's a low-resolution imaging system, so any algorithms for higher resolution image processing have interest for us. But again, since we're in a safety critical industry, we are slower to adopt these.'

Safety critical or not, Siemens sees more in the future for its imaging system. 'For the time being, we don't see the necessity to come up with low price computing systems. The focus is that the microcontroller chip set that provides us with sufficient computing power. But we want flexibility. We want to use the 3D camera for different applications, such as using driver body motion to control the radio and other automobile systems,' Zechmair said.