Imaging-Based Photometric and Color Measuring Instrumentation

As the underlying technology for machine vision continues to improve, one finds the migration of the technology into new applications. A good example of this is in photometry and in absolute color measuring instruments. In the former case, improvements in CCD performance, especially in signal-to-noise and dynamic range, has resulted in the development of some interesting photometric instruments. Other camera parameters leading to performance consistent with absolute measurements include high resolution (to 4096 x 4096) and color.

Driving these developments are applications in display evaluation and inspection. In this case these systems are able to map display properties over the entire area of the display. In other words, the imaging-based design effectively yields thousands of individual photometers or colorimeters. These systems can map contrast, uniformity of luminance and chromaticity from many angles.

Significantly these systems are based on digital cameras that are cooled and that come with up to 16-bit A/D. Much attention has to be paid to pixel repeatability both in terms of sensitivity and physical position. Hence, the cameras are calibrated typically for linearity on a pixel-by-pixel basis. Connectivity is generally based on PC-standards – USB2 and IEEE1394 (FireWire).

To gain further insights into this interesting application of essentially machine vision techniques to photometric and colorimetric measurements, suppliers of such instrumentation were canvassed for input for this article. The following participated:

• Thierry Leroux, Managing Director and David Horain – Eldim
• Tim Moggridge, President – Lumetrix
• Jouni Jussila, President – Specim

1. How would you describe your imaging-based product line that specifically addresses photometric and/or color measuring applications in manufacturing?

[Thierry Leroux/David Horain – Eldim] Our imaging photometer product line was designed to provide photometric and colorimetric accuracy in all situations, complying to CIE normalization documents in terms of spectral response. The imaging-based photometers or colorimeters combine the benefits of imaging and metrology together. There are 4 major criteria for such imaging-based devices:

1. Measurement range is important. The sensor shall have a good sensitivity and a high dynamic to light detection. We need to be able to measure very low signal as well as very bright ones. The sensor architecture and signal processing is important. ELDIM uses low noise electronics and cooled sensors in all applications. MURATest makes use of Kodak KAF CCD series sensors, used for many years in astronomy (to detect low light levels). The quantitative results rely on a specific design of the filters.

MURATest features 5 colored glass filters (multi-layered designed and manufactured by ELDIM).

2. The reliability also depends on the optical design and quality. A standard camera objective is not dedicated to scientific use. Most equipment on the market has to be recalibrated each time the working distance is changed. This is due to the angular magnification factor changing with focusing. MURATest design is based on a telecentric-on-CCD approach to ensure that calibration is stable whatever the measurement conditions.

3. The remote control software interface shall be flexible for easy integration of the camera in an automated process. MURATest software is delivered with an ActiveX function library and multiple software application such as: Mura defect, LED Wall testing.

4. Our products guarantee – by design – that no previous knowledge is required on the DUT (device under test) to be able to obtain the rated accuracy. In other words, ‘‘we offer absolute measurement products’‘ compared to ‘‘relative measurement products that are usually proposed when based on poor spectrally compensated products’‘.

[Tim Moggridge – Lumetrix] Our systems are designed for people who want quantitative measurements of a scene for photometric and chromatic content. CCD cameras are not very good detectors. They are fairly noisy and very non-linear. Lumetrix builds extensively characterized instruments which report results that are robust and correlate well with more traditional instruments.

[Jouni Jussila – Specim] Specim is developing and selling worldwide an ImSpector series of imaging spectrograph that can be easily attached to grayscale CCD or CMOS camera and turn the device to a line-scanning imaging spectrometer, Spectral Camera that can be used to measure target spectra. The measured spectra can then be used for colour calculations, material analysis, etc. on-line analysis. The ImSpector series offers extremely good price/quality ratio for the machine vision field.

2. What specifically differentiates the products you offer?

[Tim] We excel at the most difficult problems - high dynamic range and high accuracy. This is an immature market. The specification of measuring equipment in this industry is poorly articulated in my opinion. The sources of error for a two dimensional imaging system are much more complicated to describe than for a single spot measuring instrument. We like to work with our clients to discuss the measurement issues and uncertainties for their applications. These instruments offer a lot of information in short order, but are rarely close to the accuracy of state-of-the-art spot spectroradiometer/spot colorimeter/spot photometer techniques.

[Thierry/David] The basis of a video-colorimeter – the global spectral response of the system must match the CIE spectral curves. It is where technical knowledge makes the difference. The manufacturing of a video-colorimeter, hence, requires a step-by-step calibration during the design and integration steps. Basically the design of the colored filters is specific to a particular sensor so as to match perfectly the CIE colored reference curves. Moreover the number of filters used is larger than the number of CIE reference curves. A 5-filter design, delivers a better matching and, hence, a better accuracy whatever the stimulus spectrum (into the visible spectrum). Finally, the calibration of the full design, including the sensor, the filters and the optical parts, is done not only on the a-type reference illuminant but also on a set of light sources with narrow-band spectrum located near the spectrum locus. The challenge is to guarantee the same accuracy whatever the light source spectrum – accuracy and equipment response must be the same when measuring pure red, green or blue.

3. Would you characterize these instruments as ‘‘hardware-centric’‘ or ‘‘software-centric’‘ and why?

[Jouni] Definitely hardware-centric. We only manufacture spectrograph modules and integrated spectral cameras. We don't work on the applications where dedicated software is required.

[Thierry] We would say that they are both. They need a specific hardware for the above reasons. But we also do not underestimate the software part of the task. In order to have our equipment easily integrated in measurement or fabrication chains, early on (since 1998) we provided a normalized and easy to use API (Application Programming Interface) to our users.

[Tim] As a manufacturer we scrutinize the hardware that goes into our product, but the end users spend all of their time using the software. We pour our efforts into calibration techniques and analysis software tools. We get nearly all the hardware from camera companies. We are experts at evaluating cameras, not making them.

4. What are some specific photometric or color-measuring applications related specifically to manufacturing that you have addressed with your products? Are there some that can be characterized as ‘‘on-line’‘ applications?

[Thierry/David] Our company is mainly dealing with the display industry and is known in this field. Our current application fields are:

LED Tiles (for large displays) test and color compensation in production - a highly-demanding application requiring the benefits of a video-colorimeter is the calibration of the LED walls. In fact the LED characteristics suffer from dispersion and hence, an array of LED leads to a non-uniform lighting area. In this application, the 2D imaging sensor delivers the cartography of LEDs while the filters quantify the intensity of each LED. The acquisition and then the analysis return the intensity variation from a defined threshold. Each LED-driven potentiometer is automatically adjusted according to the computed dispersion factor. The result is then a uniform bright LED wall. The software interface is critical: it automates the acquisition, the analysis and the correction.

Other applications include:

• Front and rear projectors test in R&D and production
• LCDs and LCDs components (backlights, etc.) test in R&D and QA
• Uniformity defects on LCDs in production
• Dashboards tests in R&D, production and QA

[Tim] Display testing, lamp measurements, architectural measurements, transportation lighting and visibility are some applications that come to mind. Display and lamp measurements are being done in automated, production line fashions.

[Jouni] The list of current on-line applications is a long one. Our devices are used, for example, in color measurements of fabric, wood, ceramic tiles, plastic parts, etc. In addition to the visible range devices, many of our IR devices are also used in on-line process control applications to control moisture, material contents, etc.

5. What are the critical parameters of an imaging-based photometric or color measuring application that a prospective customer should understand?

[Tim] Color and photometric uncertainty is what users really would like to know. Unfortunately color and photometric ‘‘accuracy’‘ is almost always specified as reference to the source used in calibration. Thereby meaning only repeatability when measuring the same stimulus. Color and photometric accuracy for any specific application needs to be compared against a decent spectroradiometer. Color and photometric accuracy for narrow light sources such as LEDs is much worse than one could tell from most vendor's data sheets. 

[Jouni] The requirements for the camera and illumination are higher than in the normal imaging applications. Also, the measurement speed of this kind of line scan imaging spectrometer is not as good as standard line scan cameras can provide in purely imaging applications.

[Thierry/David] The most critical parameters are those that are accuracy related. This has already been expressed in question 1. The difficult issue is that many customers are new in the field of photometry and colorimetry and easily fooled by the impression they can derive from using a ‘‘simple’‘ digital camera. They often go through a cycle where they first think, or were told, or got the impression that whatever camera will do the task, and do not understand at that time the value of a specifically designed measurement instrument. They – in most cases as there can be situations where this is true – then experience repeatability and inter-comparisons issues. At that time, much money and – more important – valuable time can have been spent in vain.

6. What are the critical parameters of an imaging-based photometric or color measuring instrument that a prospective customer should understand?

[Jouni] The need is for more accurate calibration than in purely imaging applications. Also, users must pay much more attention to the optics quality, i.e., all the lenses and protection windows, etc. has to be matched properly to the used wavelength range where photometric measurements are done.

[Thierry/David] It must be understood that a nice picture is NOT a measurement. This is the main issue. It can take a lot of time to understand that all the measurements that have been done in the last 6 months cannot be used for anything other than ‘‘pictures’‘.

[Tim] First and foremost, a customer should hopefully be able to articulate what the parameters of the problem are and the expectations of the desired solution. Not all engineers are color scientists, so there is often a fair amount of technical guidance from our sales engineers. There are often several imaging solutions at hand - filter photometric, filter colorimetry, Bayer filter RGB colorimetry and hyper-spectral. The various solutions have cost-benefit trade-offs.

7. What are the skills required to program and operate a machine vision-based photometric or color measuring instrument?

[Thierry/David] I would say only few if the equipment is carefully designed. In such cases, the hardware-software package is hiding all the photometry and colorimetry complicated procedures, formula and calculations to the user (obviously she/he can also dig in the basic data if she/he wants). In addition to dedicated application software, such as EZMura software  (mura defect analysis software) or LEDsoft (LED Wall testing soft), the ActiveX function library is a common technology available in many applications such as LabVIEW, Visual Basic or Excel. Hence the user easily integrates the device components in its initial application or environment. Basic programming skills are enough. Our equipments are as easy to program as a 10th of macro lines in a spreadsheet.

[Tim] It helps if users are camera literate. Typical systems are not more difficult to use than a typical 35mm camera. The ability to use common Photoshop software would be useful for users of typical image analysis tools. Many of the techniques employed in Paint Shop are similar in our software.

[Jouni] Users must have a basic knowledge of how to make good spectral measurements. This mainly means understanding the system and data calibration and data normalization. The image processing methods that can be used are the standard ones.

8. How do you support your products – training, documentation, warranty, post-installation service, software revisions? Are these free or is there a fee?

[Tim] Software is supported by web download. Non-users can get our software too. So far we have not charged for any updates. Training is available for a fee on-site, and at our facility in Ottawa. Warranty is typically two years parts and labor. The calibration validity is one year.

[Jouni] Post- and pre-sales support is free, meaning that we help the customer to select the suitable hardware for their application and want to make sure that the system will work. We also give support after the purchase. If the customer has larger volumes, we also give training for our products. And the warranty for our products is the standard one-year, but since our products are reliable, we still give support after even a longer time period. Support and training is free, as well as the fixes during the valid warranty time.

[Thierry/David] The video-colorimeter is delivered as a turnkey solution, including the measurement unit, cables, remote PC, software and user guide. The system is intuitive enough to start without training. Optionally it may be provided on request through video conferencing to minimize the cost. The support and software updates are at extra charges but are usually billed at package rates with the annual calibration. Regarding after-sales service, an efficient network of local service engineers serves customer around the world.  First 12 months are free. Training and installation is optional as many customers can easily install units by themselves based on the printed or online documentation and the many tutorials. Our customers are usually not interested in upgrading their software every month. They do need support for new applications, training for new operators, calibration from time to time (6 to 12 months) and hardware maintenance to be sure that their line will be operating in all conditions. In the near-term we propose 2-year maintenance programs:

Gold package includes software maintenance, full phone and email support, ½ day of videoconference or on-site training (travel fees excluded in some cases) and calibration. It also comes with discounts on options, additional training, additional software licenses.

Platinum is Gold + ½ day of training + full hardware maintenance (no exclusion) + additional discounts on spare parts and other goodies.

9. Where do you see breakthroughs coming in the specific infrastructure technologies (hardware and software) that are the basis of the machine vision-based photometric and/or color measuring instruments that will result in further improvements in the near future – next three years?

[Jouni] There is a huge potential for spectral imaging in the market. More and more people have started to realize that this kind of technique is available and it can be used with reasonable costs. They also realize that this technique can be a solution for several problems that cannot be solved with current technique. It is difficult to say which are the most promising fields, since spectral imaging can be used in such a large amount of different fields.

[Thierry/David] We see improvements in accuracy and versatility of the products. Software packages also should come for specific applications (application oriented packages). On the hardware side, new devices may appear that provide 2D spectral information.

[Tim] Watch for CMOS cameras to get better. Watch for much improved image analysis software.

10. What specific performance improvements are anticipated driven by these forthcoming technological changes? How will they impact the use of these instruments?

[Thierry/David] A hardware improvement will be the shift for higher resolutions. The 0.4M pixels configuration is less and less sought. Today 1.5M pixels are the minimum resolution and the manufacturers already propose 6M pixels. Nevertheless the sensor price for large resolution is a drawback for such a shift.

Increased accuracy should allow easier inter-comparison and easier maintenance of the measurement chains. It should be the main goal of equipment manufacturers. We are often saddened to see the opposite - architecture that is not able by design to comply with standards and yet being advertised as ‘‘colorimeters’‘ or ‘‘photometers’‘.

[Tim] CMOS cameras will mean lower cost and higher resolution solutions.

[Jouni] Improved camera technology that provides usable sensors for spectral measurements with a reasonable price.

11. What are the main markets for these instruments and are there market changes within those markets that are driving the adoption of machine vision-based photometric and/or color measuring instruments?

[Tim] We will want to offer solutions anywhere color and luminance matters. In general, we all judge the quality of a product with our eyes. Imaging photometers, colorimeters and the like can provide an objective evaluation of how something will be perceived. Standards for production quality can be tested more completely and with greater ease then could be done with traditional spot meter techniques.

[Jouni] Like I mentioned, the range of applications is too large to start thinking what are the most important of those, and how they will change.

[Thierry/David] Our vision might be twisted as we are mainly focused on the display market. Obviously this is a tremendous one and one where convergence and availability of larger devices will push their use in more and more applications. Obviously (and for all these applications where human vision is the reference – as opposed to machine vision used for sorting or calibrating products) – compliance to practices imposed on photometry and colorimetry since 193X is a requirement.

Actually, applications are everywhere there is color and luminance check.

Main application regards display industry:

• LCD uniformity testing – TV, Monitor, Laptop, mobile phones
• PDP uniformity testing
• RPTV and FPTV uniformity testing
• Backlight uniformity
• Coating and surface defect

Growing application field is automotive:

• Car navigation system testing
• Dashboard
• Autoradio

12. What impact do these market changes have on the requirements for machine vision-based color and/or photometric instruments? And how will machine vision systems have to change to address these more demanding requirements?

[Thierry/David] Most of the devices coming from the machine-vision field (as opposed to the photometry and/or colorimetry field) that we have seen so far are based on the use of tri-color CCDs or tri-stimulus sensors. For most of them, they are not a real and reliable solution in our opinion. Based on what we know of the CIE (or other standards) they will never be - in their present state. This should come from the fact that no component (CCD, CMOS, diode, etc.) exists today that is compliant from the ground. Machine vision systems have then to adapt by adding the required hardware overlay to be enabled for photometry.

[Tim] Machine vision solutions are contrast solutions. The job of machine vision is to ‘‘identify an object from the background in the scene and make some decisions’‘. Measuring imaging photometer systems, and the like, quantify the amount of light at each pixel ‘‘in the scene’‘ - and hopefully with great accuracy. 8-bit and 24-bit RGB data files are probably inadequate. We use dual precision floating point image files for very high dynamic range, high precision data. I see the requirements of measuring imaging and machine vision as complementary. For example, once the image is acquired with a proper, calibrated imaging photometer system, the data can be exported to a user's machine vision software to do the calculations required.

13. As a supplier of machine vision-based photometric and/or color measuring instruments what are some challenges you face in marketing these instruments?

[Tim] Instrument specifications from different suppliers are difficult for clients to navigate. The ‘‘specmanship’‘ of vendors may mislead some customers. For example most suppliers say the linearity of their system is 1%. To me, that is possible over a very limited dynamic range - say 30:1 only. We overcome this limitation by a technique called ‘‘electronic bracketing’‘. You can read about it at www.lumetrix.com.

[Jouni] Even though spectral imaging instruments have been available for a decade, it is still challenging to help people to find and understand the technique and how it can be used and where it can give new possibilities.

[Thierry/David] Making customers understand that a nice color picture has nothing to do with quantitative photometry and colorimetry. Digital pictures being now ubiquitous, this is a tough challenge. We fear that it will take some time before users will come to a point that they will be challenged on a regular basis by inconsistent measurement chains.

Other challenges:

• Intra pixel video colorimetry?
• Video Spectrometry?
• 16 Mpixels?

14. What advice would you give to a company investigating the purchase of a machine vision-based photometric and/or color measuring instrument?

[Jouni] Spectral imaging is becoming more and more important in process control applications so now is a good time to start learning the advantages of this technique and start taking over their part of the upcoming technique.

[Theirry] Check the normalization documents. Get guarantees that the equipment are compliant (ask for normalized error figures like f’1, which show how far the device is from the normalized photopic curve for example; quality equipment must be under 5%, a conventional CCD camera is at 70% error or more). Check and re-check the data.

There is a very common trick which is to express the Luminance accuracy of a photometric device at only one wavelength or one unique spectral content (usually A type illuminant). This is an easy trick as this involves adjusting only one figure with one multiplicative coefficient, which is – as everyone can imagine – possible with a 0% accuracy without any particular effort. Even the poorer device is capable of such a performance. Such data has no meaning and luminance accuracy should always be stated for several illuminants instead. F’1 values must ALWAYS be given.

This kind of ‘‘behavior’‘ is counter-productive as it has pushed every maker to avoid harming itself by self-imposing harder goals than what is known as the norm in the industry. From our side, we are always giving the accuracy for 7 illuminants or more and always giving a guarantee on f’1 and on the quality at which we are complying with basic standards. This information is always given on our calibration certificates. Unfortunately only a few are doing it that way.

The risk of non-compliance to standard is to get inconsistent values between sites and instruments. These inconsistencies have then to be explained and debugged…this is our day-to-day task to cope with that. There isn’t a week where such a question does not come up. Our overall opinion is that this severely harms both makers and users. This is like measuring a length with an elastic string. Who would accept this?

[David] Equipment choice advice:

Vendor facilities

• Ability to control optical components manufacturing?
• Traceability of calibration equipments regarding international standards – CIE
• Calibration service

Hardware features

• CCD type – must be a scientific application CCD, B&W and blue spectrum response enhanced
• A/DC value and electronic chain quality (SNR) – for standard applications, a true dynamic of 85dB is required
• Color management – request spectral response of the whole equipment – request accuracy specifications – a stable accuracy should be guaranteed for the whole color spectrum and not only on A-type illuminant
• Luminance management – request F’1 value which is the best parameter to qualify the performance of the whole equipment
• Optic design and quality – ???
• Connection – with increasing CCD size, data transfer may take longer time – fast connection (as USB 2.0) is necessary

Software features

• Check software structure
• Is the software open for DIY customisation?

Any existing application solution offered?

[Tim] Think like a measurement expert, a metrologist. Being a metrologist means to question as much as possible the validity of the measurement(s). Get multiple vendors to consult and propose solutions. Enlist whatever help you can find to poke holes in the available solutions. When you know the solutions with so few holes that they still ‘‘float’‘ get more pragmatic on issues like price and delivery.